[PATCH 00/15] media: Add driver for the Raspberry Pi <5 CSI-2 receiver

[PATCH 00/15] media: Add driver for the Raspberry Pi <5 CSI-2 receiver

[Laurent Pinchart]

Hello everybody,

This patch series adds a new driver for the BCM2835 (and derivative)
CCP2/CSI2 camera interface named Unicam. This IP core is found in the
VC4-based Raspberry Pi, namely the Pi Zero, Pi 3 and Pi 4.

Camera support for Raspberry Pi 4 currently relies on a downstream
Unicam driver that live in the Raspberry Pi kernel tree ([1]). The
driver uses the V4L2 API, but works around the lack of features in V4L2
to properly support sensor embedded data. Since the Unicam driver
development by Raspberry Pi, some of those features have been merged in
the kernel (namely the V4L2 streams API) or are being developed (namely
generic metadata formats and subdev internal pads), with patches posted
for review on the linux-media mailing list ([2]).

This new upstream driver is based on the downstream code, extensively
reworked to use the new V4L2 APIs.

The series is based on top of a merge of

- v7 of the generic metadata and internal pads, rebased on v6.8-rc5 ([3])
- the downstream ISP driver ported to mainline ([4])

For convenience, it can be found in [5]. Note that the ISP driver is
getting upstreamed separately.

The series starts with five patches that add support for streams and
embedded data to the imx219 driver (01/15 to 05/15). Patches 06/15 to
09/15 then add the Unicam driver, with new V4L2 pixel formats (06/15 and
07/15) and DT bindings (08/15) The remaining patches cover DT
integration (10/15 to 14/15) with a sample DT overlay for the IMX219
camera module (15/15).

The patches have been tested on a Raspberry Pi 4 using an IMX219 camera
module (the Raspberry Pi camera v2), with libcamera. Updates are needed
to libcamera to use the new V4L2 APIs, patches have been posted to [6].
For manual testing with media-ctl, corresponding API updates to
v4l-utils are available at [7].

The changes to the imx219 driver effectively define the interface that
raw sensors should expose to userspace. This needs to be documented
explicitly. I would like this series to first get a review round, which
will likely raise API questions. I will then work on the documentation.

As a summary, more work is needed, but I believe we're nearing
completion. This series, along with the libcamera support, help
validating the new kernel APIs. We have in my opinion reached a point
where we can start converting other sensor drivers from the downstream
Raspberry Pi kernel to the standard APIs for embedded data, as well as
integrating the APIs in the Raspberry Pi 5 CFE driver.

[1] https://github.com/raspberrypi/linux/tree/rpi-6.1.y/drivers/media/platform/bcm2835
[2] https://lore.kernel.org/linux-media/20231106122539.1268265-1-sakari.ailus@linux.intel.com
[3] https://git.kernel.org/pub/scm/linux/kernel/git/pinchartl/linux.git/log/?h=rpi/v6.8/metadata/v7
[4] https://git.kernel.org/pub/scm/linux/kernel/git/pinchartl/linux.git/log/?h=rpi/v6.8/isp/v2
[5] https://git.kernel.org/pub/scm/linux/kernel/git/pinchartl/linux.git/log/?h=rpi/v6.8/unicam/next
[6] https://lists.libcamera.org/pipermail/libcamera-devel/2024-March/040711.html
[7] https://git.linuxtv.org/pinchartl/v4l-utils.git/log/?h=metadata

Here's the mandatory v4l2-compliance report. There are 5 errors, which
are caused by v4l2-compliance not supporting the new embedded data APIs
yet. This will be fixed and patches for v4l2-compliance will be
submitted (alongside patches to media-ctl).

v4l2-compliance 1.27.0-5180, 64 bits, 64-bit time_t
v4l2-compliance SHA: c14579f7e5f5 2024-03-01 20:29:24

Compliance test for unicam device /dev/media0:

Media Driver Info:
	Driver name      : unicam
	Model            : unicam
	Serial           :
	Bus info         : platform:fe801000.csi
	Media version    : 6.8.0
	Hardware revision: 0x00000000 (0)
	Driver version   : 6.8.0

Required ioctls:
	test MEDIA_IOC_DEVICE_INFO: OK
	test invalid ioctls: OK

Allow for multiple opens:
	test second /dev/media0 open: OK
	test MEDIA_IOC_DEVICE_INFO: OK
	test for unlimited opens: OK

Media Controller ioctls:
	test MEDIA_IOC_G_TOPOLOGY: OK
	Entities: 4 Interfaces: 4 Pads: 8 Links: 7
	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
	test MEDIA_IOC_SETUP_LINK: OK

Total for unicam device /dev/media0: 8, Succeeded: 8, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for unicam device /dev/video0:

Driver Info:
	Driver name      : unicam
	Card type        : unicam
	Bus info         : platform:fe801000.csi
	Driver version   : 6.8.0
	Capabilities     : 0xa4a00001
		Video Capture
		Metadata Capture
		I/O MC
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x24200001
		Video Capture
		I/O MC
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : unicam
	Model            : unicam
	Serial           :
	Bus info         : platform:fe801000.csi
	Media version    : 6.8.0
	Hardware revision: 0x00000000 (0)
	Driver version   : 6.8.0
Interface Info:
	ID               : 0x0300000d
	Type             : V4L Video
Entity Info:
	ID               : 0x0000000b (11)
	Name             : unicam-image
	Function         : V4L2 I/O
	Flags            : default
	Pad 0x0100000c   : 0: Sink
	  Link 0x0200000f: from remote pad 0x1000003 of entity 'unicam' (Video Interface Bridge): Data, Enabled, Immutable

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK
	test invalid ioctls: OK

Allow for multiple opens:
	test second /dev/video0 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls (Input 0):
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls (Input 0):
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test CREATE_BUFS maximum buffers: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK (Not Supported)

Total for unicam device /dev/video0: 47, Succeeded: 47, Failed: 0, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for unicam device /dev/video1:

Driver Info:
	Driver name      : unicam
	Card type        : unicam
	Bus info         : platform:fe801000.csi
	Driver version   : 6.8.0
	Capabilities     : 0xa4a00001
		Video Capture
		Metadata Capture
		I/O MC
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x24a00000
		Metadata Capture
		I/O MC
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : unicam
	Model            : unicam
	Serial           :
	Bus info         : platform:fe801000.csi
	Media version    : 6.8.0
	Hardware revision: 0x00000000 (0)
	Driver version   : 6.8.0
Interface Info:
	ID               : 0x03000013
	Type             : V4L Video
Entity Info:
	ID               : 0x00000011 (17)
	Name             : unicam-embedded
	Function         : V4L2 I/O
	Pad 0x01000012   : 0: Sink
	  Link 0x02000015: from remote pad 0x1000004 of entity 'unicam' (Video Interface Bridge): Data, Enabled, Immutable

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK
	test invalid ioctls: OK

Allow for multiple opens:
	test second /dev/video1 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls (Input 0):
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
		fail: v4l2-test-formats.cpp(590): dataformat 00003ff8 (x?) for buftype 13 not reported by ENUM_FMT
	test VIDIOC_TRY_FMT: FAIL
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls (Input 0):
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test CREATE_BUFS maximum buffers: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK (Not Supported)

Total for unicam device /dev/video1: 47, Succeeded: 46, Failed: 1, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for unicam device /dev/v4l-subdev0:

Driver Info:
	Driver version   : 6.8.0
	Capabilities     : 0x00000002
		Streams Support
	Client Capabilities: 0x0000000000000003
streams interval-uses-which Media Driver Info:
	Driver name      : unicam
	Model            : unicam
	Serial           :
	Bus info         : platform:fe801000.csi
	Media version    : 6.8.0
	Hardware revision: 0x00000000 (0)
	Driver version   : 6.8.0
Interface Info:
	ID               : 0x03000017
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000001 (1)
	Name             : unicam
	Function         : Video Interface Bridge
	Pad 0x01000002   : 0: Sink
	  Link 0x02000009: from remote pad 0x1000006 of entity 'imx219 5-0010' (Camera Sensor): Data, Enabled, Immutable
	Pad 0x01000003   : 1: Source
	  Link 0x0200000f: to remote pad 0x100000c of entity 'unicam-image' (V4L2 I/O): Data, Enabled, Immutable
	Pad 0x01000004   : 2: Source
	  Link 0x02000015: to remote pad 0x1000012 of entity 'unicam-embedded' (V4L2 I/O): Data, Enabled, Immutable

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_SUDBEV_QUERYCAP: OK
	test invalid ioctls: OK

Allow for multiple opens:
	test second /dev/v4l-subdev0 open: OK
	test VIDIOC_SUBDEV_QUERYCAP: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device routing ioctls:
		fail: v4l2-test-subdevs.cpp(631): !(source->flags & MEDIA_PAD_FL_SOURCE)
	test Try VIDIOC_SUBDEV_G_ROUTING/VIDIOC_SUBDEV_S_ROUTING: FAIL
		fail: v4l2-test-subdevs.cpp(631): !(source->flags & MEDIA_PAD_FL_SOURCE)
	test Active VIDIOC_SUBDEV_G_ROUTING/VIDIOC_SUBDEV_S_ROUTING: FAIL

Sub-Device ioctls (Sink Pad 0):

Sub-Device ioctls (Source Pad 1):

Sub-Device ioctls (Source Pad 2):

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test CREATE_BUFS maximum buffers: OK
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for unicam device /dev/v4l-subdev0: 47, Succeeded: 45, Failed: 2, Warnings: 0
--------------------------------------------------------------------------------
Compliance test for unicam device /dev/v4l-subdev1:

Driver Info:
	Driver version   : 6.8.0
	Capabilities     : 0x00000002
		Streams Support
	Client Capabilities: 0x0000000000000003
streams interval-uses-which Media Driver Info:
	Driver name      : unicam
	Model            : unicam
	Serial           :
	Bus info         : platform:fe801000.csi
	Media version    : 6.8.0
	Hardware revision: 0x00000000 (0)
	Driver version   : 6.8.0
Interface Info:
	ID               : 0x03000019
	Type             : V4L Sub-Device
Entity Info:
	ID               : 0x00000005 (5)
	Name             : imx219 5-0010
	Function         : Camera Sensor
	Pad 0x01000006   : 0: Source
	  Link 0x02000009: to remote pad 0x1000002 of entity 'unicam' (Video Interface Bridge): Data, Enabled, Immutable
	Pad 0x01000007   : 1: Sink, 00000008
	Pad 0x01000008   : 2: Sink, 00000008

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_SUDBEV_QUERYCAP: OK
	test invalid ioctls: OK

Allow for multiple opens:
	test second /dev/v4l-subdev1 open: OK
	test VIDIOC_SUBDEV_QUERYCAP: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Sub-Device routing ioctls:
		fail: v4l2-test-subdevs.cpp(630): !(sink->flags & MEDIA_PAD_FL_SINK)
	test Try VIDIOC_SUBDEV_G_ROUTING/VIDIOC_SUBDEV_S_ROUTING: FAIL
		fail: v4l2-test-subdevs.cpp(630): !(sink->flags & MEDIA_PAD_FL_SINK)
	test Active VIDIOC_SUBDEV_G_ROUTING/VIDIOC_SUBDEV_S_ROUTING: FAIL

Sub-Device ioctls (Source Pad 0):

Sub-Device ioctls (Sink Pad 1):

Sub-Device ioctls (Sink Pad 2):

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 20 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK (Not Supported)
	test VIDIOC_TRY_FMT: OK (Not Supported)
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
	test CREATE_BUFS maximum buffers: OK
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for unicam device /dev/v4l-subdev1: 47, Succeeded: 45, Failed: 2, Warnings: 0

Grand Total for unicam device /dev/media0: 196, Succeeded: 191, Failed: 5, Warnings: 0


Dave Stevenson (2):
  dt-bindings: media: Add bindings for bcm2835-unicam
  media: bcm2835-unicam: Add support for CCP2/CSI2 camera interface

Jean-Michel Hautbois (3):
  media: v4l: Add V4L2-PIX-FMT-Y12P format
  media: v4l: Add V4L2-PIX-FMT-Y14P format
  ARM: dts: bcm2835: Add Unicam CSI nodes

Laurent Pinchart (8):
  media: i2c: imx219: Inline imx219_update_pad_format() in its caller
  media: i2c: imx219: Add internal image sink pad
  media: i2c: imx219: Report internal routes to userspace
  media: i2c: imx219: Report streams using frame descriptors
  media: i2c: imx219: Add embedded data support
  ARM: dts: bcm2835-rpi: Move firmware-clocks from bcm2711 to bcm2835
  ARM: dts: bcm2711-rpi-4-b: Add CAM1 regulator
  [DNI] arm64: dts: broadcom: Add overlay for Raspberry Pi 4B IMX219
    camera

Uwe Kleine-König (2):
  ARM: dts: bcm2711-rpi: Add pinctrl-based multiplexing for I2C0
  ARM: dts: bcm2711-rpi-cm4-io: Add RTC on I2C0

 .../bindings/media/brcm,bcm2835-unicam.yaml   |  117 +
 .../media/v4l/pixfmt-yuv-luma.rst             |   48 +
 MAINTAINERS                                   |    7 +
 .../arm/boot/dts/broadcom/bcm2711-rpi-4-b.dts |    7 +
 .../boot/dts/broadcom/bcm2711-rpi-cm4-io.dts  |   17 +
 arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi   |   36 +-
 arch/arm/boot/dts/broadcom/bcm2711.dtsi       |    8 +
 arch/arm/boot/dts/broadcom/bcm2835-rpi.dtsi   |   19 +
 arch/arm/boot/dts/broadcom/bcm283x.dtsi       |   24 +
 arch/arm64/boot/dts/broadcom/Makefile         |    2 +
 .../dts/broadcom/bcm2711-rpi-4-b-imx219.dtso  |   65 +
 drivers/media/i2c/imx219.c                    |  419 ++-
 drivers/media/platform/Kconfig                |    1 +
 drivers/media/platform/Makefile               |    1 +
 drivers/media/platform/broadcom/Kconfig       |   23 +
 drivers/media/platform/broadcom/Makefile      |    3 +
 .../platform/broadcom/bcm2835-unicam-regs.h   |  255 ++
 .../media/platform/broadcom/bcm2835-unicam.c  | 2607 +++++++++++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c          |    2 +
 include/uapi/linux/videodev2.h                |    2 +
 20 files changed, 3588 insertions(+), 75 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
 create mode 100644 arch/arm64/boot/dts/broadcom/bcm2711-rpi-4-b-imx219.dtso
 create mode 100644 drivers/media/platform/broadcom/Kconfig
 create mode 100644 drivers/media/platform/broadcom/Makefile
 create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam-regs.h
 create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam.c


base-commit: 384a3aa6b48b29b4c71130c028db78484a0f0ab4
-- 
Regards,

Laurent Pinchart

[PATCH v6 01/15] media: i2c: imx219: Inline imx219_update_pad_format() in its caller

[Laurent Pinchart]

The imx219_update_pad_format() is short and called from a single place,
in imx219_set_pad_format(). Inline the code in the caller to keep all
format adjustements grouped in a single place and improve readability.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/i2c/imx219.c | 28 ++++++++++++----------------
 1 file changed, 12 insertions(+), 16 deletions(-)

diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
index e17ef2e9d9d0..3878da50860e 100644
--- a/drivers/media/i2c/imx219.c
+++ b/drivers/media/i2c/imx219.c
@@ -758,21 +758,6 @@ static int imx219_set_stream(struct v4l2_subdev *sd, int enable)
 	return ret;
 }
 
-static void imx219_update_pad_format(struct imx219 *imx219,
-				     const struct imx219_mode *mode,
-				     struct v4l2_mbus_framefmt *fmt, u32 code)
-{
-	/* Bayer order varies with flips */
-	fmt->code = imx219_get_format_code(imx219, code);
-	fmt->width = mode->width;
-	fmt->height = mode->height;
-	fmt->field = V4L2_FIELD_NONE;
-	fmt->colorspace = V4L2_COLORSPACE_RAW;
-	fmt->ycbcr_enc = V4L2_YCBCR_ENC_601;
-	fmt->quantization = V4L2_QUANTIZATION_FULL_RANGE;
-	fmt->xfer_func = V4L2_XFER_FUNC_NONE;
-}
-
 static int imx219_enum_mbus_code(struct v4l2_subdev *sd,
 				 struct v4l2_subdev_state *state,
 				 struct v4l2_subdev_mbus_code_enum *code)
@@ -819,12 +804,23 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 	struct v4l2_rect *crop;
 	unsigned int bin_h, bin_v;
 
+	/*
+	 * Adjust the requested format to match the closest mode. The Bayer
+	 * order varies with flips.
+	 */
 	mode = v4l2_find_nearest_size(supported_modes,
 				      ARRAY_SIZE(supported_modes),
 				      width, height,
 				      fmt->format.width, fmt->format.height);
 
-	imx219_update_pad_format(imx219, mode, &fmt->format, fmt->format.code);
+	fmt->format.code = imx219_get_format_code(imx219, fmt->format.code);
+	fmt->format.width = mode->width;
+	fmt->format.height = mode->height;
+	fmt->format.field = V4L2_FIELD_NONE;
+	fmt->format.colorspace = V4L2_COLORSPACE_RAW;
+	fmt->format.ycbcr_enc = V4L2_YCBCR_ENC_601;
+	fmt->format.quantization = V4L2_QUANTIZATION_FULL_RANGE;
+	fmt->format.xfer_func = V4L2_XFER_FUNC_NONE;
 
 	format = v4l2_subdev_state_get_format(state, 0);
 	*format = fmt->format;
-- 
Regards,

Laurent Pinchart

[PATCH v6 02/15] media: i2c: imx219: Add internal image sink pad

[Laurent Pinchart]

Use the newly added internal pad API to expose the internal
configuration of the sensor to userspace in a standard manner. This also
paves the way for adding support for embedded data with an additional
internal pad.

To maintain compatibility with existing userspace that may operate on
pad 0 unconditionally, keep the source pad numbered 0 and number the
internal image pad 1.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/i2c/imx219.c | 169 +++++++++++++++++++++++++++++--------
 1 file changed, 133 insertions(+), 36 deletions(-)

diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
index 3878da50860e..817bf192e4d9 100644
--- a/drivers/media/i2c/imx219.c
+++ b/drivers/media/i2c/imx219.c
@@ -140,6 +140,7 @@
 #define IMX219_DEFAULT_LINK_FREQ_4LANE	363000000
 
 /* IMX219 native and active pixel array size. */
+#define IMX219_NATIVE_FORMAT		MEDIA_BUS_FMT_SRGGB10_1X10
 #define IMX219_NATIVE_WIDTH		3296U
 #define IMX219_NATIVE_HEIGHT		2480U
 #define IMX219_PIXEL_ARRAY_LEFT		8U
@@ -312,9 +313,15 @@ static const struct imx219_mode supported_modes[] = {
 	},
 };
 
+enum imx219_pad_ids {
+	IMX219_PAD_SOURCE,
+	IMX219_PAD_IMAGE,
+	IMX219_NUM_PADS,
+};
+
 struct imx219 {
 	struct v4l2_subdev sd;
-	struct media_pad pad;
+	struct media_pad pads[IMX219_NUM_PADS];
 
 	struct regmap *regmap;
 	struct clk *xclk; /* system clock to IMX219 */
@@ -374,7 +381,7 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
 	int ret = 0;
 
 	state = v4l2_subdev_get_locked_active_state(&imx219->sd);
-	format = v4l2_subdev_state_get_format(state, 0);
+	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
 
 	if (ctrl->id == V4L2_CID_VBLANK) {
 		int exposure_max, exposure_def;
@@ -593,8 +600,8 @@ static int imx219_set_framefmt(struct imx219 *imx219,
 	u64 bin_h, bin_v;
 	int ret = 0;
 
-	format = v4l2_subdev_state_get_format(state, 0);
-	crop = v4l2_subdev_state_get_crop(state, 0);
+	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
+	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_IMAGE);
 
 	switch (format->code) {
 	case MEDIA_BUS_FMT_SRGGB8_1X8:
@@ -764,10 +771,25 @@ static int imx219_enum_mbus_code(struct v4l2_subdev *sd,
 {
 	struct imx219 *imx219 = to_imx219(sd);
 
-	if (code->index >= (ARRAY_SIZE(imx219_mbus_formats) / 4))
-		return -EINVAL;
+	if (code->pad == IMX219_PAD_IMAGE) {
+		/* The internal image pad is hardwired to the native format. */
+		if (code->index)
+			return -EINVAL;
 
-	code->code = imx219_get_format_code(imx219, imx219_mbus_formats[code->index * 4]);
+		code->code = IMX219_NATIVE_FORMAT;
+	} else {
+		/*
+		 * On the source pad, the sensor supports multiple raw formats
+		 * with different bit depths.
+		 */
+		u32 format;
+
+		if (code->index >= (ARRAY_SIZE(imx219_mbus_formats) / 4))
+			return -EINVAL;
+
+		format = imx219_mbus_formats[code->index * 4];
+		code->code = imx219_get_format_code(imx219, format);
+	}
 
 	return 0;
 }
@@ -777,19 +799,25 @@ static int imx219_enum_frame_size(struct v4l2_subdev *sd,
 				  struct v4l2_subdev_frame_size_enum *fse)
 {
 	struct imx219 *imx219 = to_imx219(sd);
-	u32 code;
 
-	if (fse->index >= ARRAY_SIZE(supported_modes))
-		return -EINVAL;
+	if (fse->pad == IMX219_PAD_IMAGE) {
+		if (fse->code != IMX219_NATIVE_FORMAT || fse->index > 0)
+			return -EINVAL;
 
-	code = imx219_get_format_code(imx219, fse->code);
-	if (fse->code != code)
-		return -EINVAL;
+		fse->min_width = IMX219_NATIVE_WIDTH;
+		fse->max_width = IMX219_NATIVE_WIDTH;
+		fse->min_height = IMX219_NATIVE_HEIGHT;
+		fse->max_height = IMX219_NATIVE_HEIGHT;
+	} else {
+		if (fse->code != imx219_get_format_code(imx219, fse->code) ||
+		    fse->index >= ARRAY_SIZE(supported_modes))
+			return -EINVAL;
 
-	fse->min_width = supported_modes[fse->index].width;
-	fse->max_width = fse->min_width;
-	fse->min_height = supported_modes[fse->index].height;
-	fse->max_height = fse->min_height;
+		fse->min_width = supported_modes[fse->index].width;
+		fse->max_width = fse->min_width;
+		fse->min_height = supported_modes[fse->index].height;
+		fse->max_height = fse->min_height;
+	}
 
 	return 0;
 }
@@ -801,9 +829,17 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 	struct imx219 *imx219 = to_imx219(sd);
 	const struct imx219_mode *mode;
 	struct v4l2_mbus_framefmt *format;
+	struct v4l2_rect *compose;
 	struct v4l2_rect *crop;
 	unsigned int bin_h, bin_v;
 
+	/*
+	 * The driver is mode-based, the format can be set on the source pad
+	 * only.
+	 */
+	if (fmt->pad != IMX219_PAD_SOURCE)
+		return v4l2_subdev_get_fmt(sd, state, fmt);
+
 	/*
 	 * Adjust the requested format to match the closest mode. The Bayer
 	 * order varies with flips.
@@ -822,22 +858,51 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 	fmt->format.quantization = V4L2_QUANTIZATION_FULL_RANGE;
 	fmt->format.xfer_func = V4L2_XFER_FUNC_NONE;
 
-	format = v4l2_subdev_state_get_format(state, 0);
+	/* Propagate the format through the sensor. */
+
+	/* The image pad models the pixel array, and thus has a fixed size. */
+	format = v4l2_subdev_state_get_format(state, IMX219_PAD_IMAGE);
 	*format = fmt->format;
+	format->code = IMX219_NATIVE_FORMAT;
+	format->width = IMX219_NATIVE_WIDTH;
+	format->height = IMX219_NATIVE_HEIGHT;
 
 	/*
 	 * Use binning to maximize the crop rectangle size, and centre it in the
 	 * sensor.
 	 */
-	bin_h = min(IMX219_PIXEL_ARRAY_WIDTH / format->width, 2U);
-	bin_v = min(IMX219_PIXEL_ARRAY_HEIGHT / format->height, 2U);
+	bin_h = min(IMX219_PIXEL_ARRAY_WIDTH / fmt->format.width, 2U);
+	bin_v = min(IMX219_PIXEL_ARRAY_HEIGHT / fmt->format.height, 2U);
 
-	crop = v4l2_subdev_state_get_crop(state, 0);
-	crop->width = format->width * bin_h;
-	crop->height = format->height * bin_v;
+	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_IMAGE);
+	crop->width = fmt->format.width * bin_h;
+	crop->height = fmt->format.height * bin_v;
 	crop->left = (IMX219_NATIVE_WIDTH - crop->width) / 2;
 	crop->top = (IMX219_NATIVE_HEIGHT - crop->height) / 2;
 
+	/*
+	 * The compose rectangle models binning, its size is the sensor output
+	 * size.
+	 */
+	compose = v4l2_subdev_state_get_compose(state, IMX219_PAD_IMAGE);
+	compose->left = 0;
+	compose->top = 0;
+	compose->width = fmt->format.width;
+	compose->height = fmt->format.height;
+
+	/*
+	 * No mode use digital crop, the source pad crop rectangle size and
+	 * format are thus identical to the image pad compose rectangle.
+	 */
+	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_SOURCE);
+	crop->left = 0;
+	crop->top = 0;
+	crop->width = fmt->format.width;
+	crop->height = fmt->format.height;
+
+	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
+	*format = fmt->format;
+
 	if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
 		int exposure_max;
 		int exposure_def;
@@ -874,13 +939,13 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_selection *sel)
 {
-	switch (sel->target) {
-	case V4L2_SEL_TGT_CROP: {
-		sel->r = *v4l2_subdev_state_get_crop(state, 0);
-		return 0;
-	}
+	struct v4l2_rect *compose;
 
+	switch (sel->target) {
 	case V4L2_SEL_TGT_NATIVE_SIZE:
+		if (sel->pad != IMX219_PAD_IMAGE)
+			return -EINVAL;
+
 		sel->r.top = 0;
 		sel->r.left = 0;
 		sel->r.width = IMX219_NATIVE_WIDTH;
@@ -890,11 +955,35 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
 
 	case V4L2_SEL_TGT_CROP_DEFAULT:
 	case V4L2_SEL_TGT_CROP_BOUNDS:
-		sel->r.top = IMX219_PIXEL_ARRAY_TOP;
-		sel->r.left = IMX219_PIXEL_ARRAY_LEFT;
-		sel->r.width = IMX219_PIXEL_ARRAY_WIDTH;
-		sel->r.height = IMX219_PIXEL_ARRAY_HEIGHT;
+		switch (sel->pad) {
+		case IMX219_PAD_IMAGE:
+			sel->r.top = IMX219_PIXEL_ARRAY_TOP;
+			sel->r.left = IMX219_PIXEL_ARRAY_LEFT;
+			sel->r.width = IMX219_PIXEL_ARRAY_WIDTH;
+			sel->r.height = IMX219_PIXEL_ARRAY_HEIGHT;
+			return 0;
 
+		case IMX219_PAD_SOURCE:
+			compose = v4l2_subdev_state_get_compose(state,
+								IMX219_PAD_IMAGE);
+			sel->r.top = 0;
+			sel->r.left = 0;
+			sel->r.width = compose->width;
+			sel->r.height = compose->height;
+			return 0;
+		}
+
+		break;
+
+	case V4L2_SEL_TGT_CROP:
+		sel->r = *v4l2_subdev_state_get_crop(state, sel->pad);
+		return 0;
+
+	case V4L2_SEL_TGT_COMPOSE:
+		if (sel->pad != IMX219_PAD_IMAGE)
+			return -EINVAL;
+
+		sel->r = *v4l2_subdev_state_get_compose(state, sel->pad);
 		return 0;
 	}
 
@@ -906,7 +995,7 @@ static int imx219_init_state(struct v4l2_subdev *sd,
 {
 	struct v4l2_subdev_format fmt = {
 		.which = V4L2_SUBDEV_FORMAT_TRY,
-		.pad = 0,
+		.pad = IMX219_PAD_SOURCE,
 		.format = {
 			.code = MEDIA_BUS_FMT_SRGGB10_1X10,
 			.width = supported_modes[0].width,
@@ -1174,10 +1263,18 @@ static int imx219_probe(struct i2c_client *client)
 			    V4L2_SUBDEV_FL_HAS_EVENTS;
 	imx219->sd.entity.function = MEDIA_ENT_F_CAM_SENSOR;
 
-	/* Initialize source pad */
-	imx219->pad.flags = MEDIA_PAD_FL_SOURCE;
+	/*
+	 * Initialize the pads. To preserve backward compatibility with
+	 * userspace that used the sensor before the introduction of the
+	 * internal image pad, the external source pad is numbered 0 and the
+	 * internal image pad numbered 1.
+	 */
+	imx219->pads[IMX219_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
+	imx219->pads[IMX219_PAD_IMAGE].flags = MEDIA_PAD_FL_SINK
+					     | MEDIA_PAD_FL_INTERNAL;
 
-	ret = media_entity_pads_init(&imx219->sd.entity, 1, &imx219->pad);
+	ret = media_entity_pads_init(&imx219->sd.entity,
+				     ARRAY_SIZE(imx219->pads), imx219->pads);
 	if (ret) {
 		dev_err(dev, "failed to init entity pads: %d\n", ret);
 		goto error_handler_free;
-- 
Regards,

Laurent Pinchart

[PATCH v6 03/15] media: i2c: imx219: Report internal routes to userspace

[Laurent Pinchart]

Usage of internal pads creates a route internal to the subdev, and the
V4L2 camera sensor API requires such routes to be reported to userspace.
Create the route in the .init_cfg() operation.

Internal routing support requires stream support, so set the
V4L2_SUBDEV_FL_HAS_STREAMS flag and switch formats and selection
rectangles access from pads to streams. As the route is immutable,
there's no need to implement the .set_routing() operation, and we can
hardcode the sink and source stream IDs to 0.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/i2c/imx219.c | 28 ++++++++++++++++++++++++++--
 1 file changed, 26 insertions(+), 2 deletions(-)

diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
index 817bf192e4d9..52afb821f667 100644
--- a/drivers/media/i2c/imx219.c
+++ b/drivers/media/i2c/imx219.c
@@ -381,7 +381,10 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
 	int ret = 0;
 
 	state = v4l2_subdev_get_locked_active_state(&imx219->sd);
-	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
+
+	format = v4l2_subdev_state_get_opposite_stream_format(state,
+							      IMX219_PAD_IMAGE,
+							      0);
 
 	if (ctrl->id == V4L2_CID_VBLANK) {
 		int exposure_max, exposure_def;
@@ -993,15 +996,35 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
 static int imx219_init_state(struct v4l2_subdev *sd,
 			     struct v4l2_subdev_state *state)
 {
+	struct v4l2_subdev_route routes[1] = {
+		{
+			.sink_pad = IMX219_PAD_IMAGE,
+			.sink_stream = 0,
+			.source_pad = IMX219_PAD_SOURCE,
+			.source_stream = 0,
+			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
+		},
+	};
+	struct v4l2_subdev_krouting routing = {
+		.len_routes = ARRAY_SIZE(routes),
+		.num_routes = ARRAY_SIZE(routes),
+		.routes = routes,
+	};
 	struct v4l2_subdev_format fmt = {
 		.which = V4L2_SUBDEV_FORMAT_TRY,
 		.pad = IMX219_PAD_SOURCE,
+		.stream = 0,
 		.format = {
 			.code = MEDIA_BUS_FMT_SRGGB10_1X10,
 			.width = supported_modes[0].width,
 			.height = supported_modes[0].height,
 		},
 	};
+	int ret;
+
+	ret = v4l2_subdev_set_routing(sd, state, &routing);
+	if (ret)
+		return ret;
 
 	imx219_set_pad_format(sd, state, &fmt);
 
@@ -1260,7 +1283,8 @@ static int imx219_probe(struct i2c_client *client)
 
 	/* Initialize subdev */
 	imx219->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE |
-			    V4L2_SUBDEV_FL_HAS_EVENTS;
+			    V4L2_SUBDEV_FL_HAS_EVENTS |
+			    V4L2_SUBDEV_FL_STREAMS;
 	imx219->sd.entity.function = MEDIA_ENT_F_CAM_SENSOR;
 
 	/*
-- 
Regards,

Laurent Pinchart

[PATCH v6 04/15] media: i2c: imx219: Report streams using frame descriptors

[Laurent Pinchart]

Implement the .get_frame_desc() subdev operation to report information
about streams to the connected CSI-2 receiver. This is required to let
the CSI-2 receiver driver know about virtual channels and data types for
each stream.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/i2c/imx219.c | 67 ++++++++++++++++++++++++++++----------
 1 file changed, 50 insertions(+), 17 deletions(-)

diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
index 52afb821f667..6e0232b6772b 100644
--- a/drivers/media/i2c/imx219.c
+++ b/drivers/media/i2c/imx219.c
@@ -23,6 +23,7 @@
 #include <linux/pm_runtime.h>
 #include <linux/regulator/consumer.h>
 
+#include <media/mipi-csi2.h>
 #include <media/v4l2-cci.h>
 #include <media/v4l2-ctrls.h>
 #include <media/v4l2-device.h>
@@ -594,6 +595,24 @@ static void imx219_free_controls(struct imx219 *imx219)
  * Subdev operations
  */
 
+static int imx219_format_bpp(u32 code)
+{
+	switch (code) {
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+		return 8;
+
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	default:
+		return 10;
+	}
+}
+
 static int imx219_set_framefmt(struct imx219 *imx219,
 			       struct v4l2_subdev_state *state)
 {
@@ -605,23 +624,7 @@ static int imx219_set_framefmt(struct imx219 *imx219,
 
 	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
 	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_IMAGE);
-
-	switch (format->code) {
-	case MEDIA_BUS_FMT_SRGGB8_1X8:
-	case MEDIA_BUS_FMT_SGRBG8_1X8:
-	case MEDIA_BUS_FMT_SGBRG8_1X8:
-	case MEDIA_BUS_FMT_SBGGR8_1X8:
-		bpp = 8;
-		break;
-
-	case MEDIA_BUS_FMT_SRGGB10_1X10:
-	case MEDIA_BUS_FMT_SGRBG10_1X10:
-	case MEDIA_BUS_FMT_SGBRG10_1X10:
-	case MEDIA_BUS_FMT_SBGGR10_1X10:
-	default:
-		bpp = 10;
-		break;
-	}
+	bpp = imx219_format_bpp(format->code);
 
 	cci_write(imx219->regmap, IMX219_REG_X_ADD_STA_A,
 		  crop->left - IMX219_PIXEL_ARRAY_LEFT, &ret);
@@ -1031,6 +1034,35 @@ static int imx219_init_state(struct v4l2_subdev *sd,
 	return 0;
 }
 
+static int imx219_get_frame_desc(struct v4l2_subdev *sd, unsigned int pad,
+				 struct v4l2_mbus_frame_desc *fd)
+{
+	const struct v4l2_mbus_framefmt *fmt;
+	struct v4l2_subdev_state *state;
+	u32 code;
+
+	if (pad != IMX219_PAD_SOURCE)
+		return -EINVAL;
+
+	state = v4l2_subdev_lock_and_get_active_state(sd);
+	fmt = v4l2_subdev_state_get_stream_format(state, IMX219_PAD_SOURCE, 0);
+	code = fmt->code;
+	v4l2_subdev_unlock_state(state);
+
+	fd->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
+	fd->num_entries = 1;
+
+	memset(fd->entry, 0, sizeof(fd->entry));
+
+	fd->entry[0].pixelcode = code;
+	fd->entry[0].stream = 0;
+	fd->entry[0].bus.csi2.vc = 0;
+	fd->entry[0].bus.csi2.dt = imx219_format_bpp(code) == 8
+				 ? MIPI_CSI2_DT_RAW8 : MIPI_CSI2_DT_RAW10;
+
+	return 0;
+}
+
 static const struct v4l2_subdev_core_ops imx219_core_ops = {
 	.subscribe_event = v4l2_ctrl_subdev_subscribe_event,
 	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
@@ -1046,6 +1078,7 @@ static const struct v4l2_subdev_pad_ops imx219_pad_ops = {
 	.set_fmt = imx219_set_pad_format,
 	.get_selection = imx219_get_selection,
 	.enum_frame_size = imx219_enum_frame_size,
+	.get_frame_desc = imx219_get_frame_desc,
 };
 
 static const struct v4l2_subdev_ops imx219_subdev_ops = {
-- 
Regards,

Laurent Pinchart

[PATCH v6 05/15] media: i2c: imx219: Add embedded data support

[Laurent Pinchart]

The IMX219 generates embedded data unconditionally. Report it as an
additional stream, with a new internal embedded data pad, and update
subdev operations accordingly.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/i2c/imx219.c | 173 ++++++++++++++++++++++++++++++++-----
 1 file changed, 151 insertions(+), 22 deletions(-)

diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
index 6e0232b6772b..cfbf4c304a9a 100644
--- a/drivers/media/i2c/imx219.c
+++ b/drivers/media/i2c/imx219.c
@@ -149,6 +149,9 @@
 #define IMX219_PIXEL_ARRAY_WIDTH	3280U
 #define IMX219_PIXEL_ARRAY_HEIGHT	2464U
 
+/* Embedded metadata stream height */
+#define IMX219_EMBEDDED_DATA_HEIGHT	2U
+
 /* Mode : resolution and related config&values */
 struct imx219_mode {
 	/* Frame width */
@@ -317,9 +320,15 @@ static const struct imx219_mode supported_modes[] = {
 enum imx219_pad_ids {
 	IMX219_PAD_SOURCE,
 	IMX219_PAD_IMAGE,
+	IMX219_PAD_EDATA,
 	IMX219_NUM_PADS,
 };
 
+enum imx219_stream_ids {
+	IMX219_STREAM_IMAGE,
+	IMX219_STREAM_EDATA,
+};
+
 struct imx219 {
 	struct v4l2_subdev sd;
 	struct media_pad pads[IMX219_NUM_PADS];
@@ -613,6 +622,25 @@ static int imx219_format_bpp(u32 code)
 	}
 }
 
+/* Return the embedded data format corresponding to an image format. */
+static u32 imx219_format_edata(u32 code)
+{
+	switch (code) {
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+		return MEDIA_BUS_FMT_META_8;
+
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	default:
+		return MEDIA_BUS_FMT_META_10;
+	}
+}
+
 static int imx219_set_framefmt(struct imx219 *imx219,
 			       struct v4l2_subdev_state *state)
 {
@@ -622,7 +650,8 @@ static int imx219_set_framefmt(struct imx219 *imx219,
 	u64 bin_h, bin_v;
 	int ret = 0;
 
-	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
+	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
+					      IMX219_STREAM_IMAGE);
 	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_IMAGE);
 	bpp = imx219_format_bpp(format->code);
 
@@ -777,17 +806,33 @@ static int imx219_enum_mbus_code(struct v4l2_subdev *sd,
 {
 	struct imx219 *imx219 = to_imx219(sd);
 
-	if (code->pad == IMX219_PAD_IMAGE) {
+	switch (code->pad) {
+	case IMX219_PAD_IMAGE:
 		/* The internal image pad is hardwired to the native format. */
-		if (code->index)
+		if (code->index > 0)
 			return -EINVAL;
 
 		code->code = IMX219_NATIVE_FORMAT;
-	} else {
-		/*
-		 * On the source pad, the sensor supports multiple raw formats
-		 * with different bit depths.
-		 */
+		return 0;
+
+	case IMX219_PAD_EDATA:
+		if (code->index > 0)
+			return -EINVAL;
+
+		code->code = MEDIA_BUS_FMT_CCS_EMBEDDED;
+		return 0;
+
+	case IMX219_PAD_SOURCE:
+	default:
+		break;
+	}
+
+	/*
+	 * On the source pad, the sensor supports multiple image raw formats
+	 * with different bit depths. The embedded data format bit depth
+	 * follows the image stream.
+	 */
+	if (code->stream == IMX219_STREAM_IMAGE) {
 		u32 format;
 
 		if (code->index >= (ARRAY_SIZE(imx219_mbus_formats) / 4))
@@ -795,6 +840,15 @@ static int imx219_enum_mbus_code(struct v4l2_subdev *sd,
 
 		format = imx219_mbus_formats[code->index * 4];
 		code->code = imx219_get_format_code(imx219, format);
+	} else {
+		struct v4l2_mbus_framefmt *fmt;
+
+		if (code->index > 0)
+			return -EINVAL;
+
+		fmt = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
+						   IMX219_STREAM_EDATA);
+		code->code = fmt->code;
 	}
 
 	return 0;
@@ -806,7 +860,8 @@ static int imx219_enum_frame_size(struct v4l2_subdev *sd,
 {
 	struct imx219 *imx219 = to_imx219(sd);
 
-	if (fse->pad == IMX219_PAD_IMAGE) {
+	switch (fse->pad) {
+	case IMX219_PAD_IMAGE:
 		if (fse->code != IMX219_NATIVE_FORMAT || fse->index > 0)
 			return -EINVAL;
 
@@ -814,7 +869,24 @@ static int imx219_enum_frame_size(struct v4l2_subdev *sd,
 		fse->max_width = IMX219_NATIVE_WIDTH;
 		fse->min_height = IMX219_NATIVE_HEIGHT;
 		fse->max_height = IMX219_NATIVE_HEIGHT;
-	} else {
+		return 0;
+
+	case IMX219_PAD_EDATA:
+		if (fse->code != MEDIA_BUS_FMT_CCS_EMBEDDED || fse->index > 0)
+			return -EINVAL;
+
+		fse->min_width = IMX219_NATIVE_WIDTH;
+		fse->max_width = IMX219_NATIVE_WIDTH;
+		fse->min_height = IMX219_EMBEDDED_DATA_HEIGHT;
+		fse->max_height = IMX219_EMBEDDED_DATA_HEIGHT;
+		return 0;
+
+	case IMX219_PAD_SOURCE:
+	default:
+		break;
+	}
+
+	if (fse->stream == IMX219_STREAM_IMAGE) {
 		if (fse->code != imx219_get_format_code(imx219, fse->code) ||
 		    fse->index >= ARRAY_SIZE(supported_modes))
 			return -EINVAL;
@@ -823,6 +895,21 @@ static int imx219_enum_frame_size(struct v4l2_subdev *sd,
 		fse->max_width = fse->min_width;
 		fse->min_height = supported_modes[fse->index].height;
 		fse->max_height = fse->min_height;
+	} else {
+		struct v4l2_mbus_framefmt *fmt;
+
+		fmt = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
+						   IMX219_STREAM_EDATA);
+		if (fse->code != fmt->code)
+			return -EINVAL;
+
+		if (fse->index)
+			return -EINVAL;
+
+		fse->min_width = fmt->width;
+		fse->max_width = fmt->width;
+		fse->min_height = IMX219_EMBEDDED_DATA_HEIGHT;
+		fse->max_height = IMX219_EMBEDDED_DATA_HEIGHT;
 	}
 
 	return 0;
@@ -834,6 +921,7 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 {
 	struct imx219 *imx219 = to_imx219(sd);
 	const struct imx219_mode *mode;
+	struct v4l2_mbus_framefmt *ed_format;
 	struct v4l2_mbus_framefmt *format;
 	struct v4l2_rect *compose;
 	struct v4l2_rect *crop;
@@ -841,9 +929,9 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 
 	/*
 	 * The driver is mode-based, the format can be set on the source pad
-	 * only.
+	 * only, and only for the image streeam.
 	 */
-	if (fmt->pad != IMX219_PAD_SOURCE)
+	if (fmt->pad != IMX219_PAD_SOURCE || fmt->stream != IMX219_STREAM_IMAGE)
 		return v4l2_subdev_get_fmt(sd, state, fmt);
 
 	/*
@@ -900,15 +988,31 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 	 * No mode use digital crop, the source pad crop rectangle size and
 	 * format are thus identical to the image pad compose rectangle.
 	 */
-	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_SOURCE);
+	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_SOURCE,
+					  IMX219_STREAM_IMAGE);
 	crop->left = 0;
 	crop->top = 0;
 	crop->width = fmt->format.width;
 	crop->height = fmt->format.height;
 
-	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
+	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
+					      IMX219_STREAM_IMAGE);
 	*format = fmt->format;
 
+	/*
+	 * Finally, update the formats on the sink and source sides of the
+	 * embedded data stream.
+	 */
+	ed_format = v4l2_subdev_state_get_format(state, IMX219_PAD_EDATA);
+	ed_format->code = imx219_format_edata(format->code);
+	ed_format->width = format->width;
+	ed_format->height = IMX219_EMBEDDED_DATA_HEIGHT;
+	ed_format->field = V4L2_FIELD_NONE;
+
+	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
+					      IMX219_STREAM_EDATA);
+	*format = *ed_format;
+
 	if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
 		int exposure_max;
 		int exposure_def;
@@ -947,6 +1051,13 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
 {
 	struct v4l2_rect *compose;
 
+	/*
+	 * The embedded data stream doesn't support selection rectangles,
+	 * neither on the embedded data pad nor on the source pad.
+	 */
+	if (sel->pad == IMX219_PAD_EDATA || sel->stream != 0)
+		return -EINVAL;
+
 	switch (sel->target) {
 	case V4L2_SEL_TGT_NATIVE_SIZE:
 		if (sel->pad != IMX219_PAD_IMAGE)
@@ -999,12 +1110,18 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
 static int imx219_init_state(struct v4l2_subdev *sd,
 			     struct v4l2_subdev_state *state)
 {
-	struct v4l2_subdev_route routes[1] = {
+	struct v4l2_subdev_route routes[2] = {
 		{
 			.sink_pad = IMX219_PAD_IMAGE,
 			.sink_stream = 0,
 			.source_pad = IMX219_PAD_SOURCE,
-			.source_stream = 0,
+			.source_stream = IMX219_STREAM_IMAGE,
+			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
+		}, {
+			.sink_pad = IMX219_PAD_EDATA,
+			.sink_stream = 0,
+			.source_pad = IMX219_PAD_SOURCE,
+			.source_stream = IMX219_STREAM_EDATA,
 			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
 		},
 	};
@@ -1016,7 +1133,7 @@ static int imx219_init_state(struct v4l2_subdev *sd,
 	struct v4l2_subdev_format fmt = {
 		.which = V4L2_SUBDEV_FORMAT_TRY,
 		.pad = IMX219_PAD_SOURCE,
-		.stream = 0,
+		.stream = IMX219_STREAM_IMAGE,
 		.format = {
 			.code = MEDIA_BUS_FMT_SRGGB10_1X10,
 			.width = supported_modes[0].width,
@@ -1029,6 +1146,10 @@ static int imx219_init_state(struct v4l2_subdev *sd,
 	if (ret)
 		return ret;
 
+	/*
+	 * Set the image stream format on the source pad. This will be
+	 * propagated to all formats and selection rectangles internally.
+	 */
 	imx219_set_pad_format(sd, state, &fmt);
 
 	return 0;
@@ -1045,21 +1166,27 @@ static int imx219_get_frame_desc(struct v4l2_subdev *sd, unsigned int pad,
 		return -EINVAL;
 
 	state = v4l2_subdev_lock_and_get_active_state(sd);
-	fmt = v4l2_subdev_state_get_stream_format(state, IMX219_PAD_SOURCE, 0);
+	fmt = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
+					   IMX219_STREAM_IMAGE);
 	code = fmt->code;
 	v4l2_subdev_unlock_state(state);
 
 	fd->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
-	fd->num_entries = 1;
+	fd->num_entries = 2;
 
 	memset(fd->entry, 0, sizeof(fd->entry));
 
 	fd->entry[0].pixelcode = code;
-	fd->entry[0].stream = 0;
+	fd->entry[0].stream = IMX219_STREAM_IMAGE;
 	fd->entry[0].bus.csi2.vc = 0;
 	fd->entry[0].bus.csi2.dt = imx219_format_bpp(code) == 8
 				 ? MIPI_CSI2_DT_RAW8 : MIPI_CSI2_DT_RAW10;
 
+	fd->entry[1].pixelcode = code;
+	fd->entry[1].stream = IMX219_STREAM_EDATA;
+	fd->entry[1].bus.csi2.vc = 0;
+	fd->entry[1].bus.csi2.dt = MIPI_CSI2_DT_EMBEDDED_8B;
+
 	return 0;
 }
 
@@ -1323,12 +1450,14 @@ static int imx219_probe(struct i2c_client *client)
 	/*
 	 * Initialize the pads. To preserve backward compatibility with
 	 * userspace that used the sensor before the introduction of the
-	 * internal image pad, the external source pad is numbered 0 and the
-	 * internal image pad numbered 1.
+	 * internal pads, the external source pad is numbered 0 and the internal
+	 * image and embedded data pads numbered 1 and 2 respectively.
 	 */
 	imx219->pads[IMX219_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
 	imx219->pads[IMX219_PAD_IMAGE].flags = MEDIA_PAD_FL_SINK
 					     | MEDIA_PAD_FL_INTERNAL;
+	imx219->pads[IMX219_PAD_EDATA].flags = MEDIA_PAD_FL_SINK
+					     | MEDIA_PAD_FL_INTERNAL;
 
 	ret = media_entity_pads_init(&imx219->sd.entity,
 				     ARRAY_SIZE(imx219->pads), imx219->pads);
-- 
Regards,

Laurent Pinchart

[PATCH v6 06/15] media: v4l: Add V4L2-PIX-FMT-Y12P format

[Laurent Pinchart]

From: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>

This is a packed grey-scale image format with a depth of 12 bits per
pixel. Two consecutive pixels are packed into 3 bytes. The first 2 bytes
contain the 8 high order bits of the pixels, and the 3rd byte contains
the 4 least significants bits of each pixel, in the same order.

Add the entry in userspace API, and document it.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 .../userspace-api/media/v4l/pixfmt-yuv-luma.rst       | 11 +++++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c                  |  1 +
 include/uapi/linux/videodev2.h                        |  1 +
 3 files changed, 13 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst b/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst
index cf8e4dfbfbd4..a650ceda7cdb 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst
@@ -114,6 +114,17 @@ are often referred to as greyscale formats.
       - ...
       - ...
 
+    * .. _V4L2-PIX-FMT-Y12P:
+
+      - ``V4L2_PIX_FMT_Y12P``
+      - 'Y12P'
+
+      - Y'\ :sub:`0`\ [11:4]
+      - Y'\ :sub:`1`\ [11:4]
+      - Y'\ :sub:`1`\ [3:0] Y'\ :sub:`0`\ [3:0]
+      - ...
+      - ...
+
     * .. _V4L2-PIX-FMT-Y14:
 
       - ``V4L2_PIX_FMT_Y14``
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index f3697ff65817..69ac4482d844 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1313,6 +1313,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_PIX_FMT_Y10BPACK:	descr = "10-bit Greyscale (Packed)"; break;
 	case V4L2_PIX_FMT_Y10P:		descr = "10-bit Greyscale (MIPI Packed)"; break;
 	case V4L2_PIX_FMT_IPU3_Y10:	descr = "10-bit greyscale (IPU3 Packed)"; break;
+	case V4L2_PIX_FMT_Y12P:		descr = "12-bit Greyscale (MIPI Packed)"; break;
 	case V4L2_PIX_FMT_Y8I:		descr = "Interleaved 8-bit Greyscale"; break;
 	case V4L2_PIX_FMT_Y12I:		descr = "Interleaved 12-bit Greyscale"; break;
 	case V4L2_PIX_FMT_Z16:		descr = "16-bit Depth"; break;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 117a081c9db9..5122ac0c6765 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -599,6 +599,7 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_Y10BPACK    v4l2_fourcc('Y', '1', '0', 'B') /* 10  Greyscale bit-packed */
 #define V4L2_PIX_FMT_Y10P    v4l2_fourcc('Y', '1', '0', 'P') /* 10  Greyscale, MIPI RAW10 packed */
 #define V4L2_PIX_FMT_IPU3_Y10		v4l2_fourcc('i', 'p', '3', 'y') /* IPU3 packed 10-bit greyscale */
+#define V4L2_PIX_FMT_Y12P    v4l2_fourcc('Y', '1', '2', 'P') /* 12  Greyscale, MIPI RAW12 packed */
 
 /* Palette formats */
 #define V4L2_PIX_FMT_PAL8    v4l2_fourcc('P', 'A', 'L', '8') /*  8  8-bit palette */
-- 
Regards,

Laurent Pinchart

[PATCH v6 07/15] media: v4l: Add V4L2-PIX-FMT-Y14P format

[Laurent Pinchart]

From: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>

This is a packed grey-scale image format with a depth of 14 bits per
pixel. Every four consecutive samples are packed into seven bytes. Each
of the first four bytes contain the eight high order bits of the pixels,
and the three following bytes contains the six least significants bits
of each pixel, in the same order.

As the other formats only needed 5 bytes before, append two bytes in the
documentation array.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 .../media/v4l/pixfmt-yuv-luma.rst             | 37 +++++++++++++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c          |  1 +
 include/uapi/linux/videodev2.h                |  1 +
 3 files changed, 39 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst b/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst
index a650ceda7cdb..b3c5779521d8 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst
@@ -36,6 +36,8 @@ are often referred to as greyscale formats.
       - Byte 2
       - Byte 3
       - Byte 4
+      - Byte 5
+      - Byte 6
 
     * .. _V4L2-PIX-FMT-GREY:
 
@@ -47,6 +49,8 @@ are often referred to as greyscale formats.
       - ...
       - ...
       - ...
+      - ...
+      - ...
 
     * .. _V4L2-PIX-FMT-IPU3-Y10:
 
@@ -58,6 +62,8 @@ are often referred to as greyscale formats.
       - Y'\ :sub:`2`\ [3:0] Y'\ :sub:`1`\ [9:6]
       - Y'\ :sub:`3`\ [1:0] Y'\ :sub:`2`\ [9:4]
       - Y'\ :sub:`3`\ [9:2]
+      - ...
+      - ...
 
     * .. _V4L2-PIX-FMT-Y10:
 
@@ -69,6 +75,8 @@ are often referred to as greyscale formats.
       - ...
       - ...
       - ...
+      - ...
+      - ...
 
     * .. _V4L2-PIX-FMT-Y10BPACK:
 
@@ -80,6 +88,8 @@ are often referred to as greyscale formats.
       - Y'\ :sub:`1`\ [3:0] Y'\ :sub:`2`\ [9:6]
       - Y'\ :sub:`2`\ [5:0] Y'\ :sub:`3`\ [9:8]
       - Y'\ :sub:`3`\ [7:0]
+      - ...
+      - ...
 
     * .. _V4L2-PIX-FMT-Y10P:
 
@@ -91,6 +101,8 @@ are often referred to as greyscale formats.
       - Y'\ :sub:`2`\ [9:2]
       - Y'\ :sub:`3`\ [9:2]
       - Y'\ :sub:`3`\ [1:0] Y'\ :sub:`2`\ [1:0] Y'\ :sub:`1`\ [1:0] Y'\ :sub:`0`\ [1:0]
+      - ...
+      - ...
 
     * .. _V4L2-PIX-FMT-Y12:
 
@@ -102,6 +114,8 @@ are often referred to as greyscale formats.
       - ...
       - ...
       - ...
+      - ...
+      - ...
 
     * .. _V4L2-PIX-FMT-Y012:
 
@@ -113,6 +127,8 @@ are often referred to as greyscale formats.
       - ...
       - ...
       - ...
+      - ...
+      - ...
 
     * .. _V4L2-PIX-FMT-Y12P:
 
@@ -124,6 +140,8 @@ are often referred to as greyscale formats.
       - Y'\ :sub:`1`\ [3:0] Y'\ :sub:`0`\ [3:0]
       - ...
       - ...
+      - ...
+      - ...
 
     * .. _V4L2-PIX-FMT-Y14:
 
@@ -135,6 +153,21 @@ are often referred to as greyscale formats.
       - ...
       - ...
       - ...
+      - ...
+      - ...
+
+    * .. _V4L2-PIX-FMT-Y14P:
+
+      - ``V4L2_PIX_FMT_Y14P``
+      - 'Y14P'
+
+      - Y'\ :sub:`0`\ [13:6]
+      - Y'\ :sub:`1`\ [13:6]
+      - Y'\ :sub:`2`\ [13:6]
+      - Y'\ :sub:`3`\ [13:6]
+      - Y'\ :sub:`1`\ [1:0] Y'\ :sub:`0`\ [5:0]
+      - Y'\ :sub:`2`\ [3:0] Y'\ :sub:`1`\ [5:2]
+      - Y'\ :sub:`3`\ [5:0] Y'\ :sub:`2`\ [5:4]
 
     * .. _V4L2-PIX-FMT-Y16:
 
@@ -146,6 +179,8 @@ are often referred to as greyscale formats.
       - ...
       - ...
       - ...
+      - ...
+      - ...
 
     * .. _V4L2-PIX-FMT-Y16-BE:
 
@@ -157,6 +192,8 @@ are often referred to as greyscale formats.
       - ...
       - ...
       - ...
+      - ...
+      - ...
 
 .. raw:: latex
 
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 69ac4482d844..5f3343802a1b 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1314,6 +1314,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_PIX_FMT_Y10P:		descr = "10-bit Greyscale (MIPI Packed)"; break;
 	case V4L2_PIX_FMT_IPU3_Y10:	descr = "10-bit greyscale (IPU3 Packed)"; break;
 	case V4L2_PIX_FMT_Y12P:		descr = "12-bit Greyscale (MIPI Packed)"; break;
+	case V4L2_PIX_FMT_Y14P:		descr = "14-bit Greyscale (MIPI Packed)"; break;
 	case V4L2_PIX_FMT_Y8I:		descr = "Interleaved 8-bit Greyscale"; break;
 	case V4L2_PIX_FMT_Y12I:		descr = "Interleaved 12-bit Greyscale"; break;
 	case V4L2_PIX_FMT_Z16:		descr = "16-bit Depth"; break;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 5122ac0c6765..2f235e52585f 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -600,6 +600,7 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_Y10P    v4l2_fourcc('Y', '1', '0', 'P') /* 10  Greyscale, MIPI RAW10 packed */
 #define V4L2_PIX_FMT_IPU3_Y10		v4l2_fourcc('i', 'p', '3', 'y') /* IPU3 packed 10-bit greyscale */
 #define V4L2_PIX_FMT_Y12P    v4l2_fourcc('Y', '1', '2', 'P') /* 12  Greyscale, MIPI RAW12 packed */
+#define V4L2_PIX_FMT_Y14P    v4l2_fourcc('Y', '1', '4', 'P') /* 14  Greyscale, MIPI RAW14 packed */
 
 /* Palette formats */
 #define V4L2_PIX_FMT_PAL8    v4l2_fourcc('P', 'A', 'L', '8') /*  8  8-bit palette */
-- 
Regards,

Laurent Pinchart

[PATCH v6 08/15] dt-bindings: media: Add bindings for bcm2835-unicam

[Laurent Pinchart]

From: Dave Stevenson <dave.stevenson@raspberrypi.com>

Introduce the dt-bindings documentation for bcm2835 CCP2/CSI2 Unicam
camera interface.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Co-developed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Co-developed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
Changes since v5:

- Squash MAINTAINERS changes in

Changes since v3:

- Make MAINTAINERS its own patch
- Describe the reg and clocks correctly
- Use a vendor entry for the number of data lanes
---
 .../bindings/media/brcm,bcm2835-unicam.yaml   | 117 ++++++++++++++++++
 MAINTAINERS                                   |   6 +
 2 files changed, 123 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml

diff --git a/Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml b/Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
new file mode 100644
index 000000000000..1938ace23b3d
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
@@ -0,0 +1,117 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/brcm,bcm2835-unicam.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Broadcom BCM283x Camera Interface (Unicam)
+
+maintainers:
+  - Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
+
+description: |-
+  The Unicam block on BCM283x SoCs is the receiver for either
+  CSI-2 or CCP2 data from image sensors or similar devices.
+
+  The main platform using this SoC is the Raspberry Pi family of boards.  On
+  the Pi the VideoCore firmware can also control this hardware block, and
+  driving it from two different processors will cause issues.  To avoid this,
+  the firmware checks the device tree configuration during boot. If it finds
+  device tree nodes whose name starts with 'csi' then it will stop the firmware
+  accessing the block, and it can then safely be used via the device tree
+  binding.
+
+properties:
+  compatible:
+    const: brcm,bcm2835-unicam
+
+  reg:
+    items:
+      - description: Unicam block.
+      - description: Clock Manager Image (CMI) block.
+
+  reg-names:
+    items:
+      - const: unicam
+      - const: cmi
+
+  interrupts:
+    maxItems: 1
+
+  clocks:
+    items:
+      - description: Clock to drive the LP state machine of Unicam.
+      - description: Clock for the VPU (core clock).
+
+  clock-names:
+    items:
+      - const: lp
+      - const: vpu
+
+  power-domains:
+    items:
+      - description: Unicam power domain
+
+  brcm,num-data-lanes:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    enum: [ 2, 4 ]
+    description: |
+      Number of CSI-2 data lanes supported by this Unicam instance. The number
+      of data lanes actively used is specified with the data-lanes endpoint
+      property.
+
+  port:
+    $ref: /schemas/graph.yaml#/$defs/port-base
+    unevaluatedProperties: false
+
+    properties:
+      endpoint:
+        $ref: /schemas/media/video-interfaces.yaml#
+        unevaluatedProperties: false
+
+        properties:
+          data-lanes: true
+
+        required:
+          - data-lanes
+
+    required:
+      - endpoint
+
+required:
+  - compatible
+  - reg
+  - reg-names
+  - interrupts
+  - clocks
+  - clock-names
+  - power-domains
+  - brcm,num-data-lanes
+  - port
+
+additionalProperties: False
+
+examples:
+  - |
+    #include <dt-bindings/clock/bcm2835.h>
+    #include <dt-bindings/interrupt-controller/arm-gic.h>
+    #include <dt-bindings/power/raspberrypi-power.h>
+    csi1: csi@7e801000 {
+        compatible = "brcm,bcm2835-unicam";
+        reg = <0x7e801000 0x800>,
+              <0x7e802004 0x4>;
+        reg-names = "unicam", "cmi";
+        interrupts = <GIC_SPI 103 IRQ_TYPE_LEVEL_HIGH>;
+        clocks = <&clocks BCM2835_CLOCK_CAM1>,
+                 <&firmware_clocks 4>;
+        clock-names = "lp", "vpu";
+        power-domains = <&power RPI_POWER_DOMAIN_UNICAM1>;
+        brcm,num-data-lanes = <2>;
+        port {
+                csi1_ep: endpoint {
+                        remote-endpoint = <&imx219_0>;
+                        data-lanes = <1 2>;
+                };
+        };
+    };
+...
diff --git a/MAINTAINERS b/MAINTAINERS
index fada59148cb5..e50a59654e6e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3997,6 +3997,12 @@ N:	bcm113*
 N:	bcm216*
 N:	kona
 
+BROADCOM BCM2835 CAMERA DRIVERS
+M:	Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
+L:	linux-media@vger.kernel.org
+S:	Maintained
+F:	Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
+
 BROADCOM BCM47XX MIPS ARCHITECTURE
 M:	Hauke Mehrtens <hauke@hauke-m.de>
 M:	Rafał Miłecki <zajec5@gmail.com>
-- 
Regards,

Laurent Pinchart

[PATCH v6 09/15] media: bcm2835-unicam: Add support for CCP2/CSI2 camera interface

[Laurent Pinchart]

From: Dave Stevenson <dave.stevenson@raspberrypi.com>

Add a driver for the Unicam camera receiver block on BCM283x processors.
It is represented as two video device nodes: unicam-image and
unicam-embedded which are connected to an internal subdev (named
unicam-subdev) in order to manage streams routing.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Co-developed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Co-developed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
Changes since v5:

- Move to drivers/media/platform/broadcom/
- Port to the upstream V4L2 streams API
- Rebase on latest metadata API proposal
- Add missing error message
- Drop unneeded documentation block for unicam_isr()
- Drop unneeded dev_dbg() and dev_err() messages
- Drop unneeded streams_mask and fmt checks
- Drop unused unicam_sd_pad_is_sink()
- Drop unneeded includes
- Drop v4l2_ctrl_subscribe_event() call
- Use pm_runtime_resume_and_get()
- Indentation and line wrap fixes
- Let the framework set bus_info
- Use v4l2_fwnode_endpoint_parse()
- Fix media device cleanup
- Drop lane reordering checks
- Fix subdev state locking
- Drop extra debug messages
- Move clock handling to runtime PM handlers
- Reorder functions
- Rename init functions for more clarity
- Initialize runtime PM earlier
- Clarify error messages
- Simplify subdev init with local variable
- Fix subdev cleanup
- Fix typos and indentation
- Don't initialize local variables needlessly
- Simplify num lanes check
- Fix metadata handling in subdev set_fmt
- Drop manual fallback to .s_stream()
- Pass v4l2_pix_format to unicam_calc_format_size_bpl()
- Simplify unicam_set_default_format()
- Fix default format settings
- Add busy check in unicam_s_fmt_meta()
- Add missing \n at end of format strings
- Fix metadata handling in subdev set_fmt
- Fix locking when starting streaming
- Return buffers from start streaming fails
- Fix format validation for metadata node
- Use video_device_pipeline_{start,stop}() helpers
- Simplify format enumeration
- Drop unset variable
- Update MAINTAINERS entry
- Update to the upstream v4l2_async_nf API
- Update to the latest subdev routing API
- Update to the latest subdev state API
- Move from subdev .init_cfg() to .init_state()
- Update to the latest videobuf2 API
- Fix v4l2_subdev_enable_streams() error check
- Use correct pad for the connected subdev
- Return buffers to vb2 when start streaming fails
- Improve debugging in start streaming handler
- Simplify DMA address management
- Drop comment about bcm2835-camera driver
- Clarify comments that explain min/max sizes
- Pass v4l2_pix_format to unicam_try_fmt()
- Drop unneeded local variables
- Rename image-related constants and functions
- Turn unicam_fmt.metadata_fmt into bool
- Rename unicam_fmt to unicam_format_info
- Rename unicam_format_info variables to fmtinfo
- Rename unicam_node.v_fmt to fmt
- Add metadata formats for RAW10, RAW12 and RAW14
- Make metadata formats line-based
- Validate format on metadata video device
- Add Co-devlopped-by tags

Changes since v3:

- Add the vendor prefix for DT name
- Use the reg-names in DT parsing
- Remove MAINTAINERS entry

Changes since v2:

- Change code organization
- Remove unused variables
- Correct the fmt_meta functions
- Rewrite the start/stop streaming
  - You can now start the image node alone, but not the metadata one
  - The buffers are allocated per-node
  - only the required stream is started, if the route exists and is
    enabled
- Prefix the macros with UNICAM_ to not have too generic names
- Drop colorspace support

Changes since v1:

- Replace the unicam_{info,debug,error} macros with dev_*()
---
 MAINTAINERS                                   |    1 +
 drivers/media/platform/Kconfig                |    1 +
 drivers/media/platform/Makefile               |    1 +
 drivers/media/platform/broadcom/Kconfig       |   23 +
 drivers/media/platform/broadcom/Makefile      |    3 +
 .../platform/broadcom/bcm2835-unicam-regs.h   |  255 ++
 .../media/platform/broadcom/bcm2835-unicam.c  | 2607 +++++++++++++++++
 7 files changed, 2891 insertions(+)
 create mode 100644 drivers/media/platform/broadcom/Kconfig
 create mode 100644 drivers/media/platform/broadcom/Makefile
 create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam-regs.h
 create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam.c

diff --git a/MAINTAINERS b/MAINTAINERS
index e50a59654e6e..cc350729f467 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4002,6 +4002,7 @@ M:	Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
 L:	linux-media@vger.kernel.org
 S:	Maintained
 F:	Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
+F:	drivers/media/platform/bcm2835/
 
 BROADCOM BCM47XX MIPS ARCHITECTURE
 M:	Hauke Mehrtens <hauke@hauke-m.de>
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 91e54215de3a..2d79bfc68c15 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -67,6 +67,7 @@ source "drivers/media/platform/amlogic/Kconfig"
 source "drivers/media/platform/amphion/Kconfig"
 source "drivers/media/platform/aspeed/Kconfig"
 source "drivers/media/platform/atmel/Kconfig"
+source "drivers/media/platform/broadcom/Kconfig"
 source "drivers/media/platform/cadence/Kconfig"
 source "drivers/media/platform/chips-media/Kconfig"
 source "drivers/media/platform/intel/Kconfig"
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 3296ec1ebe16..da17301f7439 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -10,6 +10,7 @@ obj-y += amlogic/
 obj-y += amphion/
 obj-y += aspeed/
 obj-y += atmel/
+obj-y += broadcom/
 obj-y += cadence/
 obj-y += chips-media/
 obj-y += intel/
diff --git a/drivers/media/platform/broadcom/Kconfig b/drivers/media/platform/broadcom/Kconfig
new file mode 100644
index 000000000000..cc2c9afcc948
--- /dev/null
+++ b/drivers/media/platform/broadcom/Kconfig
@@ -0,0 +1,23 @@
+# SPDX-License-Identifier: GPL-2.0
+
+config VIDEO_BCM2835_UNICAM
+	tristate "Broadcom BCM283x/BCM271x Unicam video capture driver"
+	depends on ARCH_BCM2835 || COMPILE_TEST
+	depends on PM
+	depends on VIDEO_DEV
+	select MEDIA_CONTROLLER
+	select V4L2_FWNODE
+	select VIDEO_V4L2_SUBDEV_API
+	select VIDEOBUF2_DMA_CONTIG
+	help
+	  Say Y here to enable support for the BCM283x/BCM271x CSI-2 receiver.
+	  This is a V4L2 driver that controls the CSI-2 receiver directly,
+	  independently from the VC4 firmware.
+
+	  This driver is mutually exclusive with the use of bcm2835-camera. The
+	  firmware will disable all access to the peripheral from within the
+	  firmware if it finds a DT node using it, and bcm2835-camera will
+	  therefore fail to probe.
+
+	  To compile this driver as a module, choose M here. The module will be
+	  called bcm2835-unicam.
diff --git a/drivers/media/platform/broadcom/Makefile b/drivers/media/platform/broadcom/Makefile
new file mode 100644
index 000000000000..03d2045aba2e
--- /dev/null
+++ b/drivers/media/platform/broadcom/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_VIDEO_BCM2835_UNICAM) += bcm2835-unicam.o
diff --git a/drivers/media/platform/broadcom/bcm2835-unicam-regs.h b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
new file mode 100644
index 000000000000..84775fd2fac5
--- /dev/null
+++ b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
@@ -0,0 +1,255 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+/*
+ * Copyright (C) 2017-2020 Raspberry Pi Trading.
+ * Dave Stevenson <dave.stevenson@raspberrypi.com>
+ */
+
+#ifndef VC4_REGS_UNICAM_H
+#define VC4_REGS_UNICAM_H
+
+#include <linux/bits.h>
+
+/*
+ * The following values are taken from files found within the code drop
+ * made by Broadcom for the BCM21553 Graphics Driver, predominantly in
+ * brcm_usrlib/dag/vmcsx/vcinclude/hardware_vc4.h.
+ * They have been modified to be only the register offset.
+ */
+#define UNICAM_CTRL		0x000
+#define UNICAM_STA		0x004
+#define UNICAM_ANA		0x008
+#define UNICAM_PRI		0x00c
+#define UNICAM_CLK		0x010
+#define UNICAM_CLT		0x014
+#define UNICAM_DAT0		0x018
+#define UNICAM_DAT1		0x01c
+#define UNICAM_DAT2		0x020
+#define UNICAM_DAT3		0x024
+#define UNICAM_DLT		0x028
+#define UNICAM_CMP0		0x02c
+#define UNICAM_CMP1		0x030
+#define UNICAM_CAP0		0x034
+#define UNICAM_CAP1		0x038
+#define UNICAM_ICTL		0x100
+#define UNICAM_ISTA		0x104
+#define UNICAM_IDI0		0x108
+#define UNICAM_IPIPE		0x10c
+#define UNICAM_IBSA0		0x110
+#define UNICAM_IBEA0		0x114
+#define UNICAM_IBLS		0x118
+#define UNICAM_IBWP		0x11c
+#define UNICAM_IHWIN		0x120
+#define UNICAM_IHSTA		0x124
+#define UNICAM_IVWIN		0x128
+#define UNICAM_IVSTA		0x12c
+#define UNICAM_ICC		0x130
+#define UNICAM_ICS		0x134
+#define UNICAM_IDC		0x138
+#define UNICAM_IDPO		0x13c
+#define UNICAM_IDCA		0x140
+#define UNICAM_IDCD		0x144
+#define UNICAM_IDS		0x148
+#define UNICAM_DCS		0x200
+#define UNICAM_DBSA0		0x204
+#define UNICAM_DBEA0		0x208
+#define UNICAM_DBWP		0x20c
+#define UNICAM_DBCTL		0x300
+#define UNICAM_IBSA1		0x304
+#define UNICAM_IBEA1		0x308
+#define UNICAM_IDI1		0x30c
+#define UNICAM_DBSA1		0x310
+#define UNICAM_DBEA1		0x314
+#define UNICAM_MISC		0x400
+
+/*
+ * The following bitmasks are from the kernel released by Broadcom
+ * for Android - https://android.googlesource.com/kernel/bcm/
+ * The Rhea, Hawaii, and Java chips all contain the same VideoCore4
+ * Unicam block as BCM2835, as defined in eg
+ * arch/arm/mach-rhea/include/mach/rdb_A0/brcm_rdb_cam.h and similar.
+ * Values reworked to use the kernel BIT and GENMASK macros.
+ *
+ * Some of the bit mnenomics have been amended to match the datasheet.
+ */
+/* UNICAM_CTRL Register */
+#define UNICAM_CPE		BIT(0)
+#define UNICAM_MEM		BIT(1)
+#define UNICAM_CPR		BIT(2)
+#define UNICAM_CPM_MASK		GENMASK(3, 3)
+#define UNICAM_CPM_CSI2		0
+#define UNICAM_CPM_CCP2		1
+#define UNICAM_SOE		BIT(4)
+#define UNICAM_DCM_MASK		GENMASK(5, 5)
+#define UNICAM_DCM_STROBE	0
+#define UNICAM_DCM_DATA		1
+#define UNICAM_SLS		BIT(6)
+#define UNICAM_PFT_MASK		GENMASK(11, 8)
+#define UNICAM_OET_MASK		GENMASK(20, 12)
+
+/* UNICAM_STA Register */
+#define UNICAM_SYN		BIT(0)
+#define UNICAM_CS		BIT(1)
+#define UNICAM_SBE		BIT(2)
+#define UNICAM_PBE		BIT(3)
+#define UNICAM_HOE		BIT(4)
+#define UNICAM_PLE		BIT(5)
+#define UNICAM_SSC		BIT(6)
+#define UNICAM_CRCE		BIT(7)
+#define UNICAM_OES		BIT(8)
+#define UNICAM_IFO		BIT(9)
+#define UNICAM_OFO		BIT(10)
+#define UNICAM_BFO		BIT(11)
+#define UNICAM_DL		BIT(12)
+#define UNICAM_PS		BIT(13)
+#define UNICAM_IS		BIT(14)
+#define UNICAM_PI0		BIT(15)
+#define UNICAM_PI1		BIT(16)
+#define UNICAM_FSI_S		BIT(17)
+#define UNICAM_FEI_S		BIT(18)
+#define UNICAM_LCI_S		BIT(19)
+#define UNICAM_BUF0_RDY		BIT(20)
+#define UNICAM_BUF0_NO		BIT(21)
+#define UNICAM_BUF1_RDY		BIT(22)
+#define UNICAM_BUF1_NO		BIT(23)
+#define UNICAM_DI		BIT(24)
+
+#define UNICAM_STA_MASK_ALL \
+		(UNICAM_DL | \
+		UNICAM_SBE | \
+		UNICAM_PBE | \
+		UNICAM_HOE | \
+		UNICAM_PLE | \
+		UNICAM_SSC | \
+		UNICAM_CRCE | \
+		UNICAM_IFO | \
+		UNICAM_OFO | \
+		UNICAM_PS | \
+		UNICAM_PI0 | \
+		UNICAM_PI1)
+
+/* UNICAM_ANA Register */
+#define UNICAM_APD		BIT(0)
+#define UNICAM_BPD		BIT(1)
+#define UNICAM_AR		BIT(2)
+#define UNICAM_DDL		BIT(3)
+#define UNICAM_CTATADJ_MASK	GENMASK(7, 4)
+#define UNICAM_PTATADJ_MASK	GENMASK(11, 8)
+
+/* UNICAM_PRI Register */
+#define UNICAM_PE		BIT(0)
+#define UNICAM_PT_MASK		GENMASK(2, 1)
+#define UNICAM_NP_MASK		GENMASK(7, 4)
+#define UNICAM_PP_MASK		GENMASK(11, 8)
+#define UNICAM_BS_MASK		GENMASK(15, 12)
+#define UNICAM_BL_MASK		GENMASK(17, 16)
+
+/* UNICAM_CLK Register */
+#define UNICAM_CLE		BIT(0)
+#define UNICAM_CLPD		BIT(1)
+#define UNICAM_CLLPE		BIT(2)
+#define UNICAM_CLHSE		BIT(3)
+#define UNICAM_CLTRE		BIT(4)
+#define UNICAM_CLAC_MASK	GENMASK(8, 5)
+#define UNICAM_CLSTE		BIT(29)
+
+/* UNICAM_CLT Register */
+#define UNICAM_CLT1_MASK	GENMASK(7, 0)
+#define UNICAM_CLT2_MASK	GENMASK(15, 8)
+
+/* UNICAM_DATn Registers */
+#define UNICAM_DLE		BIT(0)
+#define UNICAM_DLPD		BIT(1)
+#define UNICAM_DLLPE		BIT(2)
+#define UNICAM_DLHSE		BIT(3)
+#define UNICAM_DLTRE		BIT(4)
+#define UNICAM_DLSM		BIT(5)
+#define UNICAM_DLFO		BIT(28)
+#define UNICAM_DLSTE		BIT(29)
+
+#define UNICAM_DAT_MASK_ALL	(UNICAM_DLSTE | UNICAM_DLFO)
+
+/* UNICAM_DLT Register */
+#define UNICAM_DLT1_MASK	GENMASK(7, 0)
+#define UNICAM_DLT2_MASK	GENMASK(15, 8)
+#define UNICAM_DLT3_MASK	GENMASK(23, 16)
+
+/* UNICAM_ICTL Register */
+#define UNICAM_FSIE		BIT(0)
+#define UNICAM_FEIE		BIT(1)
+#define UNICAM_IBOB		BIT(2)
+#define UNICAM_FCM		BIT(3)
+#define UNICAM_TFC		BIT(4)
+#define UNICAM_LIP_MASK		GENMASK(6, 5)
+#define UNICAM_LCIE_MASK	GENMASK(28, 16)
+
+/* UNICAM_IDI0/1 Register */
+#define UNICAM_ID0_MASK		GENMASK(7, 0)
+#define UNICAM_ID1_MASK		GENMASK(15, 8)
+#define UNICAM_ID2_MASK		GENMASK(23, 16)
+#define UNICAM_ID3_MASK		GENMASK(31, 24)
+
+/* UNICAM_ISTA Register */
+#define UNICAM_FSI		BIT(0)
+#define UNICAM_FEI		BIT(1)
+#define UNICAM_LCI		BIT(2)
+
+#define UNICAM_ISTA_MASK_ALL	(UNICAM_FSI | UNICAM_FEI | UNICAM_LCI)
+
+/* UNICAM_IPIPE Register */
+#define UNICAM_PUM_MASK		GENMASK(2, 0)
+/* Unpacking modes */
+#define UNICAM_PUM_NONE		0
+#define UNICAM_PUM_UNPACK6	1
+#define UNICAM_PUM_UNPACK7	2
+#define UNICAM_PUM_UNPACK8	3
+#define UNICAM_PUM_UNPACK10	4
+#define UNICAM_PUM_UNPACK12	5
+#define UNICAM_PUM_UNPACK14	6
+#define UNICAM_PUM_UNPACK16	7
+#define UNICAM_DDM_MASK		GENMASK(6, 3)
+#define UNICAM_PPM_MASK		GENMASK(9, 7)
+/* Packing modes */
+#define UNICAM_PPM_NONE		0
+#define UNICAM_PPM_PACK8	1
+#define UNICAM_PPM_PACK10	2
+#define UNICAM_PPM_PACK12	3
+#define UNICAM_PPM_PACK14	4
+#define UNICAM_PPM_PACK16	5
+#define UNICAM_DEM_MASK		GENMASK(11, 10)
+#define UNICAM_DEBL_MASK	GENMASK(14, 12)
+#define UNICAM_ICM_MASK		GENMASK(16, 15)
+#define UNICAM_IDM_MASK		GENMASK(17, 17)
+
+/* UNICAM_ICC Register */
+#define UNICAM_ICFL_MASK	GENMASK(4, 0)
+#define UNICAM_ICFH_MASK	GENMASK(9, 5)
+#define UNICAM_ICST_MASK	GENMASK(12, 10)
+#define UNICAM_ICLT_MASK	GENMASK(15, 13)
+#define UNICAM_ICLL_MASK	GENMASK(31, 16)
+
+/* UNICAM_DCS Register */
+#define UNICAM_DIE		BIT(0)
+#define UNICAM_DIM		BIT(1)
+#define UNICAM_DBOB		BIT(3)
+#define UNICAM_FDE		BIT(4)
+#define UNICAM_LDP		BIT(5)
+#define UNICAM_EDL_MASK		GENMASK(15, 8)
+
+/* UNICAM_DBCTL Register */
+#define UNICAM_DBEN		BIT(0)
+#define UNICAM_BUF0_IE		BIT(1)
+#define UNICAM_BUF1_IE		BIT(2)
+
+/* UNICAM_CMP[0,1] register */
+#define UNICAM_PCE		BIT(31)
+#define UNICAM_GI		BIT(9)
+#define UNICAM_CPH		BIT(8)
+#define UNICAM_PCVC_MASK	GENMASK(7, 6)
+#define UNICAM_PCDT_MASK	GENMASK(5, 0)
+
+/* UNICAM_MISC register */
+#define UNICAM_FL0		BIT(6)
+#define UNICAM_FL1		BIT(9)
+
+#endif
diff --git a/drivers/media/platform/broadcom/bcm2835-unicam.c b/drivers/media/platform/broadcom/bcm2835-unicam.c
new file mode 100644
index 000000000000..716c89b8a217
--- /dev/null
+++ b/drivers/media/platform/broadcom/bcm2835-unicam.c
@@ -0,0 +1,2607 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * BCM283x / BCM271x Unicam Capture Driver
+ *
+ * Copyright (C) 2017-2020 - Raspberry Pi (Trading) Ltd.
+ *
+ * Dave Stevenson <dave.stevenson@raspberrypi.com>
+ *
+ * Based on TI am437x driver by
+ *   Benoit Parrot <bparrot@ti.com>
+ *   Lad, Prabhakar <prabhakar.csengg@gmail.com>
+ *
+ * and TI CAL camera interface driver by
+ *    Benoit Parrot <bparrot@ti.com>
+ *
+ *
+ * There are two camera drivers in the kernel for BCM283x - this one and
+ * bcm2835-camera (currently in staging).
+ *
+ * This driver directly controls the Unicam peripheral - there is no
+ * involvement with the VideoCore firmware. Unicam receives CSI-2 or CCP2 data
+ * and writes it into SDRAM. The only potential processing options are to
+ * repack Bayer data into an alternate format, and applying windowing. The
+ * repacking does not shift the data, so can repack V4L2_PIX_FMT_Sxxxx10P to
+ * V4L2_PIX_FMT_Sxxxx10, or V4L2_PIX_FMT_Sxxxx12P to V4L2_PIX_FMT_Sxxxx12, but
+ * not generically up to V4L2_PIX_FMT_Sxxxx16. The driver will add both formats
+ * where the relevant formats are defined, and will automatically configure the
+ * repacking as required. Support for windowing may be added later.
+ *
+ * It should be possible to connect this driver to any sensor with a suitable
+ * output interface and V4L2 subdevice driver.
+ */
+
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/err.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/slab.h>
+#include <linux/videodev2.h>
+
+#include <media/v4l2-async.h>
+#include <media/v4l2-common.h>
+#include <media/v4l2-dev.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-fwnode.h>
+#include <media/v4l2-mc.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "bcm2835-unicam-regs.h"
+
+#define UNICAM_MODULE_NAME		"unicam"
+
+/*
+ * Unicam must request a minimum of 250Mhz from the VPU clock.
+ * Otherwise the input FIFOs overrun and cause image corruption.
+ */
+#define UNICAM_MIN_VPU_CLOCK_RATE	(250 * 1000 * 1000)
+
+/* Unicam has an internal DMA alignment constraint of 16 bytes for each line. */
+#define UNICAM_DMA_BPL_ALIGNMENT	16
+
+/*
+ * The image stride is stored in a 16 bit register, and needs to be aligned to
+ * the DMA constraint. As the ISP in the same SoC has a 32 bytes alignment
+ * constraint on its input, set the image stride alignment to 32 bytes here as
+ * well to avoid incompatible configurations.
+ */
+#define UNICAM_IMAGE_BPL_ALIGNMENT	32
+#define UNICAM_IMAGE_MAX_BPL		((1 << 16) - UNICAM_IMAGE_BPL_ALIGNMENT)
+
+/*
+ * Max width is therefore determined by the max stride divided by the number of
+ * bits per pixel. Take 32bpp as a worst case. No imposed limit on the height,
+ * so adopt a square image for want of anything better.
+ */
+#define UNICAM_IMAGE_MIN_WIDTH		16
+#define UNICAM_IMAGE_MIN_HEIGHT		16
+#define UNICAM_IMAGE_MAX_WIDTH		(UNICAM_IMAGE_MAX_BPL / 4)
+#define UNICAM_IMAGE_MAX_HEIGHT		UNICAM_IMAGE_MAX_WIDTH
+
+/*
+ * There's no intrinsic limits on the width and height for embedded dat. Use
+ * the same maximum values as for the image, to avoid overflows in the image
+ * size computation.
+ */
+#define UNICAM_META_MIN_WIDTH		1
+#define UNICAM_META_MIN_HEIGHT		1
+#define UNICAM_META_MAX_WIDTH		UNICAM_IMAGE_MAX_WIDTH
+#define UNICAM_META_MAX_HEIGHT		UNICAM_IMAGE_MAX_HEIGHT
+
+/*
+ * Size of the dummy buffer. Can be any size really, but the DMA
+ * allocation works in units of page sizes.
+ */
+#define UNICAM_DUMMY_BUF_SIZE		PAGE_SIZE
+
+#define UNICAM_SD_PAD_SINK		0
+#define UNICAM_SD_PAD_SOURCE_IMAGE	1
+#define UNICAM_SD_PAD_SOURCE_METADATA	2
+#define UNICAM_SD_NUM_PADS		(1 + UNICAM_SD_PAD_SOURCE_METADATA)
+
+enum unicam_node_type {
+	UNICAM_IMAGE_NODE,
+	UNICAM_METADATA_NODE,
+	UNICAM_MAX_NODES
+};
+
+/*
+ * struct unicam_format_info - Unicam media bus format information
+ * @fourcc: V4L2 pixel format FCC identifier. 0 if n/a.
+ * @unpacked_fourcc: V4L2 pixel format FCC identifier if the data is expanded
+ * out to 16bpp. 0 if n/a.
+ * @code: V4L2 media bus format code.
+ * @depth: Bits per pixel as delivered from the source.
+ * @csi_dt: CSI data type.
+ * @metadata_fmt: This format only applies to the metadata pad.
+ */
+struct unicam_format_info {
+	u32	fourcc;
+	u32	unpacked_fourcc;
+	u32	code;
+	u8	depth;
+	u8	csi_dt;
+	bool	metadata_fmt;
+};
+
+struct unicam_buffer {
+	struct vb2_v4l2_buffer vb;
+	struct list_head list;
+	dma_addr_t dma_addr;
+	unsigned int size;
+};
+
+static inline struct unicam_buffer *to_unicam_buffer(struct vb2_buffer *vb)
+{
+	return container_of(vb, struct unicam_buffer, vb.vb2_buf);
+}
+
+struct unicam_node {
+	bool registered;
+	bool streaming;
+	unsigned int id;
+
+	/* Pointer to the current v4l2_buffer */
+	struct unicam_buffer *cur_frm;
+	/* Pointer to the next v4l2_buffer */
+	struct unicam_buffer *next_frm;
+	/* video capture */
+	const struct unicam_format_info *fmtinfo;
+	/* Used to store current pixel format */
+	struct v4l2_format fmt;
+	/* Buffer queue used in video-buf */
+	struct vb2_queue buffer_queue;
+	/* Queue of filled frames */
+	struct list_head dma_queue;
+	/* IRQ lock for DMA queue */
+	spinlock_t dma_queue_lock;
+	/* lock used to access this structure */
+	struct mutex lock;
+	/* Identifies video device for this channel */
+	struct video_device video_dev;
+	/* Pointer to the parent handle */
+	struct unicam_device *dev;
+	struct media_pad pad;
+	/*
+	 * Dummy buffer intended to be used by unicam
+	 * if we have no other queued buffers to swap to.
+	 */
+	struct unicam_buffer dummy_buf;
+	void *dummy_buf_cpu_addr;
+};
+
+struct unicam_device {
+	struct kref kref;
+
+	/* peripheral base address */
+	void __iomem *base;
+	/* clock gating base address */
+	void __iomem *clk_gate_base;
+	/* lp clock handle */
+	struct clk *clock;
+	/* vpu clock handle */
+	struct clk *vpu_clock;
+	/* V4l2 device */
+	struct v4l2_device v4l2_dev;
+	struct media_device mdev;
+
+	/* parent device */
+	struct device *dev;
+	/* subdevice async Notifier */
+	struct v4l2_async_notifier notifier;
+	unsigned int sequence;
+
+	/* Sensor node */
+	struct {
+		struct v4l2_subdev *subdev;
+		struct media_pad *pad;
+	} sensor;
+
+	/* Internal subdev */
+	struct {
+		struct v4l2_subdev sd;
+		struct media_pad pads[UNICAM_SD_NUM_PADS];
+		bool streaming;
+	} subdev;
+
+	enum v4l2_mbus_type bus_type;
+	/*
+	 * Stores bus.mipi_csi2.flags for CSI2 sensors, or
+	 * bus.mipi_csi1.strobe for CCP2.
+	 */
+	unsigned int bus_flags;
+	unsigned int max_data_lanes;
+	unsigned int active_data_lanes;
+
+	struct media_pipeline pipe;
+
+	struct unicam_node node[UNICAM_MAX_NODES];
+};
+
+static inline struct unicam_device *
+notifier_to_unicam_device(struct v4l2_async_notifier *notifier)
+{
+	return container_of(notifier, struct unicam_device, notifier);
+}
+
+static inline struct unicam_device *
+sd_to_unicam_device(struct v4l2_subdev *sd)
+{
+	return container_of(sd, struct unicam_device, subdev.sd);
+}
+
+static void unicam_release(struct kref *kref)
+{
+	struct unicam_device *unicam =
+		container_of(kref, struct unicam_device, kref);
+
+	if (unicam->mdev.dev)
+		media_device_cleanup(&unicam->mdev);
+
+	kfree(unicam);
+}
+
+static struct unicam_device *unicam_get(struct unicam_device *unicam)
+{
+	kref_get(&unicam->kref);
+	return unicam;
+}
+
+static void unicam_put(struct unicam_device *unicam)
+{
+	kref_put(&unicam->kref, unicam_release);
+}
+
+/* -----------------------------------------------------------------------------
+ * Misc helper functions
+ */
+
+static inline bool unicam_sd_pad_is_source(u32 pad)
+{
+	/* Camera RX has 1 sink pad, and N source pads */
+	return pad != UNICAM_SD_PAD_SINK;
+}
+
+static inline bool is_metadata_node(struct unicam_node *node)
+{
+	return node->video_dev.device_caps & V4L2_CAP_META_CAPTURE;
+}
+
+static inline bool is_image_node(struct unicam_node *node)
+{
+	return node->video_dev.device_caps & V4L2_CAP_VIDEO_CAPTURE;
+}
+
+/* -----------------------------------------------------------------------------
+ * Format data table and helper functions
+ */
+
+static const struct v4l2_mbus_framefmt unicam_default_image_format = {
+	.width = 640,
+	.height = 480,
+	.code = MEDIA_BUS_FMT_UYVY8_1X16,
+	.field = V4L2_FIELD_NONE,
+	.colorspace = V4L2_COLORSPACE_SRGB,
+	.ycbcr_enc = V4L2_YCBCR_ENC_601,
+	.quantization = V4L2_QUANTIZATION_LIM_RANGE,
+	.xfer_func = V4L2_XFER_FUNC_SRGB,
+	.flags = 0,
+};
+
+static const struct v4l2_mbus_framefmt unicam_default_meta_format = {
+	.width = 640,
+	.height = 2,
+	.code = MEDIA_BUS_FMT_META_8,
+	.field = V4L2_FIELD_NONE,
+};
+
+static const struct unicam_format_info unicam_image_formats[] = {
+	/* YUV Formats */
+	{
+		.fourcc		= V4L2_PIX_FMT_YUYV,
+		.code		= MEDIA_BUS_FMT_YUYV8_1X16,
+		.depth		= 16,
+		.csi_dt		= 0x1e,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_UYVY,
+		.code		= MEDIA_BUS_FMT_UYVY8_1X16,
+		.depth		= 16,
+		.csi_dt		= 0x1e,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_YVYU,
+		.code		= MEDIA_BUS_FMT_YVYU8_1X16,
+		.depth		= 16,
+		.csi_dt		= 0x1e,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_VYUY,
+		.code		= MEDIA_BUS_FMT_VYUY8_1X16,
+		.depth		= 16,
+		.csi_dt		= 0x1e,
+	}, {
+	/* RGB Formats */
+		.fourcc		= V4L2_PIX_FMT_RGB565, /* gggbbbbb rrrrrggg */
+		.code		= MEDIA_BUS_FMT_RGB565_1X16,
+		.depth		= 16,
+		.csi_dt		= 0x22,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_RGB24, /* rgb */
+		.code		= MEDIA_BUS_FMT_RGB888_1X24,
+		.depth		= 24,
+		.csi_dt		= 0x24,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_BGR24, /* bgr */
+		.code		= MEDIA_BUS_FMT_BGR888_1X24,
+		.depth		= 24,
+		.csi_dt		= 0x24,
+	}, {
+	/* Bayer Formats */
+		.fourcc		= V4L2_PIX_FMT_SBGGR8,
+		.code		= MEDIA_BUS_FMT_SBGGR8_1X8,
+		.depth		= 8,
+		.csi_dt		= 0x2a,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SGBRG8,
+		.code		= MEDIA_BUS_FMT_SGBRG8_1X8,
+		.depth		= 8,
+		.csi_dt		= 0x2a,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SGRBG8,
+		.code		= MEDIA_BUS_FMT_SGRBG8_1X8,
+		.depth		= 8,
+		.csi_dt		= 0x2a,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SRGGB8,
+		.code		= MEDIA_BUS_FMT_SRGGB8_1X8,
+		.depth		= 8,
+		.csi_dt		= 0x2a,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SBGGR10P,
+		.unpacked_fourcc = V4L2_PIX_FMT_SBGGR10,
+		.code		= MEDIA_BUS_FMT_SBGGR10_1X10,
+		.depth		= 10,
+		.csi_dt		= 0x2b,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SGBRG10P,
+		.unpacked_fourcc = V4L2_PIX_FMT_SGBRG10,
+		.code		= MEDIA_BUS_FMT_SGBRG10_1X10,
+		.depth		= 10,
+		.csi_dt		= 0x2b,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SGRBG10P,
+		.unpacked_fourcc = V4L2_PIX_FMT_SGRBG10,
+		.code		= MEDIA_BUS_FMT_SGRBG10_1X10,
+		.depth		= 10,
+		.csi_dt		= 0x2b,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SRGGB10P,
+		.unpacked_fourcc = V4L2_PIX_FMT_SRGGB10,
+		.code		= MEDIA_BUS_FMT_SRGGB10_1X10,
+		.depth		= 10,
+		.csi_dt		= 0x2b,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SBGGR12P,
+		.unpacked_fourcc = V4L2_PIX_FMT_SBGGR12,
+		.code		= MEDIA_BUS_FMT_SBGGR12_1X12,
+		.depth		= 12,
+		.csi_dt		= 0x2c,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SGBRG12P,
+		.unpacked_fourcc = V4L2_PIX_FMT_SGBRG12,
+		.code		= MEDIA_BUS_FMT_SGBRG12_1X12,
+		.depth		= 12,
+		.csi_dt		= 0x2c,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SGRBG12P,
+		.unpacked_fourcc = V4L2_PIX_FMT_SGRBG12,
+		.code		= MEDIA_BUS_FMT_SGRBG12_1X12,
+		.depth		= 12,
+		.csi_dt		= 0x2c,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SRGGB12P,
+		.unpacked_fourcc = V4L2_PIX_FMT_SRGGB12,
+		.code		= MEDIA_BUS_FMT_SRGGB12_1X12,
+		.depth		= 12,
+		.csi_dt		= 0x2c,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SBGGR14P,
+		.unpacked_fourcc = V4L2_PIX_FMT_SBGGR14,
+		.code		= MEDIA_BUS_FMT_SBGGR14_1X14,
+		.depth		= 14,
+		.csi_dt		= 0x2d,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SGBRG14P,
+		.unpacked_fourcc = V4L2_PIX_FMT_SGBRG14,
+		.code		= MEDIA_BUS_FMT_SGBRG14_1X14,
+		.depth		= 14,
+		.csi_dt		= 0x2d,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SGRBG14P,
+		.unpacked_fourcc = V4L2_PIX_FMT_SGRBG14,
+		.code		= MEDIA_BUS_FMT_SGRBG14_1X14,
+		.depth		= 14,
+		.csi_dt		= 0x2d,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SRGGB14P,
+		.unpacked_fourcc = V4L2_PIX_FMT_SRGGB14,
+		.code		= MEDIA_BUS_FMT_SRGGB14_1X14,
+		.depth		= 14,
+		.csi_dt		= 0x2d,
+	}, {
+	/* 16 bit Bayer formats could be supported. */
+
+	/* Greyscale formats */
+		.fourcc		= V4L2_PIX_FMT_GREY,
+		.code		= MEDIA_BUS_FMT_Y8_1X8,
+		.depth		= 8,
+		.csi_dt		= 0x2a,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_Y10P,
+		.unpacked_fourcc = V4L2_PIX_FMT_Y10,
+		.code		= MEDIA_BUS_FMT_Y10_1X10,
+		.depth		= 10,
+		.csi_dt		= 0x2b,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_Y12P,
+		.unpacked_fourcc = V4L2_PIX_FMT_Y12,
+		.code		= MEDIA_BUS_FMT_Y12_1X12,
+		.depth		= 12,
+		.csi_dt		= 0x2c,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_Y14P,
+		.unpacked_fourcc = V4L2_PIX_FMT_Y14,
+		.code		= MEDIA_BUS_FMT_Y14_1X14,
+		.depth		= 14,
+		.csi_dt		= 0x2d,
+	},
+};
+
+static const struct unicam_format_info unicam_meta_formats[] = {
+	{
+		.fourcc		= V4L2_META_FMT_GENERIC_8,
+		.code		= MEDIA_BUS_FMT_META_8,
+		.depth		= 8,
+		.metadata_fmt	= true,
+	}, {
+		.fourcc		= V4L2_META_FMT_GENERIC_CSI2_10,
+		.code		= MEDIA_BUS_FMT_META_10,
+		.depth		= 10,
+		.metadata_fmt	= true,
+	}, {
+		.fourcc		= V4L2_META_FMT_GENERIC_CSI2_12,
+		.code		= MEDIA_BUS_FMT_META_12,
+		.depth		= 12,
+		.metadata_fmt	= true,
+	}, {
+		.fourcc		= V4L2_META_FMT_GENERIC_CSI2_14,
+		.code		= MEDIA_BUS_FMT_META_14,
+		.depth		= 14,
+		.metadata_fmt	= true,
+	},
+};
+
+/* Format setup functions */
+static const struct unicam_format_info *
+unicam_find_format_by_code(u32 code, u32 pad)
+{
+	const struct unicam_format_info *formats;
+	unsigned int num_formats;
+	unsigned int i;
+
+	if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
+		formats = unicam_image_formats;
+		num_formats = ARRAY_SIZE(unicam_image_formats);
+	} else {
+		formats = unicam_meta_formats;
+		num_formats = ARRAY_SIZE(unicam_meta_formats);
+	}
+
+	for (i = 0; i < num_formats; i++) {
+		if (formats[i].code == code)
+			return &formats[i];
+	}
+
+	return NULL;
+}
+
+static const struct unicam_format_info *
+unicam_find_format_by_fourcc(u32 fourcc, u32 pad)
+{
+	const struct unicam_format_info *formats;
+	unsigned int num_formats;
+	unsigned int i;
+
+	if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
+		formats = unicam_image_formats;
+		num_formats = ARRAY_SIZE(unicam_image_formats);
+	} else {
+		formats = unicam_meta_formats;
+		num_formats = ARRAY_SIZE(unicam_meta_formats);
+	}
+
+	for (i = 0; i < num_formats; ++i) {
+		if (formats[i].fourcc == fourcc)
+			return &formats[i];
+	}
+
+	return NULL;
+}
+
+static void unicam_calc_image_size_bpl(struct unicam_device *unicam,
+				       const struct unicam_format_info *fmtinfo,
+				       struct v4l2_pix_format *pix)
+{
+	u32 min_bpl;
+
+	v4l_bound_align_image(&pix->width, UNICAM_IMAGE_MIN_WIDTH,
+			      UNICAM_IMAGE_MAX_WIDTH, 2,
+			      &pix->height, UNICAM_IMAGE_MIN_HEIGHT,
+			      UNICAM_IMAGE_MAX_HEIGHT, 0, 0);
+
+	/* Unpacking always goes to 16bpp */
+	if (pix->pixelformat == fmtinfo->unpacked_fourcc)
+		min_bpl = pix->width * 2;
+	else
+		min_bpl = pix->width * fmtinfo->depth / 8;
+	min_bpl = ALIGN(min_bpl, UNICAM_IMAGE_BPL_ALIGNMENT);
+
+	pix->bytesperline = ALIGN(pix->bytesperline, UNICAM_IMAGE_BPL_ALIGNMENT);
+	pix->bytesperline = clamp_t(unsigned int, pix->bytesperline, min_bpl,
+				    UNICAM_IMAGE_MAX_BPL);
+
+	pix->sizeimage = pix->height * pix->bytesperline;
+}
+
+static void unicam_calc_meta_size_bpl(struct unicam_device *unicam,
+				      const struct unicam_format_info *fmtinfo,
+				      struct v4l2_meta_format *meta)
+{
+	v4l_bound_align_image(&meta->width, UNICAM_META_MIN_WIDTH,
+			      UNICAM_META_MAX_WIDTH, 0,
+			      &meta->height, UNICAM_META_MIN_HEIGHT,
+			      UNICAM_META_MAX_HEIGHT, 0, 0);
+
+	meta->bytesperline = ALIGN(meta->width * fmtinfo->depth / 8,
+				   UNICAM_DMA_BPL_ALIGNMENT);
+	meta->buffersize = meta->height * meta->bytesperline;
+}
+
+/* -----------------------------------------------------------------------------
+ * Hardware handling
+ */
+
+static inline void unicam_clk_write(struct unicam_device *unicam, u32 val)
+{
+	/* Pass the CM_PASSWORD along with the value. */
+	writel(val | 0x5a000000, unicam->clk_gate_base);
+}
+
+static inline u32 unicam_reg_read(struct unicam_device *unicam, u32 offset)
+{
+	return readl(unicam->base + offset);
+}
+
+static inline void unicam_reg_write(struct unicam_device *unicam, u32 offset, u32 val)
+{
+	writel(val, unicam->base + offset);
+}
+
+static inline int unicam_get_field(u32 value, u32 mask)
+{
+	return (value & mask) >> __ffs(mask);
+}
+
+static inline void unicam_set_field(u32 *valp, u32 field, u32 mask)
+{
+	u32 val = *valp;
+
+	val &= ~mask;
+	val |= (field << __ffs(mask)) & mask;
+	*valp = val;
+}
+
+static inline void unicam_reg_write_field(struct unicam_device *unicam, u32 offset,
+					  u32 field, u32 mask)
+{
+	u32 val = unicam_reg_read(unicam, offset);
+
+	unicam_set_field(&val, field, mask);
+	unicam_reg_write(unicam, offset, val);
+}
+
+static void unicam_wr_dma_addr(struct unicam_node *node,
+			       struct unicam_buffer *buf)
+{
+	dma_addr_t endaddr = buf->dma_addr + buf->size;
+
+	if (node->id == UNICAM_IMAGE_NODE) {
+		unicam_reg_write(node->dev, UNICAM_IBSA0, buf->dma_addr);
+		unicam_reg_write(node->dev, UNICAM_IBEA0, endaddr);
+	} else {
+		unicam_reg_write(node->dev, UNICAM_DBSA0, buf->dma_addr);
+		unicam_reg_write(node->dev, UNICAM_DBEA0, endaddr);
+	}
+}
+
+static unsigned int unicam_get_lines_done(struct unicam_device *unicam)
+{
+	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
+	unsigned int stride = node->fmt.fmt.pix.bytesperline;
+	struct unicam_buffer *frm = node->cur_frm;
+	dma_addr_t cur_addr;
+
+	if (!frm)
+		return 0;
+
+	cur_addr = unicam_reg_read(unicam, UNICAM_IBWP);
+	return (unsigned int)(cur_addr - frm->dma_addr) / stride;
+}
+
+static void unicam_schedule_next_buffer(struct unicam_node *node)
+{
+	struct unicam_buffer *buf;
+
+	buf = list_first_entry(&node->dma_queue, struct unicam_buffer, list);
+	node->next_frm = buf;
+	list_del(&buf->list);
+
+	unicam_wr_dma_addr(node, buf);
+}
+
+static void unicam_schedule_dummy_buffer(struct unicam_node *node)
+{
+	int node_id = is_image_node(node) ? UNICAM_IMAGE_NODE : UNICAM_METADATA_NODE;
+
+	dev_dbg(node->dev->dev, "Scheduling dummy buffer for node %d\n", node_id);
+
+	unicam_wr_dma_addr(node, &node->dummy_buf);
+
+	node->next_frm = NULL;
+}
+
+static void unicam_process_buffer_complete(struct unicam_node *node,
+					   unsigned int sequence)
+{
+	node->cur_frm->vb.field = node->fmt.fmt.pix.field;
+	node->cur_frm->vb.sequence = sequence;
+
+	vb2_buffer_done(&node->cur_frm->vb.vb2_buf, VB2_BUF_STATE_DONE);
+}
+
+static void unicam_queue_event_sof(struct unicam_device *unicam)
+{
+	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
+
+	struct v4l2_event event = {
+		.type = V4L2_EVENT_FRAME_SYNC,
+		.u.frame_sync.frame_sequence = unicam->sequence,
+	};
+
+	v4l2_event_queue(&node->video_dev, &event);
+}
+
+static irqreturn_t unicam_isr(int irq, void *dev)
+{
+	struct unicam_device *unicam = dev;
+	unsigned int lines_done = unicam_get_lines_done(dev);
+	unsigned int sequence = unicam->sequence;
+	unsigned int i;
+	u32 ista, sta;
+	bool fe;
+	u64 ts;
+
+	sta = unicam_reg_read(unicam, UNICAM_STA);
+	/* Write value back to clear the interrupts */
+	unicam_reg_write(unicam, UNICAM_STA, sta);
+
+	ista = unicam_reg_read(unicam, UNICAM_ISTA);
+	/* Write value back to clear the interrupts */
+	unicam_reg_write(unicam, UNICAM_ISTA, ista);
+
+	dev_dbg(unicam->dev, "ISR: ISTA: 0x%X, STA: 0x%X, sequence %d, lines done %d\n",
+		ista, sta, sequence, lines_done);
+
+	if (!(sta & (UNICAM_IS | UNICAM_PI0)))
+		return IRQ_HANDLED;
+
+	/*
+	 * Look for either the Frame End interrupt or the Packet Capture status
+	 * to signal a frame end.
+	 */
+	fe = ista & UNICAM_FEI || sta & UNICAM_PI0;
+
+	/*
+	 * We must run the frame end handler first. If we have a valid next_frm
+	 * and we get a simultaneout FE + FS interrupt, running the FS handler
+	 * first would null out the next_frm ptr and we would have lost the
+	 * buffer forever.
+	 */
+	if (fe) {
+		/*
+		 * Ensure we have swapped buffers already as we can't
+		 * stop the peripheral. If no buffer is available, use a
+		 * dummy buffer to dump out frames until we get a new buffer
+		 * to use.
+		 */
+		for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
+			if (!unicam->node[i].streaming)
+				continue;
+
+			/*
+			 * If cur_frm == next_frm, it means we have not had
+			 * a chance to swap buffers, likely due to having
+			 * multiple interrupts occurring simultaneously (like FE
+			 * + FS + LS). In this case, we cannot signal the buffer
+			 * as complete, as the HW will reuse that buffer.
+			 */
+			if (unicam->node[i].cur_frm &&
+			    unicam->node[i].cur_frm != unicam->node[i].next_frm)
+				unicam_process_buffer_complete(&unicam->node[i],
+							       sequence);
+			unicam->node[i].cur_frm = unicam->node[i].next_frm;
+		}
+		unicam->sequence++;
+	}
+
+	if (ista & UNICAM_FSI) {
+		/*
+		 * Timestamp is to be when the first data byte was captured,
+		 * aka frame start.
+		 */
+		ts = ktime_get_ns();
+		for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
+			if (!unicam->node[i].streaming)
+				continue;
+
+			if (unicam->node[i].cur_frm)
+				unicam->node[i].cur_frm->vb.vb2_buf.timestamp =
+								ts;
+			else
+				dev_dbg(unicam->v4l2_dev.dev,
+					"ISR: [%d] Dropping frame, buffer not available at FS\n",
+					i);
+			/*
+			 * Set the next frame output to go to a dummy frame
+			 * if we have not managed to obtain another frame
+			 * from the queue.
+			 */
+			unicam_schedule_dummy_buffer(&unicam->node[i]);
+		}
+
+		unicam_queue_event_sof(unicam);
+	}
+
+	/*
+	 * Cannot swap buffer at frame end, there may be a race condition
+	 * where the HW does not actually swap it if the new frame has
+	 * already started.
+	 */
+	if (ista & (UNICAM_FSI | UNICAM_LCI) && !fe) {
+		for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
+			if (!unicam->node[i].streaming)
+				continue;
+
+			spin_lock(&unicam->node[i].dma_queue_lock);
+			if (!list_empty(&unicam->node[i].dma_queue) &&
+			    !unicam->node[i].next_frm)
+				unicam_schedule_next_buffer(&unicam->node[i]);
+			spin_unlock(&unicam->node[i].dma_queue_lock);
+		}
+	}
+
+	if (unicam_reg_read(unicam, UNICAM_ICTL) & UNICAM_FCM) {
+		/* Switch out of trigger mode if selected */
+		unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
+		unicam_reg_write_field(unicam, UNICAM_ICTL, 0, UNICAM_FCM);
+	}
+	return IRQ_HANDLED;
+}
+
+static void unicam_set_packing_config(struct unicam_device *unicam)
+{
+	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
+	u32 pack, unpack;
+	u32 val;
+
+	if (node->fmt.fmt.pix.pixelformat == node->fmtinfo->fourcc) {
+		unpack = UNICAM_PUM_NONE;
+		pack = UNICAM_PPM_NONE;
+	} else {
+		switch (node->fmtinfo->depth) {
+		case 8:
+			unpack = UNICAM_PUM_UNPACK8;
+			break;
+		case 10:
+			unpack = UNICAM_PUM_UNPACK10;
+			break;
+		case 12:
+			unpack = UNICAM_PUM_UNPACK12;
+			break;
+		case 14:
+			unpack = UNICAM_PUM_UNPACK14;
+			break;
+		case 16:
+			unpack = UNICAM_PUM_UNPACK16;
+			break;
+		default:
+			unpack = UNICAM_PUM_NONE;
+			break;
+		}
+
+		/* Repacking is always to 16bpp */
+		pack = UNICAM_PPM_PACK16;
+	}
+
+	val = 0;
+	unicam_set_field(&val, unpack, UNICAM_PUM_MASK);
+	unicam_set_field(&val, pack, UNICAM_PPM_MASK);
+	unicam_reg_write(unicam, UNICAM_IPIPE, val);
+}
+
+static void unicam_cfg_image_id(struct unicam_device *unicam)
+{
+	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
+
+	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
+		/* CSI2 mode, hardcode VC 0 for now. */
+		unicam_reg_write(unicam, UNICAM_IDI0,
+				 (0 << 6) | node->fmtinfo->csi_dt);
+	} else {
+		/* CCP2 mode */
+		unicam_reg_write(unicam, UNICAM_IDI0,
+				 0x80 | node->fmtinfo->csi_dt);
+	}
+}
+
+static void unicam_enable_ed(struct unicam_device *unicam)
+{
+	u32 val = unicam_reg_read(unicam, UNICAM_DCS);
+
+	unicam_set_field(&val, 2, UNICAM_EDL_MASK);
+	/* Do not wrap at the end of the embedded data buffer */
+	unicam_set_field(&val, 0, UNICAM_DBOB);
+
+	unicam_reg_write(unicam, UNICAM_DCS, val);
+}
+
+static void unicam_start_rx(struct unicam_device *unicam,
+			    struct unicam_buffer *buf)
+{
+	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
+	int line_int_freq = node->fmt.fmt.pix.height >> 2;
+	unsigned int i;
+	u32 val;
+
+	if (line_int_freq < 128)
+		line_int_freq = 128;
+
+	/* Enable lane clocks */
+	val = 1;
+	for (i = 0; i < unicam->active_data_lanes; i++)
+		val = val << 2 | 1;
+	unicam_clk_write(unicam, val);
+
+	/* Basic init */
+	unicam_reg_write(unicam, UNICAM_CTRL, UNICAM_MEM);
+
+	/* Enable analogue control, and leave in reset. */
+	val = UNICAM_AR;
+	unicam_set_field(&val, 7, UNICAM_CTATADJ_MASK);
+	unicam_set_field(&val, 7, UNICAM_PTATADJ_MASK);
+	unicam_reg_write(unicam, UNICAM_ANA, val);
+	usleep_range(1000, 2000);
+
+	/* Come out of reset */
+	unicam_reg_write_field(unicam, UNICAM_ANA, 0, UNICAM_AR);
+
+	/* Peripheral reset */
+	unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPR);
+	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPR);
+
+	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPE);
+
+	/* Enable Rx control. */
+	val = unicam_reg_read(unicam, UNICAM_CTRL);
+	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
+		unicam_set_field(&val, UNICAM_CPM_CSI2, UNICAM_CPM_MASK);
+		unicam_set_field(&val, UNICAM_DCM_STROBE, UNICAM_DCM_MASK);
+	} else {
+		unicam_set_field(&val, UNICAM_CPM_CCP2, UNICAM_CPM_MASK);
+		unicam_set_field(&val, unicam->bus_flags, UNICAM_DCM_MASK);
+	}
+	/* Packet framer timeout */
+	unicam_set_field(&val, 0xf, UNICAM_PFT_MASK);
+	unicam_set_field(&val, 128, UNICAM_OET_MASK);
+	unicam_reg_write(unicam, UNICAM_CTRL, val);
+
+	unicam_reg_write(unicam, UNICAM_IHWIN, 0);
+	unicam_reg_write(unicam, UNICAM_IVWIN, 0);
+
+	/* AXI bus access QoS setup */
+	val = unicam_reg_read(unicam, UNICAM_PRI);
+	unicam_set_field(&val, 0, UNICAM_BL_MASK);
+	unicam_set_field(&val, 0, UNICAM_BS_MASK);
+	unicam_set_field(&val, 0xe, UNICAM_PP_MASK);
+	unicam_set_field(&val, 8, UNICAM_NP_MASK);
+	unicam_set_field(&val, 2, UNICAM_PT_MASK);
+	unicam_set_field(&val, 1, UNICAM_PE);
+	unicam_reg_write(unicam, UNICAM_PRI, val);
+
+	unicam_reg_write_field(unicam, UNICAM_ANA, 0, UNICAM_DDL);
+
+	/* Always start in trigger frame capture mode (UNICAM_FCM set) */
+	val = UNICAM_FSIE | UNICAM_FEIE | UNICAM_FCM | UNICAM_IBOB;
+	unicam_set_field(&val, line_int_freq, UNICAM_LCIE_MASK);
+	unicam_reg_write(unicam, UNICAM_ICTL, val);
+	unicam_reg_write(unicam, UNICAM_STA, UNICAM_STA_MASK_ALL);
+	unicam_reg_write(unicam, UNICAM_ISTA, UNICAM_ISTA_MASK_ALL);
+
+	/* tclk_term_en */
+	unicam_reg_write_field(unicam, UNICAM_CLT, 2, UNICAM_CLT1_MASK);
+	/* tclk_settle */
+	unicam_reg_write_field(unicam, UNICAM_CLT, 6, UNICAM_CLT2_MASK);
+	/* td_term_en */
+	unicam_reg_write_field(unicam, UNICAM_DLT, 2, UNICAM_DLT1_MASK);
+	/* ths_settle */
+	unicam_reg_write_field(unicam, UNICAM_DLT, 6, UNICAM_DLT2_MASK);
+	/* trx_enable */
+	unicam_reg_write_field(unicam, UNICAM_DLT, 0, UNICAM_DLT3_MASK);
+
+	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_SOE);
+
+	/* Packet compare setup - required to avoid missing frame ends */
+	val = 0;
+	unicam_set_field(&val, 1, UNICAM_PCE);
+	unicam_set_field(&val, 1, UNICAM_GI);
+	unicam_set_field(&val, 1, UNICAM_CPH);
+	unicam_set_field(&val, 0, UNICAM_PCVC_MASK);
+	unicam_set_field(&val, 1, UNICAM_PCDT_MASK);
+	unicam_reg_write(unicam, UNICAM_CMP0, val);
+
+	/* Enable clock lane and set up terminations */
+	val = 0;
+	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
+		/* CSI2 */
+		unicam_set_field(&val, 1, UNICAM_CLE);
+		unicam_set_field(&val, 1, UNICAM_CLLPE);
+		if (!(unicam->bus_flags & V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK)) {
+			unicam_set_field(&val, 1, UNICAM_CLTRE);
+			unicam_set_field(&val, 1, UNICAM_CLHSE);
+		}
+	} else {
+		/* CCP2 */
+		unicam_set_field(&val, 1, UNICAM_CLE);
+		unicam_set_field(&val, 1, UNICAM_CLHSE);
+		unicam_set_field(&val, 1, UNICAM_CLTRE);
+	}
+	unicam_reg_write(unicam, UNICAM_CLK, val);
+
+	/*
+	 * Enable required data lanes with appropriate terminations.
+	 * The same value needs to be written to UNICAM_DATn registers for
+	 * the active lanes, and 0 for inactive ones.
+	 */
+	val = 0;
+	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
+		/* CSI2 */
+		unicam_set_field(&val, 1, UNICAM_DLE);
+		unicam_set_field(&val, 1, UNICAM_DLLPE);
+		if (!(unicam->bus_flags & V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK)) {
+			unicam_set_field(&val, 1, UNICAM_DLTRE);
+			unicam_set_field(&val, 1, UNICAM_DLHSE);
+		}
+	} else {
+		/* CCP2 */
+		unicam_set_field(&val, 1, UNICAM_DLE);
+		unicam_set_field(&val, 1, UNICAM_DLHSE);
+		unicam_set_field(&val, 1, UNICAM_DLTRE);
+	}
+	unicam_reg_write(unicam, UNICAM_DAT0, val);
+
+	if (unicam->active_data_lanes == 1)
+		val = 0;
+	unicam_reg_write(unicam, UNICAM_DAT1, val);
+
+	if (unicam->max_data_lanes > 2) {
+		/*
+		 * Registers UNICAM_DAT2 and UNICAM_DAT3 only valid if the
+		 * instance supports more than 2 data lanes.
+		 */
+		if (unicam->active_data_lanes == 2)
+			val = 0;
+		unicam_reg_write(unicam, UNICAM_DAT2, val);
+
+		if (unicam->active_data_lanes == 3)
+			val = 0;
+		unicam_reg_write(unicam, UNICAM_DAT3, val);
+	}
+
+	unicam_reg_write(unicam, UNICAM_IBLS,
+			 node->fmt.fmt.pix.bytesperline);
+	unicam_wr_dma_addr(&unicam->node[UNICAM_IMAGE_NODE], buf);
+	unicam_set_packing_config(unicam);
+	unicam_cfg_image_id(unicam);
+
+	val = unicam_reg_read(unicam, UNICAM_MISC);
+	unicam_set_field(&val, 1, UNICAM_FL0);
+	unicam_set_field(&val, 1, UNICAM_FL1);
+	unicam_reg_write(unicam, UNICAM_MISC, val);
+
+	/* Enable peripheral */
+	unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPE);
+
+	/* Load image pointers */
+	unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_LIP_MASK);
+
+	/*
+	 * Enable trigger only for the first frame to
+	 * sync correctly to the FS from the source.
+	 */
+	unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
+}
+
+static void unicam_start_metadata(struct unicam_device *unicam,
+				  struct unicam_buffer *buf)
+{
+	struct unicam_node *node = &unicam->node[UNICAM_METADATA_NODE];
+
+	unicam_enable_ed(unicam);
+	unicam_wr_dma_addr(node, buf);
+	unicam_reg_write_field(unicam, UNICAM_DCS, 1, UNICAM_LDP);
+}
+
+static void unicam_disable(struct unicam_device *unicam)
+{
+	/* Analogue lane control disable */
+	unicam_reg_write_field(unicam, UNICAM_ANA, 1, UNICAM_DDL);
+
+	/* Stop the output engine */
+	unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_SOE);
+
+	/* Disable the data lanes. */
+	unicam_reg_write(unicam, UNICAM_DAT0, 0);
+	unicam_reg_write(unicam, UNICAM_DAT1, 0);
+
+	if (unicam->max_data_lanes > 2) {
+		unicam_reg_write(unicam, UNICAM_DAT2, 0);
+		unicam_reg_write(unicam, UNICAM_DAT3, 0);
+	}
+
+	/* Peripheral reset */
+	unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPR);
+	usleep_range(50, 100);
+	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPR);
+
+	/* Disable peripheral */
+	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPE);
+
+	/* Clear ED setup */
+	unicam_reg_write(unicam, UNICAM_DCS, 0);
+
+	/* Disable all lane clocks */
+	unicam_clk_write(unicam, 0);
+}
+
+/* -----------------------------------------------------------------------------
+ * V4L2 subdev operations
+ */
+
+static int __unicam_subdev_set_routing(struct v4l2_subdev *sd,
+				       struct v4l2_subdev_state *state,
+				       struct v4l2_subdev_krouting *routing)
+{
+	struct v4l2_subdev_route *route;
+	int ret;
+
+	ret = v4l2_subdev_routing_validate(sd, routing,
+					   V4L2_SUBDEV_ROUTING_ONLY_1_TO_1);
+	if (ret)
+		return ret;
+
+	ret = v4l2_subdev_set_routing(sd, state, routing);
+	if (ret)
+		return ret;
+
+	for_each_active_route(&state->routing, route) {
+		const struct v4l2_mbus_framefmt *def_fmt;
+		struct v4l2_mbus_framefmt *fmt;
+
+		if (route->source_pad == UNICAM_SD_PAD_SOURCE_IMAGE)
+			def_fmt = &unicam_default_image_format;
+		else
+			def_fmt = &unicam_default_meta_format;
+
+		fmt = v4l2_subdev_state_get_format(state, route->sink_pad,
+						   route->sink_stream);
+		*fmt = *def_fmt;
+		fmt = v4l2_subdev_state_get_format(state, route->source_pad,
+						   route->source_stream);
+		*fmt = *def_fmt;
+	}
+
+	return 0;
+}
+
+static int unicam_subdev_init_state(struct v4l2_subdev *sd,
+				    struct v4l2_subdev_state *state)
+{
+	struct v4l2_subdev_route routes[] = {
+		{
+			.sink_pad = UNICAM_SD_PAD_SINK,
+			.sink_stream = 0,
+			.source_pad = UNICAM_SD_PAD_SOURCE_IMAGE,
+			.source_stream = 0,
+			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
+		},
+	};
+
+	struct v4l2_subdev_krouting routing = {
+		.len_routes = ARRAY_SIZE(routes),
+		.num_routes = ARRAY_SIZE(routes),
+		.routes = routes,
+	};
+
+	/* Initialize routing to single route to the fist source pad. */
+	return __unicam_subdev_set_routing(sd, state, &routing);
+}
+
+static int unicam_subdev_enum_mbus_code(struct v4l2_subdev *sd,
+					struct v4l2_subdev_state *state,
+					struct v4l2_subdev_mbus_code_enum *code)
+{
+	u32 pad, stream;
+	int ret;
+
+	ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
+						    code->pad, code->stream,
+						    &pad, &stream);
+	if (ret)
+		return ret;
+
+	if (unicam_sd_pad_is_source(code->pad)) {
+		/* No transcoding, source and sink codes must match. */
+		const struct v4l2_mbus_framefmt *fmt;
+
+		fmt = v4l2_subdev_state_get_format(state, pad, stream);
+		if (!fmt)
+			return -EINVAL;
+
+		if (code->index > 0)
+			return -EINVAL;
+
+		code->code = fmt->code;
+	} else {
+		const struct unicam_format_info *formats;
+		unsigned int num_formats;
+
+		if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
+			formats = unicam_image_formats;
+			num_formats = ARRAY_SIZE(unicam_image_formats);
+		} else {
+			formats = unicam_meta_formats;
+			num_formats = ARRAY_SIZE(unicam_meta_formats);
+		}
+
+		if (code->index >= num_formats)
+			return -EINVAL;
+
+		code->code = formats[code->index].code;
+	}
+
+	return 0;
+}
+
+static int unicam_subdev_enum_frame_size(struct v4l2_subdev *sd,
+					 struct v4l2_subdev_state *state,
+					 struct v4l2_subdev_frame_size_enum *fse)
+{
+	u32 pad, stream;
+	int ret;
+
+	if (fse->index > 0)
+		return -EINVAL;
+
+	ret = v4l2_subdev_routing_find_opposite_end(&state->routing, fse->pad,
+						    fse->stream, &pad,
+						    &stream);
+	if (ret)
+		return ret;
+
+	if (unicam_sd_pad_is_source(fse->pad)) {
+		/* No transcoding, source and sink formats must match. */
+		const struct v4l2_mbus_framefmt *fmt;
+
+		fmt = v4l2_subdev_state_get_format(state, pad, stream);
+		if (!fmt)
+			return -EINVAL;
+
+		if (fse->code != fmt->code)
+			return -EINVAL;
+
+		fse->min_width = fmt->width;
+		fse->max_width = fmt->width;
+		fse->min_height = fmt->height;
+		fse->max_height = fmt->height;
+	} else {
+		const struct unicam_format_info *fmtinfo;
+
+		fmtinfo = unicam_find_format_by_code(fse->code, pad);
+		if (!fmtinfo)
+			return -EINVAL;
+
+		if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
+			fse->min_width = UNICAM_IMAGE_MIN_WIDTH;
+			fse->max_width = UNICAM_IMAGE_MAX_WIDTH;
+			fse->min_height = UNICAM_IMAGE_MIN_HEIGHT;
+			fse->max_height = UNICAM_IMAGE_MAX_HEIGHT;
+		} else {
+			fse->min_width = UNICAM_META_MIN_WIDTH;
+			fse->max_width = UNICAM_META_MAX_WIDTH;
+			fse->min_height = UNICAM_META_MIN_HEIGHT;
+			fse->max_height = UNICAM_META_MAX_HEIGHT;
+		}
+	}
+
+	return 0;
+}
+
+static int unicam_subdev_set_format(struct v4l2_subdev *sd,
+				    struct v4l2_subdev_state *state,
+				    struct v4l2_subdev_format *format)
+{
+	struct unicam_device *unicam = sd_to_unicam_device(sd);
+	struct v4l2_mbus_framefmt *sink_format, *source_format;
+	const struct unicam_format_info *fmtinfo;
+	u32 source_pad, source_stream;
+	int ret;
+
+	if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE &&
+	    unicam->subdev.streaming)
+		return -EBUSY;
+
+	/* No transcoding, source and sink formats must match. */
+	if (unicam_sd_pad_is_source(format->pad))
+		return v4l2_subdev_get_fmt(sd, state, format);
+
+	/*
+	 * Allowed formats for the stream on the sink pad depend on what source
+	 * pad the stream is routed to. Find the corresponding source pad and
+	 * use it to validate the media bus code.
+	 */
+	ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
+						    format->pad, format->stream,
+						    &source_pad, &source_stream);
+	if (ret)
+		return ret;
+
+	fmtinfo = unicam_find_format_by_code(format->format.code, source_pad);
+	if (!fmtinfo) {
+		fmtinfo = source_pad == UNICAM_SD_PAD_SOURCE_IMAGE
+			? &unicam_image_formats[0] : &unicam_meta_formats[0];
+		format->format.code = fmtinfo->code;
+	}
+
+	if (source_pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
+		format->format.width = clamp_t(unsigned int,
+					       format->format.width,
+					       UNICAM_IMAGE_MIN_WIDTH,
+					       UNICAM_IMAGE_MAX_WIDTH);
+		format->format.height = clamp_t(unsigned int,
+						format->format.height,
+						UNICAM_IMAGE_MIN_HEIGHT,
+						UNICAM_IMAGE_MAX_HEIGHT);
+		format->format.field = V4L2_FIELD_NONE;
+	} else {
+		format->format.width = clamp_t(unsigned int,
+					       format->format.width,
+					       UNICAM_META_MIN_WIDTH,
+					       UNICAM_META_MAX_WIDTH);
+		format->format.height = clamp_t(unsigned int,
+						format->format.height,
+						UNICAM_META_MIN_HEIGHT,
+						UNICAM_META_MAX_HEIGHT);
+		format->format.field = V4L2_FIELD_NONE;
+
+		/* Colorspace don't apply to metadata. */
+		format->format.colorspace = 0;
+		format->format.ycbcr_enc = 0;
+		format->format.quantization = 0;
+		format->format.xfer_func = 0;
+	}
+
+	sink_format = v4l2_subdev_state_get_format(state, format->pad,
+						   format->stream);
+	source_format = v4l2_subdev_state_get_format(state, source_pad,
+						     source_stream);
+	*sink_format = format->format;
+	*source_format = format->format;
+
+	return 0;
+}
+
+static int unicam_subdev_set_routing(struct v4l2_subdev *sd,
+				     struct v4l2_subdev_state *state,
+				     enum v4l2_subdev_format_whence which,
+				     struct v4l2_subdev_krouting *routing)
+{
+	struct unicam_device *unicam = sd_to_unicam_device(sd);
+
+	if (which == V4L2_SUBDEV_FORMAT_ACTIVE && unicam->subdev.streaming)
+		return -EBUSY;
+
+	return __unicam_subdev_set_routing(sd, state, routing);
+}
+
+static int unicam_sd_enable_streams(struct v4l2_subdev *sd,
+				    struct v4l2_subdev_state *state, u32 pad,
+				    u64 streams_mask)
+{
+	struct unicam_device *unicam = sd_to_unicam_device(sd);
+	u32 other_pad, other_stream;
+	int ret;
+
+	ret = v4l2_subdev_routing_find_opposite_end(&state->routing, pad, 0,
+						    &other_pad, &other_stream);
+	if (ret)
+		return ret;
+
+	unicam->sequence = 0;
+
+	ret = v4l2_subdev_enable_streams(unicam->sensor.subdev,
+					 unicam->sensor.pad->index,
+					 BIT(other_stream));
+	if (ret) {
+		dev_err(unicam->dev, "stream on failed in subdev\n");
+		return ret;
+	}
+
+	unicam->subdev.streaming = true;
+
+	return 0;
+}
+
+static int unicam_sd_disable_streams(struct v4l2_subdev *sd,
+				     struct v4l2_subdev_state *state, u32 pad,
+				     u64 streams_mask)
+{
+	struct unicam_device *unicam = sd_to_unicam_device(sd);
+	u32 other_pad, other_stream;
+	int ret;
+
+	ret = v4l2_subdev_routing_find_opposite_end(&state->routing, pad, 0,
+						    &other_pad, &other_stream);
+	if (ret)
+		return ret;
+
+	v4l2_subdev_disable_streams(unicam->sensor.subdev,
+				    unicam->sensor.pad->index,
+				    BIT(other_stream));
+
+	unicam->subdev.streaming = false;
+
+	return 0;
+}
+
+static const struct v4l2_subdev_pad_ops unicam_subdev_pad_ops = {
+	.enum_mbus_code		= unicam_subdev_enum_mbus_code,
+	.enum_frame_size	= unicam_subdev_enum_frame_size,
+	.get_fmt		= v4l2_subdev_get_fmt,
+	.set_fmt		= unicam_subdev_set_format,
+	.set_routing		= unicam_subdev_set_routing,
+	.enable_streams		= unicam_sd_enable_streams,
+	.disable_streams	= unicam_sd_disable_streams,
+};
+
+static const struct v4l2_subdev_ops unicam_subdev_ops = {
+	.pad			= &unicam_subdev_pad_ops,
+};
+
+static const struct v4l2_subdev_internal_ops unicam_subdev_internal_ops = {
+	.init_state		= unicam_subdev_init_state,
+};
+
+static const struct media_entity_operations unicam_subdev_media_ops = {
+	.link_validate		= v4l2_subdev_link_validate,
+	.has_pad_interdep	= v4l2_subdev_has_pad_interdep,
+};
+
+static int unicam_subdev_init(struct unicam_device *unicam)
+{
+	struct v4l2_subdev *sd = &unicam->subdev.sd;
+	int ret;
+
+	v4l2_subdev_init(sd, &unicam_subdev_ops);
+	sd->internal_ops = &unicam_subdev_internal_ops;
+	v4l2_set_subdevdata(sd, unicam);
+
+	sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
+	sd->entity.ops = &unicam_subdev_media_ops;
+	sd->dev = unicam->dev;
+	sd->owner = THIS_MODULE;
+	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_STREAMS;
+
+	strscpy(sd->name, "unicam", sizeof(sd->name));
+
+	unicam->subdev.pads[UNICAM_SD_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+	unicam->subdev.pads[UNICAM_SD_PAD_SOURCE_IMAGE].flags = MEDIA_PAD_FL_SOURCE;
+	unicam->subdev.pads[UNICAM_SD_PAD_SOURCE_METADATA].flags = MEDIA_PAD_FL_SOURCE;
+
+	ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(unicam->subdev.pads),
+				     unicam->subdev.pads);
+	if (ret) {
+		dev_err(unicam->dev, "Failed to initialize media entity: %d\n",
+			ret);
+		return ret;
+	}
+
+	ret = v4l2_subdev_init_finalize(sd);
+	if (ret) {
+		dev_err(unicam->dev, "Failed to initialize subdev: %d\n", ret);
+		goto err_entity;
+	}
+
+	ret = v4l2_device_register_subdev(&unicam->v4l2_dev, sd);
+	if (ret) {
+		dev_err(unicam->dev, "Failed to register subdev: %d\n", ret);
+		goto err_subdev;
+	}
+
+	return 0;
+
+err_subdev:
+	v4l2_subdev_cleanup(sd);
+err_entity:
+	media_entity_cleanup(&sd->entity);
+	return ret;
+}
+
+static void unicam_subdev_cleanup(struct unicam_device *unicam)
+{
+	v4l2_subdev_cleanup(&unicam->subdev.sd);
+	media_entity_cleanup(&unicam->subdev.sd.entity);
+}
+
+/* -----------------------------------------------------------------------------
+ * Videobuf2 queue operations
+ */
+
+static int unicam_queue_setup(struct vb2_queue *vq,
+			      unsigned int *nbuffers,
+			      unsigned int *nplanes,
+			      unsigned int sizes[],
+			      struct device *alloc_devs[])
+{
+	struct unicam_node *node = vb2_get_drv_priv(vq);
+	u32 size = is_image_node(node) ? node->fmt.fmt.pix.sizeimage
+		 : node->fmt.fmt.meta.buffersize;
+
+	if (vq->num_buffers + *nbuffers < 3)
+		*nbuffers = 3 - vq->num_buffers;
+
+	if (*nplanes) {
+		if (sizes[0] < size) {
+			dev_dbg(node->dev->dev, "sizes[0] %i < size %u\n",
+				sizes[0], size);
+			return -EINVAL;
+		}
+		size = sizes[0];
+	}
+
+	*nplanes = 1;
+	sizes[0] = size;
+
+	return 0;
+}
+
+static int unicam_buffer_prepare(struct vb2_buffer *vb)
+{
+	struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
+	struct unicam_buffer *buf = to_unicam_buffer(vb);
+	u32 size = is_image_node(node) ? node->fmt.fmt.pix.sizeimage
+		 : node->fmt.fmt.meta.buffersize;
+
+	if (vb2_plane_size(vb, 0) < size) {
+		dev_dbg(node->dev->dev,
+			"data will not fit into plane (%lu < %u)\n",
+			vb2_plane_size(vb, 0), size);
+		return -EINVAL;
+	}
+
+	buf->dma_addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
+	buf->size = size;
+
+	vb2_set_plane_payload(&buf->vb.vb2_buf, 0, size);
+
+	return 0;
+}
+
+static void unicam_return_buffers(struct unicam_node *node,
+				  enum vb2_buffer_state state)
+{
+	struct unicam_buffer *buf, *tmp;
+
+	list_for_each_entry_safe(buf, tmp, &node->dma_queue, list) {
+		list_del(&buf->list);
+		vb2_buffer_done(&buf->vb.vb2_buf, state);
+	}
+
+	if (node->cur_frm)
+		vb2_buffer_done(&node->cur_frm->vb.vb2_buf,
+				state);
+	if (node->next_frm && node->cur_frm != node->next_frm)
+		vb2_buffer_done(&node->next_frm->vb.vb2_buf,
+				state);
+
+	node->cur_frm = NULL;
+	node->next_frm = NULL;
+}
+
+static int unicam_start_streaming(struct vb2_queue *vq, unsigned int count)
+{
+	struct unicam_node *node = vb2_get_drv_priv(vq);
+	struct unicam_device *unicam = node->dev;
+	struct v4l2_subdev_state *state;
+	struct unicam_buffer *buf;
+	unsigned long flags;
+	int ret;
+	u32 pad, stream;
+	u32 remote_pad = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
+					     : UNICAM_SD_PAD_SOURCE_METADATA;
+
+	/* Look for the route for the given pad and stream. */
+	state = v4l2_subdev_lock_and_get_active_state(&unicam->subdev.sd);
+	ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
+						    remote_pad, 0,
+						    &pad, &stream);
+	v4l2_subdev_unlock_state(state);
+
+	if (ret)
+		goto err_return_buffers;
+
+	dev_dbg(unicam->dev, "Starting stream on %s: %u/%u -> %u/%u (%s)\n",
+		unicam->subdev.sd.name, pad, stream, remote_pad, 0,
+		is_metadata_node(node) ? "metadata" : "image");
+
+	/* The metadata node can't be started alone. */
+	if (is_metadata_node(node)) {
+		if (!unicam->node[UNICAM_IMAGE_NODE].streaming) {
+			dev_err(unicam->dev,
+				"Can't start metadata without image\n");
+			ret = -EINVAL;
+			goto err_return_buffers;
+		}
+
+		spin_lock_irqsave(&node->dma_queue_lock, flags);
+		buf = list_first_entry(&node->dma_queue,
+				       struct unicam_buffer, list);
+		dev_dbg(unicam->dev, "buffer %p\n", buf);
+		node->cur_frm = buf;
+		node->next_frm = buf;
+		list_del(&buf->list);
+		spin_unlock_irqrestore(&node->dma_queue_lock, flags);
+
+		unicam_start_metadata(unicam, buf);
+		node->streaming = true;
+		return 0;
+	}
+
+	ret = pm_runtime_resume_and_get(unicam->dev);
+	if (ret < 0) {
+		dev_err(unicam->dev, "PM runtime resume failed: %d\n", ret);
+		goto err_return_buffers;
+	}
+
+	ret = video_device_pipeline_start(&node->video_dev, &unicam->pipe);
+	if (ret < 0) {
+		dev_dbg(unicam->dev, "Failed to start media pipeline: %d\n", ret);
+		goto err_pm_put;
+	}
+
+	spin_lock_irqsave(&node->dma_queue_lock, flags);
+	buf = list_first_entry(&node->dma_queue,
+			       struct unicam_buffer, list);
+	dev_dbg(unicam->dev, "buffer %p\n", buf);
+	node->cur_frm = buf;
+	node->next_frm = buf;
+	list_del(&buf->list);
+	spin_unlock_irqrestore(&node->dma_queue_lock, flags);
+
+	unicam_start_rx(unicam, buf);
+
+	ret = v4l2_subdev_enable_streams(&unicam->subdev.sd, remote_pad, BIT(0));
+	if (ret < 0) {
+		dev_err(unicam->dev, "stream on failed in subdev\n");
+		goto error_pipeline;
+	}
+
+	node->streaming = true;
+
+	return 0;
+
+error_pipeline:
+	video_device_pipeline_stop(&node->video_dev);
+err_pm_put:
+	pm_runtime_put_sync(unicam->dev);
+err_return_buffers:
+	unicam_return_buffers(node, VB2_BUF_STATE_QUEUED);
+	return ret;
+}
+
+static void unicam_stop_streaming(struct vb2_queue *vq)
+{
+	struct unicam_node *node = vb2_get_drv_priv(vq);
+	struct unicam_device *unicam = node->dev;
+	u32 remote_pad_index = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
+						   : UNICAM_SD_PAD_SOURCE_METADATA;
+
+	node->streaming = false;
+
+	v4l2_subdev_disable_streams(&unicam->subdev.sd, remote_pad_index,
+				    BIT(0));
+
+	/* We can stream only with the image node. */
+	if (is_metadata_node(node)) {
+		/*
+		 * Allow the hardware to spin in the dummy buffer.
+		 * This is only really needed if the embedded data pad is
+		 * disabled before the image pad.
+		 */
+		unicam_wr_dma_addr(node, &node->dummy_buf);
+		goto dequeue_buffers;
+	}
+
+	unicam_disable(unicam);
+
+	video_device_pipeline_stop(&node->video_dev);
+	pm_runtime_put(unicam->dev);
+
+dequeue_buffers:
+	/* Clear all queued buffers for the node */
+	unicam_return_buffers(node, VB2_BUF_STATE_ERROR);
+}
+
+static void unicam_buffer_queue(struct vb2_buffer *vb)
+{
+	struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
+	struct unicam_buffer *buf = to_unicam_buffer(vb);
+
+	spin_lock_irq(&node->dma_queue_lock);
+	list_add_tail(&buf->list, &node->dma_queue);
+	spin_unlock_irq(&node->dma_queue_lock);
+}
+
+static const struct vb2_ops unicam_video_qops = {
+	.queue_setup		= unicam_queue_setup,
+	.wait_prepare		= vb2_ops_wait_prepare,
+	.wait_finish		= vb2_ops_wait_finish,
+	.buf_prepare		= unicam_buffer_prepare,
+	.start_streaming	= unicam_start_streaming,
+	.stop_streaming		= unicam_stop_streaming,
+	.buf_queue		= unicam_buffer_queue,
+};
+
+/* -----------------------------------------------------------------------------
+ *  V4L2 video device operations
+ */
+
+static int unicam_querycap(struct file *file, void *priv,
+			   struct v4l2_capability *cap)
+{
+	strscpy(cap->driver, UNICAM_MODULE_NAME, sizeof(cap->driver));
+	strscpy(cap->card, UNICAM_MODULE_NAME, sizeof(cap->card));
+
+	cap->capabilities |= V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_META_CAPTURE;
+
+	return 0;
+}
+
+static int unicam_enum_fmt_vid(struct file *file, void  *priv,
+			       struct v4l2_fmtdesc *f)
+{
+	unsigned int index;
+	unsigned int i;
+
+	for (i = 0, index = 0; i < ARRAY_SIZE(unicam_image_formats); i++) {
+		if (f->mbus_code && unicam_image_formats[i].code != f->mbus_code)
+			continue;
+
+		if (index == f->index) {
+			f->pixelformat = unicam_image_formats[i].fourcc;
+			return 0;
+		}
+
+		index++;
+
+		if (!unicam_image_formats[i].unpacked_fourcc)
+			continue;
+
+		if (index == f->index) {
+			f->pixelformat = unicam_image_formats[i].unpacked_fourcc;
+			return 0;
+		}
+
+		index++;
+	}
+
+	return -EINVAL;
+}
+
+static int unicam_g_fmt_vid(struct file *file, void *priv,
+			    struct v4l2_format *f)
+{
+	struct unicam_node *node = video_drvdata(file);
+
+	*f = node->fmt;
+
+	return 0;
+}
+
+static const struct unicam_format_info *
+__unicam_try_fmt_vid(struct unicam_node *node, struct v4l2_pix_format *pix)
+{
+	const struct unicam_format_info *fmtinfo;
+
+	/*
+	 * Default to the first format if the requested pixel format code isn't
+	 * supported.
+	 */
+	fmtinfo = unicam_find_format_by_fourcc(pix->pixelformat,
+					       UNICAM_SD_PAD_SOURCE_IMAGE);
+	if (!fmtinfo) {
+		fmtinfo = &unicam_image_formats[0];
+		pix->pixelformat = fmtinfo->fourcc;
+	}
+
+	unicam_calc_image_size_bpl(node->dev, fmtinfo, pix);
+
+	if (pix->field == V4L2_FIELD_ANY)
+		pix->field = V4L2_FIELD_NONE;
+
+	return fmtinfo;
+}
+
+static int unicam_try_fmt_vid(struct file *file, void *priv,
+			      struct v4l2_format *f)
+{
+	struct unicam_node *node = video_drvdata(file);
+
+	__unicam_try_fmt_vid(node, &f->fmt.pix);
+	return 0;
+}
+
+static int unicam_s_fmt_vid(struct file *file, void *priv,
+			    struct v4l2_format *f)
+{
+	struct unicam_node *node = video_drvdata(file);
+
+	if (vb2_is_busy(&node->buffer_queue))
+		return -EBUSY;
+
+	node->fmtinfo = __unicam_try_fmt_vid(node, &f->fmt.pix);
+	node->fmt = *f;
+
+	return 0;
+}
+
+static int unicam_enum_fmt_meta(struct file *file, void *priv,
+				struct v4l2_fmtdesc *f)
+{
+	unsigned int i, index;
+
+	for (i = 0, index = 0; i < ARRAY_SIZE(unicam_meta_formats); i++) {
+		if (f->mbus_code && unicam_meta_formats[i].code != f->mbus_code)
+			continue;
+		if (!unicam_meta_formats[i].metadata_fmt)
+			continue;
+
+		if (index == f->index) {
+			f->pixelformat = unicam_meta_formats[i].fourcc;
+			f->type = V4L2_BUF_TYPE_META_CAPTURE;
+			return 0;
+		}
+		index++;
+	}
+
+	return -EINVAL;
+}
+
+static int unicam_g_fmt_meta(struct file *file, void *priv,
+			     struct v4l2_format *f)
+{
+	struct unicam_node *node = video_drvdata(file);
+
+	f->fmt.meta = node->fmt.fmt.meta;
+
+	return 0;
+}
+
+static const struct unicam_format_info *
+__unicam_try_fmt_meta(struct unicam_node *node, struct v4l2_meta_format *meta)
+{
+	const struct unicam_format_info *fmtinfo;
+
+	/*
+	 * Default to the first format if the requested pixel format code isn't
+	 * supported.
+	 */
+	fmtinfo = unicam_find_format_by_fourcc(meta->dataformat,
+					       UNICAM_SD_PAD_SOURCE_METADATA);
+	if (!fmtinfo) {
+		fmtinfo = &unicam_meta_formats[0];
+		meta->dataformat = fmtinfo->fourcc;
+	}
+
+	unicam_calc_meta_size_bpl(node->dev, fmtinfo, meta);
+
+	return fmtinfo;
+}
+
+static int unicam_try_fmt_meta(struct file *file, void *priv,
+			       struct v4l2_format *f)
+{
+	struct unicam_node *node = video_drvdata(file);
+
+	__unicam_try_fmt_vid(node, &f->fmt.pix);
+	return 0;
+}
+
+static int unicam_s_fmt_meta(struct file *file, void *priv,
+			     struct v4l2_format *f)
+{
+	struct unicam_node *node = video_drvdata(file);
+
+	if (vb2_is_busy(&node->buffer_queue))
+		return -EBUSY;
+
+	node->fmtinfo = __unicam_try_fmt_meta(node, &f->fmt.meta);
+	node->fmt = *f;
+
+	return 0;
+}
+
+static int unicam_enum_framesizes(struct file *file, void *fh,
+				  struct v4l2_frmsizeenum *fsize)
+{
+	struct unicam_node *node = video_drvdata(file);
+	int ret = -EINVAL;
+
+	if (fsize->index > 0)
+		return ret;
+
+	if (is_image_node(node)) {
+		if (!unicam_find_format_by_fourcc(fsize->pixel_format,
+						  UNICAM_SD_PAD_SOURCE_IMAGE))
+			return ret;
+
+		fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+		fsize->stepwise.min_width = UNICAM_IMAGE_MIN_WIDTH;
+		fsize->stepwise.max_width = UNICAM_IMAGE_MAX_WIDTH;
+		fsize->stepwise.step_width = 1;
+		fsize->stepwise.min_height = UNICAM_IMAGE_MIN_HEIGHT;
+		fsize->stepwise.max_height = UNICAM_IMAGE_MAX_HEIGHT;
+		fsize->stepwise.step_height = 1;
+	} else {
+		if (!unicam_find_format_by_fourcc(fsize->pixel_format,
+						  UNICAM_SD_PAD_SOURCE_METADATA))
+			return ret;
+
+		fsize->stepwise.min_width = UNICAM_META_MIN_WIDTH;
+		fsize->stepwise.max_width = UNICAM_META_MAX_WIDTH;
+		fsize->stepwise.step_width = 1;
+		fsize->stepwise.min_height = UNICAM_META_MIN_HEIGHT;
+		fsize->stepwise.max_height = UNICAM_META_MAX_HEIGHT;
+		fsize->stepwise.step_height = 1;
+	}
+
+	return 0;
+}
+
+static int unicam_log_status(struct file *file, void *fh)
+{
+	struct unicam_node *node = video_drvdata(file);
+	struct unicam_device *unicam = node->dev;
+	u32 reg;
+
+	/* status for sub devices */
+	v4l2_device_call_all(&unicam->v4l2_dev, 0, core, log_status);
+
+	dev_info(unicam->dev, "-----Receiver status-----\n");
+	dev_info(unicam->dev, "V4L2 width/height:   %ux%u\n",
+		 node->fmt.fmt.pix.width, node->fmt.fmt.pix.height);
+	dev_info(unicam->dev, "Mediabus format:     %08x\n",
+		 node->fmtinfo->code);
+	dev_info(unicam->dev, "V4L2 format:         %08x\n",
+		 node->fmt.fmt.pix.pixelformat);
+	reg = unicam_reg_read(unicam, UNICAM_IPIPE);
+	dev_info(unicam->dev, "Unpacking/packing:   %u / %u\n",
+		 unicam_get_field(reg, UNICAM_PUM_MASK),
+		 unicam_get_field(reg, UNICAM_PPM_MASK));
+	dev_info(unicam->dev, "----Live data----\n");
+	dev_info(unicam->dev, "Programmed stride:   %4u\n",
+		 unicam_reg_read(unicam, UNICAM_IBLS));
+	dev_info(unicam->dev, "Detected resolution: %ux%u\n",
+		 unicam_reg_read(unicam, UNICAM_IHSTA),
+		 unicam_reg_read(unicam, UNICAM_IVSTA));
+	dev_info(unicam->dev, "Write pointer:       %08x\n",
+		 unicam_reg_read(unicam, UNICAM_IBWP));
+
+	return 0;
+}
+
+static int unicam_subscribe_event(struct v4l2_fh *fh,
+				  const struct v4l2_event_subscription *sub)
+{
+	switch (sub->type) {
+	case V4L2_EVENT_FRAME_SYNC:
+		return v4l2_event_subscribe(fh, sub, 2, NULL);
+	default:
+		return -EINVAL;
+	}
+}
+
+static const struct v4l2_ioctl_ops unicam_ioctl_ops = {
+	.vidioc_querycap		= unicam_querycap,
+
+	.vidioc_enum_fmt_vid_cap	= unicam_enum_fmt_vid,
+	.vidioc_g_fmt_vid_cap		= unicam_g_fmt_vid,
+	.vidioc_try_fmt_vid_cap		= unicam_try_fmt_vid,
+	.vidioc_s_fmt_vid_cap		= unicam_s_fmt_vid,
+
+	.vidioc_enum_fmt_meta_cap	= unicam_enum_fmt_meta,
+	.vidioc_g_fmt_meta_cap		= unicam_g_fmt_meta,
+	.vidioc_try_fmt_meta_cap	= unicam_try_fmt_meta,
+	.vidioc_s_fmt_meta_cap		= unicam_s_fmt_meta,
+
+	.vidioc_enum_framesizes		= unicam_enum_framesizes,
+
+	.vidioc_reqbufs			= vb2_ioctl_reqbufs,
+	.vidioc_create_bufs		= vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf		= vb2_ioctl_prepare_buf,
+	.vidioc_querybuf		= vb2_ioctl_querybuf,
+	.vidioc_qbuf			= vb2_ioctl_qbuf,
+	.vidioc_dqbuf			= vb2_ioctl_dqbuf,
+	.vidioc_expbuf			= vb2_ioctl_expbuf,
+	.vidioc_streamon		= vb2_ioctl_streamon,
+	.vidioc_streamoff		= vb2_ioctl_streamoff,
+
+	.vidioc_log_status		= unicam_log_status,
+	.vidioc_subscribe_event		= unicam_subscribe_event,
+	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
+};
+
+/* unicam capture driver file operations */
+static const struct v4l2_file_operations unicam_fops = {
+	.owner		= THIS_MODULE,
+	.open           = v4l2_fh_open,
+	.release        = vb2_fop_release,
+	.poll		= vb2_fop_poll,
+	.unlocked_ioctl = video_ioctl2,
+	.mmap           = vb2_fop_mmap,
+};
+
+static int unicam_video_link_validate(struct media_link *link)
+{
+	struct video_device *vdev =
+		media_entity_to_video_device(link->sink->entity);
+	struct v4l2_subdev *sd =
+		media_entity_to_v4l2_subdev(link->source->entity);
+	struct unicam_node *node = video_get_drvdata(vdev);
+	const u32 pad = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
+		      : UNICAM_SD_PAD_SOURCE_METADATA;
+	const struct v4l2_mbus_framefmt *format;
+	struct v4l2_subdev_state *state;
+	int ret = 0;
+
+	state = v4l2_subdev_lock_and_get_active_state(sd);
+
+	format = v4l2_subdev_state_get_format(state, pad, 0);
+	if (!format) {
+		ret = -EINVAL;
+		goto out;
+	}
+
+	if (is_image_node(node)) {
+		const struct v4l2_pix_format *fmt = &node->fmt.fmt.pix;
+
+		if (node->fmtinfo->code != format->code ||
+		    fmt->height != format->height ||
+		    fmt->width != format->width ||
+		    fmt->field != format->field) {
+			dev_dbg(node->dev->dev,
+				"image: (%u x %u) 0x%08x %s != (%u x %u) 0x%08x %s\n",
+				fmt->width, fmt->height, node->fmtinfo->code,
+				v4l2_field_names[fmt->field],
+				format->width, format->height, format->code,
+				v4l2_field_names[format->field]);
+			ret = -EPIPE;
+		};
+	} else {
+		const struct v4l2_meta_format *fmt = &node->fmt.fmt.meta;
+
+		if (node->fmtinfo->code != format->code ||
+		    fmt->height != format->height ||
+		    fmt->width != format->width) {
+			dev_dbg(node->dev->dev,
+				"meta: (%u x %u) 0x%04x != (%u x %u) 0x%04x\n",
+				fmt->width, fmt->height, node->fmtinfo->code,
+				format->width, format->height, format->code);
+			ret = -EPIPE;
+		};
+	}
+
+out:
+	v4l2_subdev_unlock_state(state);
+	return ret;
+}
+
+static const struct media_entity_operations unicam_video_media_ops = {
+	.link_validate = unicam_video_link_validate,
+};
+
+static void unicam_node_release(struct video_device *vdev)
+{
+	struct unicam_node *node = video_get_drvdata(vdev);
+
+	unicam_put(node->dev);
+}
+
+static void unicam_set_default_format(struct unicam_node *node)
+{
+	if (is_image_node(node)) {
+		struct v4l2_pix_format *fmt = &node->fmt.fmt.pix;
+
+		node->fmtinfo = &unicam_image_formats[0];
+		node->fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+
+		v4l2_fill_pix_format(fmt, &unicam_default_image_format);
+		fmt->pixelformat = node->fmtinfo->fourcc;
+		unicam_calc_image_size_bpl(node->dev, node->fmtinfo, fmt);
+	} else {
+		struct v4l2_meta_format *fmt = &node->fmt.fmt.meta;
+
+		node->fmtinfo = &unicam_meta_formats[0];
+		node->fmt.type = V4L2_BUF_TYPE_META_CAPTURE;
+
+		fmt->dataformat = node->fmtinfo->fourcc;
+		fmt->width = unicam_default_meta_format.width;
+		fmt->height = unicam_default_meta_format.height;
+		unicam_calc_meta_size_bpl(node->dev, node->fmtinfo, fmt);
+	}
+}
+
+static int unicam_register_node(struct unicam_device *unicam,
+				enum unicam_node_type type)
+{
+	const u32 pad_index = type == UNICAM_IMAGE_NODE
+			    ? UNICAM_SD_PAD_SOURCE_IMAGE
+			    : UNICAM_SD_PAD_SOURCE_METADATA;
+	struct unicam_node *node = &unicam->node[type];
+	struct video_device *vdev = &node->video_dev;
+	struct vb2_queue *q = &node->buffer_queue;
+	int ret;
+
+	node->dev = unicam_get(unicam);
+	node->id = type;
+
+	spin_lock_init(&node->dma_queue_lock);
+	mutex_init(&node->lock);
+
+	INIT_LIST_HEAD(&node->dma_queue);
+
+	/* Initialize the videobuf2 queue. */
+	q->type = type == UNICAM_IMAGE_NODE ? V4L2_BUF_TYPE_VIDEO_CAPTURE
+					    : V4L2_BUF_TYPE_META_CAPTURE;
+	q->io_modes = VB2_MMAP | VB2_DMABUF;
+	q->drv_priv = node;
+	q->ops = &unicam_video_qops;
+	q->mem_ops = &vb2_dma_contig_memops;
+	q->buf_struct_size = sizeof(struct unicam_buffer);
+	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+	q->lock = &node->lock;
+	q->min_queued_buffers = 1;
+	q->dev = unicam->dev;
+
+	ret = vb2_queue_init(q);
+	if (ret) {
+		dev_err(unicam->dev, "vb2_queue_init() failed\n");
+		goto err_unicam_put;
+	}
+
+	/* Initialize the video device. */
+	vdev->release = unicam_node_release;
+	vdev->fops = &unicam_fops;
+	vdev->ioctl_ops = &unicam_ioctl_ops;
+	vdev->v4l2_dev = &unicam->v4l2_dev;
+	vdev->vfl_dir = VFL_DIR_RX;
+	vdev->queue = q;
+	vdev->lock = &node->lock;
+	vdev->device_caps = type == UNICAM_IMAGE_NODE
+			  ? V4L2_CAP_VIDEO_CAPTURE : V4L2_CAP_META_CAPTURE;
+	vdev->device_caps |= V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;
+	vdev->entity.ops = &unicam_video_media_ops;
+
+	snprintf(vdev->name, sizeof(vdev->name), "%s-%s", UNICAM_MODULE_NAME,
+		 type == UNICAM_IMAGE_NODE ? "image" : "embedded");
+
+	video_set_drvdata(vdev, node);
+
+	if (type == UNICAM_IMAGE_NODE)
+		vdev->entity.flags |= MEDIA_ENT_FL_DEFAULT;
+
+	node->pad.flags = MEDIA_PAD_FL_SINK;
+
+	ret = media_entity_pads_init(&vdev->entity, 1, &node->pad);
+	if (ret)
+		goto err_unicam_put;
+
+	node->dummy_buf.size = UNICAM_DUMMY_BUF_SIZE;
+	node->dummy_buf_cpu_addr = dma_alloc_coherent(unicam->dev,
+						      node->dummy_buf.size,
+						      &node->dummy_buf.dma_addr,
+						      GFP_KERNEL);
+	if (!node->dummy_buf_cpu_addr) {
+		dev_err(unicam->dev, "Unable to allocate dummy buffer.\n");
+		ret = -ENOMEM;
+		goto err_entity_cleanup;
+	}
+
+	unicam_set_default_format(node);
+
+	ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
+	if (ret) {
+		dev_err(unicam->dev, "Unable to register video device %s\n",
+			vdev->name);
+		goto err_dma_free;
+	}
+
+	node->registered = true;
+
+	ret = media_create_pad_link(&unicam->subdev.sd.entity,
+				    pad_index,
+				    &node->video_dev.entity,
+				    0,
+				    MEDIA_LNK_FL_ENABLED |
+				    MEDIA_LNK_FL_IMMUTABLE);
+	if (ret) {
+		/*
+		 * No need for cleanup, the caller will unregister the
+		 * video device, which will drop the reference on the
+		 * device and trigger the cleanup.
+		 */
+		dev_err(unicam->dev, "Unable to create pad link for %s\n",
+			unicam->sensor.subdev->name);
+		return ret;
+	}
+
+	return 0;
+
+err_dma_free:
+	dma_free_coherent(unicam->dev, node->dummy_buf.size,
+			  node->dummy_buf_cpu_addr,
+			  node->dummy_buf.dma_addr);
+err_entity_cleanup:
+	media_entity_cleanup(&vdev->entity);
+err_unicam_put:
+	unicam_put(unicam);
+	return ret;
+}
+
+static void unicam_unregister_nodes(struct unicam_device *unicam)
+{
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
+		struct unicam_node *node = &unicam->node[i];
+
+		if (node->dummy_buf_cpu_addr)
+			dma_free_coherent(unicam->dev, node->dummy_buf.size,
+					  node->dummy_buf_cpu_addr,
+					  node->dummy_buf.dma_addr);
+
+		if (node->registered) {
+			video_unregister_device(&node->video_dev);
+			node->registered = false;
+		}
+	}
+}
+
+/* -----------------------------------------------------------------------------
+ * Power management
+ */
+
+static int unicam_runtime_resume(struct device *dev)
+{
+	struct unicam_device *unicam = dev_get_drvdata(dev);
+	int ret;
+
+	ret = clk_set_min_rate(unicam->vpu_clock, UNICAM_MIN_VPU_CLOCK_RATE);
+	if (ret) {
+		dev_err(unicam->dev, "failed to set up VPU clock\n");
+		return ret;
+	}
+
+	ret = clk_prepare_enable(unicam->vpu_clock);
+	if (ret) {
+		dev_err(unicam->dev, "Failed to enable VPU clock: %d\n", ret);
+		goto err_vpu_clock;
+	}
+
+	ret = clk_set_rate(unicam->clock, 100 * 1000 * 1000);
+	if (ret) {
+		dev_err(unicam->dev, "failed to set up CSI clock\n");
+		goto err_vpu_prepare;
+	}
+
+	ret = clk_prepare_enable(unicam->clock);
+	if (ret) {
+		dev_err(unicam->dev, "Failed to enable CSI clock: %d\n", ret);
+		goto err_vpu_prepare;
+	}
+
+	return 0;
+
+err_vpu_prepare:
+	clk_disable_unprepare(unicam->vpu_clock);
+err_vpu_clock:
+	if (clk_set_min_rate(unicam->vpu_clock, 0))
+		dev_err(unicam->dev, "failed to reset the VPU clock\n");
+
+	return ret;
+}
+
+static int unicam_runtime_suspend(struct device *dev)
+{
+	struct unicam_device *unicam = dev_get_drvdata(dev);
+
+	clk_disable_unprepare(unicam->clock);
+
+	if (clk_set_min_rate(unicam->vpu_clock, 0))
+		dev_err(unicam->dev, "failed to reset the VPU clock\n");
+
+	clk_disable_unprepare(unicam->vpu_clock);
+
+	return 0;
+}
+
+static const struct dev_pm_ops unicam_pm_ops = {
+	RUNTIME_PM_OPS(unicam_runtime_suspend, unicam_runtime_resume, NULL)
+};
+
+/* -----------------------------------------------------------------------------
+ * V4L2 async notifier
+ */
+
+static int unicam_async_bound(struct v4l2_async_notifier *notifier,
+			      struct v4l2_subdev *subdev,
+			      struct v4l2_async_connection *asc)
+{
+	struct unicam_device *unicam = notifier_to_unicam_device(notifier);
+	struct media_pad *sink = &unicam->subdev.pads[UNICAM_SD_PAD_SINK];
+	struct media_pad *source;
+	int ret;
+
+	dev_dbg(unicam->dev, "Using sensor %s for capture\n",
+		subdev->name);
+
+	ret = v4l2_create_fwnode_links_to_pad(subdev, sink, MEDIA_LNK_FL_ENABLED |
+					      MEDIA_LNK_FL_IMMUTABLE);
+	if (ret)
+		return ret;
+
+	source = media_pad_remote_pad_unique(sink);
+	if (!source) {
+		dev_err(unicam->dev, "No connected sensor pad\n");
+		return -ENOTCONN;
+	}
+
+	unicam->sensor.subdev = subdev;
+	unicam->sensor.pad = source;
+
+	return 0;
+}
+
+static int unicam_async_complete(struct v4l2_async_notifier *notifier)
+{
+	struct unicam_device *unicam = notifier_to_unicam_device(notifier);
+	int ret;
+
+	ret = unicam_register_node(unicam, UNICAM_IMAGE_NODE);
+	if (ret) {
+		dev_err(unicam->dev, "Unable to register image video device.\n");
+		goto unregister;
+	}
+
+	ret = unicam_register_node(unicam, UNICAM_METADATA_NODE);
+	if (ret) {
+		dev_err(unicam->dev, "Unable to register metadata video device.\n");
+		goto unregister;
+	}
+
+	ret = v4l2_device_register_subdev_nodes(&unicam->v4l2_dev);
+	if (ret) {
+		dev_err(unicam->dev, "Unable to register subdev nodes.\n");
+		goto unregister;
+	}
+
+	return 0;
+
+unregister:
+	unicam_unregister_nodes(unicam);
+	unicam_put(unicam);
+
+	return ret;
+}
+
+static const struct v4l2_async_notifier_operations unicam_async_ops = {
+	.bound = unicam_async_bound,
+	.complete = unicam_async_complete,
+};
+
+static int unicam_async_nf_init(struct unicam_device *unicam)
+{
+	struct v4l2_fwnode_endpoint ep = { };
+	struct fwnode_handle *ep_handle;
+	struct v4l2_async_connection *asc;
+	int ret;
+
+	ret = of_property_read_u32(unicam->dev->of_node, "brcm,num-data-lanes",
+				   &unicam->max_data_lanes);
+	if (ret < 0) {
+		dev_err(unicam->dev, "Missing %s DT property\n",
+			"brcm,num-data-lanes");
+		return -EINVAL;
+	}
+
+	/* Get and parse the local endpoint. */
+	ep_handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(unicam->dev), 0, 0,
+						    FWNODE_GRAPH_ENDPOINT_NEXT);
+	if (!ep_handle) {
+		dev_err(unicam->dev, "No endpoint found\n");
+		return -ENODEV;
+	}
+
+	ret = v4l2_fwnode_endpoint_parse(ep_handle, &ep);
+	if (ret) {
+		dev_err(unicam->dev, "Failed to parse endpoint: %d\n", ret);
+		goto error;
+	}
+
+	unicam->bus_type = ep.bus_type;
+
+	switch (ep.bus_type) {
+	case V4L2_MBUS_CSI2_DPHY: {
+		unsigned int num_data_lanes = ep.bus.mipi_csi2.num_data_lanes;
+
+		if (num_data_lanes != 1 && num_data_lanes != 2 &&
+		    num_data_lanes != 4) {
+			dev_err(unicam->dev, "%u data lanes not supported\n",
+				num_data_lanes);
+			goto error;
+		}
+
+		if (num_data_lanes > unicam->max_data_lanes) {
+			dev_err(unicam->dev,
+				"Endpoint uses %u data lanes when %u are supported\n",
+				num_data_lanes, unicam->max_data_lanes);
+			goto error;
+		}
+
+		unicam->active_data_lanes = num_data_lanes;
+		unicam->bus_flags = ep.bus.mipi_csi2.flags;
+		break;
+	}
+
+	case V4L2_MBUS_CCP2:
+		unicam->max_data_lanes = 1;
+		unicam->active_data_lanes = 1;
+		unicam->bus_flags = ep.bus.mipi_csi1.strobe;
+		break;
+
+	default:
+		/* Unsupported bus type */
+		dev_err(unicam->dev, "Unsupported bus type %u\n", ep.bus_type);
+		goto error;
+	}
+
+	/* Initialize and register the async notifier. */
+	v4l2_async_nf_init(&unicam->notifier, &unicam->v4l2_dev);
+
+	asc = v4l2_async_nf_add_fwnode_remote(&unicam->notifier, ep_handle,
+					      struct v4l2_async_connection);
+	fwnode_handle_put(ep_handle);
+	ep_handle = NULL;
+
+	if (IS_ERR(asc)) {
+		ret = PTR_ERR(asc);
+		dev_err(unicam->dev, "Failed to add entry to notifier: %d\n",
+			ret);
+		goto error;
+	}
+
+	unicam->notifier.ops = &unicam_async_ops;
+
+	ret = v4l2_async_nf_register(&unicam->notifier);
+	if (ret) {
+		dev_err(unicam->dev, "Error registering device notifier: %d\n",
+			ret);
+		goto error;
+	}
+
+	return 0;
+
+error:
+	fwnode_handle_put(ep_handle);
+	return ret;
+}
+
+/* -----------------------------------------------------------------------------
+ * Probe & remove
+ */
+
+static int unicam_media_init(struct unicam_device *unicam)
+{
+	int ret;
+
+	unicam->mdev.dev = unicam->dev;
+	strscpy(unicam->mdev.model, UNICAM_MODULE_NAME,
+		sizeof(unicam->mdev.model));
+	strscpy(unicam->mdev.serial, "", sizeof(unicam->mdev.serial));
+	unicam->mdev.hw_revision = 0;
+
+	media_device_init(&unicam->mdev);
+
+	unicam->v4l2_dev.mdev = &unicam->mdev;
+
+	ret = v4l2_device_register(unicam->dev, &unicam->v4l2_dev);
+	if (ret < 0) {
+		dev_err(unicam->dev, "Unable to register v4l2 device\n");
+		goto err_media_cleanup;
+	}
+
+	ret = media_device_register(&unicam->mdev);
+	if (ret < 0) {
+		dev_err(unicam->dev,
+			"Unable to register media-controller device\n");
+		goto err_v4l2_unregister;
+	}
+
+	return 0;
+
+err_v4l2_unregister:
+	v4l2_device_unregister(&unicam->v4l2_dev);
+err_media_cleanup:
+	media_device_cleanup(&unicam->mdev);
+	return ret;
+}
+
+static int unicam_probe(struct platform_device *pdev)
+{
+	struct unicam_device *unicam;
+	int ret;
+
+	unicam = kzalloc(sizeof(*unicam), GFP_KERNEL);
+	if (!unicam)
+		return -ENOMEM;
+
+	kref_init(&unicam->kref);
+	unicam->dev = &pdev->dev;
+	platform_set_drvdata(pdev, unicam);
+
+	unicam->base = devm_platform_ioremap_resource_byname(pdev, "unicam");
+	if (IS_ERR(unicam->base)) {
+		ret = PTR_ERR(unicam->base);
+		goto err_unicam_put;
+	}
+
+	unicam->clk_gate_base = devm_platform_ioremap_resource_byname(pdev, "cmi");
+	if (IS_ERR(unicam->clk_gate_base)) {
+		ret = PTR_ERR(unicam->clk_gate_base);
+		goto err_unicam_put;
+	}
+
+	unicam->clock = devm_clk_get(&pdev->dev, "lp");
+	if (IS_ERR(unicam->clock)) {
+		dev_err(unicam->dev, "Failed to get lp clock\n");
+		ret = PTR_ERR(unicam->clock);
+		goto err_unicam_put;
+	}
+
+	unicam->vpu_clock = devm_clk_get(&pdev->dev, "vpu");
+	if (IS_ERR(unicam->vpu_clock)) {
+		dev_err(unicam->dev, "Failed to get vpu clock\n");
+		ret = PTR_ERR(unicam->vpu_clock);
+		goto err_unicam_put;
+	}
+
+	ret = platform_get_irq(pdev, 0);
+	if (ret <= 0) {
+		dev_err(&pdev->dev, "No IRQ resource\n");
+		ret = -EINVAL;
+		goto err_unicam_put;
+	}
+
+	ret = devm_request_irq(&pdev->dev, ret, unicam_isr, 0,
+			       "unicam_capture0", unicam);
+	if (ret) {
+		dev_err(&pdev->dev, "Unable to request interrupt\n");
+		ret = -EINVAL;
+		goto err_unicam_put;
+	}
+
+	/* Enable the block power domain. */
+	pm_runtime_enable(&pdev->dev);
+
+	ret = unicam_media_init(unicam);
+	if (ret)
+		goto err_pm_runtime;
+
+	ret = unicam_subdev_init(unicam);
+	if (ret)
+		goto err_media_unregister;
+
+	ret = unicam_async_nf_init(unicam);
+	if (ret)
+		goto err_subdev_unregister;
+
+	return 0;
+
+err_subdev_unregister:
+	unicam_subdev_cleanup(unicam);
+err_media_unregister:
+	media_device_unregister(&unicam->mdev);
+err_pm_runtime:
+	pm_runtime_disable(&pdev->dev);
+err_unicam_put:
+	unicam_put(unicam);
+
+	return ret;
+}
+
+static int unicam_remove(struct platform_device *pdev)
+{
+	struct unicam_device *unicam = platform_get_drvdata(pdev);
+
+	unicam_unregister_nodes(unicam);
+	v4l2_device_unregister(&unicam->v4l2_dev);
+	media_device_unregister(&unicam->mdev);
+	v4l2_async_nf_unregister(&unicam->notifier);
+
+	unicam_subdev_cleanup(unicam);
+
+	unicam_put(unicam);
+
+	pm_runtime_disable(&pdev->dev);
+
+	return 0;
+}
+
+static const struct of_device_id unicam_of_match[] = {
+	{ .compatible = "brcm,bcm2835-unicam", },
+	{ /* sentinel */ },
+};
+MODULE_DEVICE_TABLE(of, unicam_of_match);
+
+static struct platform_driver unicam_driver = {
+	.probe		= unicam_probe,
+	.remove		= unicam_remove,
+	.driver = {
+		.name	= UNICAM_MODULE_NAME,
+		.pm	= pm_ptr(&unicam_pm_ops),
+		.of_match_table = of_match_ptr(unicam_of_match),
+	},
+};
+
+module_platform_driver(unicam_driver);
+
+MODULE_AUTHOR("Dave Stevenson <dave.stevenson@raspberrypi.com>");
+MODULE_DESCRIPTION("BCM2835 Unicam driver");
+MODULE_LICENSE("GPL");
-- 
Regards,

Laurent Pinchart

[PATCH v6 10/15] ARM: dts: bcm2835-rpi: Move firmware-clocks from bcm2711 to bcm2835

[Laurent Pinchart]

The Raspberry Pi firmware handles clocks on all BCM2835-derived SoCs,
not just on the BCM2711. Move the corresponding DT node from
bcm2711-rpi.dtsi to bcm2835-rpi.dtsi.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi | 5 -----
 arch/arm/boot/dts/broadcom/bcm2835-rpi.dtsi | 5 +++++
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi b/arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi
index d233a191c139..86188eabeb24 100644
--- a/arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi
+++ b/arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi
@@ -20,11 +20,6 @@ aliases {
 };
 
 &firmware {
-	firmware_clocks: clocks {
-		compatible = "raspberrypi,firmware-clocks";
-		#clock-cells = <1>;
-	};
-
 	expgpio: gpio {
 		compatible = "raspberrypi,firmware-gpio";
 		gpio-controller;
diff --git a/arch/arm/boot/dts/broadcom/bcm2835-rpi.dtsi b/arch/arm/boot/dts/broadcom/bcm2835-rpi.dtsi
index f0acc9390f31..7e752a66bf8d 100644
--- a/arch/arm/boot/dts/broadcom/bcm2835-rpi.dtsi
+++ b/arch/arm/boot/dts/broadcom/bcm2835-rpi.dtsi
@@ -9,6 +9,11 @@ firmware: firmware {
 
 			mboxes = <&mailbox>;
 			dma-ranges;
+
+			firmware_clocks: clocks {
+				compatible = "raspberrypi,firmware-clocks";
+				#clock-cells = <1>;
+			};
 		};
 
 		power: power {
-- 
Regards,

Laurent Pinchart

[PATCH v6 11/15] ARM: dts: bcm2835: Add Unicam CSI nodes

[Laurent Pinchart]

From: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>

Add both MIPI CSI-2 nodes in the bcm283x tree and take care of the
Raspberry Pi / BCM2711 specific in the related files.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Stefan Wahren <stefan.wahren@i2se.com>
---
 arch/arm/boot/dts/broadcom/bcm2711.dtsi     |  8 +++++++
 arch/arm/boot/dts/broadcom/bcm2835-rpi.dtsi | 14 ++++++++++++
 arch/arm/boot/dts/broadcom/bcm283x.dtsi     | 24 +++++++++++++++++++++
 3 files changed, 46 insertions(+)

diff --git a/arch/arm/boot/dts/broadcom/bcm2711.dtsi b/arch/arm/boot/dts/broadcom/bcm2711.dtsi
index 22c7f1561344..1d8f9f80f935 100644
--- a/arch/arm/boot/dts/broadcom/bcm2711.dtsi
+++ b/arch/arm/boot/dts/broadcom/bcm2711.dtsi
@@ -1114,6 +1114,14 @@ &rmem {
 	#address-cells = <2>;
 };
 
+&csi0 {
+	interrupts = <GIC_SPI 102 IRQ_TYPE_LEVEL_HIGH>;
+};
+
+&csi1 {
+	interrupts = <GIC_SPI 103 IRQ_TYPE_LEVEL_HIGH>;
+};
+
 &cma {
 	/*
 	 * arm64 reserves the CMA by default somewhere in ZONE_DMA32,
diff --git a/arch/arm/boot/dts/broadcom/bcm2835-rpi.dtsi b/arch/arm/boot/dts/broadcom/bcm2835-rpi.dtsi
index 7e752a66bf8d..f2aad209187f 100644
--- a/arch/arm/boot/dts/broadcom/bcm2835-rpi.dtsi
+++ b/arch/arm/boot/dts/broadcom/bcm2835-rpi.dtsi
@@ -30,6 +30,20 @@ vchiq: mailbox@7e00b840 {
 	};
 };
 
+&csi0 {
+	clocks = <&clocks BCM2835_CLOCK_CAM0>,
+		 <&firmware_clocks 4>;
+	clock-names = "lp", "vpu";
+	power-domains = <&power RPI_POWER_DOMAIN_UNICAM0>;
+};
+
+&csi1 {
+	clocks = <&clocks BCM2835_CLOCK_CAM1>,
+		 <&firmware_clocks 4>;
+	clock-names = "lp", "vpu";
+	power-domains = <&power RPI_POWER_DOMAIN_UNICAM1>;
+};
+
 &gpio {
 	gpioout: gpioout {
 		brcm,pins = <6>;
diff --git a/arch/arm/boot/dts/broadcom/bcm283x.dtsi b/arch/arm/boot/dts/broadcom/bcm283x.dtsi
index 2ca8a2505a4d..69b0919f1324 100644
--- a/arch/arm/boot/dts/broadcom/bcm283x.dtsi
+++ b/arch/arm/boot/dts/broadcom/bcm283x.dtsi
@@ -454,6 +454,30 @@ dsi1: dsi@7e700000 {
 			status = "disabled";
 		};
 
+		csi0: csi@7e800000 {
+			compatible = "brcm,bcm2835-unicam";
+			reg = <0x7e800000 0x800>,
+			      <0x7e802000 0x4>;
+			reg-names = "unicam", "cmi";
+			interrupts = <2 6>;
+			brcm,num-data-lanes = <2>;
+			status = "disabled";
+			port {
+			};
+		};
+
+		csi1: csi@7e801000 {
+			compatible = "brcm,bcm2835-unicam";
+			reg = <0x7e801000 0x800>,
+			      <0x7e802004 0x4>;
+			reg-names = "unicam", "cmi";
+			interrupts = <2 7>;
+			brcm,num-data-lanes = <4>;
+			status = "disabled";
+			port {
+			};
+		};
+
 		i2c1: i2c@7e804000 {
 			compatible = "brcm,bcm2835-i2c";
 			reg = <0x7e804000 0x1000>;
-- 
Regards,

Laurent Pinchart

[PATCH v6 12/15] ARM: dts: bcm2711-rpi: Add pinctrl-based multiplexing for I2C0

[Laurent Pinchart]

From: Uwe Kleine-König <uwe@kleine-koenig.org>

BCM2711-based Raspberry Pi boards (4B, CM4 and 400) multiplex the I2C0
controller over two sets of pins, GPIO0+1 and GPIO44+45. The former is
exposed on the 40-pin header, while the latter is used for the CSI and
DSI connectors.

Add a pinctrl-based I2C bus multiplexer to bcm2711-rpi.dtsi to model
this multiplexing. The two child buses are named i2c0_0 and i2c0_1.

Note that if you modified the dts before to add devices to the i2c bus
appearing on pins gpio0 + gpio1 (either directly in the dts or using an
overlay), you have to put these into the i2c0_0 node introduced here
now.

Signed-off-by: Uwe Kleine-König <uwe@kleine-koenig.org>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
Changes since v3:

- Split addition of the RTC to a separate patch
- Move the mux to bcm2711-rpi.dtsi
---
 arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi | 31 +++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi b/arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi
index 86188eabeb24..826ed6efa9ff 100644
--- a/arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi
+++ b/arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi
@@ -17,6 +17,32 @@ aliases {
 		pcie0 = &pcie0;
 		blconfig = &blconfig;
 	};
+
+	i2c0mux: i2c0mux {
+		compatible = "i2c-mux-pinctrl";
+		#address-cells = <1>;
+		#size-cells = <0>;
+
+		i2c-parent = <&i2c0>;
+
+		pinctrl-names = "i2c0", "i2c0-vc";
+		pinctrl-0 = <&i2c0_gpio0>;
+		pinctrl-1 = <&i2c0_gpio44>;
+
+		status = "disabled";
+
+		i2c0_0: i2c@0 {
+			reg = <0>;
+			#address-cells = <1>;
+			#size-cells = <0>;
+		};
+
+		i2c0_1: i2c@1 {
+			reg = <1>;
+			#address-cells = <1>;
+			#size-cells = <0>;
+		};
+	};
 };
 
 &firmware {
@@ -49,6 +75,11 @@ &hvs {
 	clocks = <&firmware_clocks 4>;
 };
 
+&i2c0 {
+	/delete-property/ pinctrl-names;
+	/delete-property/ pinctrl-0;
+};
+
 &rmem {
 	/*
 	 * RPi4's co-processor will copy the board's bootloader configuration
-- 
Regards,

Laurent Pinchart

[PATCH v6 13/15] ARM: dts: bcm2711-rpi-cm4-io: Add RTC on I2C0

[Laurent Pinchart]

From: Uwe Kleine-König <uwe@kleine-koenig.org>

The cm4-io board comes with a PCF85063 on I2C0, connected to the GPIO44
and GPIO45 pins. Add it to the device tree.

Signed-off-by: Uwe Kleine-König <uwe@kleine-koenig.org>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
Changes since v4:

- Use the right part number in the compatible string
- Add the quartz-load-femtofarads property

Changes since v3:

- Separate addition of the RTC to a patch of its own
---
 .../boot/dts/broadcom/bcm2711-rpi-cm4-io.dts    | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/arch/arm/boot/dts/broadcom/bcm2711-rpi-cm4-io.dts b/arch/arm/boot/dts/broadcom/bcm2711-rpi-cm4-io.dts
index d7ba02f586d3..d5ebb535afd2 100644
--- a/arch/arm/boot/dts/broadcom/bcm2711-rpi-cm4-io.dts
+++ b/arch/arm/boot/dts/broadcom/bcm2711-rpi-cm4-io.dts
@@ -101,6 +101,23 @@ &genet {
 	status = "okay";
 };
 
+&i2c0 {
+	status = "okay";
+};
+
+&i2c0_1 {
+	rtc@51 {
+		/* Attention: An alarm resets the machine */
+		compatible = "nxp,pcf85063a";
+		reg = <0x51>;
+		quartz-load-femtofarads = <7000>;
+	};
+};
+
+&i2c0mux {
+	status = "okay";
+};
+
 &led_act {
 	gpios = <&gpio 42 GPIO_ACTIVE_HIGH>;
 };
-- 
Regards,

Laurent Pinchart

[PATCH v6 14/15] ARM: dts: bcm2711-rpi-4-b: Add CAM1 regulator

[Laurent Pinchart]

Add a fixed regulator to model the power supply to the camera connector.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 arch/arm/boot/dts/broadcom/bcm2711-rpi-4-b.dts | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/arm/boot/dts/broadcom/bcm2711-rpi-4-b.dts b/arch/arm/boot/dts/broadcom/bcm2711-rpi-4-b.dts
index d5f8823230db..cfc8cb5e10ba 100644
--- a/arch/arm/boot/dts/broadcom/bcm2711-rpi-4-b.dts
+++ b/arch/arm/boot/dts/broadcom/bcm2711-rpi-4-b.dts
@@ -15,6 +15,13 @@ chosen {
 		stdout-path = "serial1:115200n8";
 	};
 
+	cam1_reg: regulator-cam1 {
+		compatible = "regulator-fixed";
+		regulator-name = "cam1-reg";
+		enable-active-high;
+		gpio = <&expgpio 5 GPIO_ACTIVE_HIGH>;
+	};
+
 	sd_io_1v8_reg: regulator-sd-io-1v8 {
 		compatible = "regulator-gpio";
 		regulator-name = "vdd-sd-io";
-- 
Regards,

Laurent Pinchart

[PATCH v6 15/15] [DNI] arm64: dts: broadcom: Add overlay for Raspberry Pi 4B IMX219 camera

[Laurent Pinchart]

For testing only at this point.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 arch/arm64/boot/dts/broadcom/Makefile         |  2 +
 .../dts/broadcom/bcm2711-rpi-4-b-imx219.dtso  | 65 +++++++++++++++++++
 2 files changed, 67 insertions(+)
 create mode 100644 arch/arm64/boot/dts/broadcom/bcm2711-rpi-4-b-imx219.dtso

diff --git a/arch/arm64/boot/dts/broadcom/Makefile b/arch/arm64/boot/dts/broadcom/Makefile
index 8b4591ddd27c..a64cfef8bd9a 100644
--- a/arch/arm64/boot/dts/broadcom/Makefile
+++ b/arch/arm64/boot/dts/broadcom/Makefile
@@ -12,6 +12,8 @@ dtb-$(CONFIG_ARCH_BCM2835) += bcm2711-rpi-400.dtb \
 			      bcm2837-rpi-cm3-io3.dtb \
 			      bcm2837-rpi-zero-2-w.dtb
 
+bcm2711-rpi-4-b-imx219-dtbs := bcm2711-rpi-4-b.dtb bcm2711-rpi-4-b-imx219.dtbo
+
 subdir-y	+= bcmbca
 subdir-y	+= northstar2
 subdir-y	+= stingray
diff --git a/arch/arm64/boot/dts/broadcom/bcm2711-rpi-4-b-imx219.dtso b/arch/arm64/boot/dts/broadcom/bcm2711-rpi-4-b-imx219.dtso
new file mode 100644
index 000000000000..33c3219ca4c3
--- /dev/null
+++ b/arch/arm64/boot/dts/broadcom/bcm2711-rpi-4-b-imx219.dtso
@@ -0,0 +1,65 @@
+// SPDX-License-Identifier: GPL-2.0-only
+// Definitions for IMX219 camera module on VC I2C bus
+
+/dts-v1/;
+/plugin/;
+
+&{/} {
+	cam1_clk: cam1_clk {
+		compatible = "fixed-clock";
+		#clock-cells = <0>;
+		clock-frequency = <24000000>;
+	};
+};
+
+&csi1 {
+	status = "okay";
+
+	port {
+		csi_ep: endpoint {
+			remote-endpoint = <&cam_endpoint>;
+			clock-lanes = <0>;
+			data-lanes = <1 2>;
+			clock-noncontinuous;
+		};
+	};
+};
+
+&i2c0 {
+	status = "okay";
+};
+
+&i2c0_1 {
+	#address-cells = <1>;
+	#size-cells = <0>;
+	status = "okay";
+
+	camera@10 {
+		compatible = "sony,imx219";
+		reg = <0x10>;
+
+		clocks = <&cam1_clk>;
+		clock-names = "xclk";
+
+		VANA-supply = <&cam1_reg>;	/* 2.8v */
+		VDIG-supply = <&cam1_reg>;	/* 1.8v */
+		VDDL-supply = <&cam1_reg>;	/* 1.2v */
+
+		rotation = <180>;
+		orientation = <2>;
+
+		port {
+			cam_endpoint: endpoint {
+				remote-endpoint = <&csi_ep>;
+				clock-lanes = <0>;
+				data-lanes = <1 2>;
+				clock-noncontinuous;
+				link-frequencies = /bits/ 64 <456000000>;
+			};
+		};
+	};
+};
+
+&i2c0mux {
+	status = "okay";
+};
-- 
Regards,

Laurent Pinchart

Re: [PATCH 00/15] media: Add driver for the Raspberry Pi <5 CSI-2 receiver

[Laurent Pinchart]

The subject line should obviously have read 'PATCH v6', and I should
have updated Jean-Michel's e-mail address instead of blindly relying on
get-maintainer.pl. Maybe sending patches on a Friday evening isn't he
best idea after all. Sorry about that.

On Fri, Mar 01, 2024 at 11:32:15PM +0200, Laurent Pinchart wrote:
> Hello everybody,
> 
> This patch series adds a new driver for the BCM2835 (and derivative)
> CCP2/CSI2 camera interface named Unicam. This IP core is found in the
> VC4-based Raspberry Pi, namely the Pi Zero, Pi 3 and Pi 4.
> 
> Camera support for Raspberry Pi 4 currently relies on a downstream
> Unicam driver that live in the Raspberry Pi kernel tree ([1]). The
> driver uses the V4L2 API, but works around the lack of features in V4L2
> to properly support sensor embedded data. Since the Unicam driver
> development by Raspberry Pi, some of those features have been merged in
> the kernel (namely the V4L2 streams API) or are being developed (namely
> generic metadata formats and subdev internal pads), with patches posted
> for review on the linux-media mailing list ([2]).
> 
> This new upstream driver is based on the downstream code, extensively
> reworked to use the new V4L2 APIs.
> 
> The series is based on top of a merge of
> 
> - v7 of the generic metadata and internal pads, rebased on v6.8-rc5 ([3])
> - the downstream ISP driver ported to mainline ([4])
> 
> For convenience, it can be found in [5]. Note that the ISP driver is
> getting upstreamed separately.
> 
> The series starts with five patches that add support for streams and
> embedded data to the imx219 driver (01/15 to 05/15). Patches 06/15 to
> 09/15 then add the Unicam driver, with new V4L2 pixel formats (06/15 and
> 07/15) and DT bindings (08/15) The remaining patches cover DT
> integration (10/15 to 14/15) with a sample DT overlay for the IMX219
> camera module (15/15).
> 
> The patches have been tested on a Raspberry Pi 4 using an IMX219 camera
> module (the Raspberry Pi camera v2), with libcamera. Updates are needed
> to libcamera to use the new V4L2 APIs, patches have been posted to [6].
> For manual testing with media-ctl, corresponding API updates to
> v4l-utils are available at [7].
> 
> The changes to the imx219 driver effectively define the interface that
> raw sensors should expose to userspace. This needs to be documented
> explicitly. I would like this series to first get a review round, which
> will likely raise API questions. I will then work on the documentation.
> 
> As a summary, more work is needed, but I believe we're nearing
> completion. This series, along with the libcamera support, help
> validating the new kernel APIs. We have in my opinion reached a point
> where we can start converting other sensor drivers from the downstream
> Raspberry Pi kernel to the standard APIs for embedded data, as well as
> integrating the APIs in the Raspberry Pi 5 CFE driver.
> 
> [1] https://github.com/raspberrypi/linux/tree/rpi-6.1.y/drivers/media/platform/bcm2835
> [2] https://lore.kernel.org/linux-media/20231106122539.1268265-1-sakari.ailus@linux.intel.com
> [3] https://git.kernel.org/pub/scm/linux/kernel/git/pinchartl/linux.git/log/?h=rpi/v6.8/metadata/v7
> [4] https://git.kernel.org/pub/scm/linux/kernel/git/pinchartl/linux.git/log/?h=rpi/v6.8/isp/v2
> [5] https://git.kernel.org/pub/scm/linux/kernel/git/pinchartl/linux.git/log/?h=rpi/v6.8/unicam/next
> [6] https://lists.libcamera.org/pipermail/libcamera-devel/2024-March/040711.html
> [7] https://git.linuxtv.org/pinchartl/v4l-utils.git/log/?h=metadata
> 
> Here's the mandatory v4l2-compliance report. There are 5 errors, which
> are caused by v4l2-compliance not supporting the new embedded data APIs
> yet. This will be fixed and patches for v4l2-compliance will be
> submitted (alongside patches to media-ctl).
> 
> v4l2-compliance 1.27.0-5180, 64 bits, 64-bit time_t
> v4l2-compliance SHA: c14579f7e5f5 2024-03-01 20:29:24
> 
> Compliance test for unicam device /dev/media0:
> 
> Media Driver Info:
> 	Driver name      : unicam
> 	Model            : unicam
> 	Serial           :
> 	Bus info         : platform:fe801000.csi
> 	Media version    : 6.8.0
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 6.8.0
> 
> Required ioctls:
> 	test MEDIA_IOC_DEVICE_INFO: OK
> 	test invalid ioctls: OK
> 
> Allow for multiple opens:
> 	test second /dev/media0 open: OK
> 	test MEDIA_IOC_DEVICE_INFO: OK
> 	test for unlimited opens: OK
> 
> Media Controller ioctls:
> 	test MEDIA_IOC_G_TOPOLOGY: OK
> 	Entities: 4 Interfaces: 4 Pads: 8 Links: 7
> 	test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> 	test MEDIA_IOC_SETUP_LINK: OK
> 
> Total for unicam device /dev/media0: 8, Succeeded: 8, Failed: 0, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for unicam device /dev/video0:
> 
> Driver Info:
> 	Driver name      : unicam
> 	Card type        : unicam
> 	Bus info         : platform:fe801000.csi
> 	Driver version   : 6.8.0
> 	Capabilities     : 0xa4a00001
> 		Video Capture
> 		Metadata Capture
> 		I/O MC
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x24200001
> 		Video Capture
> 		I/O MC
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : unicam
> 	Model            : unicam
> 	Serial           :
> 	Bus info         : platform:fe801000.csi
> 	Media version    : 6.8.0
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 6.8.0
> Interface Info:
> 	ID               : 0x0300000d
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x0000000b (11)
> 	Name             : unicam-image
> 	Function         : V4L2 I/O
> 	Flags            : default
> 	Pad 0x0100000c   : 0: Sink
> 	  Link 0x0200000f: from remote pad 0x1000003 of entity 'unicam' (Video Interface Bridge): Data, Enabled, Immutable
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 	test invalid ioctls: OK
> 
> Allow for multiple opens:
> 	test second /dev/video0 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 1 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls (Input 0):
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls (Input 0):
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK
> 
> Codec ioctls (Input 0):
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls (Input 0):
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> 	test CREATE_BUFS maximum buffers: OK
> 	test VIDIOC_EXPBUF: OK
> 	test Requests: OK (Not Supported)
> 
> Total for unicam device /dev/video0: 47, Succeeded: 47, Failed: 0, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for unicam device /dev/video1:
> 
> Driver Info:
> 	Driver name      : unicam
> 	Card type        : unicam
> 	Bus info         : platform:fe801000.csi
> 	Driver version   : 6.8.0
> 	Capabilities     : 0xa4a00001
> 		Video Capture
> 		Metadata Capture
> 		I/O MC
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x24a00000
> 		Metadata Capture
> 		I/O MC
> 		Streaming
> 		Extended Pix Format
> Media Driver Info:
> 	Driver name      : unicam
> 	Model            : unicam
> 	Serial           :
> 	Bus info         : platform:fe801000.csi
> 	Media version    : 6.8.0
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 6.8.0
> Interface Info:
> 	ID               : 0x03000013
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x00000011 (17)
> 	Name             : unicam-embedded
> 	Function         : V4L2 I/O
> 	Pad 0x01000012   : 0: Sink
> 	  Link 0x02000015: from remote pad 0x1000004 of entity 'unicam' (Video Interface Bridge): Data, Enabled, Immutable
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 	test invalid ioctls: OK
> 
> Allow for multiple opens:
> 	test second /dev/video1 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 1 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls (Input 0):
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls (Input 0):
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 		fail: v4l2-test-formats.cpp(590): dataformat 00003ff8 (x?) for buftype 13 not reported by ENUM_FMT
> 	test VIDIOC_TRY_FMT: FAIL
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls (Input 0):
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls (Input 0):
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> 	test CREATE_BUFS maximum buffers: OK
> 	test VIDIOC_EXPBUF: OK
> 	test Requests: OK (Not Supported)
> 
> Total for unicam device /dev/video1: 47, Succeeded: 46, Failed: 1, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for unicam device /dev/v4l-subdev0:
> 
> Driver Info:
> 	Driver version   : 6.8.0
> 	Capabilities     : 0x00000002
> 		Streams Support
> 	Client Capabilities: 0x0000000000000003
> streams interval-uses-which Media Driver Info:
> 	Driver name      : unicam
> 	Model            : unicam
> 	Serial           :
> 	Bus info         : platform:fe801000.csi
> 	Media version    : 6.8.0
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 6.8.0
> Interface Info:
> 	ID               : 0x03000017
> 	Type             : V4L Sub-Device
> Entity Info:
> 	ID               : 0x00000001 (1)
> 	Name             : unicam
> 	Function         : Video Interface Bridge
> 	Pad 0x01000002   : 0: Sink
> 	  Link 0x02000009: from remote pad 0x1000006 of entity 'imx219 5-0010' (Camera Sensor): Data, Enabled, Immutable
> 	Pad 0x01000003   : 1: Source
> 	  Link 0x0200000f: to remote pad 0x100000c of entity 'unicam-image' (V4L2 I/O): Data, Enabled, Immutable
> 	Pad 0x01000004   : 2: Source
> 	  Link 0x02000015: to remote pad 0x1000012 of entity 'unicam-embedded' (V4L2 I/O): Data, Enabled, Immutable
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_SUDBEV_QUERYCAP: OK
> 	test invalid ioctls: OK
> 
> Allow for multiple opens:
> 	test second /dev/v4l-subdev0 open: OK
> 	test VIDIOC_SUBDEV_QUERYCAP: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Sub-Device routing ioctls:
> 		fail: v4l2-test-subdevs.cpp(631): !(source->flags & MEDIA_PAD_FL_SOURCE)
> 	test Try VIDIOC_SUBDEV_G_ROUTING/VIDIOC_SUBDEV_S_ROUTING: FAIL
> 		fail: v4l2-test-subdevs.cpp(631): !(source->flags & MEDIA_PAD_FL_SOURCE)
> 	test Active VIDIOC_SUBDEV_G_ROUTING/VIDIOC_SUBDEV_S_ROUTING: FAIL
> 
> Sub-Device ioctls (Sink Pad 0):
> 
> Sub-Device ioctls (Source Pad 1):
> 
> Sub-Device ioctls (Source Pad 2):
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK (Not Supported)
> 	test VIDIOC_TRY_FMT: OK (Not Supported)
> 	test VIDIOC_S_FMT: OK (Not Supported)
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> 	test CREATE_BUFS maximum buffers: OK
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK (Not Supported)
> 
> Total for unicam device /dev/v4l-subdev0: 47, Succeeded: 45, Failed: 2, Warnings: 0
> --------------------------------------------------------------------------------
> Compliance test for unicam device /dev/v4l-subdev1:
> 
> Driver Info:
> 	Driver version   : 6.8.0
> 	Capabilities     : 0x00000002
> 		Streams Support
> 	Client Capabilities: 0x0000000000000003
> streams interval-uses-which Media Driver Info:
> 	Driver name      : unicam
> 	Model            : unicam
> 	Serial           :
> 	Bus info         : platform:fe801000.csi
> 	Media version    : 6.8.0
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 6.8.0
> Interface Info:
> 	ID               : 0x03000019
> 	Type             : V4L Sub-Device
> Entity Info:
> 	ID               : 0x00000005 (5)
> 	Name             : imx219 5-0010
> 	Function         : Camera Sensor
> 	Pad 0x01000006   : 0: Source
> 	  Link 0x02000009: to remote pad 0x1000002 of entity 'unicam' (Video Interface Bridge): Data, Enabled, Immutable
> 	Pad 0x01000007   : 1: Sink, 00000008
> 	Pad 0x01000008   : 2: Sink, 00000008
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_SUDBEV_QUERYCAP: OK
> 	test invalid ioctls: OK
> 
> Allow for multiple opens:
> 	test second /dev/v4l-subdev1 open: OK
> 	test VIDIOC_SUBDEV_QUERYCAP: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Sub-Device routing ioctls:
> 		fail: v4l2-test-subdevs.cpp(630): !(sink->flags & MEDIA_PAD_FL_SINK)
> 	test Try VIDIOC_SUBDEV_G_ROUTING/VIDIOC_SUBDEV_S_ROUTING: FAIL
> 		fail: v4l2-test-subdevs.cpp(630): !(sink->flags & MEDIA_PAD_FL_SINK)
> 	test Active VIDIOC_SUBDEV_G_ROUTING/VIDIOC_SUBDEV_S_ROUTING: FAIL
> 
> Sub-Device ioctls (Source Pad 0):
> 
> Sub-Device ioctls (Sink Pad 1):
> 
> Sub-Device ioctls (Sink Pad 2):
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> 	test VIDIOC_QUERYCTRL: OK
> 	test VIDIOC_G/S_CTRL: OK
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 20 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK (Not Supported)
> 	test VIDIOC_TRY_FMT: OK (Not Supported)
> 	test VIDIOC_S_FMT: OK (Not Supported)
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> 	test CREATE_BUFS maximum buffers: OK
> 	test VIDIOC_EXPBUF: OK (Not Supported)
> 	test Requests: OK (Not Supported)
> 
> Total for unicam device /dev/v4l-subdev1: 47, Succeeded: 45, Failed: 2, Warnings: 0
> 
> Grand Total for unicam device /dev/media0: 196, Succeeded: 191, Failed: 5, Warnings: 0
> 
> 
> Dave Stevenson (2):
>   dt-bindings: media: Add bindings for bcm2835-unicam
>   media: bcm2835-unicam: Add support for CCP2/CSI2 camera interface
> 
> Jean-Michel Hautbois (3):
>   media: v4l: Add V4L2-PIX-FMT-Y12P format
>   media: v4l: Add V4L2-PIX-FMT-Y14P format
>   ARM: dts: bcm2835: Add Unicam CSI nodes
> 
> Laurent Pinchart (8):
>   media: i2c: imx219: Inline imx219_update_pad_format() in its caller
>   media: i2c: imx219: Add internal image sink pad
>   media: i2c: imx219: Report internal routes to userspace
>   media: i2c: imx219: Report streams using frame descriptors
>   media: i2c: imx219: Add embedded data support
>   ARM: dts: bcm2835-rpi: Move firmware-clocks from bcm2711 to bcm2835
>   ARM: dts: bcm2711-rpi-4-b: Add CAM1 regulator
>   [DNI] arm64: dts: broadcom: Add overlay for Raspberry Pi 4B IMX219
>     camera
> 
> Uwe Kleine-König (2):
>   ARM: dts: bcm2711-rpi: Add pinctrl-based multiplexing for I2C0
>   ARM: dts: bcm2711-rpi-cm4-io: Add RTC on I2C0
> 
>  .../bindings/media/brcm,bcm2835-unicam.yaml   |  117 +
>  .../media/v4l/pixfmt-yuv-luma.rst             |   48 +
>  MAINTAINERS                                   |    7 +
>  .../arm/boot/dts/broadcom/bcm2711-rpi-4-b.dts |    7 +
>  .../boot/dts/broadcom/bcm2711-rpi-cm4-io.dts  |   17 +
>  arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi   |   36 +-
>  arch/arm/boot/dts/broadcom/bcm2711.dtsi       |    8 +
>  arch/arm/boot/dts/broadcom/bcm2835-rpi.dtsi   |   19 +
>  arch/arm/boot/dts/broadcom/bcm283x.dtsi       |   24 +
>  arch/arm64/boot/dts/broadcom/Makefile         |    2 +
>  .../dts/broadcom/bcm2711-rpi-4-b-imx219.dtso  |   65 +
>  drivers/media/i2c/imx219.c                    |  419 ++-
>  drivers/media/platform/Kconfig                |    1 +
>  drivers/media/platform/Makefile               |    1 +
>  drivers/media/platform/broadcom/Kconfig       |   23 +
>  drivers/media/platform/broadcom/Makefile      |    3 +
>  .../platform/broadcom/bcm2835-unicam-regs.h   |  255 ++
>  .../media/platform/broadcom/bcm2835-unicam.c  | 2607 +++++++++++++++++
>  drivers/media/v4l2-core/v4l2-ioctl.c          |    2 +
>  include/uapi/linux/videodev2.h                |    2 +
>  20 files changed, 3588 insertions(+), 75 deletions(-)
>  create mode 100644 Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
>  create mode 100644 arch/arm64/boot/dts/broadcom/bcm2711-rpi-4-b-imx219.dtso
>  create mode 100644 drivers/media/platform/broadcom/Kconfig
>  create mode 100644 drivers/media/platform/broadcom/Makefile
>  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam-regs.h
>  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam.c
> 
> 
> base-commit: 384a3aa6b48b29b4c71130c028db78484a0f0ab4

-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 01/15] media: i2c: imx219: Inline imx219_update_pad_format() in its caller

[Jacopo Mondi]

Hi Laurent

On Fri, Mar 01, 2024 at 11:32:16PM +0200, Laurent Pinchart wrote:
> The imx219_update_pad_format() is short and called from a single place,
> in imx219_set_pad_format(). Inline the code in the caller to keep all
> format adjustements grouped in a single place and improve readability.
>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

> ---
>  drivers/media/i2c/imx219.c | 28 ++++++++++++----------------
>  1 file changed, 12 insertions(+), 16 deletions(-)
>
> diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> index e17ef2e9d9d0..3878da50860e 100644
> --- a/drivers/media/i2c/imx219.c
> +++ b/drivers/media/i2c/imx219.c
> @@ -758,21 +758,6 @@ static int imx219_set_stream(struct v4l2_subdev *sd, int enable)
>  	return ret;
>  }
>
> -static void imx219_update_pad_format(struct imx219 *imx219,
> -				     const struct imx219_mode *mode,
> -				     struct v4l2_mbus_framefmt *fmt, u32 code)
> -{
> -	/* Bayer order varies with flips */
> -	fmt->code = imx219_get_format_code(imx219, code);
> -	fmt->width = mode->width;
> -	fmt->height = mode->height;
> -	fmt->field = V4L2_FIELD_NONE;
> -	fmt->colorspace = V4L2_COLORSPACE_RAW;
> -	fmt->ycbcr_enc = V4L2_YCBCR_ENC_601;
> -	fmt->quantization = V4L2_QUANTIZATION_FULL_RANGE;
> -	fmt->xfer_func = V4L2_XFER_FUNC_NONE;
> -}
> -
>  static int imx219_enum_mbus_code(struct v4l2_subdev *sd,
>  				 struct v4l2_subdev_state *state,
>  				 struct v4l2_subdev_mbus_code_enum *code)
> @@ -819,12 +804,23 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
>  	struct v4l2_rect *crop;
>  	unsigned int bin_h, bin_v;
>
> +	/*
> +	 * Adjust the requested format to match the closest mode. The Bayer
> +	 * order varies with flips.
> +	 */
>  	mode = v4l2_find_nearest_size(supported_modes,
>  				      ARRAY_SIZE(supported_modes),
>  				      width, height,
>  				      fmt->format.width, fmt->format.height);
>
> -	imx219_update_pad_format(imx219, mode, &fmt->format, fmt->format.code);
> +	fmt->format.code = imx219_get_format_code(imx219, fmt->format.code);
> +	fmt->format.width = mode->width;
> +	fmt->format.height = mode->height;
> +	fmt->format.field = V4L2_FIELD_NONE;
> +	fmt->format.colorspace = V4L2_COLORSPACE_RAW;
> +	fmt->format.ycbcr_enc = V4L2_YCBCR_ENC_601;
> +	fmt->format.quantization = V4L2_QUANTIZATION_FULL_RANGE;
> +	fmt->format.xfer_func = V4L2_XFER_FUNC_NONE;
>
>  	format = v4l2_subdev_state_get_format(state, 0);
>  	*format = fmt->format;
> --
> Regards,
>
> Laurent Pinchart
>
>

Re: [PATCH v6 02/15] media: i2c: imx219: Add internal image sink pad

[Jacopo Mondi]

Hi Laurent

On Fri, Mar 01, 2024 at 11:32:17PM +0200, Laurent Pinchart wrote:
> Use the newly added internal pad API to expose the internal
> configuration of the sensor to userspace in a standard manner. This also
> paves the way for adding support for embedded data with an additional
> internal pad.
>
> To maintain compatibility with existing userspace that may operate on
> pad 0 unconditionally, keep the source pad numbered 0 and number the
> internal image pad 1.
>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
>  drivers/media/i2c/imx219.c | 169 +++++++++++++++++++++++++++++--------
>  1 file changed, 133 insertions(+), 36 deletions(-)
>
> diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> index 3878da50860e..817bf192e4d9 100644
> --- a/drivers/media/i2c/imx219.c
> +++ b/drivers/media/i2c/imx219.c
> @@ -140,6 +140,7 @@
>  #define IMX219_DEFAULT_LINK_FREQ_4LANE	363000000
>
>  /* IMX219 native and active pixel array size. */
> +#define IMX219_NATIVE_FORMAT		MEDIA_BUS_FMT_SRGGB10_1X10
>  #define IMX219_NATIVE_WIDTH		3296U
>  #define IMX219_NATIVE_HEIGHT		2480U
>  #define IMX219_PIXEL_ARRAY_LEFT		8U
> @@ -312,9 +313,15 @@ static const struct imx219_mode supported_modes[] = {
>  	},
>  };
>
> +enum imx219_pad_ids {
> +	IMX219_PAD_SOURCE,
> +	IMX219_PAD_IMAGE,

Feels a bit weird for the internal source pad to be named "Image" as
it will (if I understand correctly) host both the image and metadata
streams.

However, I have an hard time proposing a different name, but we should
find something that should be used in all drivers ported to this new
API.

Enough with bikeshedding anyway, this is a tiny detail.

> +	IMX219_NUM_PADS,
> +};
> +
>  struct imx219 {
>  	struct v4l2_subdev sd;
> -	struct media_pad pad;
> +	struct media_pad pads[IMX219_NUM_PADS];
>
>  	struct regmap *regmap;
>  	struct clk *xclk; /* system clock to IMX219 */
> @@ -374,7 +381,7 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
>  	int ret = 0;
>
>  	state = v4l2_subdev_get_locked_active_state(&imx219->sd);
> -	format = v4l2_subdev_state_get_format(state, 0);
> +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
>
>  	if (ctrl->id == V4L2_CID_VBLANK) {
>  		int exposure_max, exposure_def;
> @@ -593,8 +600,8 @@ static int imx219_set_framefmt(struct imx219 *imx219,
>  	u64 bin_h, bin_v;
>  	int ret = 0;
>
> -	format = v4l2_subdev_state_get_format(state, 0);
> -	crop = v4l2_subdev_state_get_crop(state, 0);
> +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
> +	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_IMAGE);
>
>  	switch (format->code) {
>  	case MEDIA_BUS_FMT_SRGGB8_1X8:
> @@ -764,10 +771,25 @@ static int imx219_enum_mbus_code(struct v4l2_subdev *sd,
>  {
>  	struct imx219 *imx219 = to_imx219(sd);
>
> -	if (code->index >= (ARRAY_SIZE(imx219_mbus_formats) / 4))
> -		return -EINVAL;
> +	if (code->pad == IMX219_PAD_IMAGE) {
> +		/* The internal image pad is hardwired to the native format. */
> +		if (code->index)
> +			return -EINVAL;
>
> -	code->code = imx219_get_format_code(imx219, imx219_mbus_formats[code->index * 4]);
> +		code->code = IMX219_NATIVE_FORMAT;

If you return 0 here you can spare the else {} branch. Same in the
function below.

> +	} else {
> +		/*
> +		 * On the source pad, the sensor supports multiple raw formats
> +		 * with different bit depths.
> +		 */
> +		u32 format;
> +
> +		if (code->index >= (ARRAY_SIZE(imx219_mbus_formats) / 4))
> +			return -EINVAL;
> +
> +		format = imx219_mbus_formats[code->index * 4];
> +		code->code = imx219_get_format_code(imx219, format);
> +	}
>
>  	return 0;
>  }
> @@ -777,19 +799,25 @@ static int imx219_enum_frame_size(struct v4l2_subdev *sd,
>  				  struct v4l2_subdev_frame_size_enum *fse)
>  {
>  	struct imx219 *imx219 = to_imx219(sd);
> -	u32 code;
>
> -	if (fse->index >= ARRAY_SIZE(supported_modes))
> -		return -EINVAL;
> +	if (fse->pad == IMX219_PAD_IMAGE) {
> +		if (fse->code != IMX219_NATIVE_FORMAT || fse->index > 0)
> +			return -EINVAL;
>
> -	code = imx219_get_format_code(imx219, fse->code);
> -	if (fse->code != code)
> -		return -EINVAL;
> +		fse->min_width = IMX219_NATIVE_WIDTH;
> +		fse->max_width = IMX219_NATIVE_WIDTH;
> +		fse->min_height = IMX219_NATIVE_HEIGHT;
> +		fse->max_height = IMX219_NATIVE_HEIGHT;
> +	} else {
> +		if (fse->code != imx219_get_format_code(imx219, fse->code) ||
> +		    fse->index >= ARRAY_SIZE(supported_modes))
> +			return -EINVAL;
>
> -	fse->min_width = supported_modes[fse->index].width;
> -	fse->max_width = fse->min_width;
> -	fse->min_height = supported_modes[fse->index].height;
> -	fse->max_height = fse->min_height;
> +		fse->min_width = supported_modes[fse->index].width;
> +		fse->max_width = fse->min_width;
> +		fse->min_height = supported_modes[fse->index].height;
> +		fse->max_height = fse->min_height;
> +	}
>
>  	return 0;
>  }
> @@ -801,9 +829,17 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
>  	struct imx219 *imx219 = to_imx219(sd);
>  	const struct imx219_mode *mode;
>  	struct v4l2_mbus_framefmt *format;
> +	struct v4l2_rect *compose;
>  	struct v4l2_rect *crop;
>  	unsigned int bin_h, bin_v;
>
> +	/*
> +	 * The driver is mode-based, the format can be set on the source pad
> +	 * only.
> +	 */
> +	if (fmt->pad != IMX219_PAD_SOURCE)
> +		return v4l2_subdev_get_fmt(sd, state, fmt);
> +
>  	/*
>  	 * Adjust the requested format to match the closest mode. The Bayer
>  	 * order varies with flips.
> @@ -822,22 +858,51 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
>  	fmt->format.quantization = V4L2_QUANTIZATION_FULL_RANGE;
>  	fmt->format.xfer_func = V4L2_XFER_FUNC_NONE;
>
> -	format = v4l2_subdev_state_get_format(state, 0);
> +	/* Propagate the format through the sensor. */
> +
> +	/* The image pad models the pixel array, and thus has a fixed size. */
> +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_IMAGE);
>  	*format = fmt->format;

Is this assignment needed ?

Isn't 'fmt' meant to be applied at pad #0 ? Also, you overwrite the
code and size below, do the rest of the 'struct v4l2_mbus_framefmt'
fields apply to pad #1 (field, colorspace, quantization etc ? If pad
#1 represents the pixel array, I don't think they do).

Also, can't the pad #1 format be initialized by .init_state() and never
touched again ?


> +	format->code = IMX219_NATIVE_FORMAT;
> +	format->width = IMX219_NATIVE_WIDTH;
> +	format->height = IMX219_NATIVE_HEIGHT;
>
>  	/*
>  	 * Use binning to maximize the crop rectangle size, and centre it in the
>  	 * sensor.
>  	 */
> -	bin_h = min(IMX219_PIXEL_ARRAY_WIDTH / format->width, 2U);
> -	bin_v = min(IMX219_PIXEL_ARRAY_HEIGHT / format->height, 2U);
> +	bin_h = min(IMX219_PIXEL_ARRAY_WIDTH / fmt->format.width, 2U);
> +	bin_v = min(IMX219_PIXEL_ARRAY_HEIGHT / fmt->format.height, 2U);
>
> -	crop = v4l2_subdev_state_get_crop(state, 0);
> -	crop->width = format->width * bin_h;
> -	crop->height = format->height * bin_v;
> +	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_IMAGE);
> +	crop->width = fmt->format.width * bin_h;
> +	crop->height = fmt->format.height * bin_v;
>  	crop->left = (IMX219_NATIVE_WIDTH - crop->width) / 2;
>  	crop->top = (IMX219_NATIVE_HEIGHT - crop->height) / 2;

This is no different than before, but I now wonder if VGA is actually
obtained by cropping down to 1280x960 and then by a 2x2 binning, or is
there a 4x4 binning mode. I presume this is correct though, as this has been
validated by many people.

>
> +	/*
> +	 * The compose rectangle models binning, its size is the sensor output
> +	 * size.
> +	 */
> +	compose = v4l2_subdev_state_get_compose(state, IMX219_PAD_IMAGE);
> +	compose->left = 0;
> +	compose->top = 0;
> +	compose->width = fmt->format.width;
> +	compose->height = fmt->format.height;
> +
> +	/*
> +	 * No mode use digital crop, the source pad crop rectangle size and
> +	 * format are thus identical to the image pad compose rectangle.
> +	 */
> +	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_SOURCE);
> +	crop->left = 0;
> +	crop->top = 0;
> +	crop->width = fmt->format.width;
> +	crop->height = fmt->format.height;
> +
> +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
> +	*format = fmt->format;
> +
>  	if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
>  		int exposure_max;
>  		int exposure_def;
> @@ -874,13 +939,13 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
>  				struct v4l2_subdev_state *state,
>  				struct v4l2_subdev_selection *sel)
>  {
> -	switch (sel->target) {
> -	case V4L2_SEL_TGT_CROP: {
> -		sel->r = *v4l2_subdev_state_get_crop(state, 0);
> -		return 0;
> -	}
> +	struct v4l2_rect *compose;
>
> +	switch (sel->target) {
>  	case V4L2_SEL_TGT_NATIVE_SIZE:
> +		if (sel->pad != IMX219_PAD_IMAGE)
> +			return -EINVAL;
> +
>  		sel->r.top = 0;
>  		sel->r.left = 0;
>  		sel->r.width = IMX219_NATIVE_WIDTH;
> @@ -890,11 +955,35 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
>
>  	case V4L2_SEL_TGT_CROP_DEFAULT:
>  	case V4L2_SEL_TGT_CROP_BOUNDS:
> -		sel->r.top = IMX219_PIXEL_ARRAY_TOP;
> -		sel->r.left = IMX219_PIXEL_ARRAY_LEFT;
> -		sel->r.width = IMX219_PIXEL_ARRAY_WIDTH;
> -		sel->r.height = IMX219_PIXEL_ARRAY_HEIGHT;
> +		switch (sel->pad) {
> +		case IMX219_PAD_IMAGE:
> +			sel->r.top = IMX219_PIXEL_ARRAY_TOP;
> +			sel->r.left = IMX219_PIXEL_ARRAY_LEFT;
> +			sel->r.width = IMX219_PIXEL_ARRAY_WIDTH;
> +			sel->r.height = IMX219_PIXEL_ARRAY_HEIGHT;
> +			return 0;
>
> +		case IMX219_PAD_SOURCE:
> +			compose = v4l2_subdev_state_get_compose(state,
> +								IMX219_PAD_IMAGE);
> +			sel->r.top = 0;
> +			sel->r.left = 0;
> +			sel->r.width = compose->width;
> +			sel->r.height = compose->height;
> +			return 0;
> +		}
> +
> +		break;
> +
> +	case V4L2_SEL_TGT_CROP:
> +		sel->r = *v4l2_subdev_state_get_crop(state, sel->pad);
> +		return 0;
> +
> +	case V4L2_SEL_TGT_COMPOSE:
> +		if (sel->pad != IMX219_PAD_IMAGE)
> +			return -EINVAL;
> +
> +		sel->r = *v4l2_subdev_state_get_compose(state, sel->pad);
>  		return 0;

Do we need to support TGT_COMPOSE_BOUNDS for PAD_IMAGE ? I would have
suggested V4L2_SEL_TGT_COMPOSE_DEFAULT too, but according to the spec
it does not apply to subdevices.

>  	}
>
> @@ -906,7 +995,7 @@ static int imx219_init_state(struct v4l2_subdev *sd,
>  {
>  	struct v4l2_subdev_format fmt = {
>  		.which = V4L2_SUBDEV_FORMAT_TRY,
> -		.pad = 0,
> +		.pad = IMX219_PAD_SOURCE,
>  		.format = {
>  			.code = MEDIA_BUS_FMT_SRGGB10_1X10,
>  			.width = supported_modes[0].width,

Why not intialize pad#1 format here an never touch it again ? Should
also the pad#1 crop and compose rectangles and pad#0 crop be
initialized here ?

> @@ -1174,10 +1263,18 @@ static int imx219_probe(struct i2c_client *client)
>  			    V4L2_SUBDEV_FL_HAS_EVENTS;
>  	imx219->sd.entity.function = MEDIA_ENT_F_CAM_SENSOR;
>
> -	/* Initialize source pad */
> -	imx219->pad.flags = MEDIA_PAD_FL_SOURCE;
> +	/*
> +	 * Initialize the pads. To preserve backward compatibility with
> +	 * userspace that used the sensor before the introduction of the
> +	 * internal image pad, the external source pad is numbered 0 and the
> +	 * internal image pad numbered 1.
> +	 */
> +	imx219->pads[IMX219_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
> +	imx219->pads[IMX219_PAD_IMAGE].flags = MEDIA_PAD_FL_SINK
> +					     | MEDIA_PAD_FL_INTERNAL;
>
> -	ret = media_entity_pads_init(&imx219->sd.entity, 1, &imx219->pad);
> +	ret = media_entity_pads_init(&imx219->sd.entity,
> +				     ARRAY_SIZE(imx219->pads), imx219->pads);
>  	if (ret) {
>  		dev_err(dev, "failed to init entity pads: %d\n", ret);
>  		goto error_handler_free;
> --
> Regards,
>
> Laurent Pinchart
>
>

Re: [PATCH v6 03/15] media: i2c: imx219: Report internal routes to userspace

[Jacopo Mondi]

Hi Laurent

On Fri, Mar 01, 2024 at 11:32:18PM +0200, Laurent Pinchart wrote:
> Usage of internal pads creates a route internal to the subdev, and the
> V4L2 camera sensor API requires such routes to be reported to userspace.
> Create the route in the .init_cfg() operation.

It's now "init_state()"

>
> Internal routing support requires stream support, so set the
> V4L2_SUBDEV_FL_HAS_STREAMS flag and switch formats and selection
> rectangles access from pads to streams. As the route is immutable,
> there's no need to implement the .set_routing() operation, and we can
> hardcode the sink and source stream IDs to 0.
>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
>  drivers/media/i2c/imx219.c | 28 ++++++++++++++++++++++++++--
>  1 file changed, 26 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> index 817bf192e4d9..52afb821f667 100644
> --- a/drivers/media/i2c/imx219.c
> +++ b/drivers/media/i2c/imx219.c
> @@ -381,7 +381,10 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
>  	int ret = 0;
>
>  	state = v4l2_subdev_get_locked_active_state(&imx219->sd);
> -	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
> +
> +	format = v4l2_subdev_state_get_opposite_stream_format(state,
> +							      IMX219_PAD_IMAGE,
> +							      0);

Is this change necessary ?

Apart from this
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Thanks
  j

>
>  	if (ctrl->id == V4L2_CID_VBLANK) {
>  		int exposure_max, exposure_def;
> @@ -993,15 +996,35 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
>  static int imx219_init_state(struct v4l2_subdev *sd,
>  			     struct v4l2_subdev_state *state)
>  {
> +	struct v4l2_subdev_route routes[1] = {
> +		{
> +			.sink_pad = IMX219_PAD_IMAGE,
> +			.sink_stream = 0,
> +			.source_pad = IMX219_PAD_SOURCE,
> +			.source_stream = 0,
> +			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
> +		},
> +	};
> +	struct v4l2_subdev_krouting routing = {
> +		.len_routes = ARRAY_SIZE(routes),
> +		.num_routes = ARRAY_SIZE(routes),
> +		.routes = routes,
> +	};
>  	struct v4l2_subdev_format fmt = {
>  		.which = V4L2_SUBDEV_FORMAT_TRY,
>  		.pad = IMX219_PAD_SOURCE,
> +		.stream = 0,
>  		.format = {
>  			.code = MEDIA_BUS_FMT_SRGGB10_1X10,
>  			.width = supported_modes[0].width,
>  			.height = supported_modes[0].height,
>  		},
>  	};
> +	int ret;
> +
> +	ret = v4l2_subdev_set_routing(sd, state, &routing);
> +	if (ret)
> +		return ret;
>
>  	imx219_set_pad_format(sd, state, &fmt);
>
> @@ -1260,7 +1283,8 @@ static int imx219_probe(struct i2c_client *client)
>
>  	/* Initialize subdev */
>  	imx219->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE |
> -			    V4L2_SUBDEV_FL_HAS_EVENTS;
> +			    V4L2_SUBDEV_FL_HAS_EVENTS |
> +			    V4L2_SUBDEV_FL_STREAMS;
>  	imx219->sd.entity.function = MEDIA_ENT_F_CAM_SENSOR;
>
>  	/*
> --
> Regards,
>
> Laurent Pinchart
>
>

Re: [PATCH v6 04/15] media: i2c: imx219: Report streams using frame descriptors

[Jacopo Mondi]

Hi Laurent

On Fri, Mar 01, 2024 at 11:32:19PM +0200, Laurent Pinchart wrote:
> Implement the .get_frame_desc() subdev operation to report information
> about streams to the connected CSI-2 receiver. This is required to let
> the CSI-2 receiver driver know about virtual channels and data types for
> each stream.
>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
>  drivers/media/i2c/imx219.c | 67 ++++++++++++++++++++++++++++----------
>  1 file changed, 50 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> index 52afb821f667..6e0232b6772b 100644
> --- a/drivers/media/i2c/imx219.c
> +++ b/drivers/media/i2c/imx219.c
> @@ -23,6 +23,7 @@
>  #include <linux/pm_runtime.h>
>  #include <linux/regulator/consumer.h>
>
> +#include <media/mipi-csi2.h>
>  #include <media/v4l2-cci.h>
>  #include <media/v4l2-ctrls.h>
>  #include <media/v4l2-device.h>
> @@ -594,6 +595,24 @@ static void imx219_free_controls(struct imx219 *imx219)
>   * Subdev operations
>   */
>
> +static int imx219_format_bpp(u32 code)

This can be unsigned as the functions does not return errrors

Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

> +{
> +	switch (code) {
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +		return 8;
> +
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	default:
> +		return 10;
> +	}
> +}
> +
>  static int imx219_set_framefmt(struct imx219 *imx219,
>  			       struct v4l2_subdev_state *state)
>  {
> @@ -605,23 +624,7 @@ static int imx219_set_framefmt(struct imx219 *imx219,
>
>  	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
>  	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_IMAGE);
> -
> -	switch (format->code) {
> -	case MEDIA_BUS_FMT_SRGGB8_1X8:
> -	case MEDIA_BUS_FMT_SGRBG8_1X8:
> -	case MEDIA_BUS_FMT_SGBRG8_1X8:
> -	case MEDIA_BUS_FMT_SBGGR8_1X8:
> -		bpp = 8;
> -		break;
> -
> -	case MEDIA_BUS_FMT_SRGGB10_1X10:
> -	case MEDIA_BUS_FMT_SGRBG10_1X10:
> -	case MEDIA_BUS_FMT_SGBRG10_1X10:
> -	case MEDIA_BUS_FMT_SBGGR10_1X10:
> -	default:
> -		bpp = 10;
> -		break;
> -	}
> +	bpp = imx219_format_bpp(format->code);
>
>  	cci_write(imx219->regmap, IMX219_REG_X_ADD_STA_A,
>  		  crop->left - IMX219_PIXEL_ARRAY_LEFT, &ret);
> @@ -1031,6 +1034,35 @@ static int imx219_init_state(struct v4l2_subdev *sd,
>  	return 0;
>  }
>
> +static int imx219_get_frame_desc(struct v4l2_subdev *sd, unsigned int pad,
> +				 struct v4l2_mbus_frame_desc *fd)
> +{
> +	const struct v4l2_mbus_framefmt *fmt;
> +	struct v4l2_subdev_state *state;
> +	u32 code;
> +
> +	if (pad != IMX219_PAD_SOURCE)
> +		return -EINVAL;
> +
> +	state = v4l2_subdev_lock_and_get_active_state(sd);
> +	fmt = v4l2_subdev_state_get_stream_format(state, IMX219_PAD_SOURCE, 0);
> +	code = fmt->code;
> +	v4l2_subdev_unlock_state(state);
> +
> +	fd->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
> +	fd->num_entries = 1;
> +
> +	memset(fd->entry, 0, sizeof(fd->entry));
> +
> +	fd->entry[0].pixelcode = code;
> +	fd->entry[0].stream = 0;
> +	fd->entry[0].bus.csi2.vc = 0;
> +	fd->entry[0].bus.csi2.dt = imx219_format_bpp(code) == 8
> +				 ? MIPI_CSI2_DT_RAW8 : MIPI_CSI2_DT_RAW10;
> +
> +	return 0;
> +}
> +
>  static const struct v4l2_subdev_core_ops imx219_core_ops = {
>  	.subscribe_event = v4l2_ctrl_subdev_subscribe_event,
>  	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> @@ -1046,6 +1078,7 @@ static const struct v4l2_subdev_pad_ops imx219_pad_ops = {
>  	.set_fmt = imx219_set_pad_format,
>  	.get_selection = imx219_get_selection,
>  	.enum_frame_size = imx219_enum_frame_size,
> +	.get_frame_desc = imx219_get_frame_desc,
>  };
>
>  static const struct v4l2_subdev_ops imx219_subdev_ops = {
> --
> Regards,
>
> Laurent Pinchart
>
>

Re: [PATCH v6 02/15] media: i2c: imx219: Add internal image sink pad

[Jacopo Mondi]

Hi Laurent

On Mon, Mar 04, 2024 at 10:13:47AM +0100, Jacopo Mondi wrote:
> Hi Laurent
>
> On Fri, Mar 01, 2024 at 11:32:17PM +0200, Laurent Pinchart wrote:
> > Use the newly added internal pad API to expose the internal
> > configuration of the sensor to userspace in a standard manner. This also
> > paves the way for adding support for embedded data with an additional
> > internal pad.
> >
> > To maintain compatibility with existing userspace that may operate on
> > pad 0 unconditionally, keep the source pad numbered 0 and number the
> > internal image pad 1.
> >
> > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > ---
> >  drivers/media/i2c/imx219.c | 169 +++++++++++++++++++++++++++++--------
> >  1 file changed, 133 insertions(+), 36 deletions(-)
> >
> > diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> > index 3878da50860e..817bf192e4d9 100644
> > --- a/drivers/media/i2c/imx219.c
> > +++ b/drivers/media/i2c/imx219.c
> > @@ -140,6 +140,7 @@
> >  #define IMX219_DEFAULT_LINK_FREQ_4LANE	363000000
> >
> >  /* IMX219 native and active pixel array size. */
> > +#define IMX219_NATIVE_FORMAT		MEDIA_BUS_FMT_SRGGB10_1X10
> >  #define IMX219_NATIVE_WIDTH		3296U
> >  #define IMX219_NATIVE_HEIGHT		2480U
> >  #define IMX219_PIXEL_ARRAY_LEFT		8U
> > @@ -312,9 +313,15 @@ static const struct imx219_mode supported_modes[] = {
> >  	},
> >  };
> >
> > +enum imx219_pad_ids {
> > +	IMX219_PAD_SOURCE,
> > +	IMX219_PAD_IMAGE,
>
> Feels a bit weird for the internal source pad to be named "Image" as
> it will (if I understand correctly) host both the image and metadata
> streams.
>
> However, I have an hard time proposing a different name, but we should
> find something that should be used in all drivers ported to this new
> API.
>
> Enough with bikeshedding anyway, this is a tiny detail.
>

Now that I've read 5/15 I now understand why this is called "IMAGE" :)

> > +	IMX219_NUM_PADS,
> > +};
> > +
> >  struct imx219 {
> >  	struct v4l2_subdev sd;
> > -	struct media_pad pad;
> > +	struct media_pad pads[IMX219_NUM_PADS];
> >
> >  	struct regmap *regmap;
> >  	struct clk *xclk; /* system clock to IMX219 */
> > @@ -374,7 +381,7 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
> >  	int ret = 0;
> >
> >  	state = v4l2_subdev_get_locked_active_state(&imx219->sd);
> > -	format = v4l2_subdev_state_get_format(state, 0);
> > +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
> >
> >  	if (ctrl->id == V4L2_CID_VBLANK) {
> >  		int exposure_max, exposure_def;
> > @@ -593,8 +600,8 @@ static int imx219_set_framefmt(struct imx219 *imx219,
> >  	u64 bin_h, bin_v;
> >  	int ret = 0;
> >
> > -	format = v4l2_subdev_state_get_format(state, 0);
> > -	crop = v4l2_subdev_state_get_crop(state, 0);
> > +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
> > +	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_IMAGE);
> >
> >  	switch (format->code) {
> >  	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > @@ -764,10 +771,25 @@ static int imx219_enum_mbus_code(struct v4l2_subdev *sd,
> >  {
> >  	struct imx219 *imx219 = to_imx219(sd);
> >
> > -	if (code->index >= (ARRAY_SIZE(imx219_mbus_formats) / 4))
> > -		return -EINVAL;
> > +	if (code->pad == IMX219_PAD_IMAGE) {
> > +		/* The internal image pad is hardwired to the native format. */
> > +		if (code->index)
> > +			return -EINVAL;
> >
> > -	code->code = imx219_get_format_code(imx219, imx219_mbus_formats[code->index * 4]);
> > +		code->code = IMX219_NATIVE_FORMAT;
>
> If you return 0 here you can spare the else {} branch. Same in the
> function below.
>

You're reworking this in 5/15, so no need to change this and cause
rebase conflicts.

> > +	} else {
> > +		/*
> > +		 * On the source pad, the sensor supports multiple raw formats
> > +		 * with different bit depths.
> > +		 */
> > +		u32 format;
> > +
> > +		if (code->index >= (ARRAY_SIZE(imx219_mbus_formats) / 4))
> > +			return -EINVAL;
> > +
> > +		format = imx219_mbus_formats[code->index * 4];
> > +		code->code = imx219_get_format_code(imx219, format);
> > +	}
> >
> >  	return 0;
> >  }
> > @@ -777,19 +799,25 @@ static int imx219_enum_frame_size(struct v4l2_subdev *sd,
> >  				  struct v4l2_subdev_frame_size_enum *fse)
> >  {
> >  	struct imx219 *imx219 = to_imx219(sd);
> > -	u32 code;
> >
> > -	if (fse->index >= ARRAY_SIZE(supported_modes))
> > -		return -EINVAL;
> > +	if (fse->pad == IMX219_PAD_IMAGE) {
> > +		if (fse->code != IMX219_NATIVE_FORMAT || fse->index > 0)
> > +			return -EINVAL;
> >
> > -	code = imx219_get_format_code(imx219, fse->code);
> > -	if (fse->code != code)
> > -		return -EINVAL;
> > +		fse->min_width = IMX219_NATIVE_WIDTH;
> > +		fse->max_width = IMX219_NATIVE_WIDTH;
> > +		fse->min_height = IMX219_NATIVE_HEIGHT;
> > +		fse->max_height = IMX219_NATIVE_HEIGHT;
> > +	} else {
> > +		if (fse->code != imx219_get_format_code(imx219, fse->code) ||
> > +		    fse->index >= ARRAY_SIZE(supported_modes))
> > +			return -EINVAL;
> >
> > -	fse->min_width = supported_modes[fse->index].width;
> > -	fse->max_width = fse->min_width;
> > -	fse->min_height = supported_modes[fse->index].height;
> > -	fse->max_height = fse->min_height;
> > +		fse->min_width = supported_modes[fse->index].width;
> > +		fse->max_width = fse->min_width;
> > +		fse->min_height = supported_modes[fse->index].height;
> > +		fse->max_height = fse->min_height;
> > +	}
> >
> >  	return 0;
> >  }
> > @@ -801,9 +829,17 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
> >  	struct imx219 *imx219 = to_imx219(sd);
> >  	const struct imx219_mode *mode;
> >  	struct v4l2_mbus_framefmt *format;
> > +	struct v4l2_rect *compose;
> >  	struct v4l2_rect *crop;
> >  	unsigned int bin_h, bin_v;
> >
> > +	/*
> > +	 * The driver is mode-based, the format can be set on the source pad
> > +	 * only.
> > +	 */
> > +	if (fmt->pad != IMX219_PAD_SOURCE)
> > +		return v4l2_subdev_get_fmt(sd, state, fmt);
> > +
> >  	/*
> >  	 * Adjust the requested format to match the closest mode. The Bayer
> >  	 * order varies with flips.
> > @@ -822,22 +858,51 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
> >  	fmt->format.quantization = V4L2_QUANTIZATION_FULL_RANGE;
> >  	fmt->format.xfer_func = V4L2_XFER_FUNC_NONE;
> >
> > -	format = v4l2_subdev_state_get_format(state, 0);
> > +	/* Propagate the format through the sensor. */
> > +
> > +	/* The image pad models the pixel array, and thus has a fixed size. */
> > +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_IMAGE);
> >  	*format = fmt->format;
>
> Is this assignment needed ?
>
> Isn't 'fmt' meant to be applied at pad #0 ? Also, you overwrite the
> code and size below, do the rest of the 'struct v4l2_mbus_framefmt'
> fields apply to pad #1 (field, colorspace, quantization etc ? If pad
> #1 represents the pixel array, I don't think they do).
>
> Also, can't the pad #1 format be initialized by .init_state() and never
> touched again ?
>
>
> > +	format->code = IMX219_NATIVE_FORMAT;
> > +	format->width = IMX219_NATIVE_WIDTH;
> > +	format->height = IMX219_NATIVE_HEIGHT;
> >
> >  	/*
> >  	 * Use binning to maximize the crop rectangle size, and centre it in the
> >  	 * sensor.
> >  	 */
> > -	bin_h = min(IMX219_PIXEL_ARRAY_WIDTH / format->width, 2U);
> > -	bin_v = min(IMX219_PIXEL_ARRAY_HEIGHT / format->height, 2U);
> > +	bin_h = min(IMX219_PIXEL_ARRAY_WIDTH / fmt->format.width, 2U);
> > +	bin_v = min(IMX219_PIXEL_ARRAY_HEIGHT / fmt->format.height, 2U);
> >
> > -	crop = v4l2_subdev_state_get_crop(state, 0);
> > -	crop->width = format->width * bin_h;
> > -	crop->height = format->height * bin_v;
> > +	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_IMAGE);
> > +	crop->width = fmt->format.width * bin_h;
> > +	crop->height = fmt->format.height * bin_v;
> >  	crop->left = (IMX219_NATIVE_WIDTH - crop->width) / 2;
> >  	crop->top = (IMX219_NATIVE_HEIGHT - crop->height) / 2;
>
> This is no different than before, but I now wonder if VGA is actually
> obtained by cropping down to 1280x960 and then by a 2x2 binning, or is
> there a 4x4 binning mode. I presume this is correct though, as this has been
> validated by many people.
>
> >
> > +	/*
> > +	 * The compose rectangle models binning, its size is the sensor output
> > +	 * size.
> > +	 */
> > +	compose = v4l2_subdev_state_get_compose(state, IMX219_PAD_IMAGE);
> > +	compose->left = 0;
> > +	compose->top = 0;
> > +	compose->width = fmt->format.width;
> > +	compose->height = fmt->format.height;
> > +
> > +	/*
> > +	 * No mode use digital crop, the source pad crop rectangle size and
> > +	 * format are thus identical to the image pad compose rectangle.
> > +	 */
> > +	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_SOURCE);
> > +	crop->left = 0;
> > +	crop->top = 0;
> > +	crop->width = fmt->format.width;
> > +	crop->height = fmt->format.height;
> > +
> > +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
> > +	*format = fmt->format;
> > +
> >  	if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
> >  		int exposure_max;
> >  		int exposure_def;
> > @@ -874,13 +939,13 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
> >  				struct v4l2_subdev_state *state,
> >  				struct v4l2_subdev_selection *sel)
> >  {
> > -	switch (sel->target) {
> > -	case V4L2_SEL_TGT_CROP: {
> > -		sel->r = *v4l2_subdev_state_get_crop(state, 0);
> > -		return 0;
> > -	}
> > +	struct v4l2_rect *compose;
> >
> > +	switch (sel->target) {
> >  	case V4L2_SEL_TGT_NATIVE_SIZE:
> > +		if (sel->pad != IMX219_PAD_IMAGE)
> > +			return -EINVAL;
> > +
> >  		sel->r.top = 0;
> >  		sel->r.left = 0;
> >  		sel->r.width = IMX219_NATIVE_WIDTH;
> > @@ -890,11 +955,35 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
> >
> >  	case V4L2_SEL_TGT_CROP_DEFAULT:
> >  	case V4L2_SEL_TGT_CROP_BOUNDS:
> > -		sel->r.top = IMX219_PIXEL_ARRAY_TOP;
> > -		sel->r.left = IMX219_PIXEL_ARRAY_LEFT;
> > -		sel->r.width = IMX219_PIXEL_ARRAY_WIDTH;
> > -		sel->r.height = IMX219_PIXEL_ARRAY_HEIGHT;
> > +		switch (sel->pad) {
> > +		case IMX219_PAD_IMAGE:
> > +			sel->r.top = IMX219_PIXEL_ARRAY_TOP;
> > +			sel->r.left = IMX219_PIXEL_ARRAY_LEFT;
> > +			sel->r.width = IMX219_PIXEL_ARRAY_WIDTH;
> > +			sel->r.height = IMX219_PIXEL_ARRAY_HEIGHT;
> > +			return 0;
> >
> > +		case IMX219_PAD_SOURCE:
> > +			compose = v4l2_subdev_state_get_compose(state,
> > +								IMX219_PAD_IMAGE);
> > +			sel->r.top = 0;
> > +			sel->r.left = 0;
> > +			sel->r.width = compose->width;
> > +			sel->r.height = compose->height;
> > +			return 0;
> > +		}
> > +
> > +		break;
> > +
> > +	case V4L2_SEL_TGT_CROP:
> > +		sel->r = *v4l2_subdev_state_get_crop(state, sel->pad);
> > +		return 0;
> > +
> > +	case V4L2_SEL_TGT_COMPOSE:
> > +		if (sel->pad != IMX219_PAD_IMAGE)
> > +			return -EINVAL;
> > +
> > +		sel->r = *v4l2_subdev_state_get_compose(state, sel->pad);
> >  		return 0;
>
> Do we need to support TGT_COMPOSE_BOUNDS for PAD_IMAGE ? I would have
> suggested V4L2_SEL_TGT_COMPOSE_DEFAULT too, but according to the spec
> it does not apply to subdevices.
>
> >  	}
> >
> > @@ -906,7 +995,7 @@ static int imx219_init_state(struct v4l2_subdev *sd,
> >  {
> >  	struct v4l2_subdev_format fmt = {
> >  		.which = V4L2_SUBDEV_FORMAT_TRY,
> > -		.pad = 0,
> > +		.pad = IMX219_PAD_SOURCE,
> >  		.format = {
> >  			.code = MEDIA_BUS_FMT_SRGGB10_1X10,
> >  			.width = supported_modes[0].width,
>
> Why not intialize pad#1 format here an never touch it again ? Should
> also the pad#1 crop and compose rectangles and pad#0 crop be
> initialized here ?
>
> > @@ -1174,10 +1263,18 @@ static int imx219_probe(struct i2c_client *client)
> >  			    V4L2_SUBDEV_FL_HAS_EVENTS;
> >  	imx219->sd.entity.function = MEDIA_ENT_F_CAM_SENSOR;
> >
> > -	/* Initialize source pad */
> > -	imx219->pad.flags = MEDIA_PAD_FL_SOURCE;
> > +	/*
> > +	 * Initialize the pads. To preserve backward compatibility with
> > +	 * userspace that used the sensor before the introduction of the
> > +	 * internal image pad, the external source pad is numbered 0 and the
> > +	 * internal image pad numbered 1.
> > +	 */
> > +	imx219->pads[IMX219_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
> > +	imx219->pads[IMX219_PAD_IMAGE].flags = MEDIA_PAD_FL_SINK
> > +					     | MEDIA_PAD_FL_INTERNAL;
> >
> > -	ret = media_entity_pads_init(&imx219->sd.entity, 1, &imx219->pad);
> > +	ret = media_entity_pads_init(&imx219->sd.entity,
> > +				     ARRAY_SIZE(imx219->pads), imx219->pads);
> >  	if (ret) {
> >  		dev_err(dev, "failed to init entity pads: %d\n", ret);
> >  		goto error_handler_free;
> > --
> > Regards,
> >
> > Laurent Pinchart
> >
> >
>

Re: [PATCH v6 05/15] media: i2c: imx219: Add embedded data support

[Jacopo Mondi]

Hi Laurent

On Fri, Mar 01, 2024 at 11:32:20PM +0200, Laurent Pinchart wrote:
> The IMX219 generates embedded data unconditionally. Report it as an
> additional stream, with a new internal embedded data pad, and update
> subdev operations accordingly.
>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
>  drivers/media/i2c/imx219.c | 173 ++++++++++++++++++++++++++++++++-----
>  1 file changed, 151 insertions(+), 22 deletions(-)
>
> diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> index 6e0232b6772b..cfbf4c304a9a 100644
> --- a/drivers/media/i2c/imx219.c
> +++ b/drivers/media/i2c/imx219.c
> @@ -149,6 +149,9 @@
>  #define IMX219_PIXEL_ARRAY_WIDTH	3280U
>  #define IMX219_PIXEL_ARRAY_HEIGHT	2464U
>
> +/* Embedded metadata stream height */
> +#define IMX219_EMBEDDED_DATA_HEIGHT	2U
> +
>  /* Mode : resolution and related config&values */
>  struct imx219_mode {
>  	/* Frame width */
> @@ -317,9 +320,15 @@ static const struct imx219_mode supported_modes[] = {
>  enum imx219_pad_ids {
>  	IMX219_PAD_SOURCE,
>  	IMX219_PAD_IMAGE,
> +	IMX219_PAD_EDATA,
>  	IMX219_NUM_PADS,
>  };
>
> +enum imx219_stream_ids {
> +	IMX219_STREAM_IMAGE,
> +	IMX219_STREAM_EDATA,
> +};
> +
>  struct imx219 {
>  	struct v4l2_subdev sd;
>  	struct media_pad pads[IMX219_NUM_PADS];
> @@ -613,6 +622,25 @@ static int imx219_format_bpp(u32 code)
>  	}
>  }
>
> +/* Return the embedded data format corresponding to an image format. */
> +static u32 imx219_format_edata(u32 code)
> +{
> +	switch (code) {
> +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> +		return MEDIA_BUS_FMT_META_8;
> +
> +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> +	default:
> +		return MEDIA_BUS_FMT_META_10;
> +	}
> +}
> +
>  static int imx219_set_framefmt(struct imx219 *imx219,
>  			       struct v4l2_subdev_state *state)
>  {
> @@ -622,7 +650,8 @@ static int imx219_set_framefmt(struct imx219 *imx219,
>  	u64 bin_h, bin_v;
>  	int ret = 0;
>
> -	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
> +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
> +					      IMX219_STREAM_IMAGE);
>  	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_IMAGE);
>  	bpp = imx219_format_bpp(format->code);
>
> @@ -777,17 +806,33 @@ static int imx219_enum_mbus_code(struct v4l2_subdev *sd,
>  {
>  	struct imx219 *imx219 = to_imx219(sd);
>
> -	if (code->pad == IMX219_PAD_IMAGE) {
> +	switch (code->pad) {
> +	case IMX219_PAD_IMAGE:
>  		/* The internal image pad is hardwired to the native format. */
> -		if (code->index)
> +		if (code->index > 0)
>  			return -EINVAL;
>
>  		code->code = IMX219_NATIVE_FORMAT;
> -	} else {
> -		/*
> -		 * On the source pad, the sensor supports multiple raw formats
> -		 * with different bit depths.
> -		 */
> +		return 0;
> +
> +	case IMX219_PAD_EDATA:
> +		if (code->index > 0)
> +			return -EINVAL;
> +
> +		code->code = MEDIA_BUS_FMT_CCS_EMBEDDED;
> +		return 0;
> +
> +	case IMX219_PAD_SOURCE:
> +	default:
> +		break;

Do you need these ?

> +	}
> +
> +	/*
> +	 * On the source pad, the sensor supports multiple image raw formats
> +	 * with different bit depths. The embedded data format bit depth
> +	 * follows the image stream.
> +	 */
> +	if (code->stream == IMX219_STREAM_IMAGE) {
>  		u32 format;
>
>  		if (code->index >= (ARRAY_SIZE(imx219_mbus_formats) / 4))
> @@ -795,6 +840,15 @@ static int imx219_enum_mbus_code(struct v4l2_subdev *sd,
>
>  		format = imx219_mbus_formats[code->index * 4];
>  		code->code = imx219_get_format_code(imx219, format);
> +	} else {
> +		struct v4l2_mbus_framefmt *fmt;
> +
> +		if (code->index > 0)
> +			return -EINVAL;
> +
> +		fmt = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
> +						   IMX219_STREAM_EDATA);
> +		code->code = fmt->code;
>  	}
>
>  	return 0;
> @@ -806,7 +860,8 @@ static int imx219_enum_frame_size(struct v4l2_subdev *sd,
>  {
>  	struct imx219 *imx219 = to_imx219(sd);
>
> -	if (fse->pad == IMX219_PAD_IMAGE) {
> +	switch (fse->pad) {
> +	case IMX219_PAD_IMAGE:
>  		if (fse->code != IMX219_NATIVE_FORMAT || fse->index > 0)
>  			return -EINVAL;
>
> @@ -814,7 +869,24 @@ static int imx219_enum_frame_size(struct v4l2_subdev *sd,
>  		fse->max_width = IMX219_NATIVE_WIDTH;
>  		fse->min_height = IMX219_NATIVE_HEIGHT;
>  		fse->max_height = IMX219_NATIVE_HEIGHT;
> -	} else {
> +		return 0;
> +
> +	case IMX219_PAD_EDATA:
> +		if (fse->code != MEDIA_BUS_FMT_CCS_EMBEDDED || fse->index > 0)
> +			return -EINVAL;
> +
> +		fse->min_width = IMX219_NATIVE_WIDTH;
> +		fse->max_width = IMX219_NATIVE_WIDTH;
> +		fse->min_height = IMX219_EMBEDDED_DATA_HEIGHT;
> +		fse->max_height = IMX219_EMBEDDED_DATA_HEIGHT;
> +		return 0;
> +
> +	case IMX219_PAD_SOURCE:
> +	default:
> +		break;
> +	}
> +
> +	if (fse->stream == IMX219_STREAM_IMAGE) {
>  		if (fse->code != imx219_get_format_code(imx219, fse->code) ||
>  		    fse->index >= ARRAY_SIZE(supported_modes))
>  			return -EINVAL;
> @@ -823,6 +895,21 @@ static int imx219_enum_frame_size(struct v4l2_subdev *sd,
>  		fse->max_width = fse->min_width;
>  		fse->min_height = supported_modes[fse->index].height;
>  		fse->max_height = fse->min_height;
> +	} else {
> +		struct v4l2_mbus_framefmt *fmt;
> +
> +		fmt = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
> +						   IMX219_STREAM_EDATA);
> +		if (fse->code != fmt->code)
> +			return -EINVAL;
> +
> +		if (fse->index)
> +			return -EINVAL;
> +
> +		fse->min_width = fmt->width;
> +		fse->max_width = fmt->width;
> +		fse->min_height = IMX219_EMBEDDED_DATA_HEIGHT;
> +		fse->max_height = IMX219_EMBEDDED_DATA_HEIGHT;
>  	}
>
>  	return 0;
> @@ -834,6 +921,7 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
>  {
>  	struct imx219 *imx219 = to_imx219(sd);
>  	const struct imx219_mode *mode;
> +	struct v4l2_mbus_framefmt *ed_format;
>  	struct v4l2_mbus_framefmt *format;
>  	struct v4l2_rect *compose;
>  	struct v4l2_rect *crop;
> @@ -841,9 +929,9 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
>
>  	/*
>  	 * The driver is mode-based, the format can be set on the source pad
> -	 * only.
> +	 * only, and only for the image streeam.
>  	 */
> -	if (fmt->pad != IMX219_PAD_SOURCE)
> +	if (fmt->pad != IMX219_PAD_SOURCE || fmt->stream != IMX219_STREAM_IMAGE)
>  		return v4l2_subdev_get_fmt(sd, state, fmt);
>
>  	/*
> @@ -900,15 +988,31 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
>  	 * No mode use digital crop, the source pad crop rectangle size and
>  	 * format are thus identical to the image pad compose rectangle.
>  	 */
> -	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_SOURCE);
> +	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_SOURCE,
> +					  IMX219_STREAM_IMAGE);
>  	crop->left = 0;
>  	crop->top = 0;
>  	crop->width = fmt->format.width;
>  	crop->height = fmt->format.height;
>
> -	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
> +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
> +					      IMX219_STREAM_IMAGE);
>  	*format = fmt->format;
>
> +	/*
> +	 * Finally, update the formats on the sink and source sides of the
> +	 * embedded data stream.
> +	 */
> +	ed_format = v4l2_subdev_state_get_format(state, IMX219_PAD_EDATA);
> +	ed_format->code = imx219_format_edata(format->code);

One could consider making imx219_format_edata() retrieve the
[PAD_SOURCE/STREAM_IMAGE] format internally instead of passing it from
outside, to avoid passing in a from format

> +	ed_format->width = format->width;
> +	ed_format->height = IMX219_EMBEDDED_DATA_HEIGHT;
> +	ed_format->field = V4L2_FIELD_NONE;
> +
> +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
> +					      IMX219_STREAM_EDATA);
> +	*format = *ed_format;

Or
        *v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
                                      IMX219_STREAM_EDATA) = *ed_format;

To avoid re-assigning 'format' which now points to the [0/0] format
and might be reused later to update controls ?

> +
>  	if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
>  		int exposure_max;
>  		int exposure_def;
> @@ -947,6 +1051,13 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
>  {
>  	struct v4l2_rect *compose;
>
> +	/*
> +	 * The embedded data stream doesn't support selection rectangles,
> +	 * neither on the embedded data pad nor on the source pad.
> +	 */
> +	if (sel->pad == IMX219_PAD_EDATA || sel->stream != 0)
> +		return -EINVAL;
> +
>  	switch (sel->target) {
>  	case V4L2_SEL_TGT_NATIVE_SIZE:
>  		if (sel->pad != IMX219_PAD_IMAGE)
> @@ -999,12 +1110,18 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
>  static int imx219_init_state(struct v4l2_subdev *sd,
>  			     struct v4l2_subdev_state *state)
>  {
> -	struct v4l2_subdev_route routes[1] = {
> +	struct v4l2_subdev_route routes[2] = {
>  		{
>  			.sink_pad = IMX219_PAD_IMAGE,
>  			.sink_stream = 0,
>  			.source_pad = IMX219_PAD_SOURCE,
> -			.source_stream = 0,
> +			.source_stream = IMX219_STREAM_IMAGE,
> +			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
> +		}, {
> +			.sink_pad = IMX219_PAD_EDATA,
> +			.sink_stream = 0,
> +			.source_pad = IMX219_PAD_SOURCE,
> +			.source_stream = IMX219_STREAM_EDATA,
>  			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
>  		},
>  	};
> @@ -1016,7 +1133,7 @@ static int imx219_init_state(struct v4l2_subdev *sd,
>  	struct v4l2_subdev_format fmt = {
>  		.which = V4L2_SUBDEV_FORMAT_TRY,
>  		.pad = IMX219_PAD_SOURCE,
> -		.stream = 0,
> +		.stream = IMX219_STREAM_IMAGE,
>  		.format = {
>  			.code = MEDIA_BUS_FMT_SRGGB10_1X10,
>  			.width = supported_modes[0].width,
> @@ -1029,6 +1146,10 @@ static int imx219_init_state(struct v4l2_subdev *sd,
>  	if (ret)
>  		return ret;
>
> +	/*
> +	 * Set the image stream format on the source pad. This will be
> +	 * propagated to all formats and selection rectangles internally.
> +	 */
>  	imx219_set_pad_format(sd, state, &fmt);
>
>  	return 0;
> @@ -1045,21 +1166,27 @@ static int imx219_get_frame_desc(struct v4l2_subdev *sd, unsigned int pad,
>  		return -EINVAL;
>
>  	state = v4l2_subdev_lock_and_get_active_state(sd);
> -	fmt = v4l2_subdev_state_get_stream_format(state, IMX219_PAD_SOURCE, 0);
> +	fmt = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
> +					   IMX219_STREAM_IMAGE);
>  	code = fmt->code;
>  	v4l2_subdev_unlock_state(state);
>
>  	fd->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
> -	fd->num_entries = 1;
> +	fd->num_entries = 2;
>
>  	memset(fd->entry, 0, sizeof(fd->entry));
>
>  	fd->entry[0].pixelcode = code;
> -	fd->entry[0].stream = 0;
> +	fd->entry[0].stream = IMX219_STREAM_IMAGE;
>  	fd->entry[0].bus.csi2.vc = 0;
>  	fd->entry[0].bus.csi2.dt = imx219_format_bpp(code) == 8
>  				 ? MIPI_CSI2_DT_RAW8 : MIPI_CSI2_DT_RAW10;
>
> +	fd->entry[1].pixelcode = code;

Is the mbus_code on this second entry the same as the first one ?

> +	fd->entry[1].stream = IMX219_STREAM_EDATA;
> +	fd->entry[1].bus.csi2.vc = 0;
> +	fd->entry[1].bus.csi2.dt = MIPI_CSI2_DT_EMBEDDED_8B;
> +
>  	return 0;
>  }
>
> @@ -1323,12 +1450,14 @@ static int imx219_probe(struct i2c_client *client)
>  	/*
>  	 * Initialize the pads. To preserve backward compatibility with
>  	 * userspace that used the sensor before the introduction of the
> -	 * internal image pad, the external source pad is numbered 0 and the
> -	 * internal image pad numbered 1.
> +	 * internal pads, the external source pad is numbered 0 and the internal
> +	 * image and embedded data pads numbered 1 and 2 respectively.
>  	 */
>  	imx219->pads[IMX219_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
>  	imx219->pads[IMX219_PAD_IMAGE].flags = MEDIA_PAD_FL_SINK
>  					     | MEDIA_PAD_FL_INTERNAL;
> +	imx219->pads[IMX219_PAD_EDATA].flags = MEDIA_PAD_FL_SINK
> +					     | MEDIA_PAD_FL_INTERNAL;
>
>  	ret = media_entity_pads_init(&imx219->sd.entity,
>  				     ARRAY_SIZE(imx219->pads), imx219->pads);
> --
> Regards,
>
> Laurent Pinchart
>
>

Re: [PATCH v6 06/15] media: v4l: Add V4L2-PIX-FMT-Y12P format

[Jacopo Mondi]

Hi Laurent

Ah
https://patchwork.linuxtv.org/project/linux-media/patch/20240223163012.300763-2-jacopo.mondi@ideasonboard.com/

same for the Y14 one

I guess we have upported the same patch twice ?

Whichever series gets in first wins

On Fri, Mar 01, 2024 at 11:32:21PM +0200, Laurent Pinchart wrote:
> From: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
>
> This is a packed grey-scale image format with a depth of 12 bits per
> pixel. Two consecutive pixels are packed into 3 bytes. The first 2 bytes
> contain the 8 high order bits of the pixels, and the 3rd byte contains
> the 4 least significants bits of each pixel, in the same order.
>
> Add the entry in userspace API, and document it.
>
> Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
>  .../userspace-api/media/v4l/pixfmt-yuv-luma.rst       | 11 +++++++++++
>  drivers/media/v4l2-core/v4l2-ioctl.c                  |  1 +
>  include/uapi/linux/videodev2.h                        |  1 +
>  3 files changed, 13 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst b/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst
> index cf8e4dfbfbd4..a650ceda7cdb 100644
> --- a/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst
> +++ b/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst
> @@ -114,6 +114,17 @@ are often referred to as greyscale formats.
>        - ...
>        - ...
>
> +    * .. _V4L2-PIX-FMT-Y12P:
> +
> +      - ``V4L2_PIX_FMT_Y12P``
> +      - 'Y12P'
> +
> +      - Y'\ :sub:`0`\ [11:4]
> +      - Y'\ :sub:`1`\ [11:4]
> +      - Y'\ :sub:`1`\ [3:0] Y'\ :sub:`0`\ [3:0]
> +      - ...
> +      - ...
> +
>      * .. _V4L2-PIX-FMT-Y14:
>
>        - ``V4L2_PIX_FMT_Y14``
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index f3697ff65817..69ac4482d844 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1313,6 +1313,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>  	case V4L2_PIX_FMT_Y10BPACK:	descr = "10-bit Greyscale (Packed)"; break;
>  	case V4L2_PIX_FMT_Y10P:		descr = "10-bit Greyscale (MIPI Packed)"; break;
>  	case V4L2_PIX_FMT_IPU3_Y10:	descr = "10-bit greyscale (IPU3 Packed)"; break;
> +	case V4L2_PIX_FMT_Y12P:		descr = "12-bit Greyscale (MIPI Packed)"; break;
>  	case V4L2_PIX_FMT_Y8I:		descr = "Interleaved 8-bit Greyscale"; break;
>  	case V4L2_PIX_FMT_Y12I:		descr = "Interleaved 12-bit Greyscale"; break;
>  	case V4L2_PIX_FMT_Z16:		descr = "16-bit Depth"; break;
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 117a081c9db9..5122ac0c6765 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -599,6 +599,7 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_Y10BPACK    v4l2_fourcc('Y', '1', '0', 'B') /* 10  Greyscale bit-packed */
>  #define V4L2_PIX_FMT_Y10P    v4l2_fourcc('Y', '1', '0', 'P') /* 10  Greyscale, MIPI RAW10 packed */
>  #define V4L2_PIX_FMT_IPU3_Y10		v4l2_fourcc('i', 'p', '3', 'y') /* IPU3 packed 10-bit greyscale */
> +#define V4L2_PIX_FMT_Y12P    v4l2_fourcc('Y', '1', '2', 'P') /* 12  Greyscale, MIPI RAW12 packed */
>
>  /* Palette formats */
>  #define V4L2_PIX_FMT_PAL8    v4l2_fourcc('P', 'A', 'L', '8') /*  8  8-bit palette */
> --
> Regards,
>
> Laurent Pinchart
>
>

Re: [PATCH v6 03/15] media: i2c: imx219: Report internal routes to userspace

[Laurent Pinchart]

Hi Jacopo,

On Mon, Mar 04, 2024 at 10:30:34AM +0100, Jacopo Mondi wrote:
> On Fri, Mar 01, 2024 at 11:32:18PM +0200, Laurent Pinchart wrote:
> > Usage of internal pads creates a route internal to the subdev, and the
> > V4L2 camera sensor API requires such routes to be reported to userspace.
> > Create the route in the .init_cfg() operation.
> 
> It's now "init_state()"

Oops. Will fix.

> > Internal routing support requires stream support, so set the
> > V4L2_SUBDEV_FL_HAS_STREAMS flag and switch formats and selection
> > rectangles access from pads to streams. As the route is immutable,
> > there's no need to implement the .set_routing() operation, and we can
> > hardcode the sink and source stream IDs to 0.
> >
> > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > ---
> >  drivers/media/i2c/imx219.c | 28 ++++++++++++++++++++++++++--
> >  1 file changed, 26 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> > index 817bf192e4d9..52afb821f667 100644
> > --- a/drivers/media/i2c/imx219.c
> > +++ b/drivers/media/i2c/imx219.c
> > @@ -381,7 +381,10 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
> >  	int ret = 0;
> >
> >  	state = v4l2_subdev_get_locked_active_state(&imx219->sd);
> > -	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
> > +
> > +	format = v4l2_subdev_state_get_opposite_stream_format(state,
> > +							      IMX219_PAD_IMAGE,
> > +							      0);
> 
> Is this change necessary ?

Now that we have routes, we need to get the format on the right stream.
There's only one at the moment, so there's no need to change it here.
I'll drop this change and update patch 05/15 accordingly.

> Apart from this
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> 
> >
> >  	if (ctrl->id == V4L2_CID_VBLANK) {
> >  		int exposure_max, exposure_def;
> > @@ -993,15 +996,35 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
> >  static int imx219_init_state(struct v4l2_subdev *sd,
> >  			     struct v4l2_subdev_state *state)
> >  {
> > +	struct v4l2_subdev_route routes[1] = {
> > +		{
> > +			.sink_pad = IMX219_PAD_IMAGE,
> > +			.sink_stream = 0,
> > +			.source_pad = IMX219_PAD_SOURCE,
> > +			.source_stream = 0,
> > +			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
> > +		},
> > +	};
> > +	struct v4l2_subdev_krouting routing = {
> > +		.len_routes = ARRAY_SIZE(routes),
> > +		.num_routes = ARRAY_SIZE(routes),
> > +		.routes = routes,
> > +	};
> >  	struct v4l2_subdev_format fmt = {
> >  		.which = V4L2_SUBDEV_FORMAT_TRY,
> >  		.pad = IMX219_PAD_SOURCE,
> > +		.stream = 0,
> >  		.format = {
> >  			.code = MEDIA_BUS_FMT_SRGGB10_1X10,
> >  			.width = supported_modes[0].width,
> >  			.height = supported_modes[0].height,
> >  		},
> >  	};
> > +	int ret;
> > +
> > +	ret = v4l2_subdev_set_routing(sd, state, &routing);
> > +	if (ret)
> > +		return ret;
> >
> >  	imx219_set_pad_format(sd, state, &fmt);
> >
> > @@ -1260,7 +1283,8 @@ static int imx219_probe(struct i2c_client *client)
> >
> >  	/* Initialize subdev */
> >  	imx219->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE |
> > -			    V4L2_SUBDEV_FL_HAS_EVENTS;
> > +			    V4L2_SUBDEV_FL_HAS_EVENTS |
> > +			    V4L2_SUBDEV_FL_STREAMS;
> >  	imx219->sd.entity.function = MEDIA_ENT_F_CAM_SENSOR;
> >
> >  	/*

-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 02/15] media: i2c: imx219: Add internal image sink pad

[Laurent Pinchart]

Hi Jacopo,

On Mon, Mar 04, 2024 at 10:38:12AM +0100, Jacopo Mondi wrote:
> On Mon, Mar 04, 2024 at 10:13:47AM +0100, Jacopo Mondi wrote:
> > On Fri, Mar 01, 2024 at 11:32:17PM +0200, Laurent Pinchart wrote:
> > > Use the newly added internal pad API to expose the internal
> > > configuration of the sensor to userspace in a standard manner. This also
> > > paves the way for adding support for embedded data with an additional
> > > internal pad.
> > >
> > > To maintain compatibility with existing userspace that may operate on
> > > pad 0 unconditionally, keep the source pad numbered 0 and number the
> > > internal image pad 1.
> > >
> > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > ---
> > >  drivers/media/i2c/imx219.c | 169 +++++++++++++++++++++++++++++--------
> > >  1 file changed, 133 insertions(+), 36 deletions(-)
> > >
> > > diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> > > index 3878da50860e..817bf192e4d9 100644
> > > --- a/drivers/media/i2c/imx219.c
> > > +++ b/drivers/media/i2c/imx219.c
> > > @@ -140,6 +140,7 @@
> > >  #define IMX219_DEFAULT_LINK_FREQ_4LANE	363000000
> > >
> > >  /* IMX219 native and active pixel array size. */
> > > +#define IMX219_NATIVE_FORMAT		MEDIA_BUS_FMT_SRGGB10_1X10
> > >  #define IMX219_NATIVE_WIDTH		3296U
> > >  #define IMX219_NATIVE_HEIGHT		2480U
> > >  #define IMX219_PIXEL_ARRAY_LEFT		8U
> > > @@ -312,9 +313,15 @@ static const struct imx219_mode supported_modes[] = {
> > >  	},
> > >  };
> > >
> > > +enum imx219_pad_ids {
> > > +	IMX219_PAD_SOURCE,
> > > +	IMX219_PAD_IMAGE,
> >
> > Feels a bit weird for the internal source pad to be named "Image" as
> > it will (if I understand correctly) host both the image and metadata
> > streams.
> >
> > However, I have an hard time proposing a different name, but we should
> > find something that should be used in all drivers ported to this new
> > API.
> >
> > Enough with bikeshedding anyway, this is a tiny detail.
> 
> Now that I've read 5/15 I now understand why this is called "IMAGE" :)

:-)

> > > +	IMX219_NUM_PADS,
> > > +};
> > > +
> > >  struct imx219 {
> > >  	struct v4l2_subdev sd;
> > > -	struct media_pad pad;
> > > +	struct media_pad pads[IMX219_NUM_PADS];
> > >
> > >  	struct regmap *regmap;
> > >  	struct clk *xclk; /* system clock to IMX219 */
> > > @@ -374,7 +381,7 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
> > >  	int ret = 0;
> > >
> > >  	state = v4l2_subdev_get_locked_active_state(&imx219->sd);
> > > -	format = v4l2_subdev_state_get_format(state, 0);
> > > +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
> > >
> > >  	if (ctrl->id == V4L2_CID_VBLANK) {
> > >  		int exposure_max, exposure_def;
> > > @@ -593,8 +600,8 @@ static int imx219_set_framefmt(struct imx219 *imx219,
> > >  	u64 bin_h, bin_v;
> > >  	int ret = 0;
> > >
> > > -	format = v4l2_subdev_state_get_format(state, 0);
> > > -	crop = v4l2_subdev_state_get_crop(state, 0);
> > > +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
> > > +	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_IMAGE);
> > >
> > >  	switch (format->code) {
> > >  	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > > @@ -764,10 +771,25 @@ static int imx219_enum_mbus_code(struct v4l2_subdev *sd,
> > >  {
> > >  	struct imx219 *imx219 = to_imx219(sd);
> > >
> > > -	if (code->index >= (ARRAY_SIZE(imx219_mbus_formats) / 4))
> > > -		return -EINVAL;
> > > +	if (code->pad == IMX219_PAD_IMAGE) {
> > > +		/* The internal image pad is hardwired to the native format. */
> > > +		if (code->index)
> > > +			return -EINVAL;
> > >
> > > -	code->code = imx219_get_format_code(imx219, imx219_mbus_formats[code->index * 4]);
> > > +		code->code = IMX219_NATIVE_FORMAT;
> >
> > If you return 0 here you can spare the else {} branch. Same in the
> > function below.
> >
> 
> You're reworking this in 5/15, so no need to change this and cause
> rebase conflicts.

It's true that I'm reworking it further in 05/15, but I think the change
here still makes sense, as it updates the driver to support the internal
pad properly. Beside, if I were to drop this change, then I would get a
rebase conflict :-)

> > > +	} else {
> > > +		/*
> > > +		 * On the source pad, the sensor supports multiple raw formats
> > > +		 * with different bit depths.
> > > +		 */
> > > +		u32 format;
> > > +
> > > +		if (code->index >= (ARRAY_SIZE(imx219_mbus_formats) / 4))
> > > +			return -EINVAL;
> > > +
> > > +		format = imx219_mbus_formats[code->index * 4];
> > > +		code->code = imx219_get_format_code(imx219, format);
> > > +	}
> > >
> > >  	return 0;
> > >  }
> > > @@ -777,19 +799,25 @@ static int imx219_enum_frame_size(struct v4l2_subdev *sd,
> > >  				  struct v4l2_subdev_frame_size_enum *fse)
> > >  {
> > >  	struct imx219 *imx219 = to_imx219(sd);
> > > -	u32 code;
> > >
> > > -	if (fse->index >= ARRAY_SIZE(supported_modes))
> > > -		return -EINVAL;
> > > +	if (fse->pad == IMX219_PAD_IMAGE) {
> > > +		if (fse->code != IMX219_NATIVE_FORMAT || fse->index > 0)
> > > +			return -EINVAL;
> > >
> > > -	code = imx219_get_format_code(imx219, fse->code);
> > > -	if (fse->code != code)
> > > -		return -EINVAL;
> > > +		fse->min_width = IMX219_NATIVE_WIDTH;
> > > +		fse->max_width = IMX219_NATIVE_WIDTH;
> > > +		fse->min_height = IMX219_NATIVE_HEIGHT;
> > > +		fse->max_height = IMX219_NATIVE_HEIGHT;
> > > +	} else {
> > > +		if (fse->code != imx219_get_format_code(imx219, fse->code) ||
> > > +		    fse->index >= ARRAY_SIZE(supported_modes))
> > > +			return -EINVAL;
> > >
> > > -	fse->min_width = supported_modes[fse->index].width;
> > > -	fse->max_width = fse->min_width;
> > > -	fse->min_height = supported_modes[fse->index].height;
> > > -	fse->max_height = fse->min_height;
> > > +		fse->min_width = supported_modes[fse->index].width;
> > > +		fse->max_width = fse->min_width;
> > > +		fse->min_height = supported_modes[fse->index].height;
> > > +		fse->max_height = fse->min_height;
> > > +	}
> > >
> > >  	return 0;
> > >  }
> > > @@ -801,9 +829,17 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
> > >  	struct imx219 *imx219 = to_imx219(sd);
> > >  	const struct imx219_mode *mode;
> > >  	struct v4l2_mbus_framefmt *format;
> > > +	struct v4l2_rect *compose;
> > >  	struct v4l2_rect *crop;
> > >  	unsigned int bin_h, bin_v;
> > >
> > > +	/*
> > > +	 * The driver is mode-based, the format can be set on the source pad
> > > +	 * only.
> > > +	 */
> > > +	if (fmt->pad != IMX219_PAD_SOURCE)
> > > +		return v4l2_subdev_get_fmt(sd, state, fmt);
> > > +
> > >  	/*
> > >  	 * Adjust the requested format to match the closest mode. The Bayer
> > >  	 * order varies with flips.
> > > @@ -822,22 +858,51 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
> > >  	fmt->format.quantization = V4L2_QUANTIZATION_FULL_RANGE;
> > >  	fmt->format.xfer_func = V4L2_XFER_FUNC_NONE;
> > >
> > > -	format = v4l2_subdev_state_get_format(state, 0);
> > > +	/* Propagate the format through the sensor. */
> > > +
> > > +	/* The image pad models the pixel array, and thus has a fixed size. */
> > > +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_IMAGE);
> > >  	*format = fmt->format;
> >
> > Is this assignment needed ?
> >
> > Isn't 'fmt' meant to be applied at pad #0 ? Also, you overwrite the
> > code and size below, do the rest of the 'struct v4l2_mbus_framefmt'
> > fields apply to pad #1 (field, colorspace, quantization etc ? If pad
> > #1 represents the pixel array, I don't think they do).

It's a shortcut to avoid setting field, colorspace, ycbcr_enc,
quantization and xfer_func manually, as there's set a few lines above in
fmt. I can set the fields manually if you prefer.

> > Also, can't the pad #1 format be initialized by .init_state() and never
> > touched again ?

They could, but then .init_state() couldn't be implemented by simply
calling imx219_set_pad_format(). I think it is clearer to store all the
format setting logic in one place, especially when adding support for
the embedded data stream.

> > > +	format->code = IMX219_NATIVE_FORMAT;
> > > +	format->width = IMX219_NATIVE_WIDTH;
> > > +	format->height = IMX219_NATIVE_HEIGHT;
> > >
> > >  	/*
> > >  	 * Use binning to maximize the crop rectangle size, and centre it in the
> > >  	 * sensor.
> > >  	 */
> > > -	bin_h = min(IMX219_PIXEL_ARRAY_WIDTH / format->width, 2U);
> > > -	bin_v = min(IMX219_PIXEL_ARRAY_HEIGHT / format->height, 2U);
> > > +	bin_h = min(IMX219_PIXEL_ARRAY_WIDTH / fmt->format.width, 2U);
> > > +	bin_v = min(IMX219_PIXEL_ARRAY_HEIGHT / fmt->format.height, 2U);
> > >
> > > -	crop = v4l2_subdev_state_get_crop(state, 0);
> > > -	crop->width = format->width * bin_h;
> > > -	crop->height = format->height * bin_v;
> > > +	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_IMAGE);
> > > +	crop->width = fmt->format.width * bin_h;
> > > +	crop->height = fmt->format.height * bin_v;
> > >  	crop->left = (IMX219_NATIVE_WIDTH - crop->width) / 2;
> > >  	crop->top = (IMX219_NATIVE_HEIGHT - crop->height) / 2;
> >
> > This is no different than before, but I now wonder if VGA is actually
> > obtained by cropping down to 1280x960 and then by a 2x2 binning, or is
> > there a 4x4 binning mode. I presume this is correct though, as this has been
> > validated by many people.

It would be worth experimenting with the /4 binning mode.

> > > +	/*
> > > +	 * The compose rectangle models binning, its size is the sensor output
> > > +	 * size.
> > > +	 */
> > > +	compose = v4l2_subdev_state_get_compose(state, IMX219_PAD_IMAGE);
> > > +	compose->left = 0;
> > > +	compose->top = 0;
> > > +	compose->width = fmt->format.width;
> > > +	compose->height = fmt->format.height;
> > > +
> > > +	/*
> > > +	 * No mode use digital crop, the source pad crop rectangle size and
> > > +	 * format are thus identical to the image pad compose rectangle.
> > > +	 */
> > > +	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_SOURCE);
> > > +	crop->left = 0;
> > > +	crop->top = 0;
> > > +	crop->width = fmt->format.width;
> > > +	crop->height = fmt->format.height;
> > > +
> > > +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
> > > +	*format = fmt->format;
> > > +
> > >  	if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
> > >  		int exposure_max;
> > >  		int exposure_def;
> > > @@ -874,13 +939,13 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
> > >  				struct v4l2_subdev_state *state,
> > >  				struct v4l2_subdev_selection *sel)
> > >  {
> > > -	switch (sel->target) {
> > > -	case V4L2_SEL_TGT_CROP: {
> > > -		sel->r = *v4l2_subdev_state_get_crop(state, 0);
> > > -		return 0;
> > > -	}
> > > +	struct v4l2_rect *compose;
> > >
> > > +	switch (sel->target) {
> > >  	case V4L2_SEL_TGT_NATIVE_SIZE:
> > > +		if (sel->pad != IMX219_PAD_IMAGE)
> > > +			return -EINVAL;
> > > +
> > >  		sel->r.top = 0;
> > >  		sel->r.left = 0;
> > >  		sel->r.width = IMX219_NATIVE_WIDTH;
> > > @@ -890,11 +955,35 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
> > >
> > >  	case V4L2_SEL_TGT_CROP_DEFAULT:
> > >  	case V4L2_SEL_TGT_CROP_BOUNDS:
> > > -		sel->r.top = IMX219_PIXEL_ARRAY_TOP;
> > > -		sel->r.left = IMX219_PIXEL_ARRAY_LEFT;
> > > -		sel->r.width = IMX219_PIXEL_ARRAY_WIDTH;
> > > -		sel->r.height = IMX219_PIXEL_ARRAY_HEIGHT;
> > > +		switch (sel->pad) {
> > > +		case IMX219_PAD_IMAGE:
> > > +			sel->r.top = IMX219_PIXEL_ARRAY_TOP;
> > > +			sel->r.left = IMX219_PIXEL_ARRAY_LEFT;
> > > +			sel->r.width = IMX219_PIXEL_ARRAY_WIDTH;
> > > +			sel->r.height = IMX219_PIXEL_ARRAY_HEIGHT;
> > > +			return 0;
> > >
> > > +		case IMX219_PAD_SOURCE:
> > > +			compose = v4l2_subdev_state_get_compose(state,
> > > +								IMX219_PAD_IMAGE);
> > > +			sel->r.top = 0;
> > > +			sel->r.left = 0;
> > > +			sel->r.width = compose->width;
> > > +			sel->r.height = compose->height;
> > > +			return 0;
> > > +		}
> > > +
> > > +		break;
> > > +
> > > +	case V4L2_SEL_TGT_CROP:
> > > +		sel->r = *v4l2_subdev_state_get_crop(state, sel->pad);
> > > +		return 0;
> > > +
> > > +	case V4L2_SEL_TGT_COMPOSE:
> > > +		if (sel->pad != IMX219_PAD_IMAGE)
> > > +			return -EINVAL;
> > > +
> > > +		sel->r = *v4l2_subdev_state_get_compose(state, sel->pad);
> > >  		return 0;
> >
> > Do we need to support TGT_COMPOSE_BOUNDS for PAD_IMAGE ? I would have
> > suggested V4L2_SEL_TGT_COMPOSE_DEFAULT too, but according to the spec
> > it does not apply to subdevices.

Good question. V4L2_SEL_TGT_COMPOSE_BOUNDS is a bit under-documented for
V4L2 subdevs. I wonder if it wouldn't be simpler for drivers to not
implement it when the compose bounds are identical to the crop rectangle
size, as that could be considered a default ?

> > >  	}
> > >
> > > @@ -906,7 +995,7 @@ static int imx219_init_state(struct v4l2_subdev *sd,
> > >  {
> > >  	struct v4l2_subdev_format fmt = {
> > >  		.which = V4L2_SUBDEV_FORMAT_TRY,
> > > -		.pad = 0,
> > > +		.pad = IMX219_PAD_SOURCE,
> > >  		.format = {
> > >  			.code = MEDIA_BUS_FMT_SRGGB10_1X10,
> > >  			.width = supported_modes[0].width,
> >
> > Why not intialize pad#1 format here an never touch it again ? Should
> > also the pad#1 crop and compose rectangles and pad#0 crop be
> > initialized here ?

As written above, I think it's cleaner to have all the format-related
constraints in one place, especially with support for embedded data. It
may be a style preference, I'm not sure.

> > > @@ -1174,10 +1263,18 @@ static int imx219_probe(struct i2c_client *client)
> > >  			    V4L2_SUBDEV_FL_HAS_EVENTS;
> > >  	imx219->sd.entity.function = MEDIA_ENT_F_CAM_SENSOR;
> > >
> > > -	/* Initialize source pad */
> > > -	imx219->pad.flags = MEDIA_PAD_FL_SOURCE;
> > > +	/*
> > > +	 * Initialize the pads. To preserve backward compatibility with
> > > +	 * userspace that used the sensor before the introduction of the
> > > +	 * internal image pad, the external source pad is numbered 0 and the
> > > +	 * internal image pad numbered 1.
> > > +	 */
> > > +	imx219->pads[IMX219_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
> > > +	imx219->pads[IMX219_PAD_IMAGE].flags = MEDIA_PAD_FL_SINK
> > > +					     | MEDIA_PAD_FL_INTERNAL;
> > >
> > > -	ret = media_entity_pads_init(&imx219->sd.entity, 1, &imx219->pad);
> > > +	ret = media_entity_pads_init(&imx219->sd.entity,
> > > +				     ARRAY_SIZE(imx219->pads), imx219->pads);
> > >  	if (ret) {
> > >  		dev_err(dev, "failed to init entity pads: %d\n", ret);
> > >  		goto error_handler_free;

-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 04/15] media: i2c: imx219: Report streams using frame descriptors

[Laurent Pinchart]

On Mon, Mar 04, 2024 at 10:33:37AM +0100, Jacopo Mondi wrote:
> Hi Laurent
> 
> On Fri, Mar 01, 2024 at 11:32:19PM +0200, Laurent Pinchart wrote:
> > Implement the .get_frame_desc() subdev operation to report information
> > about streams to the connected CSI-2 receiver. This is required to let
> > the CSI-2 receiver driver know about virtual channels and data types for
> > each stream.
> >
> > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > ---
> >  drivers/media/i2c/imx219.c | 67 ++++++++++++++++++++++++++++----------
> >  1 file changed, 50 insertions(+), 17 deletions(-)
> >
> > diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> > index 52afb821f667..6e0232b6772b 100644
> > --- a/drivers/media/i2c/imx219.c
> > +++ b/drivers/media/i2c/imx219.c
> > @@ -23,6 +23,7 @@
> >  #include <linux/pm_runtime.h>
> >  #include <linux/regulator/consumer.h>
> >
> > +#include <media/mipi-csi2.h>
> >  #include <media/v4l2-cci.h>
> >  #include <media/v4l2-ctrls.h>
> >  #include <media/v4l2-device.h>
> > @@ -594,6 +595,24 @@ static void imx219_free_controls(struct imx219 *imx219)
> >   * Subdev operations
> >   */
> >
> > +static int imx219_format_bpp(u32 code)
> 
> This can be unsigned as the functions does not return errrors

I'll change that.

> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> 
> > +{
> > +	switch (code) {
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +		return 8;
> > +
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	default:
> > +		return 10;
> > +	}
> > +}
> > +
> >  static int imx219_set_framefmt(struct imx219 *imx219,
> >  			       struct v4l2_subdev_state *state)
> >  {
> > @@ -605,23 +624,7 @@ static int imx219_set_framefmt(struct imx219 *imx219,
> >
> >  	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
> >  	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_IMAGE);
> > -
> > -	switch (format->code) {
> > -	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > -	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > -	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > -	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > -		bpp = 8;
> > -		break;
> > -
> > -	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > -	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > -	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > -	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > -	default:
> > -		bpp = 10;
> > -		break;
> > -	}
> > +	bpp = imx219_format_bpp(format->code);
> >
> >  	cci_write(imx219->regmap, IMX219_REG_X_ADD_STA_A,
> >  		  crop->left - IMX219_PIXEL_ARRAY_LEFT, &ret);
> > @@ -1031,6 +1034,35 @@ static int imx219_init_state(struct v4l2_subdev *sd,
> >  	return 0;
> >  }
> >
> > +static int imx219_get_frame_desc(struct v4l2_subdev *sd, unsigned int pad,
> > +				 struct v4l2_mbus_frame_desc *fd)
> > +{
> > +	const struct v4l2_mbus_framefmt *fmt;
> > +	struct v4l2_subdev_state *state;
> > +	u32 code;
> > +
> > +	if (pad != IMX219_PAD_SOURCE)
> > +		return -EINVAL;
> > +
> > +	state = v4l2_subdev_lock_and_get_active_state(sd);
> > +	fmt = v4l2_subdev_state_get_stream_format(state, IMX219_PAD_SOURCE, 0);
> > +	code = fmt->code;
> > +	v4l2_subdev_unlock_state(state);
> > +
> > +	fd->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
> > +	fd->num_entries = 1;
> > +
> > +	memset(fd->entry, 0, sizeof(fd->entry));
> > +
> > +	fd->entry[0].pixelcode = code;
> > +	fd->entry[0].stream = 0;
> > +	fd->entry[0].bus.csi2.vc = 0;
> > +	fd->entry[0].bus.csi2.dt = imx219_format_bpp(code) == 8
> > +				 ? MIPI_CSI2_DT_RAW8 : MIPI_CSI2_DT_RAW10;
> > +
> > +	return 0;
> > +}
> > +
> >  static const struct v4l2_subdev_core_ops imx219_core_ops = {
> >  	.subscribe_event = v4l2_ctrl_subdev_subscribe_event,
> >  	.unsubscribe_event = v4l2_event_subdev_unsubscribe,
> > @@ -1046,6 +1078,7 @@ static const struct v4l2_subdev_pad_ops imx219_pad_ops = {
> >  	.set_fmt = imx219_set_pad_format,
> >  	.get_selection = imx219_get_selection,
> >  	.enum_frame_size = imx219_enum_frame_size,
> > +	.get_frame_desc = imx219_get_frame_desc,
> >  };
> >
> >  static const struct v4l2_subdev_ops imx219_subdev_ops = {

-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 02/15] media: i2c: imx219: Add internal image sink pad

[Jacopo Mondi]

Hi Laurent

On Mon, Mar 04, 2024 at 05:14:26PM +0200, Laurent Pinchart wrote:
> Hi Jacopo,
>
> On Mon, Mar 04, 2024 at 10:38:12AM +0100, Jacopo Mondi wrote:
> > On Mon, Mar 04, 2024 at 10:13:47AM +0100, Jacopo Mondi wrote:
> > > On Fri, Mar 01, 2024 at 11:32:17PM +0200, Laurent Pinchart wrote:
> > > > Use the newly added internal pad API to expose the internal
> > > > configuration of the sensor to userspace in a standard manner. This also
> > > > paves the way for adding support for embedded data with an additional
> > > > internal pad.
> > > >
> > > > To maintain compatibility with existing userspace that may operate on
> > > > pad 0 unconditionally, keep the source pad numbered 0 and number the
> > > > internal image pad 1.
> > > >
> > > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > > ---
> > > >  drivers/media/i2c/imx219.c | 169 +++++++++++++++++++++++++++++--------
> > > >  1 file changed, 133 insertions(+), 36 deletions(-)
> > > >
> > > > diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> > > > index 3878da50860e..817bf192e4d9 100644
> > > > --- a/drivers/media/i2c/imx219.c
> > > > +++ b/drivers/media/i2c/imx219.c
> > > > @@ -140,6 +140,7 @@
> > > >  #define IMX219_DEFAULT_LINK_FREQ_4LANE	363000000
> > > >
> > > >  /* IMX219 native and active pixel array size. */
> > > > +#define IMX219_NATIVE_FORMAT		MEDIA_BUS_FMT_SRGGB10_1X10
> > > >  #define IMX219_NATIVE_WIDTH		3296U
> > > >  #define IMX219_NATIVE_HEIGHT		2480U
> > > >  #define IMX219_PIXEL_ARRAY_LEFT		8U
> > > > @@ -312,9 +313,15 @@ static const struct imx219_mode supported_modes[] = {
> > > >  	},
> > > >  };
> > > >
> > > > +enum imx219_pad_ids {
> > > > +	IMX219_PAD_SOURCE,
> > > > +	IMX219_PAD_IMAGE,
> > >
> > > Feels a bit weird for the internal source pad to be named "Image" as
> > > it will (if I understand correctly) host both the image and metadata
> > > streams.
> > >
> > > However, I have an hard time proposing a different name, but we should
> > > find something that should be used in all drivers ported to this new
> > > API.
> > >
> > > Enough with bikeshedding anyway, this is a tiny detail.
> >
> > Now that I've read 5/15 I now understand why this is called "IMAGE" :)
>
> :-)
>
> > > > +	IMX219_NUM_PADS,
> > > > +};
> > > > +
> > > >  struct imx219 {
> > > >  	struct v4l2_subdev sd;
> > > > -	struct media_pad pad;
> > > > +	struct media_pad pads[IMX219_NUM_PADS];
> > > >
> > > >  	struct regmap *regmap;
> > > >  	struct clk *xclk; /* system clock to IMX219 */
> > > > @@ -374,7 +381,7 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
> > > >  	int ret = 0;
> > > >
> > > >  	state = v4l2_subdev_get_locked_active_state(&imx219->sd);
> > > > -	format = v4l2_subdev_state_get_format(state, 0);
> > > > +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
> > > >
> > > >  	if (ctrl->id == V4L2_CID_VBLANK) {
> > > >  		int exposure_max, exposure_def;
> > > > @@ -593,8 +600,8 @@ static int imx219_set_framefmt(struct imx219 *imx219,
> > > >  	u64 bin_h, bin_v;
> > > >  	int ret = 0;
> > > >
> > > > -	format = v4l2_subdev_state_get_format(state, 0);
> > > > -	crop = v4l2_subdev_state_get_crop(state, 0);
> > > > +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
> > > > +	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_IMAGE);
> > > >
> > > >  	switch (format->code) {
> > > >  	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > > > @@ -764,10 +771,25 @@ static int imx219_enum_mbus_code(struct v4l2_subdev *sd,
> > > >  {
> > > >  	struct imx219 *imx219 = to_imx219(sd);
> > > >
> > > > -	if (code->index >= (ARRAY_SIZE(imx219_mbus_formats) / 4))
> > > > -		return -EINVAL;
> > > > +	if (code->pad == IMX219_PAD_IMAGE) {
> > > > +		/* The internal image pad is hardwired to the native format. */
> > > > +		if (code->index)
> > > > +			return -EINVAL;
> > > >
> > > > -	code->code = imx219_get_format_code(imx219, imx219_mbus_formats[code->index * 4]);
> > > > +		code->code = IMX219_NATIVE_FORMAT;
> > >
> > > If you return 0 here you can spare the else {} branch. Same in the
> > > function below.
> > >
> >
> > You're reworking this in 5/15, so no need to change this and cause
> > rebase conflicts.
>
> It's true that I'm reworking it further in 05/15, but I think the change
> here still makes sense, as it updates the driver to support the internal
> pad properly. Beside, if I were to drop this change, then I would get a
> rebase conflict :-)
>

I was suggesting to ignore my comment, not to drop the change ;)

> > > > +	} else {
> > > > +		/*
> > > > +		 * On the source pad, the sensor supports multiple raw formats
> > > > +		 * with different bit depths.
> > > > +		 */
> > > > +		u32 format;
> > > > +
> > > > +		if (code->index >= (ARRAY_SIZE(imx219_mbus_formats) / 4))
> > > > +			return -EINVAL;
> > > > +
> > > > +		format = imx219_mbus_formats[code->index * 4];
> > > > +		code->code = imx219_get_format_code(imx219, format);
> > > > +	}
> > > >
> > > >  	return 0;
> > > >  }
> > > > @@ -777,19 +799,25 @@ static int imx219_enum_frame_size(struct v4l2_subdev *sd,
> > > >  				  struct v4l2_subdev_frame_size_enum *fse)
> > > >  {
> > > >  	struct imx219 *imx219 = to_imx219(sd);
> > > > -	u32 code;
> > > >
> > > > -	if (fse->index >= ARRAY_SIZE(supported_modes))
> > > > -		return -EINVAL;
> > > > +	if (fse->pad == IMX219_PAD_IMAGE) {
> > > > +		if (fse->code != IMX219_NATIVE_FORMAT || fse->index > 0)
> > > > +			return -EINVAL;
> > > >
> > > > -	code = imx219_get_format_code(imx219, fse->code);
> > > > -	if (fse->code != code)
> > > > -		return -EINVAL;
> > > > +		fse->min_width = IMX219_NATIVE_WIDTH;
> > > > +		fse->max_width = IMX219_NATIVE_WIDTH;
> > > > +		fse->min_height = IMX219_NATIVE_HEIGHT;
> > > > +		fse->max_height = IMX219_NATIVE_HEIGHT;
> > > > +	} else {
> > > > +		if (fse->code != imx219_get_format_code(imx219, fse->code) ||
> > > > +		    fse->index >= ARRAY_SIZE(supported_modes))
> > > > +			return -EINVAL;
> > > >
> > > > -	fse->min_width = supported_modes[fse->index].width;
> > > > -	fse->max_width = fse->min_width;
> > > > -	fse->min_height = supported_modes[fse->index].height;
> > > > -	fse->max_height = fse->min_height;
> > > > +		fse->min_width = supported_modes[fse->index].width;
> > > > +		fse->max_width = fse->min_width;
> > > > +		fse->min_height = supported_modes[fse->index].height;
> > > > +		fse->max_height = fse->min_height;
> > > > +	}
> > > >
> > > >  	return 0;
> > > >  }
> > > > @@ -801,9 +829,17 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
> > > >  	struct imx219 *imx219 = to_imx219(sd);
> > > >  	const struct imx219_mode *mode;
> > > >  	struct v4l2_mbus_framefmt *format;
> > > > +	struct v4l2_rect *compose;
> > > >  	struct v4l2_rect *crop;
> > > >  	unsigned int bin_h, bin_v;
> > > >
> > > > +	/*
> > > > +	 * The driver is mode-based, the format can be set on the source pad
> > > > +	 * only.
> > > > +	 */
> > > > +	if (fmt->pad != IMX219_PAD_SOURCE)
> > > > +		return v4l2_subdev_get_fmt(sd, state, fmt);
> > > > +
> > > >  	/*
> > > >  	 * Adjust the requested format to match the closest mode. The Bayer
> > > >  	 * order varies with flips.
> > > > @@ -822,22 +858,51 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
> > > >  	fmt->format.quantization = V4L2_QUANTIZATION_FULL_RANGE;
> > > >  	fmt->format.xfer_func = V4L2_XFER_FUNC_NONE;
> > > >
> > > > -	format = v4l2_subdev_state_get_format(state, 0);
> > > > +	/* Propagate the format through the sensor. */
> > > > +
> > > > +	/* The image pad models the pixel array, and thus has a fixed size. */
> > > > +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_IMAGE);
> > > >  	*format = fmt->format;
> > >
> > > Is this assignment needed ?
> > >
> > > Isn't 'fmt' meant to be applied at pad #0 ? Also, you overwrite the
> > > code and size below, do the rest of the 'struct v4l2_mbus_framefmt'
> > > fields apply to pad #1 (field, colorspace, quantization etc ? If pad
> > > #1 represents the pixel array, I don't think they do).
>
> It's a shortcut to avoid setting field, colorspace, ycbcr_enc,
> quantization and xfer_func manually, as there's set a few lines above in
> fmt. I can set the fields manually if you prefer.
>

Yes, and my question was related to the fact those information as
received from userspace might not be valid for pad #1, but as they are
corrected a few lines above, and are not different between pad #0 and
pad #1, I think the version you have is fine

> > > Also, can't the pad #1 format be initialized by .init_state() and never
> > > touched again ?
>
> They could, but then .init_state() couldn't be implemented by simply
> calling imx219_set_pad_format(). I think it is clearer to store all the
> format setting logic in one place, especially when adding support for
> the embedded data stream.
>

Ack then!

> > > > +	format->code = IMX219_NATIVE_FORMAT;
> > > > +	format->width = IMX219_NATIVE_WIDTH;
> > > > +	format->height = IMX219_NATIVE_HEIGHT;
> > > >
> > > >  	/*
> > > >  	 * Use binning to maximize the crop rectangle size, and centre it in the
> > > >  	 * sensor.
> > > >  	 */
> > > > -	bin_h = min(IMX219_PIXEL_ARRAY_WIDTH / format->width, 2U);
> > > > -	bin_v = min(IMX219_PIXEL_ARRAY_HEIGHT / format->height, 2U);
> > > > +	bin_h = min(IMX219_PIXEL_ARRAY_WIDTH / fmt->format.width, 2U);
> > > > +	bin_v = min(IMX219_PIXEL_ARRAY_HEIGHT / fmt->format.height, 2U);
> > > >
> > > > -	crop = v4l2_subdev_state_get_crop(state, 0);
> > > > -	crop->width = format->width * bin_h;
> > > > -	crop->height = format->height * bin_v;
> > > > +	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_IMAGE);
> > > > +	crop->width = fmt->format.width * bin_h;
> > > > +	crop->height = fmt->format.height * bin_v;
> > > >  	crop->left = (IMX219_NATIVE_WIDTH - crop->width) / 2;
> > > >  	crop->top = (IMX219_NATIVE_HEIGHT - crop->height) / 2;
> > >
> > > This is no different than before, but I now wonder if VGA is actually
> > > obtained by cropping down to 1280x960 and then by a 2x2 binning, or is
> > > there a 4x4 binning mode. I presume this is correct though, as this has been
> > > validated by many people.
>
> It would be worth experimenting with the /4 binning mode.
>

Maybe not as part of this series

> > > > +	/*
> > > > +	 * The compose rectangle models binning, its size is the sensor output
> > > > +	 * size.
> > > > +	 */
> > > > +	compose = v4l2_subdev_state_get_compose(state, IMX219_PAD_IMAGE);
> > > > +	compose->left = 0;
> > > > +	compose->top = 0;
> > > > +	compose->width = fmt->format.width;
> > > > +	compose->height = fmt->format.height;
> > > > +
> > > > +	/*
> > > > +	 * No mode use digital crop, the source pad crop rectangle size and
> > > > +	 * format are thus identical to the image pad compose rectangle.
> > > > +	 */
> > > > +	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_SOURCE);
> > > > +	crop->left = 0;
> > > > +	crop->top = 0;
> > > > +	crop->width = fmt->format.width;
> > > > +	crop->height = fmt->format.height;
> > > > +
> > > > +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
> > > > +	*format = fmt->format;
> > > > +
> > > >  	if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
> > > >  		int exposure_max;
> > > >  		int exposure_def;
> > > > @@ -874,13 +939,13 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
> > > >  				struct v4l2_subdev_state *state,
> > > >  				struct v4l2_subdev_selection *sel)
> > > >  {
> > > > -	switch (sel->target) {
> > > > -	case V4L2_SEL_TGT_CROP: {
> > > > -		sel->r = *v4l2_subdev_state_get_crop(state, 0);
> > > > -		return 0;
> > > > -	}
> > > > +	struct v4l2_rect *compose;
> > > >
> > > > +	switch (sel->target) {
> > > >  	case V4L2_SEL_TGT_NATIVE_SIZE:
> > > > +		if (sel->pad != IMX219_PAD_IMAGE)
> > > > +			return -EINVAL;
> > > > +
> > > >  		sel->r.top = 0;
> > > >  		sel->r.left = 0;
> > > >  		sel->r.width = IMX219_NATIVE_WIDTH;
> > > > @@ -890,11 +955,35 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
> > > >
> > > >  	case V4L2_SEL_TGT_CROP_DEFAULT:
> > > >  	case V4L2_SEL_TGT_CROP_BOUNDS:
> > > > -		sel->r.top = IMX219_PIXEL_ARRAY_TOP;
> > > > -		sel->r.left = IMX219_PIXEL_ARRAY_LEFT;
> > > > -		sel->r.width = IMX219_PIXEL_ARRAY_WIDTH;
> > > > -		sel->r.height = IMX219_PIXEL_ARRAY_HEIGHT;
> > > > +		switch (sel->pad) {
> > > > +		case IMX219_PAD_IMAGE:
> > > > +			sel->r.top = IMX219_PIXEL_ARRAY_TOP;
> > > > +			sel->r.left = IMX219_PIXEL_ARRAY_LEFT;
> > > > +			sel->r.width = IMX219_PIXEL_ARRAY_WIDTH;
> > > > +			sel->r.height = IMX219_PIXEL_ARRAY_HEIGHT;
> > > > +			return 0;
> > > >
> > > > +		case IMX219_PAD_SOURCE:
> > > > +			compose = v4l2_subdev_state_get_compose(state,
> > > > +								IMX219_PAD_IMAGE);
> > > > +			sel->r.top = 0;
> > > > +			sel->r.left = 0;
> > > > +			sel->r.width = compose->width;
> > > > +			sel->r.height = compose->height;
> > > > +			return 0;
> > > > +		}
> > > > +
> > > > +		break;
> > > > +
> > > > +	case V4L2_SEL_TGT_CROP:
> > > > +		sel->r = *v4l2_subdev_state_get_crop(state, sel->pad);
> > > > +		return 0;
> > > > +
> > > > +	case V4L2_SEL_TGT_COMPOSE:
> > > > +		if (sel->pad != IMX219_PAD_IMAGE)
> > > > +			return -EINVAL;
> > > > +
> > > > +		sel->r = *v4l2_subdev_state_get_compose(state, sel->pad);
> > > >  		return 0;
> > >
> > > Do we need to support TGT_COMPOSE_BOUNDS for PAD_IMAGE ? I would have
> > > suggested V4L2_SEL_TGT_COMPOSE_DEFAULT too, but according to the spec
> > > it does not apply to subdevices.
>
> Good question. V4L2_SEL_TGT_COMPOSE_BOUNDS is a bit under-documented for
> V4L2 subdevs. I wonder if it wouldn't be simpler for drivers to not
> implement it when the compose bounds are identical to the crop rectangle
> size, as that could be considered a default ?
>

I'm fine with that, it should maybe be documented

> > > >  	}
> > > >
> > > > @@ -906,7 +995,7 @@ static int imx219_init_state(struct v4l2_subdev *sd,
> > > >  {
> > > >  	struct v4l2_subdev_format fmt = {
> > > >  		.which = V4L2_SUBDEV_FORMAT_TRY,
> > > > -		.pad = 0,
> > > > +		.pad = IMX219_PAD_SOURCE,
> > > >  		.format = {
> > > >  			.code = MEDIA_BUS_FMT_SRGGB10_1X10,
> > > >  			.width = supported_modes[0].width,
> > >
> > > Why not intialize pad#1 format here an never touch it again ? Should
> > > also the pad#1 crop and compose rectangles and pad#0 crop be
> > > initialized here ?
>
> As written above, I think it's cleaner to have all the format-related
> constraints in one place, especially with support for embedded data. It
> may be a style preference, I'm not sure.
>

Fine with me!

Thanks
  j

> > > > @@ -1174,10 +1263,18 @@ static int imx219_probe(struct i2c_client *client)
> > > >  			    V4L2_SUBDEV_FL_HAS_EVENTS;
> > > >  	imx219->sd.entity.function = MEDIA_ENT_F_CAM_SENSOR;
> > > >
> > > > -	/* Initialize source pad */
> > > > -	imx219->pad.flags = MEDIA_PAD_FL_SOURCE;
> > > > +	/*
> > > > +	 * Initialize the pads. To preserve backward compatibility with
> > > > +	 * userspace that used the sensor before the introduction of the
> > > > +	 * internal image pad, the external source pad is numbered 0 and the
> > > > +	 * internal image pad numbered 1.
> > > > +	 */
> > > > +	imx219->pads[IMX219_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
> > > > +	imx219->pads[IMX219_PAD_IMAGE].flags = MEDIA_PAD_FL_SINK
> > > > +					     | MEDIA_PAD_FL_INTERNAL;
> > > >
> > > > -	ret = media_entity_pads_init(&imx219->sd.entity, 1, &imx219->pad);
> > > > +	ret = media_entity_pads_init(&imx219->sd.entity,
> > > > +				     ARRAY_SIZE(imx219->pads), imx219->pads);
> > > >  	if (ret) {
> > > >  		dev_err(dev, "failed to init entity pads: %d\n", ret);
> > > >  		goto error_handler_free;
>
> --
> Regards,
>
> Laurent Pinchart
>

Re: [PATCH v6 09/15] media: bcm2835-unicam: Add support for CCP2/CSI2 camera interface

[Jacopo Mondi]

Hi Laurent

On Fri, Mar 01, 2024 at 11:32:24PM +0200, Laurent Pinchart wrote:
> From: Dave Stevenson <dave.stevenson@raspberrypi.com>
>
> Add a driver for the Unicam camera receiver block on BCM283x processors.
> It is represented as two video device nodes: unicam-image and
> unicam-embedded which are connected to an internal subdev (named
> unicam-subdev) in order to manage streams routing.
>
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> Co-developed-by: Naushir Patuck <naush@raspberrypi.com>
> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> Co-developed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
> Changes since v5:
>
> - Move to drivers/media/platform/broadcom/
> - Port to the upstream V4L2 streams API
> - Rebase on latest metadata API proposal
> - Add missing error message
> - Drop unneeded documentation block for unicam_isr()
> - Drop unneeded dev_dbg() and dev_err() messages
> - Drop unneeded streams_mask and fmt checks
> - Drop unused unicam_sd_pad_is_sink()
> - Drop unneeded includes
> - Drop v4l2_ctrl_subscribe_event() call
> - Use pm_runtime_resume_and_get()
> - Indentation and line wrap fixes
> - Let the framework set bus_info
> - Use v4l2_fwnode_endpoint_parse()
> - Fix media device cleanup
> - Drop lane reordering checks
> - Fix subdev state locking
> - Drop extra debug messages
> - Move clock handling to runtime PM handlers
> - Reorder functions
> - Rename init functions for more clarity
> - Initialize runtime PM earlier
> - Clarify error messages
> - Simplify subdev init with local variable
> - Fix subdev cleanup
> - Fix typos and indentation
> - Don't initialize local variables needlessly
> - Simplify num lanes check
> - Fix metadata handling in subdev set_fmt
> - Drop manual fallback to .s_stream()
> - Pass v4l2_pix_format to unicam_calc_format_size_bpl()
> - Simplify unicam_set_default_format()
> - Fix default format settings
> - Add busy check in unicam_s_fmt_meta()
> - Add missing \n at end of format strings
> - Fix metadata handling in subdev set_fmt
> - Fix locking when starting streaming
> - Return buffers from start streaming fails
> - Fix format validation for metadata node
> - Use video_device_pipeline_{start,stop}() helpers
> - Simplify format enumeration
> - Drop unset variable
> - Update MAINTAINERS entry
> - Update to the upstream v4l2_async_nf API
> - Update to the latest subdev routing API
> - Update to the latest subdev state API
> - Move from subdev .init_cfg() to .init_state()
> - Update to the latest videobuf2 API
> - Fix v4l2_subdev_enable_streams() error check
> - Use correct pad for the connected subdev
> - Return buffers to vb2 when start streaming fails
> - Improve debugging in start streaming handler
> - Simplify DMA address management
> - Drop comment about bcm2835-camera driver
> - Clarify comments that explain min/max sizes
> - Pass v4l2_pix_format to unicam_try_fmt()
> - Drop unneeded local variables
> - Rename image-related constants and functions
> - Turn unicam_fmt.metadata_fmt into bool
> - Rename unicam_fmt to unicam_format_info
> - Rename unicam_format_info variables to fmtinfo
> - Rename unicam_node.v_fmt to fmt
> - Add metadata formats for RAW10, RAW12 and RAW14
> - Make metadata formats line-based
> - Validate format on metadata video device
> - Add Co-devlopped-by tags
>
> Changes since v3:
>
> - Add the vendor prefix for DT name
> - Use the reg-names in DT parsing
> - Remove MAINTAINERS entry
>
> Changes since v2:
>
> - Change code organization
> - Remove unused variables
> - Correct the fmt_meta functions
> - Rewrite the start/stop streaming
>   - You can now start the image node alone, but not the metadata one
>   - The buffers are allocated per-node
>   - only the required stream is started, if the route exists and is
>     enabled
> - Prefix the macros with UNICAM_ to not have too generic names
> - Drop colorspace support
>
> Changes since v1:
>
> - Replace the unicam_{info,debug,error} macros with dev_*()
> ---
>  MAINTAINERS                                   |    1 +
>  drivers/media/platform/Kconfig                |    1 +
>  drivers/media/platform/Makefile               |    1 +
>  drivers/media/platform/broadcom/Kconfig       |   23 +
>  drivers/media/platform/broadcom/Makefile      |    3 +
>  .../platform/broadcom/bcm2835-unicam-regs.h   |  255 ++
>  .../media/platform/broadcom/bcm2835-unicam.c  | 2607 +++++++++++++++++
>  7 files changed, 2891 insertions(+)
>  create mode 100644 drivers/media/platform/broadcom/Kconfig
>  create mode 100644 drivers/media/platform/broadcom/Makefile
>  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam-regs.h
>  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam.c
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index e50a59654e6e..cc350729f467 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -4002,6 +4002,7 @@ M:	Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
>  L:	linux-media@vger.kernel.org
>  S:	Maintained
>  F:	Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> +F:	drivers/media/platform/bcm2835/
>
>  BROADCOM BCM47XX MIPS ARCHITECTURE
>  M:	Hauke Mehrtens <hauke@hauke-m.de>
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index 91e54215de3a..2d79bfc68c15 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -67,6 +67,7 @@ source "drivers/media/platform/amlogic/Kconfig"
>  source "drivers/media/platform/amphion/Kconfig"
>  source "drivers/media/platform/aspeed/Kconfig"
>  source "drivers/media/platform/atmel/Kconfig"
> +source "drivers/media/platform/broadcom/Kconfig"
>  source "drivers/media/platform/cadence/Kconfig"
>  source "drivers/media/platform/chips-media/Kconfig"
>  source "drivers/media/platform/intel/Kconfig"
> diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> index 3296ec1ebe16..da17301f7439 100644
> --- a/drivers/media/platform/Makefile
> +++ b/drivers/media/platform/Makefile
> @@ -10,6 +10,7 @@ obj-y += amlogic/
>  obj-y += amphion/
>  obj-y += aspeed/
>  obj-y += atmel/
> +obj-y += broadcom/
>  obj-y += cadence/
>  obj-y += chips-media/
>  obj-y += intel/
> diff --git a/drivers/media/platform/broadcom/Kconfig b/drivers/media/platform/broadcom/Kconfig
> new file mode 100644
> index 000000000000..cc2c9afcc948
> --- /dev/null
> +++ b/drivers/media/platform/broadcom/Kconfig
> @@ -0,0 +1,23 @@
> +# SPDX-License-Identifier: GPL-2.0
> +
> +config VIDEO_BCM2835_UNICAM
> +	tristate "Broadcom BCM283x/BCM271x Unicam video capture driver"
> +	depends on ARCH_BCM2835 || COMPILE_TEST
> +	depends on PM
> +	depends on VIDEO_DEV
> +	select MEDIA_CONTROLLER
> +	select V4L2_FWNODE
> +	select VIDEO_V4L2_SUBDEV_API
> +	select VIDEOBUF2_DMA_CONTIG
> +	help
> +	  Say Y here to enable support for the BCM283x/BCM271x CSI-2 receiver.
> +	  This is a V4L2 driver that controls the CSI-2 receiver directly,
> +	  independently from the VC4 firmware.
> +
> +	  This driver is mutually exclusive with the use of bcm2835-camera. The
> +	  firmware will disable all access to the peripheral from within the
> +	  firmware if it finds a DT node using it, and bcm2835-camera will
> +	  therefore fail to probe.
> +
> +	  To compile this driver as a module, choose M here. The module will be
> +	  called bcm2835-unicam.
> diff --git a/drivers/media/platform/broadcom/Makefile b/drivers/media/platform/broadcom/Makefile
> new file mode 100644
> index 000000000000..03d2045aba2e
> --- /dev/null
> +++ b/drivers/media/platform/broadcom/Makefile
> @@ -0,0 +1,3 @@
> +# SPDX-License-Identifier: GPL-2.0
> +
> +obj-$(CONFIG_VIDEO_BCM2835_UNICAM) += bcm2835-unicam.o
> diff --git a/drivers/media/platform/broadcom/bcm2835-unicam-regs.h b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> new file mode 100644
> index 000000000000..84775fd2fac5
> --- /dev/null
> +++ b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> @@ -0,0 +1,255 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +
> +/*
> + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> + */
> +
> +#ifndef VC4_REGS_UNICAM_H
> +#define VC4_REGS_UNICAM_H
> +
> +#include <linux/bits.h>
> +
> +/*
> + * The following values are taken from files found within the code drop
> + * made by Broadcom for the BCM21553 Graphics Driver, predominantly in
> + * brcm_usrlib/dag/vmcsx/vcinclude/hardware_vc4.h.
> + * They have been modified to be only the register offset.
> + */
> +#define UNICAM_CTRL		0x000
> +#define UNICAM_STA		0x004
> +#define UNICAM_ANA		0x008
> +#define UNICAM_PRI		0x00c
> +#define UNICAM_CLK		0x010
> +#define UNICAM_CLT		0x014
> +#define UNICAM_DAT0		0x018
> +#define UNICAM_DAT1		0x01c
> +#define UNICAM_DAT2		0x020
> +#define UNICAM_DAT3		0x024
> +#define UNICAM_DLT		0x028
> +#define UNICAM_CMP0		0x02c
> +#define UNICAM_CMP1		0x030
> +#define UNICAM_CAP0		0x034
> +#define UNICAM_CAP1		0x038
> +#define UNICAM_ICTL		0x100
> +#define UNICAM_ISTA		0x104
> +#define UNICAM_IDI0		0x108
> +#define UNICAM_IPIPE		0x10c
> +#define UNICAM_IBSA0		0x110
> +#define UNICAM_IBEA0		0x114
> +#define UNICAM_IBLS		0x118
> +#define UNICAM_IBWP		0x11c
> +#define UNICAM_IHWIN		0x120
> +#define UNICAM_IHSTA		0x124
> +#define UNICAM_IVWIN		0x128
> +#define UNICAM_IVSTA		0x12c
> +#define UNICAM_ICC		0x130
> +#define UNICAM_ICS		0x134
> +#define UNICAM_IDC		0x138
> +#define UNICAM_IDPO		0x13c
> +#define UNICAM_IDCA		0x140
> +#define UNICAM_IDCD		0x144
> +#define UNICAM_IDS		0x148
> +#define UNICAM_DCS		0x200
> +#define UNICAM_DBSA0		0x204
> +#define UNICAM_DBEA0		0x208
> +#define UNICAM_DBWP		0x20c
> +#define UNICAM_DBCTL		0x300
> +#define UNICAM_IBSA1		0x304
> +#define UNICAM_IBEA1		0x308
> +#define UNICAM_IDI1		0x30c
> +#define UNICAM_DBSA1		0x310
> +#define UNICAM_DBEA1		0x314
> +#define UNICAM_MISC		0x400
> +
> +/*
> + * The following bitmasks are from the kernel released by Broadcom
> + * for Android - https://android.googlesource.com/kernel/bcm/
> + * The Rhea, Hawaii, and Java chips all contain the same VideoCore4
> + * Unicam block as BCM2835, as defined in eg
> + * arch/arm/mach-rhea/include/mach/rdb_A0/brcm_rdb_cam.h and similar.
> + * Values reworked to use the kernel BIT and GENMASK macros.
> + *
> + * Some of the bit mnenomics have been amended to match the datasheet.
> + */
> +/* UNICAM_CTRL Register */
> +#define UNICAM_CPE		BIT(0)
> +#define UNICAM_MEM		BIT(1)
> +#define UNICAM_CPR		BIT(2)
> +#define UNICAM_CPM_MASK		GENMASK(3, 3)
> +#define UNICAM_CPM_CSI2		0
> +#define UNICAM_CPM_CCP2		1
> +#define UNICAM_SOE		BIT(4)
> +#define UNICAM_DCM_MASK		GENMASK(5, 5)
> +#define UNICAM_DCM_STROBE	0
> +#define UNICAM_DCM_DATA		1
> +#define UNICAM_SLS		BIT(6)
> +#define UNICAM_PFT_MASK		GENMASK(11, 8)
> +#define UNICAM_OET_MASK		GENMASK(20, 12)
> +
> +/* UNICAM_STA Register */
> +#define UNICAM_SYN		BIT(0)
> +#define UNICAM_CS		BIT(1)
> +#define UNICAM_SBE		BIT(2)
> +#define UNICAM_PBE		BIT(3)
> +#define UNICAM_HOE		BIT(4)
> +#define UNICAM_PLE		BIT(5)
> +#define UNICAM_SSC		BIT(6)
> +#define UNICAM_CRCE		BIT(7)
> +#define UNICAM_OES		BIT(8)
> +#define UNICAM_IFO		BIT(9)
> +#define UNICAM_OFO		BIT(10)
> +#define UNICAM_BFO		BIT(11)
> +#define UNICAM_DL		BIT(12)
> +#define UNICAM_PS		BIT(13)
> +#define UNICAM_IS		BIT(14)
> +#define UNICAM_PI0		BIT(15)
> +#define UNICAM_PI1		BIT(16)
> +#define UNICAM_FSI_S		BIT(17)
> +#define UNICAM_FEI_S		BIT(18)
> +#define UNICAM_LCI_S		BIT(19)
> +#define UNICAM_BUF0_RDY		BIT(20)
> +#define UNICAM_BUF0_NO		BIT(21)
> +#define UNICAM_BUF1_RDY		BIT(22)
> +#define UNICAM_BUF1_NO		BIT(23)
> +#define UNICAM_DI		BIT(24)
> +
> +#define UNICAM_STA_MASK_ALL \
> +		(UNICAM_DL | \
> +		UNICAM_SBE | \
> +		UNICAM_PBE | \
> +		UNICAM_HOE | \
> +		UNICAM_PLE | \
> +		UNICAM_SSC | \
> +		UNICAM_CRCE | \
> +		UNICAM_IFO | \
> +		UNICAM_OFO | \
> +		UNICAM_PS | \
> +		UNICAM_PI0 | \
> +		UNICAM_PI1)
> +
> +/* UNICAM_ANA Register */
> +#define UNICAM_APD		BIT(0)
> +#define UNICAM_BPD		BIT(1)
> +#define UNICAM_AR		BIT(2)
> +#define UNICAM_DDL		BIT(3)
> +#define UNICAM_CTATADJ_MASK	GENMASK(7, 4)
> +#define UNICAM_PTATADJ_MASK	GENMASK(11, 8)
> +
> +/* UNICAM_PRI Register */
> +#define UNICAM_PE		BIT(0)
> +#define UNICAM_PT_MASK		GENMASK(2, 1)
> +#define UNICAM_NP_MASK		GENMASK(7, 4)
> +#define UNICAM_PP_MASK		GENMASK(11, 8)
> +#define UNICAM_BS_MASK		GENMASK(15, 12)
> +#define UNICAM_BL_MASK		GENMASK(17, 16)
> +
> +/* UNICAM_CLK Register */
> +#define UNICAM_CLE		BIT(0)
> +#define UNICAM_CLPD		BIT(1)
> +#define UNICAM_CLLPE		BIT(2)
> +#define UNICAM_CLHSE		BIT(3)
> +#define UNICAM_CLTRE		BIT(4)
> +#define UNICAM_CLAC_MASK	GENMASK(8, 5)
> +#define UNICAM_CLSTE		BIT(29)
> +
> +/* UNICAM_CLT Register */
> +#define UNICAM_CLT1_MASK	GENMASK(7, 0)
> +#define UNICAM_CLT2_MASK	GENMASK(15, 8)
> +
> +/* UNICAM_DATn Registers */
> +#define UNICAM_DLE		BIT(0)
> +#define UNICAM_DLPD		BIT(1)
> +#define UNICAM_DLLPE		BIT(2)
> +#define UNICAM_DLHSE		BIT(3)
> +#define UNICAM_DLTRE		BIT(4)
> +#define UNICAM_DLSM		BIT(5)
> +#define UNICAM_DLFO		BIT(28)
> +#define UNICAM_DLSTE		BIT(29)
> +
> +#define UNICAM_DAT_MASK_ALL	(UNICAM_DLSTE | UNICAM_DLFO)
> +
> +/* UNICAM_DLT Register */
> +#define UNICAM_DLT1_MASK	GENMASK(7, 0)
> +#define UNICAM_DLT2_MASK	GENMASK(15, 8)
> +#define UNICAM_DLT3_MASK	GENMASK(23, 16)
> +
> +/* UNICAM_ICTL Register */
> +#define UNICAM_FSIE		BIT(0)
> +#define UNICAM_FEIE		BIT(1)
> +#define UNICAM_IBOB		BIT(2)
> +#define UNICAM_FCM		BIT(3)
> +#define UNICAM_TFC		BIT(4)
> +#define UNICAM_LIP_MASK		GENMASK(6, 5)
> +#define UNICAM_LCIE_MASK	GENMASK(28, 16)
> +
> +/* UNICAM_IDI0/1 Register */
> +#define UNICAM_ID0_MASK		GENMASK(7, 0)
> +#define UNICAM_ID1_MASK		GENMASK(15, 8)
> +#define UNICAM_ID2_MASK		GENMASK(23, 16)
> +#define UNICAM_ID3_MASK		GENMASK(31, 24)
> +
> +/* UNICAM_ISTA Register */
> +#define UNICAM_FSI		BIT(0)
> +#define UNICAM_FEI		BIT(1)
> +#define UNICAM_LCI		BIT(2)
> +
> +#define UNICAM_ISTA_MASK_ALL	(UNICAM_FSI | UNICAM_FEI | UNICAM_LCI)
> +
> +/* UNICAM_IPIPE Register */
> +#define UNICAM_PUM_MASK		GENMASK(2, 0)
> +/* Unpacking modes */
> +#define UNICAM_PUM_NONE		0
> +#define UNICAM_PUM_UNPACK6	1
> +#define UNICAM_PUM_UNPACK7	2
> +#define UNICAM_PUM_UNPACK8	3
> +#define UNICAM_PUM_UNPACK10	4
> +#define UNICAM_PUM_UNPACK12	5
> +#define UNICAM_PUM_UNPACK14	6
> +#define UNICAM_PUM_UNPACK16	7
> +#define UNICAM_DDM_MASK		GENMASK(6, 3)
> +#define UNICAM_PPM_MASK		GENMASK(9, 7)
> +/* Packing modes */
> +#define UNICAM_PPM_NONE		0
> +#define UNICAM_PPM_PACK8	1
> +#define UNICAM_PPM_PACK10	2
> +#define UNICAM_PPM_PACK12	3
> +#define UNICAM_PPM_PACK14	4
> +#define UNICAM_PPM_PACK16	5
> +#define UNICAM_DEM_MASK		GENMASK(11, 10)
> +#define UNICAM_DEBL_MASK	GENMASK(14, 12)
> +#define UNICAM_ICM_MASK		GENMASK(16, 15)
> +#define UNICAM_IDM_MASK		GENMASK(17, 17)
> +
> +/* UNICAM_ICC Register */
> +#define UNICAM_ICFL_MASK	GENMASK(4, 0)
> +#define UNICAM_ICFH_MASK	GENMASK(9, 5)
> +#define UNICAM_ICST_MASK	GENMASK(12, 10)
> +#define UNICAM_ICLT_MASK	GENMASK(15, 13)
> +#define UNICAM_ICLL_MASK	GENMASK(31, 16)
> +
> +/* UNICAM_DCS Register */
> +#define UNICAM_DIE		BIT(0)
> +#define UNICAM_DIM		BIT(1)
> +#define UNICAM_DBOB		BIT(3)
> +#define UNICAM_FDE		BIT(4)
> +#define UNICAM_LDP		BIT(5)
> +#define UNICAM_EDL_MASK		GENMASK(15, 8)
> +
> +/* UNICAM_DBCTL Register */
> +#define UNICAM_DBEN		BIT(0)
> +#define UNICAM_BUF0_IE		BIT(1)
> +#define UNICAM_BUF1_IE		BIT(2)
> +
> +/* UNICAM_CMP[0,1] register */
> +#define UNICAM_PCE		BIT(31)
> +#define UNICAM_GI		BIT(9)
> +#define UNICAM_CPH		BIT(8)
> +#define UNICAM_PCVC_MASK	GENMASK(7, 6)
> +#define UNICAM_PCDT_MASK	GENMASK(5, 0)
> +
> +/* UNICAM_MISC register */
> +#define UNICAM_FL0		BIT(6)
> +#define UNICAM_FL1		BIT(9)
> +
> +#endif
> diff --git a/drivers/media/platform/broadcom/bcm2835-unicam.c b/drivers/media/platform/broadcom/bcm2835-unicam.c
> new file mode 100644
> index 000000000000..716c89b8a217
> --- /dev/null
> +++ b/drivers/media/platform/broadcom/bcm2835-unicam.c
> @@ -0,0 +1,2607 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * BCM283x / BCM271x Unicam Capture Driver
> + *
> + * Copyright (C) 2017-2020 - Raspberry Pi (Trading) Ltd.
> + *
> + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> + *
> + * Based on TI am437x driver by
> + *   Benoit Parrot <bparrot@ti.com>
> + *   Lad, Prabhakar <prabhakar.csengg@gmail.com>
> + *
> + * and TI CAL camera interface driver by
> + *    Benoit Parrot <bparrot@ti.com>
> + *
> + *
> + * There are two camera drivers in the kernel for BCM283x - this one and
> + * bcm2835-camera (currently in staging).
> + *
> + * This driver directly controls the Unicam peripheral - there is no
> + * involvement with the VideoCore firmware. Unicam receives CSI-2 or CCP2 data
> + * and writes it into SDRAM. The only potential processing options are to
> + * repack Bayer data into an alternate format, and applying windowing. The
> + * repacking does not shift the data, so can repack V4L2_PIX_FMT_Sxxxx10P to
> + * V4L2_PIX_FMT_Sxxxx10, or V4L2_PIX_FMT_Sxxxx12P to V4L2_PIX_FMT_Sxxxx12, but
> + * not generically up to V4L2_PIX_FMT_Sxxxx16. The driver will add both formats
> + * where the relevant formats are defined, and will automatically configure the
> + * repacking as required. Support for windowing may be added later.

Does this last paragraph applies ? I see all the supported packed raw
format having an 'unpacked_fourcc' associated (all but the 8 bit ones
ofc)

> + *
> + * It should be possible to connect this driver to any sensor with a suitable
> + * output interface and V4L2 subdevice driver.
> + */
> +
> +#include <linux/clk.h>
> +#include <linux/delay.h>
> +#include <linux/device.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/err.h>
> +#include <linux/interrupt.h>
> +#include <linux/io.h>
> +#include <linux/module.h>
> +#include <linux/of.h>
> +#include <linux/of_device.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/slab.h>
> +#include <linux/videodev2.h>
> +
> +#include <media/v4l2-async.h>
> +#include <media/v4l2-common.h>
> +#include <media/v4l2-dev.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/v4l2-fwnode.h>
> +#include <media/v4l2-mc.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "bcm2835-unicam-regs.h"
> +
> +#define UNICAM_MODULE_NAME		"unicam"
> +
> +/*
> + * Unicam must request a minimum of 250Mhz from the VPU clock.
> + * Otherwise the input FIFOs overrun and cause image corruption.
> + */
> +#define UNICAM_MIN_VPU_CLOCK_RATE	(250 * 1000 * 1000)

Shouldn't this be set in DT ?

> +
> +/* Unicam has an internal DMA alignment constraint of 16 bytes for each line. */
> +#define UNICAM_DMA_BPL_ALIGNMENT	16
> +
> +/*
> + * The image stride is stored in a 16 bit register, and needs to be aligned to
> + * the DMA constraint. As the ISP in the same SoC has a 32 bytes alignment
> + * constraint on its input, set the image stride alignment to 32 bytes here as
> + * well to avoid incompatible configurations.
> + */
> +#define UNICAM_IMAGE_BPL_ALIGNMENT	32
> +#define UNICAM_IMAGE_MAX_BPL		((1 << 16) - UNICAM_IMAGE_BPL_ALIGNMENT)
> +
> +/*
> + * Max width is therefore determined by the max stride divided by the number of
> + * bits per pixel. Take 32bpp as a worst case. No imposed limit on the height,
> + * so adopt a square image for want of anything better.
> + */
> +#define UNICAM_IMAGE_MIN_WIDTH		16
> +#define UNICAM_IMAGE_MIN_HEIGHT		16
> +#define UNICAM_IMAGE_MAX_WIDTH		(UNICAM_IMAGE_MAX_BPL / 4)
> +#define UNICAM_IMAGE_MAX_HEIGHT		UNICAM_IMAGE_MAX_WIDTH
> +
> +/*
> + * There's no intrinsic limits on the width and height for embedded dat. Use
> + * the same maximum values as for the image, to avoid overflows in the image
> + * size computation.
> + */
> +#define UNICAM_META_MIN_WIDTH		1
> +#define UNICAM_META_MIN_HEIGHT		1
> +#define UNICAM_META_MAX_WIDTH		UNICAM_IMAGE_MAX_WIDTH
> +#define UNICAM_META_MAX_HEIGHT		UNICAM_IMAGE_MAX_HEIGHT
> +
> +/*
> + * Size of the dummy buffer. Can be any size really, but the DMA
> + * allocation works in units of page sizes.
> + */
> +#define UNICAM_DUMMY_BUF_SIZE		PAGE_SIZE
> +
> +#define UNICAM_SD_PAD_SINK		0
> +#define UNICAM_SD_PAD_SOURCE_IMAGE	1
> +#define UNICAM_SD_PAD_SOURCE_METADATA	2
> +#define UNICAM_SD_NUM_PADS		(1 + UNICAM_SD_PAD_SOURCE_METADATA)

How about an enum then ?

> +
> +enum unicam_node_type {
> +	UNICAM_IMAGE_NODE,
> +	UNICAM_METADATA_NODE,
> +	UNICAM_MAX_NODES
> +};
> +
> +/*
> + * struct unicam_format_info - Unicam media bus format information
> + * @fourcc: V4L2 pixel format FCC identifier. 0 if n/a.
> + * @unpacked_fourcc: V4L2 pixel format FCC identifier if the data is expanded
> + * out to 16bpp. 0 if n/a.
> + * @code: V4L2 media bus format code.
> + * @depth: Bits per pixel as delivered from the source.
> + * @csi_dt: CSI data type.
> + * @metadata_fmt: This format only applies to the metadata pad.
> + */
> +struct unicam_format_info {
> +	u32	fourcc;
> +	u32	unpacked_fourcc;
> +	u32	code;
> +	u8	depth;
> +	u8	csi_dt;
> +	bool	metadata_fmt;
> +};
> +
> +struct unicam_buffer {
> +	struct vb2_v4l2_buffer vb;
> +	struct list_head list;
> +	dma_addr_t dma_addr;
> +	unsigned int size;
> +};
> +
> +static inline struct unicam_buffer *to_unicam_buffer(struct vb2_buffer *vb)
> +{
> +	return container_of(vb, struct unicam_buffer, vb.vb2_buf);
> +}
> +
> +struct unicam_node {
> +	bool registered;
> +	bool streaming;
> +	unsigned int id;
> +
> +	/* Pointer to the current v4l2_buffer */
> +	struct unicam_buffer *cur_frm;
> +	/* Pointer to the next v4l2_buffer */
> +	struct unicam_buffer *next_frm;
> +	/* video capture */
> +	const struct unicam_format_info *fmtinfo;
> +	/* Used to store current pixel format */
> +	struct v4l2_format fmt;
> +	/* Buffer queue used in video-buf */
> +	struct vb2_queue buffer_queue;
> +	/* Queue of filled frames */
> +	struct list_head dma_queue;
> +	/* IRQ lock for DMA queue */
> +	spinlock_t dma_queue_lock;
> +	/* lock used to access this structure */
> +	struct mutex lock;
> +	/* Identifies video device for this channel */
> +	struct video_device video_dev;
> +	/* Pointer to the parent handle */
> +	struct unicam_device *dev;
> +	struct media_pad pad;
> +	/*
> +	 * Dummy buffer intended to be used by unicam
> +	 * if we have no other queued buffers to swap to.
> +	 */
> +	struct unicam_buffer dummy_buf;
> +	void *dummy_buf_cpu_addr;
> +};
> +
> +struct unicam_device {
> +	struct kref kref;
> +
> +	/* peripheral base address */
> +	void __iomem *base;
> +	/* clock gating base address */
> +	void __iomem *clk_gate_base;

Is the clock gating part of unicam or should this be expressed as a
clock provide and be referenced in DT through a phandle ?

> +	/* lp clock handle */
> +	struct clk *clock;
> +	/* vpu clock handle */
> +	struct clk *vpu_clock;
> +	/* V4l2 device */
> +	struct v4l2_device v4l2_dev;
> +	struct media_device mdev;
> +
> +	/* parent device */
> +	struct device *dev;
> +	/* subdevice async Notifier */

s/Notifier/notifier

> +	struct v4l2_async_notifier notifier;
> +	unsigned int sequence;
> +
> +	/* Sensor node */
> +	struct {
> +		struct v4l2_subdev *subdev;
> +		struct media_pad *pad;
> +	} sensor;
> +
> +	/* Internal subdev */
> +	struct {
> +		struct v4l2_subdev sd;
> +		struct media_pad pads[UNICAM_SD_NUM_PADS];
> +		bool streaming;
> +	} subdev;
> +
> +	enum v4l2_mbus_type bus_type;
> +	/*
> +	 * Stores bus.mipi_csi2.flags for CSI2 sensors, or
> +	 * bus.mipi_csi1.strobe for CCP2.
> +	 */
> +	unsigned int bus_flags;
> +	unsigned int max_data_lanes;
> +	unsigned int active_data_lanes;
> +
> +	struct media_pipeline pipe;
> +
> +	struct unicam_node node[UNICAM_MAX_NODES];
> +};
> +
> +static inline struct unicam_device *
> +notifier_to_unicam_device(struct v4l2_async_notifier *notifier)
> +{
> +	return container_of(notifier, struct unicam_device, notifier);
> +}
> +
> +static inline struct unicam_device *

I thought using inline is disouraged as the compiler knows what to do

> +sd_to_unicam_device(struct v4l2_subdev *sd)
> +{
> +	return container_of(sd, struct unicam_device, subdev.sd);

You can also use container_of_const() if you need it

> +}
> +
> +static void unicam_release(struct kref *kref)
> +{
> +	struct unicam_device *unicam =
> +		container_of(kref, struct unicam_device, kref);
> +
> +	if (unicam->mdev.dev)
> +		media_device_cleanup(&unicam->mdev);
> +
> +	kfree(unicam);
> +}
> +
> +static struct unicam_device *unicam_get(struct unicam_device *unicam)
> +{
> +	kref_get(&unicam->kref);
> +	return unicam;
> +}
> +
> +static void unicam_put(struct unicam_device *unicam)
> +{
> +	kref_put(&unicam->kref, unicam_release);
> +}
> +

Does this handle a bit more gracefully the pesky lifetime management
issue in v4l2 ?

> +/* -----------------------------------------------------------------------------
> + * Misc helper functions
> + */
> +
> +static inline bool unicam_sd_pad_is_source(u32 pad)
> +{
> +	/* Camera RX has 1 sink pad, and N source pads */
> +	return pad != UNICAM_SD_PAD_SINK;
> +}
> +
> +static inline bool is_metadata_node(struct unicam_node *node)
> +{
> +	return node->video_dev.device_caps & V4L2_CAP_META_CAPTURE;
> +}
> +
> +static inline bool is_image_node(struct unicam_node *node)
> +{
> +	return node->video_dev.device_caps & V4L2_CAP_VIDEO_CAPTURE;
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * Format data table and helper functions
> + */
> +
> +static const struct v4l2_mbus_framefmt unicam_default_image_format = {
> +	.width = 640,
> +	.height = 480,
> +	.code = MEDIA_BUS_FMT_UYVY8_1X16,
> +	.field = V4L2_FIELD_NONE,
> +	.colorspace = V4L2_COLORSPACE_SRGB,
> +	.ycbcr_enc = V4L2_YCBCR_ENC_601,
> +	.quantization = V4L2_QUANTIZATION_LIM_RANGE,
> +	.xfer_func = V4L2_XFER_FUNC_SRGB,
> +	.flags = 0,
> +};
> +
> +static const struct v4l2_mbus_framefmt unicam_default_meta_format = {
> +	.width = 640,
> +	.height = 2,
> +	.code = MEDIA_BUS_FMT_META_8,
> +	.field = V4L2_FIELD_NONE,
> +};
> +
> +static const struct unicam_format_info unicam_image_formats[] = {
> +	/* YUV Formats */
> +	{
> +		.fourcc		= V4L2_PIX_FMT_YUYV,
> +		.code		= MEDIA_BUS_FMT_YUYV8_1X16,
> +		.depth		= 16,
> +		.csi_dt		= 0x1e,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_UYVY,
> +		.code		= MEDIA_BUS_FMT_UYVY8_1X16,
> +		.depth		= 16,
> +		.csi_dt		= 0x1e,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_YVYU,
> +		.code		= MEDIA_BUS_FMT_YVYU8_1X16,
> +		.depth		= 16,
> +		.csi_dt		= 0x1e,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_VYUY,
> +		.code		= MEDIA_BUS_FMT_VYUY8_1X16,
> +		.depth		= 16,
> +		.csi_dt		= 0x1e,
> +	}, {
> +	/* RGB Formats */
> +		.fourcc		= V4L2_PIX_FMT_RGB565, /* gggbbbbb rrrrrggg */
> +		.code		= MEDIA_BUS_FMT_RGB565_1X16,
> +		.depth		= 16,
> +		.csi_dt		= 0x22,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_RGB24, /* rgb */
> +		.code		= MEDIA_BUS_FMT_RGB888_1X24,
> +		.depth		= 24,
> +		.csi_dt		= 0x24,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_BGR24, /* bgr */
> +		.code		= MEDIA_BUS_FMT_BGR888_1X24,
> +		.depth		= 24,
> +		.csi_dt		= 0x24,
> +	}, {
> +	/* Bayer Formats */
> +		.fourcc		= V4L2_PIX_FMT_SBGGR8,
> +		.code		= MEDIA_BUS_FMT_SBGGR8_1X8,
> +		.depth		= 8,
> +		.csi_dt		= 0x2a,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SGBRG8,
> +		.code		= MEDIA_BUS_FMT_SGBRG8_1X8,
> +		.depth		= 8,
> +		.csi_dt		= 0x2a,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SGRBG8,
> +		.code		= MEDIA_BUS_FMT_SGRBG8_1X8,
> +		.depth		= 8,
> +		.csi_dt		= 0x2a,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SRGGB8,
> +		.code		= MEDIA_BUS_FMT_SRGGB8_1X8,
> +		.depth		= 8,
> +		.csi_dt		= 0x2a,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SBGGR10P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SBGGR10,
> +		.code		= MEDIA_BUS_FMT_SBGGR10_1X10,
> +		.depth		= 10,
> +		.csi_dt		= 0x2b,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SGBRG10P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SGBRG10,
> +		.code		= MEDIA_BUS_FMT_SGBRG10_1X10,
> +		.depth		= 10,
> +		.csi_dt		= 0x2b,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SGRBG10P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SGRBG10,
> +		.code		= MEDIA_BUS_FMT_SGRBG10_1X10,
> +		.depth		= 10,
> +		.csi_dt		= 0x2b,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SRGGB10P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SRGGB10,
> +		.code		= MEDIA_BUS_FMT_SRGGB10_1X10,
> +		.depth		= 10,
> +		.csi_dt		= 0x2b,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SBGGR12P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SBGGR12,
> +		.code		= MEDIA_BUS_FMT_SBGGR12_1X12,
> +		.depth		= 12,
> +		.csi_dt		= 0x2c,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SGBRG12P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SGBRG12,
> +		.code		= MEDIA_BUS_FMT_SGBRG12_1X12,
> +		.depth		= 12,
> +		.csi_dt		= 0x2c,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SGRBG12P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SGRBG12,
> +		.code		= MEDIA_BUS_FMT_SGRBG12_1X12,
> +		.depth		= 12,
> +		.csi_dt		= 0x2c,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SRGGB12P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SRGGB12,
> +		.code		= MEDIA_BUS_FMT_SRGGB12_1X12,
> +		.depth		= 12,
> +		.csi_dt		= 0x2c,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SBGGR14P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SBGGR14,
> +		.code		= MEDIA_BUS_FMT_SBGGR14_1X14,
> +		.depth		= 14,
> +		.csi_dt		= 0x2d,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SGBRG14P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SGBRG14,
> +		.code		= MEDIA_BUS_FMT_SGBRG14_1X14,
> +		.depth		= 14,
> +		.csi_dt		= 0x2d,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SGRBG14P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SGRBG14,
> +		.code		= MEDIA_BUS_FMT_SGRBG14_1X14,
> +		.depth		= 14,
> +		.csi_dt		= 0x2d,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SRGGB14P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SRGGB14,
> +		.code		= MEDIA_BUS_FMT_SRGGB14_1X14,
> +		.depth		= 14,
> +		.csi_dt		= 0x2d,
> +	}, {
> +	/* 16 bit Bayer formats could be supported. */
> +
> +	/* Greyscale formats */
> +		.fourcc		= V4L2_PIX_FMT_GREY,
> +		.code		= MEDIA_BUS_FMT_Y8_1X8,
> +		.depth		= 8,
> +		.csi_dt		= 0x2a,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_Y10P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_Y10,
> +		.code		= MEDIA_BUS_FMT_Y10_1X10,
> +		.depth		= 10,
> +		.csi_dt		= 0x2b,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_Y12P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_Y12,
> +		.code		= MEDIA_BUS_FMT_Y12_1X12,
> +		.depth		= 12,
> +		.csi_dt		= 0x2c,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_Y14P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_Y14,
> +		.code		= MEDIA_BUS_FMT_Y14_1X14,
> +		.depth		= 14,
> +		.csi_dt		= 0x2d,
> +	},
> +};
> +
> +static const struct unicam_format_info unicam_meta_formats[] = {
> +	{
> +		.fourcc		= V4L2_META_FMT_GENERIC_8,
> +		.code		= MEDIA_BUS_FMT_META_8,
> +		.depth		= 8,
> +		.metadata_fmt	= true,
> +	}, {
> +		.fourcc		= V4L2_META_FMT_GENERIC_CSI2_10,
> +		.code		= MEDIA_BUS_FMT_META_10,
> +		.depth		= 10,
> +		.metadata_fmt	= true,
> +	}, {
> +		.fourcc		= V4L2_META_FMT_GENERIC_CSI2_12,
> +		.code		= MEDIA_BUS_FMT_META_12,
> +		.depth		= 12,
> +		.metadata_fmt	= true,
> +	}, {
> +		.fourcc		= V4L2_META_FMT_GENERIC_CSI2_14,
> +		.code		= MEDIA_BUS_FMT_META_14,
> +		.depth		= 14,
> +		.metadata_fmt	= true,
> +	},
> +};
> +
> +/* Format setup functions */
> +static const struct unicam_format_info *
> +unicam_find_format_by_code(u32 code, u32 pad)
> +{
> +	const struct unicam_format_info *formats;
> +	unsigned int num_formats;
> +	unsigned int i;
> +
> +	if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> +		formats = unicam_image_formats;
> +		num_formats = ARRAY_SIZE(unicam_image_formats);
> +	} else {
> +		formats = unicam_meta_formats;
> +		num_formats = ARRAY_SIZE(unicam_meta_formats);
> +	}
> +
> +	for (i = 0; i < num_formats; i++) {
> +		if (formats[i].code == code)
> +			return &formats[i];
> +	}
> +
> +	return NULL;
> +}
> +
> +static const struct unicam_format_info *
> +unicam_find_format_by_fourcc(u32 fourcc, u32 pad)
> +{
> +	const struct unicam_format_info *formats;
> +	unsigned int num_formats;
> +	unsigned int i;
> +
> +	if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> +		formats = unicam_image_formats;
> +		num_formats = ARRAY_SIZE(unicam_image_formats);
> +	} else {
> +		formats = unicam_meta_formats;
> +		num_formats = ARRAY_SIZE(unicam_meta_formats);
> +	}
> +
> +	for (i = 0; i < num_formats; ++i) {
> +		if (formats[i].fourcc == fourcc)
> +			return &formats[i];
> +	}
> +
> +	return NULL;
> +}
> +
> +static void unicam_calc_image_size_bpl(struct unicam_device *unicam,
> +				       const struct unicam_format_info *fmtinfo,
> +				       struct v4l2_pix_format *pix)
> +{
> +	u32 min_bpl;
> +
> +	v4l_bound_align_image(&pix->width, UNICAM_IMAGE_MIN_WIDTH,
> +			      UNICAM_IMAGE_MAX_WIDTH, 2,
> +			      &pix->height, UNICAM_IMAGE_MIN_HEIGHT,
> +			      UNICAM_IMAGE_MAX_HEIGHT, 0, 0);
> +
> +	/* Unpacking always goes to 16bpp */
> +	if (pix->pixelformat == fmtinfo->unpacked_fourcc)
> +		min_bpl = pix->width * 2;
> +	else
> +		min_bpl = pix->width * fmtinfo->depth / 8;
> +	min_bpl = ALIGN(min_bpl, UNICAM_IMAGE_BPL_ALIGNMENT);
> +
> +	pix->bytesperline = ALIGN(pix->bytesperline, UNICAM_IMAGE_BPL_ALIGNMENT);
> +	pix->bytesperline = clamp_t(unsigned int, pix->bytesperline, min_bpl,
> +				    UNICAM_IMAGE_MAX_BPL);
> +
> +	pix->sizeimage = pix->height * pix->bytesperline;
> +}
> +
> +static void unicam_calc_meta_size_bpl(struct unicam_device *unicam,
> +				      const struct unicam_format_info *fmtinfo,
> +				      struct v4l2_meta_format *meta)
> +{
> +	v4l_bound_align_image(&meta->width, UNICAM_META_MIN_WIDTH,
> +			      UNICAM_META_MAX_WIDTH, 0,
> +			      &meta->height, UNICAM_META_MIN_HEIGHT,
> +			      UNICAM_META_MAX_HEIGHT, 0, 0);
> +
> +	meta->bytesperline = ALIGN(meta->width * fmtinfo->depth / 8,
> +				   UNICAM_DMA_BPL_ALIGNMENT);
> +	meta->buffersize = meta->height * meta->bytesperline;
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * Hardware handling
> + */
> +

As this has been used for years in BSP driver, I won't go in detail in
the HW handling functions

> +static inline void unicam_clk_write(struct unicam_device *unicam, u32 val)
> +{
> +	/* Pass the CM_PASSWORD along with the value. */
> +	writel(val | 0x5a000000, unicam->clk_gate_base);
> +}
> +
> +static inline u32 unicam_reg_read(struct unicam_device *unicam, u32 offset)
> +{
> +	return readl(unicam->base + offset);
> +}
> +
> +static inline void unicam_reg_write(struct unicam_device *unicam, u32 offset, u32 val)
> +{
> +	writel(val, unicam->base + offset);
> +}
> +
> +static inline int unicam_get_field(u32 value, u32 mask)
> +{
> +	return (value & mask) >> __ffs(mask);
> +}
> +
> +static inline void unicam_set_field(u32 *valp, u32 field, u32 mask)
> +{
> +	u32 val = *valp;
> +
> +	val &= ~mask;
> +	val |= (field << __ffs(mask)) & mask;
> +	*valp = val;
> +}
> +
> +static inline void unicam_reg_write_field(struct unicam_device *unicam, u32 offset,
> +					  u32 field, u32 mask)
> +{
> +	u32 val = unicam_reg_read(unicam, offset);
> +
> +	unicam_set_field(&val, field, mask);
> +	unicam_reg_write(unicam, offset, val);
> +}
> +
> +static void unicam_wr_dma_addr(struct unicam_node *node,
> +			       struct unicam_buffer *buf)
> +{
> +	dma_addr_t endaddr = buf->dma_addr + buf->size;
> +
> +	if (node->id == UNICAM_IMAGE_NODE) {
> +		unicam_reg_write(node->dev, UNICAM_IBSA0, buf->dma_addr);
> +		unicam_reg_write(node->dev, UNICAM_IBEA0, endaddr);
> +	} else {
> +		unicam_reg_write(node->dev, UNICAM_DBSA0, buf->dma_addr);
> +		unicam_reg_write(node->dev, UNICAM_DBEA0, endaddr);
> +	}
> +}
> +
> +static unsigned int unicam_get_lines_done(struct unicam_device *unicam)
> +{
> +	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> +	unsigned int stride = node->fmt.fmt.pix.bytesperline;
> +	struct unicam_buffer *frm = node->cur_frm;
> +	dma_addr_t cur_addr;
> +
> +	if (!frm)
> +		return 0;
> +
> +	cur_addr = unicam_reg_read(unicam, UNICAM_IBWP);
> +	return (unsigned int)(cur_addr - frm->dma_addr) / stride;
> +}
> +
> +static void unicam_schedule_next_buffer(struct unicam_node *node)
> +{
> +	struct unicam_buffer *buf;
> +
> +	buf = list_first_entry(&node->dma_queue, struct unicam_buffer, list);
> +	node->next_frm = buf;
> +	list_del(&buf->list);
> +
> +	unicam_wr_dma_addr(node, buf);
> +}
> +
> +static void unicam_schedule_dummy_buffer(struct unicam_node *node)
> +{
> +	int node_id = is_image_node(node) ? UNICAM_IMAGE_NODE : UNICAM_METADATA_NODE;
> +
> +	dev_dbg(node->dev->dev, "Scheduling dummy buffer for node %d\n", node_id);
> +
> +	unicam_wr_dma_addr(node, &node->dummy_buf);
> +
> +	node->next_frm = NULL;
> +}
> +
> +static void unicam_process_buffer_complete(struct unicam_node *node,
> +					   unsigned int sequence)
> +{
> +	node->cur_frm->vb.field = node->fmt.fmt.pix.field;
> +	node->cur_frm->vb.sequence = sequence;
> +
> +	vb2_buffer_done(&node->cur_frm->vb.vb2_buf, VB2_BUF_STATE_DONE);
> +}
> +
> +static void unicam_queue_event_sof(struct unicam_device *unicam)
> +{
> +	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> +
> +	struct v4l2_event event = {
> +		.type = V4L2_EVENT_FRAME_SYNC,
> +		.u.frame_sync.frame_sequence = unicam->sequence,
> +	};
> +
> +	v4l2_event_queue(&node->video_dev, &event);
> +}
> +
> +static irqreturn_t unicam_isr(int irq, void *dev)
> +{
> +	struct unicam_device *unicam = dev;
> +	unsigned int lines_done = unicam_get_lines_done(dev);
> +	unsigned int sequence = unicam->sequence;
> +	unsigned int i;
> +	u32 ista, sta;
> +	bool fe;
> +	u64 ts;
> +
> +	sta = unicam_reg_read(unicam, UNICAM_STA);
> +	/* Write value back to clear the interrupts */
> +	unicam_reg_write(unicam, UNICAM_STA, sta);
> +
> +	ista = unicam_reg_read(unicam, UNICAM_ISTA);
> +	/* Write value back to clear the interrupts */
> +	unicam_reg_write(unicam, UNICAM_ISTA, ista);
> +
> +	dev_dbg(unicam->dev, "ISR: ISTA: 0x%X, STA: 0x%X, sequence %d, lines done %d\n",
> +		ista, sta, sequence, lines_done);
> +
> +	if (!(sta & (UNICAM_IS | UNICAM_PI0)))
> +		return IRQ_HANDLED;
> +
> +	/*
> +	 * Look for either the Frame End interrupt or the Packet Capture status
> +	 * to signal a frame end.
> +	 */
> +	fe = ista & UNICAM_FEI || sta & UNICAM_PI0;
> +
> +	/*
> +	 * We must run the frame end handler first. If we have a valid next_frm
> +	 * and we get a simultaneout FE + FS interrupt, running the FS handler
> +	 * first would null out the next_frm ptr and we would have lost the
> +	 * buffer forever.
> +	 */
> +	if (fe) {
> +		/*
> +		 * Ensure we have swapped buffers already as we can't
> +		 * stop the peripheral. If no buffer is available, use a
> +		 * dummy buffer to dump out frames until we get a new buffer
> +		 * to use.
> +		 */
> +		for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> +			if (!unicam->node[i].streaming)
> +				continue;
> +
> +			/*
> +			 * If cur_frm == next_frm, it means we have not had
> +			 * a chance to swap buffers, likely due to having
> +			 * multiple interrupts occurring simultaneously (like FE
> +			 * + FS + LS). In this case, we cannot signal the buffer
> +			 * as complete, as the HW will reuse that buffer.
> +			 */
> +			if (unicam->node[i].cur_frm &&
> +			    unicam->node[i].cur_frm != unicam->node[i].next_frm)
> +				unicam_process_buffer_complete(&unicam->node[i],
> +							       sequence);
> +			unicam->node[i].cur_frm = unicam->node[i].next_frm;
> +		}
> +		unicam->sequence++;
> +	}
> +
> +	if (ista & UNICAM_FSI) {
> +		/*
> +		 * Timestamp is to be when the first data byte was captured,
> +		 * aka frame start.
> +		 */
> +		ts = ktime_get_ns();
> +		for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> +			if (!unicam->node[i].streaming)
> +				continue;
> +
> +			if (unicam->node[i].cur_frm)
> +				unicam->node[i].cur_frm->vb.vb2_buf.timestamp =
> +								ts;
> +			else
> +				dev_dbg(unicam->v4l2_dev.dev,
> +					"ISR: [%d] Dropping frame, buffer not available at FS\n",
> +					i);
> +			/*
> +			 * Set the next frame output to go to a dummy frame
> +			 * if we have not managed to obtain another frame
> +			 * from the queue.
> +			 */
> +			unicam_schedule_dummy_buffer(&unicam->node[i]);
> +		}
> +
> +		unicam_queue_event_sof(unicam);
> +	}
> +
> +	/*
> +	 * Cannot swap buffer at frame end, there may be a race condition
> +	 * where the HW does not actually swap it if the new frame has
> +	 * already started.
> +	 */
> +	if (ista & (UNICAM_FSI | UNICAM_LCI) && !fe) {
> +		for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> +			if (!unicam->node[i].streaming)
> +				continue;
> +
> +			spin_lock(&unicam->node[i].dma_queue_lock);
> +			if (!list_empty(&unicam->node[i].dma_queue) &&
> +			    !unicam->node[i].next_frm)
> +				unicam_schedule_next_buffer(&unicam->node[i]);
> +			spin_unlock(&unicam->node[i].dma_queue_lock);
> +		}
> +	}
> +
> +	if (unicam_reg_read(unicam, UNICAM_ICTL) & UNICAM_FCM) {
> +		/* Switch out of trigger mode if selected */
> +		unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> +		unicam_reg_write_field(unicam, UNICAM_ICTL, 0, UNICAM_FCM);
> +	}
> +	return IRQ_HANDLED;
> +}
> +
> +static void unicam_set_packing_config(struct unicam_device *unicam)
> +{
> +	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> +	u32 pack, unpack;
> +	u32 val;
> +
> +	if (node->fmt.fmt.pix.pixelformat == node->fmtinfo->fourcc) {
> +		unpack = UNICAM_PUM_NONE;
> +		pack = UNICAM_PPM_NONE;
> +	} else {
> +		switch (node->fmtinfo->depth) {
> +		case 8:
> +			unpack = UNICAM_PUM_UNPACK8;
> +			break;
> +		case 10:
> +			unpack = UNICAM_PUM_UNPACK10;
> +			break;
> +		case 12:
> +			unpack = UNICAM_PUM_UNPACK12;
> +			break;
> +		case 14:
> +			unpack = UNICAM_PUM_UNPACK14;
> +			break;
> +		case 16:
> +			unpack = UNICAM_PUM_UNPACK16;
> +			break;
> +		default:
> +			unpack = UNICAM_PUM_NONE;
> +			break;
> +		}
> +
> +		/* Repacking is always to 16bpp */
> +		pack = UNICAM_PPM_PACK16;
> +	}
> +
> +	val = 0;
> +	unicam_set_field(&val, unpack, UNICAM_PUM_MASK);
> +	unicam_set_field(&val, pack, UNICAM_PPM_MASK);
> +	unicam_reg_write(unicam, UNICAM_IPIPE, val);
> +}
> +
> +static void unicam_cfg_image_id(struct unicam_device *unicam)
> +{
> +	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> +
> +	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> +		/* CSI2 mode, hardcode VC 0 for now. */
> +		unicam_reg_write(unicam, UNICAM_IDI0,
> +				 (0 << 6) | node->fmtinfo->csi_dt);
> +	} else {
> +		/* CCP2 mode */
> +		unicam_reg_write(unicam, UNICAM_IDI0,
> +				 0x80 | node->fmtinfo->csi_dt);
> +	}
> +}
> +
> +static void unicam_enable_ed(struct unicam_device *unicam)
> +{
> +	u32 val = unicam_reg_read(unicam, UNICAM_DCS);
> +
> +	unicam_set_field(&val, 2, UNICAM_EDL_MASK);
> +	/* Do not wrap at the end of the embedded data buffer */
> +	unicam_set_field(&val, 0, UNICAM_DBOB);
> +
> +	unicam_reg_write(unicam, UNICAM_DCS, val);
> +}
> +
> +static void unicam_start_rx(struct unicam_device *unicam,
> +			    struct unicam_buffer *buf)
> +{
> +	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> +	int line_int_freq = node->fmt.fmt.pix.height >> 2;
> +	unsigned int i;
> +	u32 val;
> +
> +	if (line_int_freq < 128)
> +		line_int_freq = 128;
> +
> +	/* Enable lane clocks */
> +	val = 1;
> +	for (i = 0; i < unicam->active_data_lanes; i++)
> +		val = val << 2 | 1;
> +	unicam_clk_write(unicam, val);
> +
> +	/* Basic init */
> +	unicam_reg_write(unicam, UNICAM_CTRL, UNICAM_MEM);
> +
> +	/* Enable analogue control, and leave in reset. */
> +	val = UNICAM_AR;
> +	unicam_set_field(&val, 7, UNICAM_CTATADJ_MASK);
> +	unicam_set_field(&val, 7, UNICAM_PTATADJ_MASK);
> +	unicam_reg_write(unicam, UNICAM_ANA, val);
> +	usleep_range(1000, 2000);
> +
> +	/* Come out of reset */
> +	unicam_reg_write_field(unicam, UNICAM_ANA, 0, UNICAM_AR);
> +
> +	/* Peripheral reset */
> +	unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPR);
> +	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPR);
> +
> +	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPE);
> +
> +	/* Enable Rx control. */
> +	val = unicam_reg_read(unicam, UNICAM_CTRL);
> +	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> +		unicam_set_field(&val, UNICAM_CPM_CSI2, UNICAM_CPM_MASK);
> +		unicam_set_field(&val, UNICAM_DCM_STROBE, UNICAM_DCM_MASK);
> +	} else {
> +		unicam_set_field(&val, UNICAM_CPM_CCP2, UNICAM_CPM_MASK);
> +		unicam_set_field(&val, unicam->bus_flags, UNICAM_DCM_MASK);
> +	}
> +	/* Packet framer timeout */
> +	unicam_set_field(&val, 0xf, UNICAM_PFT_MASK);
> +	unicam_set_field(&val, 128, UNICAM_OET_MASK);
> +	unicam_reg_write(unicam, UNICAM_CTRL, val);
> +
> +	unicam_reg_write(unicam, UNICAM_IHWIN, 0);
> +	unicam_reg_write(unicam, UNICAM_IVWIN, 0);
> +
> +	/* AXI bus access QoS setup */
> +	val = unicam_reg_read(unicam, UNICAM_PRI);
> +	unicam_set_field(&val, 0, UNICAM_BL_MASK);
> +	unicam_set_field(&val, 0, UNICAM_BS_MASK);
> +	unicam_set_field(&val, 0xe, UNICAM_PP_MASK);
> +	unicam_set_field(&val, 8, UNICAM_NP_MASK);
> +	unicam_set_field(&val, 2, UNICAM_PT_MASK);
> +	unicam_set_field(&val, 1, UNICAM_PE);
> +	unicam_reg_write(unicam, UNICAM_PRI, val);
> +
> +	unicam_reg_write_field(unicam, UNICAM_ANA, 0, UNICAM_DDL);
> +
> +	/* Always start in trigger frame capture mode (UNICAM_FCM set) */
> +	val = UNICAM_FSIE | UNICAM_FEIE | UNICAM_FCM | UNICAM_IBOB;
> +	unicam_set_field(&val, line_int_freq, UNICAM_LCIE_MASK);
> +	unicam_reg_write(unicam, UNICAM_ICTL, val);
> +	unicam_reg_write(unicam, UNICAM_STA, UNICAM_STA_MASK_ALL);
> +	unicam_reg_write(unicam, UNICAM_ISTA, UNICAM_ISTA_MASK_ALL);
> +
> +	/* tclk_term_en */
> +	unicam_reg_write_field(unicam, UNICAM_CLT, 2, UNICAM_CLT1_MASK);
> +	/* tclk_settle */
> +	unicam_reg_write_field(unicam, UNICAM_CLT, 6, UNICAM_CLT2_MASK);
> +	/* td_term_en */
> +	unicam_reg_write_field(unicam, UNICAM_DLT, 2, UNICAM_DLT1_MASK);
> +	/* ths_settle */
> +	unicam_reg_write_field(unicam, UNICAM_DLT, 6, UNICAM_DLT2_MASK);
> +	/* trx_enable */
> +	unicam_reg_write_field(unicam, UNICAM_DLT, 0, UNICAM_DLT3_MASK);
> +
> +	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_SOE);
> +
> +	/* Packet compare setup - required to avoid missing frame ends */
> +	val = 0;
> +	unicam_set_field(&val, 1, UNICAM_PCE);
> +	unicam_set_field(&val, 1, UNICAM_GI);
> +	unicam_set_field(&val, 1, UNICAM_CPH);
> +	unicam_set_field(&val, 0, UNICAM_PCVC_MASK);
> +	unicam_set_field(&val, 1, UNICAM_PCDT_MASK);
> +	unicam_reg_write(unicam, UNICAM_CMP0, val);
> +
> +	/* Enable clock lane and set up terminations */
> +	val = 0;
> +	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> +		/* CSI2 */
> +		unicam_set_field(&val, 1, UNICAM_CLE);
> +		unicam_set_field(&val, 1, UNICAM_CLLPE);
> +		if (!(unicam->bus_flags & V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK)) {
> +			unicam_set_field(&val, 1, UNICAM_CLTRE);
> +			unicam_set_field(&val, 1, UNICAM_CLHSE);
> +		}
> +	} else {
> +		/* CCP2 */
> +		unicam_set_field(&val, 1, UNICAM_CLE);
> +		unicam_set_field(&val, 1, UNICAM_CLHSE);
> +		unicam_set_field(&val, 1, UNICAM_CLTRE);
> +	}
> +	unicam_reg_write(unicam, UNICAM_CLK, val);
> +
> +	/*
> +	 * Enable required data lanes with appropriate terminations.
> +	 * The same value needs to be written to UNICAM_DATn registers for
> +	 * the active lanes, and 0 for inactive ones.
> +	 */
> +	val = 0;
> +	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> +		/* CSI2 */
> +		unicam_set_field(&val, 1, UNICAM_DLE);
> +		unicam_set_field(&val, 1, UNICAM_DLLPE);
> +		if (!(unicam->bus_flags & V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK)) {
> +			unicam_set_field(&val, 1, UNICAM_DLTRE);
> +			unicam_set_field(&val, 1, UNICAM_DLHSE);
> +		}
> +	} else {
> +		/* CCP2 */
> +		unicam_set_field(&val, 1, UNICAM_DLE);
> +		unicam_set_field(&val, 1, UNICAM_DLHSE);
> +		unicam_set_field(&val, 1, UNICAM_DLTRE);
> +	}
> +	unicam_reg_write(unicam, UNICAM_DAT0, val);
> +
> +	if (unicam->active_data_lanes == 1)
> +		val = 0;
> +	unicam_reg_write(unicam, UNICAM_DAT1, val);
> +
> +	if (unicam->max_data_lanes > 2) {
> +		/*
> +		 * Registers UNICAM_DAT2 and UNICAM_DAT3 only valid if the
> +		 * instance supports more than 2 data lanes.
> +		 */
> +		if (unicam->active_data_lanes == 2)
> +			val = 0;
> +		unicam_reg_write(unicam, UNICAM_DAT2, val);
> +
> +		if (unicam->active_data_lanes == 3)
> +			val = 0;
> +		unicam_reg_write(unicam, UNICAM_DAT3, val);
> +	}
> +
> +	unicam_reg_write(unicam, UNICAM_IBLS,
> +			 node->fmt.fmt.pix.bytesperline);
> +	unicam_wr_dma_addr(&unicam->node[UNICAM_IMAGE_NODE], buf);
> +	unicam_set_packing_config(unicam);
> +	unicam_cfg_image_id(unicam);
> +
> +	val = unicam_reg_read(unicam, UNICAM_MISC);
> +	unicam_set_field(&val, 1, UNICAM_FL0);
> +	unicam_set_field(&val, 1, UNICAM_FL1);
> +	unicam_reg_write(unicam, UNICAM_MISC, val);
> +
> +	/* Enable peripheral */
> +	unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPE);
> +
> +	/* Load image pointers */
> +	unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_LIP_MASK);
> +
> +	/*
> +	 * Enable trigger only for the first frame to
> +	 * sync correctly to the FS from the source.
> +	 */
> +	unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> +}
> +
> +static void unicam_start_metadata(struct unicam_device *unicam,
> +				  struct unicam_buffer *buf)
> +{
> +	struct unicam_node *node = &unicam->node[UNICAM_METADATA_NODE];
> +
> +	unicam_enable_ed(unicam);
> +	unicam_wr_dma_addr(node, buf);
> +	unicam_reg_write_field(unicam, UNICAM_DCS, 1, UNICAM_LDP);
> +}
> +
> +static void unicam_disable(struct unicam_device *unicam)
> +{
> +	/* Analogue lane control disable */
> +	unicam_reg_write_field(unicam, UNICAM_ANA, 1, UNICAM_DDL);
> +
> +	/* Stop the output engine */
> +	unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_SOE);
> +
> +	/* Disable the data lanes. */
> +	unicam_reg_write(unicam, UNICAM_DAT0, 0);
> +	unicam_reg_write(unicam, UNICAM_DAT1, 0);
> +
> +	if (unicam->max_data_lanes > 2) {
> +		unicam_reg_write(unicam, UNICAM_DAT2, 0);
> +		unicam_reg_write(unicam, UNICAM_DAT3, 0);
> +	}
> +
> +	/* Peripheral reset */
> +	unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPR);
> +	usleep_range(50, 100);
> +	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPR);
> +
> +	/* Disable peripheral */
> +	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPE);
> +
> +	/* Clear ED setup */
> +	unicam_reg_write(unicam, UNICAM_DCS, 0);
> +
> +	/* Disable all lane clocks */
> +	unicam_clk_write(unicam, 0);
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * V4L2 subdev operations
> + */
> +
> +static int __unicam_subdev_set_routing(struct v4l2_subdev *sd,
> +				       struct v4l2_subdev_state *state,
> +				       struct v4l2_subdev_krouting *routing)
> +{
> +	struct v4l2_subdev_route *route;
> +	int ret;
> +
> +	ret = v4l2_subdev_routing_validate(sd, routing,
> +					   V4L2_SUBDEV_ROUTING_ONLY_1_TO_1);
> +	if (ret)
> +		return ret;
> +
> +	ret = v4l2_subdev_set_routing(sd, state, routing);
> +	if (ret)
> +		return ret;
> +
> +	for_each_active_route(&state->routing, route) {
> +		const struct v4l2_mbus_framefmt *def_fmt;
> +		struct v4l2_mbus_framefmt *fmt;
> +
> +		if (route->source_pad == UNICAM_SD_PAD_SOURCE_IMAGE)
> +			def_fmt = &unicam_default_image_format;
> +		else
> +			def_fmt = &unicam_default_meta_format;
> +
> +		fmt = v4l2_subdev_state_get_format(state, route->sink_pad,
> +						   route->sink_stream);
> +		*fmt = *def_fmt;
> +		fmt = v4l2_subdev_state_get_format(state, route->source_pad,
> +						   route->source_stream);
> +		*fmt = *def_fmt;
> +	}
> +
> +	return 0;
> +}
> +
> +static int unicam_subdev_init_state(struct v4l2_subdev *sd,
> +				    struct v4l2_subdev_state *state)
> +{
> +	struct v4l2_subdev_route routes[] = {
> +		{
> +			.sink_pad = UNICAM_SD_PAD_SINK,
> +			.sink_stream = 0,
> +			.source_pad = UNICAM_SD_PAD_SOURCE_IMAGE,
> +			.source_stream = 0,
> +			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
> +		},
> +	};
> +
> +	struct v4l2_subdev_krouting routing = {
> +		.len_routes = ARRAY_SIZE(routes),
> +		.num_routes = ARRAY_SIZE(routes),
> +		.routes = routes,
> +	};
> +
> +	/* Initialize routing to single route to the fist source pad. */
> +	return __unicam_subdev_set_routing(sd, state, &routing);
> +}
> +
> +static int unicam_subdev_enum_mbus_code(struct v4l2_subdev *sd,
> +					struct v4l2_subdev_state *state,
> +					struct v4l2_subdev_mbus_code_enum *code)
> +{
> +	u32 pad, stream;
> +	int ret;
> +
> +	ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
> +						    code->pad, code->stream,
> +						    &pad, &stream);
> +	if (ret)
> +		return ret;
> +
> +	if (unicam_sd_pad_is_source(code->pad)) {
> +		/* No transcoding, source and sink codes must match. */
> +		const struct v4l2_mbus_framefmt *fmt;
> +
> +		fmt = v4l2_subdev_state_get_format(state, pad, stream);
> +		if (!fmt)
> +			return -EINVAL;
> +
> +		if (code->index > 0)
> +			return -EINVAL;
> +
> +		code->code = fmt->code;
> +	} else {
> +		const struct unicam_format_info *formats;
> +		unsigned int num_formats;
> +
> +		if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> +			formats = unicam_image_formats;
> +			num_formats = ARRAY_SIZE(unicam_image_formats);
> +		} else {
> +			formats = unicam_meta_formats;
> +			num_formats = ARRAY_SIZE(unicam_meta_formats);
> +		}
> +
> +		if (code->index >= num_formats)
> +			return -EINVAL;
> +
> +		code->code = formats[code->index].code;
> +	}
> +
> +	return 0;
> +}
> +
> +static int unicam_subdev_enum_frame_size(struct v4l2_subdev *sd,
> +					 struct v4l2_subdev_state *state,
> +					 struct v4l2_subdev_frame_size_enum *fse)
> +{
> +	u32 pad, stream;
> +	int ret;
> +
> +	if (fse->index > 0)
> +		return -EINVAL;
> +
> +	ret = v4l2_subdev_routing_find_opposite_end(&state->routing, fse->pad,
> +						    fse->stream, &pad,
> +						    &stream);
> +	if (ret)
> +		return ret;
> +
> +	if (unicam_sd_pad_is_source(fse->pad)) {
> +		/* No transcoding, source and sink formats must match. */
> +		const struct v4l2_mbus_framefmt *fmt;
> +
> +		fmt = v4l2_subdev_state_get_format(state, pad, stream);
> +		if (!fmt)
> +			return -EINVAL;
> +
> +		if (fse->code != fmt->code)
> +			return -EINVAL;
> +
> +		fse->min_width = fmt->width;
> +		fse->max_width = fmt->width;
> +		fse->min_height = fmt->height;
> +		fse->max_height = fmt->height;
> +	} else {
> +		const struct unicam_format_info *fmtinfo;
> +
> +		fmtinfo = unicam_find_format_by_code(fse->code, pad);
> +		if (!fmtinfo)
> +			return -EINVAL;
> +
> +		if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> +			fse->min_width = UNICAM_IMAGE_MIN_WIDTH;
> +			fse->max_width = UNICAM_IMAGE_MAX_WIDTH;
> +			fse->min_height = UNICAM_IMAGE_MIN_HEIGHT;
> +			fse->max_height = UNICAM_IMAGE_MAX_HEIGHT;
> +		} else {
> +			fse->min_width = UNICAM_META_MIN_WIDTH;
> +			fse->max_width = UNICAM_META_MAX_WIDTH;
> +			fse->min_height = UNICAM_META_MIN_HEIGHT;
> +			fse->max_height = UNICAM_META_MAX_HEIGHT;
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +static int unicam_subdev_set_format(struct v4l2_subdev *sd,
> +				    struct v4l2_subdev_state *state,
> +				    struct v4l2_subdev_format *format)
> +{
> +	struct unicam_device *unicam = sd_to_unicam_device(sd);
> +	struct v4l2_mbus_framefmt *sink_format, *source_format;
> +	const struct unicam_format_info *fmtinfo;
> +	u32 source_pad, source_stream;
> +	int ret;
> +
> +	if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE &&
> +	    unicam->subdev.streaming)
> +		return -EBUSY;
> +
> +	/* No transcoding, source and sink formats must match. */
> +	if (unicam_sd_pad_is_source(format->pad))
> +		return v4l2_subdev_get_fmt(sd, state, format);
> +
> +	/*
> +	 * Allowed formats for the stream on the sink pad depend on what source
> +	 * pad the stream is routed to. Find the corresponding source pad and
> +	 * use it to validate the media bus code.
> +	 */
> +	ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
> +						    format->pad, format->stream,
> +						    &source_pad, &source_stream);
> +	if (ret)
> +		return ret;
> +
> +	fmtinfo = unicam_find_format_by_code(format->format.code, source_pad);
> +	if (!fmtinfo) {
> +		fmtinfo = source_pad == UNICAM_SD_PAD_SOURCE_IMAGE
> +			? &unicam_image_formats[0] : &unicam_meta_formats[0];
> +		format->format.code = fmtinfo->code;
> +	}
> +
> +	if (source_pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> +		format->format.width = clamp_t(unsigned int,
> +					       format->format.width,
> +					       UNICAM_IMAGE_MIN_WIDTH,
> +					       UNICAM_IMAGE_MAX_WIDTH);
> +		format->format.height = clamp_t(unsigned int,
> +						format->format.height,
> +						UNICAM_IMAGE_MIN_HEIGHT,
> +						UNICAM_IMAGE_MAX_HEIGHT);
> +		format->format.field = V4L2_FIELD_NONE;
> +	} else {
> +		format->format.width = clamp_t(unsigned int,
> +					       format->format.width,
> +					       UNICAM_META_MIN_WIDTH,
> +					       UNICAM_META_MAX_WIDTH);
> +		format->format.height = clamp_t(unsigned int,
> +						format->format.height,
> +						UNICAM_META_MIN_HEIGHT,
> +						UNICAM_META_MAX_HEIGHT);
> +		format->format.field = V4L2_FIELD_NONE;
> +
> +		/* Colorspace don't apply to metadata. */
> +		format->format.colorspace = 0;
> +		format->format.ycbcr_enc = 0;
> +		format->format.quantization = 0;
> +		format->format.xfer_func = 0;
> +	}
> +
> +	sink_format = v4l2_subdev_state_get_format(state, format->pad,
> +						   format->stream);
> +	source_format = v4l2_subdev_state_get_format(state, source_pad,
> +						     source_stream);
> +	*sink_format = format->format;
> +	*source_format = format->format;
> +
> +	return 0;
> +}
> +
> +static int unicam_subdev_set_routing(struct v4l2_subdev *sd,
> +				     struct v4l2_subdev_state *state,
> +				     enum v4l2_subdev_format_whence which,
> +				     struct v4l2_subdev_krouting *routing)
> +{
> +	struct unicam_device *unicam = sd_to_unicam_device(sd);
> +
> +	if (which == V4L2_SUBDEV_FORMAT_ACTIVE && unicam->subdev.streaming)
> +		return -EBUSY;
> +
> +	return __unicam_subdev_set_routing(sd, state, routing);
> +}
> +
> +static int unicam_sd_enable_streams(struct v4l2_subdev *sd,
> +				    struct v4l2_subdev_state *state, u32 pad,
> +				    u64 streams_mask)
> +{
> +	struct unicam_device *unicam = sd_to_unicam_device(sd);
> +	u32 other_pad, other_stream;
> +	int ret;
> +
> +	ret = v4l2_subdev_routing_find_opposite_end(&state->routing, pad, 0,
> +						    &other_pad, &other_stream);
> +	if (ret)
> +		return ret;
> +
> +	unicam->sequence = 0;
> +
> +	ret = v4l2_subdev_enable_streams(unicam->sensor.subdev,
> +					 unicam->sensor.pad->index,
> +					 BIT(other_stream));
> +	if (ret) {
> +		dev_err(unicam->dev, "stream on failed in subdev\n");
> +		return ret;
> +	}
> +
> +	unicam->subdev.streaming = true;
> +
> +	return 0;
> +}
> +
> +static int unicam_sd_disable_streams(struct v4l2_subdev *sd,
> +				     struct v4l2_subdev_state *state, u32 pad,
> +				     u64 streams_mask)
> +{
> +	struct unicam_device *unicam = sd_to_unicam_device(sd);
> +	u32 other_pad, other_stream;
> +	int ret;
> +
> +	ret = v4l2_subdev_routing_find_opposite_end(&state->routing, pad, 0,
> +						    &other_pad, &other_stream);
> +	if (ret)
> +		return ret;
> +
> +	v4l2_subdev_disable_streams(unicam->sensor.subdev,
> +				    unicam->sensor.pad->index,
> +				    BIT(other_stream));
> +
> +	unicam->subdev.streaming = false;
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_subdev_pad_ops unicam_subdev_pad_ops = {
> +	.enum_mbus_code		= unicam_subdev_enum_mbus_code,
> +	.enum_frame_size	= unicam_subdev_enum_frame_size,
> +	.get_fmt		= v4l2_subdev_get_fmt,
> +	.set_fmt		= unicam_subdev_set_format,
> +	.set_routing		= unicam_subdev_set_routing,
> +	.enable_streams		= unicam_sd_enable_streams,
> +	.disable_streams	= unicam_sd_disable_streams,
> +};
> +
> +static const struct v4l2_subdev_ops unicam_subdev_ops = {
> +	.pad			= &unicam_subdev_pad_ops,
> +};
> +
> +static const struct v4l2_subdev_internal_ops unicam_subdev_internal_ops = {
> +	.init_state		= unicam_subdev_init_state,
> +};
> +
> +static const struct media_entity_operations unicam_subdev_media_ops = {
> +	.link_validate		= v4l2_subdev_link_validate,
> +	.has_pad_interdep	= v4l2_subdev_has_pad_interdep,
> +};
> +
> +static int unicam_subdev_init(struct unicam_device *unicam)
> +{
> +	struct v4l2_subdev *sd = &unicam->subdev.sd;
> +	int ret;
> +
> +	v4l2_subdev_init(sd, &unicam_subdev_ops);
> +	sd->internal_ops = &unicam_subdev_internal_ops;
> +	v4l2_set_subdevdata(sd, unicam);
> +
> +	sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> +	sd->entity.ops = &unicam_subdev_media_ops;
> +	sd->dev = unicam->dev;
> +	sd->owner = THIS_MODULE;
> +	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_STREAMS;
> +
> +	strscpy(sd->name, "unicam", sizeof(sd->name));
> +
> +	unicam->subdev.pads[UNICAM_SD_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> +	unicam->subdev.pads[UNICAM_SD_PAD_SOURCE_IMAGE].flags = MEDIA_PAD_FL_SOURCE;
> +	unicam->subdev.pads[UNICAM_SD_PAD_SOURCE_METADATA].flags = MEDIA_PAD_FL_SOURCE;
> +
> +	ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(unicam->subdev.pads),
> +				     unicam->subdev.pads);
> +	if (ret) {
> +		dev_err(unicam->dev, "Failed to initialize media entity: %d\n",
> +			ret);
> +		return ret;
> +	}
> +
> +	ret = v4l2_subdev_init_finalize(sd);
> +	if (ret) {
> +		dev_err(unicam->dev, "Failed to initialize subdev: %d\n", ret);
> +		goto err_entity;
> +	}
> +
> +	ret = v4l2_device_register_subdev(&unicam->v4l2_dev, sd);
> +	if (ret) {
> +		dev_err(unicam->dev, "Failed to register subdev: %d\n", ret);
> +		goto err_subdev;
> +	}
> +
> +	return 0;
> +
> +err_subdev:
> +	v4l2_subdev_cleanup(sd);
> +err_entity:
> +	media_entity_cleanup(&sd->entity);
> +	return ret;
> +}
> +
> +static void unicam_subdev_cleanup(struct unicam_device *unicam)
> +{
> +	v4l2_subdev_cleanup(&unicam->subdev.sd);
> +	media_entity_cleanup(&unicam->subdev.sd.entity);
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * Videobuf2 queue operations
> + */
> +
> +static int unicam_queue_setup(struct vb2_queue *vq,
> +			      unsigned int *nbuffers,
> +			      unsigned int *nplanes,
> +			      unsigned int sizes[],
> +			      struct device *alloc_devs[])
> +{
> +	struct unicam_node *node = vb2_get_drv_priv(vq);
> +	u32 size = is_image_node(node) ? node->fmt.fmt.pix.sizeimage
> +		 : node->fmt.fmt.meta.buffersize;
> +
> +	if (vq->num_buffers + *nbuffers < 3)

Why 3 ? Are these dummies ?

> +		*nbuffers = 3 - vq->num_buffers;
> +
> +	if (*nplanes) {
> +		if (sizes[0] < size) {
> +			dev_dbg(node->dev->dev, "sizes[0] %i < size %u\n",
> +				sizes[0], size);
> +			return -EINVAL;
> +		}
> +		size = sizes[0];
> +	}
> +
> +	*nplanes = 1;
> +	sizes[0] = size;
> +
> +	return 0;
> +}
> +
> +static int unicam_buffer_prepare(struct vb2_buffer *vb)
> +{
> +	struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
> +	struct unicam_buffer *buf = to_unicam_buffer(vb);
> +	u32 size = is_image_node(node) ? node->fmt.fmt.pix.sizeimage
> +		 : node->fmt.fmt.meta.buffersize;
> +
> +	if (vb2_plane_size(vb, 0) < size) {
> +		dev_dbg(node->dev->dev,
> +			"data will not fit into plane (%lu < %u)\n",
> +			vb2_plane_size(vb, 0), size);
> +		return -EINVAL;
> +	}
> +
> +	buf->dma_addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
> +	buf->size = size;
> +
> +	vb2_set_plane_payload(&buf->vb.vb2_buf, 0, size);
> +
> +	return 0;
> +}
> +
> +static void unicam_return_buffers(struct unicam_node *node,
> +				  enum vb2_buffer_state state)
> +{
> +	struct unicam_buffer *buf, *tmp;
> +
> +	list_for_each_entry_safe(buf, tmp, &node->dma_queue, list) {
> +		list_del(&buf->list);
> +		vb2_buffer_done(&buf->vb.vb2_buf, state);
> +	}
> +
> +	if (node->cur_frm)
> +		vb2_buffer_done(&node->cur_frm->vb.vb2_buf,
> +				state);
> +	if (node->next_frm && node->cur_frm != node->next_frm)
> +		vb2_buffer_done(&node->next_frm->vb.vb2_buf,
> +				state);
> +
> +	node->cur_frm = NULL;
> +	node->next_frm = NULL;
> +}
> +
> +static int unicam_start_streaming(struct vb2_queue *vq, unsigned int count)
> +{
> +	struct unicam_node *node = vb2_get_drv_priv(vq);
> +	struct unicam_device *unicam = node->dev;
> +	struct v4l2_subdev_state *state;
> +	struct unicam_buffer *buf;
> +	unsigned long flags;
> +	int ret;
> +	u32 pad, stream;
> +	u32 remote_pad = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
> +					     : UNICAM_SD_PAD_SOURCE_METADATA;
> +
> +	/* Look for the route for the given pad and stream. */
> +	state = v4l2_subdev_lock_and_get_active_state(&unicam->subdev.sd);
> +	ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
> +						    remote_pad, 0,
> +						    &pad, &stream);
> +	v4l2_subdev_unlock_state(state);
> +
> +	if (ret)
> +		goto err_return_buffers;
> +
> +	dev_dbg(unicam->dev, "Starting stream on %s: %u/%u -> %u/%u (%s)\n",
> +		unicam->subdev.sd.name, pad, stream, remote_pad, 0,
> +		is_metadata_node(node) ? "metadata" : "image");
> +
> +	/* The metadata node can't be started alone. */
> +	if (is_metadata_node(node)) {
> +		if (!unicam->node[UNICAM_IMAGE_NODE].streaming) {

Does it mean the metadata node has to be started second, or should
this be made a nop and the metadata node gets started once the image
node is started too ? I'm fine with the constraint of having the
metadata node being started second fwiw

> +			dev_err(unicam->dev,
> +				"Can't start metadata without image\n");
> +			ret = -EINVAL;
> +			goto err_return_buffers;
> +		}
> +
> +		spin_lock_irqsave(&node->dma_queue_lock, flags);
> +		buf = list_first_entry(&node->dma_queue,
> +				       struct unicam_buffer, list);
> +		dev_dbg(unicam->dev, "buffer %p\n", buf);

Is this useful ?

> +		node->cur_frm = buf;
> +		node->next_frm = buf;
> +		list_del(&buf->list);
> +		spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> +
> +		unicam_start_metadata(unicam, buf);
> +		node->streaming = true;
> +		return 0;
> +	}
> +
> +	ret = pm_runtime_resume_and_get(unicam->dev);
> +	if (ret < 0) {
> +		dev_err(unicam->dev, "PM runtime resume failed: %d\n", ret);
> +		goto err_return_buffers;
> +	}
> +
> +	ret = video_device_pipeline_start(&node->video_dev, &unicam->pipe);
> +	if (ret < 0) {
> +		dev_dbg(unicam->dev, "Failed to start media pipeline: %d\n", ret);

Isn't this an err ?

> +		goto err_pm_put;
> +	}
> +
> +	spin_lock_irqsave(&node->dma_queue_lock, flags);
> +	buf = list_first_entry(&node->dma_queue,
> +			       struct unicam_buffer, list);
> +	dev_dbg(unicam->dev, "buffer %p\n", buf);
> +	node->cur_frm = buf;
> +	node->next_frm = buf;
> +	list_del(&buf->list);
> +	spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> +
> +	unicam_start_rx(unicam, buf);
> +
> +	ret = v4l2_subdev_enable_streams(&unicam->subdev.sd, remote_pad, BIT(0));

A bit confused by the API here, shouldn't we also do this for embedded
data ?

> +	if (ret < 0) {
> +		dev_err(unicam->dev, "stream on failed in subdev\n");
> +		goto error_pipeline;
> +	}
> +
> +	node->streaming = true;
> +
> +	return 0;
> +
> +error_pipeline:
> +	video_device_pipeline_stop(&node->video_dev);
> +err_pm_put:
> +	pm_runtime_put_sync(unicam->dev);
> +err_return_buffers:
> +	unicam_return_buffers(node, VB2_BUF_STATE_QUEUED);
> +	return ret;
> +}
> +
> +static void unicam_stop_streaming(struct vb2_queue *vq)
> +{
> +	struct unicam_node *node = vb2_get_drv_priv(vq);
> +	struct unicam_device *unicam = node->dev;
> +	u32 remote_pad_index = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
> +						   : UNICAM_SD_PAD_SOURCE_METADATA;
> +
> +	node->streaming = false;
> +
> +	v4l2_subdev_disable_streams(&unicam->subdev.sd, remote_pad_index,
> +				    BIT(0));
> +
> +	/* We can stream only with the image node. */
> +	if (is_metadata_node(node)) {
> +		/*
> +		 * Allow the hardware to spin in the dummy buffer.
> +		 * This is only really needed if the embedded data pad is
> +		 * disabled before the image pad.
> +		 */
> +		unicam_wr_dma_addr(node, &node->dummy_buf);
> +		goto dequeue_buffers;
> +	}
> +
> +	unicam_disable(unicam);
> +
> +	video_device_pipeline_stop(&node->video_dev);
> +	pm_runtime_put(unicam->dev);
> +
> +dequeue_buffers:
> +	/* Clear all queued buffers for the node */
> +	unicam_return_buffers(node, VB2_BUF_STATE_ERROR);
> +}
> +
> +static void unicam_buffer_queue(struct vb2_buffer *vb)
> +{
> +	struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
> +	struct unicam_buffer *buf = to_unicam_buffer(vb);
> +
> +	spin_lock_irq(&node->dma_queue_lock);
> +	list_add_tail(&buf->list, &node->dma_queue);
> +	spin_unlock_irq(&node->dma_queue_lock);
> +}
> +
> +static const struct vb2_ops unicam_video_qops = {
> +	.queue_setup		= unicam_queue_setup,
> +	.wait_prepare		= vb2_ops_wait_prepare,
> +	.wait_finish		= vb2_ops_wait_finish,
> +	.buf_prepare		= unicam_buffer_prepare,
> +	.start_streaming	= unicam_start_streaming,
> +	.stop_streaming		= unicam_stop_streaming,
> +	.buf_queue		= unicam_buffer_queue,
> +};
> +
> +/* -----------------------------------------------------------------------------
> + *  V4L2 video device operations
> + */
> +
> +static int unicam_querycap(struct file *file, void *priv,
> +			   struct v4l2_capability *cap)
> +{
> +	strscpy(cap->driver, UNICAM_MODULE_NAME, sizeof(cap->driver));
> +	strscpy(cap->card, UNICAM_MODULE_NAME, sizeof(cap->card));
> +
> +	cap->capabilities |= V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_META_CAPTURE;
> +
> +	return 0;
> +}
> +
> +static int unicam_enum_fmt_vid(struct file *file, void  *priv,
> +			       struct v4l2_fmtdesc *f)
> +{
> +	unsigned int index;
> +	unsigned int i;
> +
> +	for (i = 0, index = 0; i < ARRAY_SIZE(unicam_image_formats); i++) {
> +		if (f->mbus_code && unicam_image_formats[i].code != f->mbus_code)
> +			continue;
> +
> +		if (index == f->index) {
> +			f->pixelformat = unicam_image_formats[i].fourcc;
> +			return 0;
> +		}
> +
> +		index++;
> +
> +		if (!unicam_image_formats[i].unpacked_fourcc)
> +			continue;
> +
> +		if (index == f->index) {
> +			f->pixelformat = unicam_image_formats[i].unpacked_fourcc;
> +			return 0;
> +		}
> +
> +		index++;
> +	}
> +
> +	return -EINVAL;
> +}
> +
> +static int unicam_g_fmt_vid(struct file *file, void *priv,
> +			    struct v4l2_format *f)
> +{
> +	struct unicam_node *node = video_drvdata(file);
> +
> +	*f = node->fmt;
> +
> +	return 0;
> +}
> +
> +static const struct unicam_format_info *
> +__unicam_try_fmt_vid(struct unicam_node *node, struct v4l2_pix_format *pix)
> +{
> +	const struct unicam_format_info *fmtinfo;
> +
> +	/*
> +	 * Default to the first format if the requested pixel format code isn't
> +	 * supported.
> +	 */
> +	fmtinfo = unicam_find_format_by_fourcc(pix->pixelformat,
> +					       UNICAM_SD_PAD_SOURCE_IMAGE);
> +	if (!fmtinfo) {
> +		fmtinfo = &unicam_image_formats[0];
> +		pix->pixelformat = fmtinfo->fourcc;
> +	}
> +
> +	unicam_calc_image_size_bpl(node->dev, fmtinfo, pix);
> +
> +	if (pix->field == V4L2_FIELD_ANY)
> +		pix->field = V4L2_FIELD_NONE;
> +
> +	return fmtinfo;
> +}
> +
> +static int unicam_try_fmt_vid(struct file *file, void *priv,
> +			      struct v4l2_format *f)
> +{
> +	struct unicam_node *node = video_drvdata(file);
> +
> +	__unicam_try_fmt_vid(node, &f->fmt.pix);
> +	return 0;
> +}
> +
> +static int unicam_s_fmt_vid(struct file *file, void *priv,
> +			    struct v4l2_format *f)
> +{
> +	struct unicam_node *node = video_drvdata(file);
> +
> +	if (vb2_is_busy(&node->buffer_queue))
> +		return -EBUSY;
> +
> +	node->fmtinfo = __unicam_try_fmt_vid(node, &f->fmt.pix);
> +	node->fmt = *f;
> +
> +	return 0;
> +}
> +
> +static int unicam_enum_fmt_meta(struct file *file, void *priv,
> +				struct v4l2_fmtdesc *f)
> +{
> +	unsigned int i, index;
> +
> +	for (i = 0, index = 0; i < ARRAY_SIZE(unicam_meta_formats); i++) {
> +		if (f->mbus_code && unicam_meta_formats[i].code != f->mbus_code)

Do we want to allow mbus_code filtering for metadata ? There's a
1-to-1 relationship between mbus codes and pixel formats

> +			continue;
> +		if (!unicam_meta_formats[i].metadata_fmt)
> +			continue;

How can this be false if we're iterating on unicam_meta_formats[] ?

> +
> +		if (index == f->index) {
> +			f->pixelformat = unicam_meta_formats[i].fourcc;
> +			f->type = V4L2_BUF_TYPE_META_CAPTURE;
> +			return 0;
> +		}
> +		index++;
> +	}
> +
> +	return -EINVAL;
> +}
> +
> +static int unicam_g_fmt_meta(struct file *file, void *priv,
> +			     struct v4l2_format *f)
> +{
> +	struct unicam_node *node = video_drvdata(file);
> +
> +	f->fmt.meta = node->fmt.fmt.meta;
> +
> +	return 0;
> +}
> +
> +static const struct unicam_format_info *
> +__unicam_try_fmt_meta(struct unicam_node *node, struct v4l2_meta_format *meta)
> +{
> +	const struct unicam_format_info *fmtinfo;
> +
> +	/*
> +	 * Default to the first format if the requested pixel format code isn't
> +	 * supported.
> +	 */
> +	fmtinfo = unicam_find_format_by_fourcc(meta->dataformat,
> +					       UNICAM_SD_PAD_SOURCE_METADATA);
> +	if (!fmtinfo) {
> +		fmtinfo = &unicam_meta_formats[0];
> +		meta->dataformat = fmtinfo->fourcc;
> +	}
> +
> +	unicam_calc_meta_size_bpl(node->dev, fmtinfo, meta);
> +
> +	return fmtinfo;
> +}
> +
> +static int unicam_try_fmt_meta(struct file *file, void *priv,
> +			       struct v4l2_format *f)
> +{
> +	struct unicam_node *node = video_drvdata(file);
> +
> +	__unicam_try_fmt_vid(node, &f->fmt.pix);
> +	return 0;
> +}
> +
> +static int unicam_s_fmt_meta(struct file *file, void *priv,
> +			     struct v4l2_format *f)
> +{
> +	struct unicam_node *node = video_drvdata(file);
> +
> +	if (vb2_is_busy(&node->buffer_queue))
> +		return -EBUSY;
> +
> +	node->fmtinfo = __unicam_try_fmt_meta(node, &f->fmt.meta);
> +	node->fmt = *f;
> +
> +	return 0;
> +}
> +
> +static int unicam_enum_framesizes(struct file *file, void *fh,
> +				  struct v4l2_frmsizeenum *fsize)
> +{
> +	struct unicam_node *node = video_drvdata(file);
> +	int ret = -EINVAL;
> +
> +	if (fsize->index > 0)
> +		return ret;
> +
> +	if (is_image_node(node)) {
> +		if (!unicam_find_format_by_fourcc(fsize->pixel_format,
> +						  UNICAM_SD_PAD_SOURCE_IMAGE))
> +			return ret;
> +
> +		fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> +		fsize->stepwise.min_width = UNICAM_IMAGE_MIN_WIDTH;
> +		fsize->stepwise.max_width = UNICAM_IMAGE_MAX_WIDTH;
> +		fsize->stepwise.step_width = 1;
> +		fsize->stepwise.min_height = UNICAM_IMAGE_MIN_HEIGHT;
> +		fsize->stepwise.max_height = UNICAM_IMAGE_MAX_HEIGHT;
> +		fsize->stepwise.step_height = 1;
> +	} else {
> +		if (!unicam_find_format_by_fourcc(fsize->pixel_format,
> +						  UNICAM_SD_PAD_SOURCE_METADATA))
> +			return ret;
> +
Isn't this V4L2_FRMSIZE_TYPE_STEPWISE as well ?

> +		fsize->stepwise.min_width = UNICAM_META_MIN_WIDTH;
> +		fsize->stepwise.max_width = UNICAM_META_MAX_WIDTH;
> +		fsize->stepwise.step_width = 1;
> +		fsize->stepwise.min_height = UNICAM_META_MIN_HEIGHT;
> +		fsize->stepwise.max_height = UNICAM_META_MAX_HEIGHT;
> +		fsize->stepwise.step_height = 1;
> +	}
> +
> +	return 0;
> +}
> +
> +static int unicam_log_status(struct file *file, void *fh)
> +{
> +	struct unicam_node *node = video_drvdata(file);
> +	struct unicam_device *unicam = node->dev;
> +	u32 reg;
> +
> +	/* status for sub devices */
> +	v4l2_device_call_all(&unicam->v4l2_dev, 0, core, log_status);
> +
> +	dev_info(unicam->dev, "-----Receiver status-----\n");
> +	dev_info(unicam->dev, "V4L2 width/height:   %ux%u\n",
> +		 node->fmt.fmt.pix.width, node->fmt.fmt.pix.height);
> +	dev_info(unicam->dev, "Mediabus format:     %08x\n",
> +		 node->fmtinfo->code);
> +	dev_info(unicam->dev, "V4L2 format:         %08x\n",
> +		 node->fmt.fmt.pix.pixelformat);
> +	reg = unicam_reg_read(unicam, UNICAM_IPIPE);
> +	dev_info(unicam->dev, "Unpacking/packing:   %u / %u\n",
> +		 unicam_get_field(reg, UNICAM_PUM_MASK),
> +		 unicam_get_field(reg, UNICAM_PPM_MASK));
> +	dev_info(unicam->dev, "----Live data----\n");
> +	dev_info(unicam->dev, "Programmed stride:   %4u\n",
> +		 unicam_reg_read(unicam, UNICAM_IBLS));
> +	dev_info(unicam->dev, "Detected resolution: %ux%u\n",
> +		 unicam_reg_read(unicam, UNICAM_IHSTA),
> +		 unicam_reg_read(unicam, UNICAM_IVSTA));
> +	dev_info(unicam->dev, "Write pointer:       %08x\n",
> +		 unicam_reg_read(unicam, UNICAM_IBWP));
> +
> +	return 0;
> +}
> +
> +static int unicam_subscribe_event(struct v4l2_fh *fh,
> +				  const struct v4l2_event_subscription *sub)
> +{
> +	switch (sub->type) {
> +	case V4L2_EVENT_FRAME_SYNC:
> +		return v4l2_event_subscribe(fh, sub, 2, NULL);
> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +static const struct v4l2_ioctl_ops unicam_ioctl_ops = {
> +	.vidioc_querycap		= unicam_querycap,
> +
> +	.vidioc_enum_fmt_vid_cap	= unicam_enum_fmt_vid,
> +	.vidioc_g_fmt_vid_cap		= unicam_g_fmt_vid,
> +	.vidioc_try_fmt_vid_cap		= unicam_try_fmt_vid,
> +	.vidioc_s_fmt_vid_cap		= unicam_s_fmt_vid,
> +
> +	.vidioc_enum_fmt_meta_cap	= unicam_enum_fmt_meta,
> +	.vidioc_g_fmt_meta_cap		= unicam_g_fmt_meta,
> +	.vidioc_try_fmt_meta_cap	= unicam_try_fmt_meta,
> +	.vidioc_s_fmt_meta_cap		= unicam_s_fmt_meta,
> +
> +	.vidioc_enum_framesizes		= unicam_enum_framesizes,
> +
> +	.vidioc_reqbufs			= vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs		= vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf		= vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf		= vb2_ioctl_querybuf,
> +	.vidioc_qbuf			= vb2_ioctl_qbuf,
> +	.vidioc_dqbuf			= vb2_ioctl_dqbuf,
> +	.vidioc_expbuf			= vb2_ioctl_expbuf,
> +	.vidioc_streamon		= vb2_ioctl_streamon,
> +	.vidioc_streamoff		= vb2_ioctl_streamoff,
> +
> +	.vidioc_log_status		= unicam_log_status,
> +	.vidioc_subscribe_event		= unicam_subscribe_event,
> +	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
> +};
> +
> +/* unicam capture driver file operations */
> +static const struct v4l2_file_operations unicam_fops = {
> +	.owner		= THIS_MODULE,
> +	.open           = v4l2_fh_open,
> +	.release        = vb2_fop_release,
> +	.poll		= vb2_fop_poll,
> +	.unlocked_ioctl = video_ioctl2,
> +	.mmap           = vb2_fop_mmap,
> +};
> +
> +static int unicam_video_link_validate(struct media_link *link)
> +{
> +	struct video_device *vdev =
> +		media_entity_to_video_device(link->sink->entity);
> +	struct v4l2_subdev *sd =
> +		media_entity_to_v4l2_subdev(link->source->entity);
> +	struct unicam_node *node = video_get_drvdata(vdev);
> +	const u32 pad = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
> +		      : UNICAM_SD_PAD_SOURCE_METADATA;
> +	const struct v4l2_mbus_framefmt *format;
> +	struct v4l2_subdev_state *state;
> +	int ret = 0;
> +
> +	state = v4l2_subdev_lock_and_get_active_state(sd);
> +
> +	format = v4l2_subdev_state_get_format(state, pad, 0);
> +	if (!format) {
> +		ret = -EINVAL;
> +		goto out;
> +	}
> +
> +	if (is_image_node(node)) {
> +		const struct v4l2_pix_format *fmt = &node->fmt.fmt.pix;
> +
> +		if (node->fmtinfo->code != format->code ||
> +		    fmt->height != format->height ||
> +		    fmt->width != format->width ||
> +		    fmt->field != format->field) {
> +			dev_dbg(node->dev->dev,
> +				"image: (%u x %u) 0x%08x %s != (%u x %u) 0x%08x %s\n",
> +				fmt->width, fmt->height, node->fmtinfo->code,
> +				v4l2_field_names[fmt->field],
> +				format->width, format->height, format->code,
> +				v4l2_field_names[format->field]);
> +			ret = -EPIPE;
> +		};

No need for ; here and in the next loop

> +	} else {
> +		const struct v4l2_meta_format *fmt = &node->fmt.fmt.meta;
> +
> +		if (node->fmtinfo->code != format->code ||
> +		    fmt->height != format->height ||
> +		    fmt->width != format->width) {
> +			dev_dbg(node->dev->dev,
> +				"meta: (%u x %u) 0x%04x != (%u x %u) 0x%04x\n",
> +				fmt->width, fmt->height, node->fmtinfo->code,
> +				format->width, format->height, format->code);
> +			ret = -EPIPE;
> +		};
> +	}
> +
> +out:
> +	v4l2_subdev_unlock_state(state);
> +	return ret;
> +}
> +
> +static const struct media_entity_operations unicam_video_media_ops = {
> +	.link_validate = unicam_video_link_validate,
> +};
> +
> +static void unicam_node_release(struct video_device *vdev)
> +{
> +	struct unicam_node *node = video_get_drvdata(vdev);
> +
> +	unicam_put(node->dev);
> +}
> +
> +static void unicam_set_default_format(struct unicam_node *node)
> +{
> +	if (is_image_node(node)) {
> +		struct v4l2_pix_format *fmt = &node->fmt.fmt.pix;
> +
> +		node->fmtinfo = &unicam_image_formats[0];
> +		node->fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> +
> +		v4l2_fill_pix_format(fmt, &unicam_default_image_format);
> +		fmt->pixelformat = node->fmtinfo->fourcc;
> +		unicam_calc_image_size_bpl(node->dev, node->fmtinfo, fmt);
> +	} else {
> +		struct v4l2_meta_format *fmt = &node->fmt.fmt.meta;
> +
> +		node->fmtinfo = &unicam_meta_formats[0];
> +		node->fmt.type = V4L2_BUF_TYPE_META_CAPTURE;
> +
> +		fmt->dataformat = node->fmtinfo->fourcc;
> +		fmt->width = unicam_default_meta_format.width;
> +		fmt->height = unicam_default_meta_format.height;
> +		unicam_calc_meta_size_bpl(node->dev, node->fmtinfo, fmt);
> +	}
> +}
> +
> +static int unicam_register_node(struct unicam_device *unicam,
> +				enum unicam_node_type type)
> +{
> +	const u32 pad_index = type == UNICAM_IMAGE_NODE
> +			    ? UNICAM_SD_PAD_SOURCE_IMAGE
> +			    : UNICAM_SD_PAD_SOURCE_METADATA;
> +	struct unicam_node *node = &unicam->node[type];
> +	struct video_device *vdev = &node->video_dev;
> +	struct vb2_queue *q = &node->buffer_queue;
> +	int ret;
> +
> +	node->dev = unicam_get(unicam);
> +	node->id = type;
> +
> +	spin_lock_init(&node->dma_queue_lock);
> +	mutex_init(&node->lock);
> +
> +	INIT_LIST_HEAD(&node->dma_queue);
> +
> +	/* Initialize the videobuf2 queue. */
> +	q->type = type == UNICAM_IMAGE_NODE ? V4L2_BUF_TYPE_VIDEO_CAPTURE
> +					    : V4L2_BUF_TYPE_META_CAPTURE;
> +	q->io_modes = VB2_MMAP | VB2_DMABUF;
> +	q->drv_priv = node;
> +	q->ops = &unicam_video_qops;
> +	q->mem_ops = &vb2_dma_contig_memops;
> +	q->buf_struct_size = sizeof(struct unicam_buffer);
> +	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> +	q->lock = &node->lock;
> +	q->min_queued_buffers = 1;
> +	q->dev = unicam->dev;
> +
> +	ret = vb2_queue_init(q);
> +	if (ret) {
> +		dev_err(unicam->dev, "vb2_queue_init() failed\n");
> +		goto err_unicam_put;
> +	}
> +
> +	/* Initialize the video device. */
> +	vdev->release = unicam_node_release;
> +	vdev->fops = &unicam_fops;
> +	vdev->ioctl_ops = &unicam_ioctl_ops;
> +	vdev->v4l2_dev = &unicam->v4l2_dev;
> +	vdev->vfl_dir = VFL_DIR_RX;
> +	vdev->queue = q;
> +	vdev->lock = &node->lock;
> +	vdev->device_caps = type == UNICAM_IMAGE_NODE
> +			  ? V4L2_CAP_VIDEO_CAPTURE : V4L2_CAP_META_CAPTURE;
> +	vdev->device_caps |= V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;
> +	vdev->entity.ops = &unicam_video_media_ops;
> +
> +	snprintf(vdev->name, sizeof(vdev->name), "%s-%s", UNICAM_MODULE_NAME,
> +		 type == UNICAM_IMAGE_NODE ? "image" : "embedded");
> +
> +	video_set_drvdata(vdev, node);
> +
> +	if (type == UNICAM_IMAGE_NODE)
> +		vdev->entity.flags |= MEDIA_ENT_FL_DEFAULT;
> +
> +	node->pad.flags = MEDIA_PAD_FL_SINK;
> +
> +	ret = media_entity_pads_init(&vdev->entity, 1, &node->pad);
> +	if (ret)
> +		goto err_unicam_put;
> +
> +	node->dummy_buf.size = UNICAM_DUMMY_BUF_SIZE;
> +	node->dummy_buf_cpu_addr = dma_alloc_coherent(unicam->dev,
> +						      node->dummy_buf.size,
> +						      &node->dummy_buf.dma_addr,
> +						      GFP_KERNEL);
> +	if (!node->dummy_buf_cpu_addr) {
> +		dev_err(unicam->dev, "Unable to allocate dummy buffer.\n");
> +		ret = -ENOMEM;
> +		goto err_entity_cleanup;
> +	}
> +
> +	unicam_set_default_format(node);
> +
> +	ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> +	if (ret) {
> +		dev_err(unicam->dev, "Unable to register video device %s\n",
> +			vdev->name);
> +		goto err_dma_free;
> +	}
> +
> +	node->registered = true;
> +
> +	ret = media_create_pad_link(&unicam->subdev.sd.entity,
> +				    pad_index,
> +				    &node->video_dev.entity,
> +				    0,
> +				    MEDIA_LNK_FL_ENABLED |
> +				    MEDIA_LNK_FL_IMMUTABLE);
> +	if (ret) {
> +		/*
> +		 * No need for cleanup, the caller will unregister the
> +		 * video device, which will drop the reference on the
> +		 * device and trigger the cleanup.
> +		 */
> +		dev_err(unicam->dev, "Unable to create pad link for %s\n",
> +			unicam->sensor.subdev->name);
> +		return ret;
> +	}
> +
> +	return 0;
> +
> +err_dma_free:
> +	dma_free_coherent(unicam->dev, node->dummy_buf.size,
> +			  node->dummy_buf_cpu_addr,
> +			  node->dummy_buf.dma_addr);
> +err_entity_cleanup:
> +	media_entity_cleanup(&vdev->entity);
> +err_unicam_put:
> +	unicam_put(unicam);
> +	return ret;
> +}
> +
> +static void unicam_unregister_nodes(struct unicam_device *unicam)
> +{
> +	unsigned int i;
> +
> +	for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> +		struct unicam_node *node = &unicam->node[i];
> +
> +		if (node->dummy_buf_cpu_addr)
> +			dma_free_coherent(unicam->dev, node->dummy_buf.size,
> +					  node->dummy_buf_cpu_addr,
> +					  node->dummy_buf.dma_addr);
> +
> +		if (node->registered) {
> +			video_unregister_device(&node->video_dev);
> +			node->registered = false;
> +		}
> +	}
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * Power management
> + */
> +
> +static int unicam_runtime_resume(struct device *dev)
> +{
> +	struct unicam_device *unicam = dev_get_drvdata(dev);
> +	int ret;
> +
> +	ret = clk_set_min_rate(unicam->vpu_clock, UNICAM_MIN_VPU_CLOCK_RATE);
> +	if (ret) {
> +		dev_err(unicam->dev, "failed to set up VPU clock\n");
> +		return ret;
> +	}

Same question as above, shouldn't this be an 'assigned-clock-rates in
dts ?

> +
> +	ret = clk_prepare_enable(unicam->vpu_clock);
> +	if (ret) {
> +		dev_err(unicam->dev, "Failed to enable VPU clock: %d\n", ret);
> +		goto err_vpu_clock;
> +	}
> +
> +	ret = clk_set_rate(unicam->clock, 100 * 1000 * 1000);
> +	if (ret) {
> +		dev_err(unicam->dev, "failed to set up CSI clock\n");
> +		goto err_vpu_prepare;
> +	}

same question

> +
> +	ret = clk_prepare_enable(unicam->clock);
> +	if (ret) {
> +		dev_err(unicam->dev, "Failed to enable CSI clock: %d\n", ret);
> +		goto err_vpu_prepare;

Do you need to 'unset' the rate as done for the VPU clock ?

> +	}
> +
> +	return 0;
> +
> +err_vpu_prepare:
> +	clk_disable_unprepare(unicam->vpu_clock);
> +err_vpu_clock:
> +	if (clk_set_min_rate(unicam->vpu_clock, 0))
> +		dev_err(unicam->dev, "failed to reset the VPU clock\n");
> +
> +	return ret;
> +}
> +
> +static int unicam_runtime_suspend(struct device *dev)
> +{
> +	struct unicam_device *unicam = dev_get_drvdata(dev);
> +
> +	clk_disable_unprepare(unicam->clock);

(continuing  with the above question: probably not, as the clock is
not 'unset' neither here)

The rest looks good, nice to see this progressing!

Thanks
  j

> +
> +	if (clk_set_min_rate(unicam->vpu_clock, 0))
> +		dev_err(unicam->dev, "failed to reset the VPU clock\n");
> +
> +	clk_disable_unprepare(unicam->vpu_clock);
> +
> +	return 0;
> +}
> +
> +static const struct dev_pm_ops unicam_pm_ops = {
> +	RUNTIME_PM_OPS(unicam_runtime_suspend, unicam_runtime_resume, NULL)
> +};
> +
> +/* -----------------------------------------------------------------------------
> + * V4L2 async notifier
> + */
> +
> +static int unicam_async_bound(struct v4l2_async_notifier *notifier,
> +			      struct v4l2_subdev *subdev,
> +			      struct v4l2_async_connection *asc)
> +{
> +	struct unicam_device *unicam = notifier_to_unicam_device(notifier);
> +	struct media_pad *sink = &unicam->subdev.pads[UNICAM_SD_PAD_SINK];
> +	struct media_pad *source;
> +	int ret;
> +
> +	dev_dbg(unicam->dev, "Using sensor %s for capture\n",
> +		subdev->name);
> +
> +	ret = v4l2_create_fwnode_links_to_pad(subdev, sink, MEDIA_LNK_FL_ENABLED |
> +					      MEDIA_LNK_FL_IMMUTABLE);
> +	if (ret)
> +		return ret;
> +
> +	source = media_pad_remote_pad_unique(sink);
> +	if (!source) {
> +		dev_err(unicam->dev, "No connected sensor pad\n");
> +		return -ENOTCONN;
> +	}
> +
> +	unicam->sensor.subdev = subdev;
> +	unicam->sensor.pad = source;
> +
> +	return 0;
> +}
> +
> +static int unicam_async_complete(struct v4l2_async_notifier *notifier)
> +{
> +	struct unicam_device *unicam = notifier_to_unicam_device(notifier);
> +	int ret;
> +
> +	ret = unicam_register_node(unicam, UNICAM_IMAGE_NODE);
> +	if (ret) {
> +		dev_err(unicam->dev, "Unable to register image video device.\n");
> +		goto unregister;
> +	}
> +
> +	ret = unicam_register_node(unicam, UNICAM_METADATA_NODE);
> +	if (ret) {
> +		dev_err(unicam->dev, "Unable to register metadata video device.\n");
> +		goto unregister;
> +	}
> +
> +	ret = v4l2_device_register_subdev_nodes(&unicam->v4l2_dev);
> +	if (ret) {
> +		dev_err(unicam->dev, "Unable to register subdev nodes.\n");
> +		goto unregister;
> +	}
> +
> +	return 0;
> +
> +unregister:
> +	unicam_unregister_nodes(unicam);
> +	unicam_put(unicam);
> +
> +	return ret;
> +}
> +
> +static const struct v4l2_async_notifier_operations unicam_async_ops = {
> +	.bound = unicam_async_bound,
> +	.complete = unicam_async_complete,
> +};
> +
> +static int unicam_async_nf_init(struct unicam_device *unicam)
> +{
> +	struct v4l2_fwnode_endpoint ep = { };
> +	struct fwnode_handle *ep_handle;
> +	struct v4l2_async_connection *asc;
> +	int ret;
> +
> +	ret = of_property_read_u32(unicam->dev->of_node, "brcm,num-data-lanes",
> +				   &unicam->max_data_lanes);
> +	if (ret < 0) {
> +		dev_err(unicam->dev, "Missing %s DT property\n",
> +			"brcm,num-data-lanes");
> +		return -EINVAL;
> +	}
> +
> +	/* Get and parse the local endpoint. */
> +	ep_handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(unicam->dev), 0, 0,
> +						    FWNODE_GRAPH_ENDPOINT_NEXT);
> +	if (!ep_handle) {
> +		dev_err(unicam->dev, "No endpoint found\n");
> +		return -ENODEV;
> +	}
> +
> +	ret = v4l2_fwnode_endpoint_parse(ep_handle, &ep);
> +	if (ret) {
> +		dev_err(unicam->dev, "Failed to parse endpoint: %d\n", ret);
> +		goto error;
> +	}
> +
> +	unicam->bus_type = ep.bus_type;
> +
> +	switch (ep.bus_type) {
> +	case V4L2_MBUS_CSI2_DPHY: {
> +		unsigned int num_data_lanes = ep.bus.mipi_csi2.num_data_lanes;
> +
> +		if (num_data_lanes != 1 && num_data_lanes != 2 &&
> +		    num_data_lanes != 4) {
> +			dev_err(unicam->dev, "%u data lanes not supported\n",
> +				num_data_lanes);
> +			goto error;
> +		}
> +
> +		if (num_data_lanes > unicam->max_data_lanes) {
> +			dev_err(unicam->dev,
> +				"Endpoint uses %u data lanes when %u are supported\n",
> +				num_data_lanes, unicam->max_data_lanes);
> +			goto error;
> +		}
> +
> +		unicam->active_data_lanes = num_data_lanes;
> +		unicam->bus_flags = ep.bus.mipi_csi2.flags;
> +		break;
> +	}
> +
> +	case V4L2_MBUS_CCP2:
> +		unicam->max_data_lanes = 1;
> +		unicam->active_data_lanes = 1;
> +		unicam->bus_flags = ep.bus.mipi_csi1.strobe;
> +		break;
> +
> +	default:
> +		/* Unsupported bus type */
> +		dev_err(unicam->dev, "Unsupported bus type %u\n", ep.bus_type);
> +		goto error;
> +	}
> +
> +	/* Initialize and register the async notifier. */
> +	v4l2_async_nf_init(&unicam->notifier, &unicam->v4l2_dev);
> +
> +	asc = v4l2_async_nf_add_fwnode_remote(&unicam->notifier, ep_handle,
> +					      struct v4l2_async_connection);
> +	fwnode_handle_put(ep_handle);
> +	ep_handle = NULL;
> +
> +	if (IS_ERR(asc)) {
> +		ret = PTR_ERR(asc);
> +		dev_err(unicam->dev, "Failed to add entry to notifier: %d\n",
> +			ret);
> +		goto error;
> +	}
> +
> +	unicam->notifier.ops = &unicam_async_ops;
> +
> +	ret = v4l2_async_nf_register(&unicam->notifier);
> +	if (ret) {
> +		dev_err(unicam->dev, "Error registering device notifier: %d\n",
> +			ret);
> +		goto error;
> +	}
> +
> +	return 0;
> +
> +error:
> +	fwnode_handle_put(ep_handle);
> +	return ret;
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * Probe & remove
> + */
> +
> +static int unicam_media_init(struct unicam_device *unicam)
> +{
> +	int ret;
> +
> +	unicam->mdev.dev = unicam->dev;
> +	strscpy(unicam->mdev.model, UNICAM_MODULE_NAME,
> +		sizeof(unicam->mdev.model));
> +	strscpy(unicam->mdev.serial, "", sizeof(unicam->mdev.serial));
> +	unicam->mdev.hw_revision = 0;
> +
> +	media_device_init(&unicam->mdev);
> +
> +	unicam->v4l2_dev.mdev = &unicam->mdev;
> +
> +	ret = v4l2_device_register(unicam->dev, &unicam->v4l2_dev);
> +	if (ret < 0) {
> +		dev_err(unicam->dev, "Unable to register v4l2 device\n");
> +		goto err_media_cleanup;
> +	}
> +
> +	ret = media_device_register(&unicam->mdev);
> +	if (ret < 0) {
> +		dev_err(unicam->dev,
> +			"Unable to register media-controller device\n");
> +		goto err_v4l2_unregister;
> +	}
> +
> +	return 0;
> +
> +err_v4l2_unregister:
> +	v4l2_device_unregister(&unicam->v4l2_dev);
> +err_media_cleanup:
> +	media_device_cleanup(&unicam->mdev);
> +	return ret;
> +}
> +
> +static int unicam_probe(struct platform_device *pdev)
> +{
> +	struct unicam_device *unicam;
> +	int ret;
> +
> +	unicam = kzalloc(sizeof(*unicam), GFP_KERNEL);
> +	if (!unicam)
> +		return -ENOMEM;
> +
> +	kref_init(&unicam->kref);
> +	unicam->dev = &pdev->dev;
> +	platform_set_drvdata(pdev, unicam);
> +
> +	unicam->base = devm_platform_ioremap_resource_byname(pdev, "unicam");
> +	if (IS_ERR(unicam->base)) {
> +		ret = PTR_ERR(unicam->base);
> +		goto err_unicam_put;
> +	}
> +
> +	unicam->clk_gate_base = devm_platform_ioremap_resource_byname(pdev, "cmi");
> +	if (IS_ERR(unicam->clk_gate_base)) {
> +		ret = PTR_ERR(unicam->clk_gate_base);
> +		goto err_unicam_put;
> +	}
> +
> +	unicam->clock = devm_clk_get(&pdev->dev, "lp");
> +	if (IS_ERR(unicam->clock)) {
> +		dev_err(unicam->dev, "Failed to get lp clock\n");
> +		ret = PTR_ERR(unicam->clock);
> +		goto err_unicam_put;
> +	}
> +
> +	unicam->vpu_clock = devm_clk_get(&pdev->dev, "vpu");
> +	if (IS_ERR(unicam->vpu_clock)) {
> +		dev_err(unicam->dev, "Failed to get vpu clock\n");
> +		ret = PTR_ERR(unicam->vpu_clock);
> +		goto err_unicam_put;
> +	}
> +
> +	ret = platform_get_irq(pdev, 0);
> +	if (ret <= 0) {
> +		dev_err(&pdev->dev, "No IRQ resource\n");
> +		ret = -EINVAL;
> +		goto err_unicam_put;
> +	}
> +
> +	ret = devm_request_irq(&pdev->dev, ret, unicam_isr, 0,
> +			       "unicam_capture0", unicam);
> +	if (ret) {
> +		dev_err(&pdev->dev, "Unable to request interrupt\n");
> +		ret = -EINVAL;
> +		goto err_unicam_put;
> +	}
> +
> +	/* Enable the block power domain. */
> +	pm_runtime_enable(&pdev->dev);
> +
> +	ret = unicam_media_init(unicam);
> +	if (ret)
> +		goto err_pm_runtime;
> +
> +	ret = unicam_subdev_init(unicam);
> +	if (ret)
> +		goto err_media_unregister;
> +
> +	ret = unicam_async_nf_init(unicam);
> +	if (ret)
> +		goto err_subdev_unregister;
> +
> +	return 0;
> +
> +err_subdev_unregister:
> +	unicam_subdev_cleanup(unicam);
> +err_media_unregister:
> +	media_device_unregister(&unicam->mdev);
> +err_pm_runtime:
> +	pm_runtime_disable(&pdev->dev);
> +err_unicam_put:
> +	unicam_put(unicam);
> +
> +	return ret;
> +}
> +
> +static int unicam_remove(struct platform_device *pdev)
> +{
> +	struct unicam_device *unicam = platform_get_drvdata(pdev);
> +
> +	unicam_unregister_nodes(unicam);
> +	v4l2_device_unregister(&unicam->v4l2_dev);
> +	media_device_unregister(&unicam->mdev);
> +	v4l2_async_nf_unregister(&unicam->notifier);
> +
> +	unicam_subdev_cleanup(unicam);
> +
> +	unicam_put(unicam);
> +
> +	pm_runtime_disable(&pdev->dev);
> +
> +	return 0;
> +}
> +
> +static const struct of_device_id unicam_of_match[] = {
> +	{ .compatible = "brcm,bcm2835-unicam", },
> +	{ /* sentinel */ },
> +};
> +MODULE_DEVICE_TABLE(of, unicam_of_match);
> +
> +static struct platform_driver unicam_driver = {
> +	.probe		= unicam_probe,
> +	.remove		= unicam_remove,
> +	.driver = {
> +		.name	= UNICAM_MODULE_NAME,
> +		.pm	= pm_ptr(&unicam_pm_ops),
> +		.of_match_table = of_match_ptr(unicam_of_match),
> +	},
> +};
> +
> +module_platform_driver(unicam_driver);
> +
> +MODULE_AUTHOR("Dave Stevenson <dave.stevenson@raspberrypi.com>");
> +MODULE_DESCRIPTION("BCM2835 Unicam driver");
> +MODULE_LICENSE("GPL");
> --
> Regards,
>
> Laurent Pinchart
>
>

Re: [PATCH v6 05/15] media: i2c: imx219: Add embedded data support

[Laurent Pinchart]

Hi Jacopo,

On Mon, Mar 04, 2024 at 10:47:19AM +0100, Jacopo Mondi wrote:
> On Fri, Mar 01, 2024 at 11:32:20PM +0200, Laurent Pinchart wrote:
> > The IMX219 generates embedded data unconditionally. Report it as an
> > additional stream, with a new internal embedded data pad, and update
> > subdev operations accordingly.
> >
> > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > ---
> >  drivers/media/i2c/imx219.c | 173 ++++++++++++++++++++++++++++++++-----
> >  1 file changed, 151 insertions(+), 22 deletions(-)
> >
> > diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> > index 6e0232b6772b..cfbf4c304a9a 100644
> > --- a/drivers/media/i2c/imx219.c
> > +++ b/drivers/media/i2c/imx219.c
> > @@ -149,6 +149,9 @@
> >  #define IMX219_PIXEL_ARRAY_WIDTH	3280U
> >  #define IMX219_PIXEL_ARRAY_HEIGHT	2464U
> >
> > +/* Embedded metadata stream height */
> > +#define IMX219_EMBEDDED_DATA_HEIGHT	2U
> > +
> >  /* Mode : resolution and related config&values */
> >  struct imx219_mode {
> >  	/* Frame width */
> > @@ -317,9 +320,15 @@ static const struct imx219_mode supported_modes[] = {
> >  enum imx219_pad_ids {
> >  	IMX219_PAD_SOURCE,
> >  	IMX219_PAD_IMAGE,
> > +	IMX219_PAD_EDATA,
> >  	IMX219_NUM_PADS,
> >  };
> >
> > +enum imx219_stream_ids {
> > +	IMX219_STREAM_IMAGE,
> > +	IMX219_STREAM_EDATA,
> > +};
> > +
> >  struct imx219 {
> >  	struct v4l2_subdev sd;
> >  	struct media_pad pads[IMX219_NUM_PADS];
> > @@ -613,6 +622,25 @@ static int imx219_format_bpp(u32 code)
> >  	}
> >  }
> >
> > +/* Return the embedded data format corresponding to an image format. */
> > +static u32 imx219_format_edata(u32 code)
> > +{
> > +	switch (code) {
> > +	case MEDIA_BUS_FMT_SRGGB8_1X8:
> > +	case MEDIA_BUS_FMT_SGRBG8_1X8:
> > +	case MEDIA_BUS_FMT_SGBRG8_1X8:
> > +	case MEDIA_BUS_FMT_SBGGR8_1X8:
> > +		return MEDIA_BUS_FMT_META_8;
> > +
> > +	case MEDIA_BUS_FMT_SRGGB10_1X10:
> > +	case MEDIA_BUS_FMT_SGRBG10_1X10:
> > +	case MEDIA_BUS_FMT_SGBRG10_1X10:
> > +	case MEDIA_BUS_FMT_SBGGR10_1X10:
> > +	default:
> > +		return MEDIA_BUS_FMT_META_10;
> > +	}
> > +}
> > +
> >  static int imx219_set_framefmt(struct imx219 *imx219,
> >  			       struct v4l2_subdev_state *state)
> >  {
> > @@ -622,7 +650,8 @@ static int imx219_set_framefmt(struct imx219 *imx219,
> >  	u64 bin_h, bin_v;
> >  	int ret = 0;
> >
> > -	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
> > +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
> > +					      IMX219_STREAM_IMAGE);
> >  	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_IMAGE);
> >  	bpp = imx219_format_bpp(format->code);
> >
> > @@ -777,17 +806,33 @@ static int imx219_enum_mbus_code(struct v4l2_subdev *sd,
> >  {
> >  	struct imx219 *imx219 = to_imx219(sd);
> >
> > -	if (code->pad == IMX219_PAD_IMAGE) {
> > +	switch (code->pad) {
> > +	case IMX219_PAD_IMAGE:
> >  		/* The internal image pad is hardwired to the native format. */
> > -		if (code->index)
> > +		if (code->index > 0)
> >  			return -EINVAL;
> >
> >  		code->code = IMX219_NATIVE_FORMAT;
> > -	} else {
> > -		/*
> > -		 * On the source pad, the sensor supports multiple raw formats
> > -		 * with different bit depths.
> > -		 */
> > +		return 0;
> > +
> > +	case IMX219_PAD_EDATA:
> > +		if (code->index > 0)
> > +			return -EINVAL;
> > +
> > +		code->code = MEDIA_BUS_FMT_CCS_EMBEDDED;
> > +		return 0;
> > +
> > +	case IMX219_PAD_SOURCE:
> > +	default:
> > +		break;
> 
> Do you need these ?

Not strictly speaking, but I think it better conveys the intention of
the code by showing that I want the function to continue for the source
pad. I can drop them if you prefer.

> > +	}
> > +
> > +	/*
> > +	 * On the source pad, the sensor supports multiple image raw formats
> > +	 * with different bit depths. The embedded data format bit depth
> > +	 * follows the image stream.
> > +	 */
> > +	if (code->stream == IMX219_STREAM_IMAGE) {
> >  		u32 format;
> >
> >  		if (code->index >= (ARRAY_SIZE(imx219_mbus_formats) / 4))
> > @@ -795,6 +840,15 @@ static int imx219_enum_mbus_code(struct v4l2_subdev *sd,
> >
> >  		format = imx219_mbus_formats[code->index * 4];
> >  		code->code = imx219_get_format_code(imx219, format);
> > +	} else {
> > +		struct v4l2_mbus_framefmt *fmt;
> > +
> > +		if (code->index > 0)
> > +			return -EINVAL;
> > +
> > +		fmt = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
> > +						   IMX219_STREAM_EDATA);
> > +		code->code = fmt->code;
> >  	}
> >
> >  	return 0;
> > @@ -806,7 +860,8 @@ static int imx219_enum_frame_size(struct v4l2_subdev *sd,
> >  {
> >  	struct imx219 *imx219 = to_imx219(sd);
> >
> > -	if (fse->pad == IMX219_PAD_IMAGE) {
> > +	switch (fse->pad) {
> > +	case IMX219_PAD_IMAGE:
> >  		if (fse->code != IMX219_NATIVE_FORMAT || fse->index > 0)
> >  			return -EINVAL;
> >
> > @@ -814,7 +869,24 @@ static int imx219_enum_frame_size(struct v4l2_subdev *sd,
> >  		fse->max_width = IMX219_NATIVE_WIDTH;
> >  		fse->min_height = IMX219_NATIVE_HEIGHT;
> >  		fse->max_height = IMX219_NATIVE_HEIGHT;
> > -	} else {
> > +		return 0;
> > +
> > +	case IMX219_PAD_EDATA:
> > +		if (fse->code != MEDIA_BUS_FMT_CCS_EMBEDDED || fse->index > 0)
> > +			return -EINVAL;
> > +
> > +		fse->min_width = IMX219_NATIVE_WIDTH;
> > +		fse->max_width = IMX219_NATIVE_WIDTH;
> > +		fse->min_height = IMX219_EMBEDDED_DATA_HEIGHT;
> > +		fse->max_height = IMX219_EMBEDDED_DATA_HEIGHT;
> > +		return 0;
> > +
> > +	case IMX219_PAD_SOURCE:
> > +	default:
> > +		break;
> > +	}
> > +
> > +	if (fse->stream == IMX219_STREAM_IMAGE) {
> >  		if (fse->code != imx219_get_format_code(imx219, fse->code) ||
> >  		    fse->index >= ARRAY_SIZE(supported_modes))
> >  			return -EINVAL;
> > @@ -823,6 +895,21 @@ static int imx219_enum_frame_size(struct v4l2_subdev *sd,
> >  		fse->max_width = fse->min_width;
> >  		fse->min_height = supported_modes[fse->index].height;
> >  		fse->max_height = fse->min_height;
> > +	} else {
> > +		struct v4l2_mbus_framefmt *fmt;
> > +
> > +		fmt = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
> > +						   IMX219_STREAM_EDATA);
> > +		if (fse->code != fmt->code)
> > +			return -EINVAL;
> > +
> > +		if (fse->index)
> > +			return -EINVAL;
> > +
> > +		fse->min_width = fmt->width;
> > +		fse->max_width = fmt->width;
> > +		fse->min_height = IMX219_EMBEDDED_DATA_HEIGHT;
> > +		fse->max_height = IMX219_EMBEDDED_DATA_HEIGHT;
> >  	}
> >
> >  	return 0;
> > @@ -834,6 +921,7 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
> >  {
> >  	struct imx219 *imx219 = to_imx219(sd);
> >  	const struct imx219_mode *mode;
> > +	struct v4l2_mbus_framefmt *ed_format;
> >  	struct v4l2_mbus_framefmt *format;
> >  	struct v4l2_rect *compose;
> >  	struct v4l2_rect *crop;
> > @@ -841,9 +929,9 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
> >
> >  	/*
> >  	 * The driver is mode-based, the format can be set on the source pad
> > -	 * only.
> > +	 * only, and only for the image streeam.
> >  	 */
> > -	if (fmt->pad != IMX219_PAD_SOURCE)
> > +	if (fmt->pad != IMX219_PAD_SOURCE || fmt->stream != IMX219_STREAM_IMAGE)
> >  		return v4l2_subdev_get_fmt(sd, state, fmt);
> >
> >  	/*
> > @@ -900,15 +988,31 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
> >  	 * No mode use digital crop, the source pad crop rectangle size and
> >  	 * format are thus identical to the image pad compose rectangle.
> >  	 */
> > -	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_SOURCE);
> > +	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_SOURCE,
> > +					  IMX219_STREAM_IMAGE);
> >  	crop->left = 0;
> >  	crop->top = 0;
> >  	crop->width = fmt->format.width;
> >  	crop->height = fmt->format.height;
> >
> > -	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
> > +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
> > +					      IMX219_STREAM_IMAGE);
> >  	*format = fmt->format;
> >
> > +	/*
> > +	 * Finally, update the formats on the sink and source sides of the
> > +	 * embedded data stream.
> > +	 */
> > +	ed_format = v4l2_subdev_state_get_format(state, IMX219_PAD_EDATA);
> > +	ed_format->code = imx219_format_edata(format->code);
> 
> One could consider making imx219_format_edata() retrieve the
> [PAD_SOURCE/STREAM_IMAGE] format internally instead of passing it from
> outside, to avoid passing in a from format

True, but I need to retrieve that format above anyway, so it would be
less efficient. The function has a single caller, I don't want to
presume how it could evolve in the future and try to optimize for that.

> > +	ed_format->width = format->width;
> > +	ed_format->height = IMX219_EMBEDDED_DATA_HEIGHT;
> > +	ed_format->field = V4L2_FIELD_NONE;
> > +
> > +	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
> > +					      IMX219_STREAM_EDATA);
> > +	*format = *ed_format;
> 
> Or
>         *v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
>                                       IMX219_STREAM_EDATA) = *ed_format;
> 
> To avoid re-assigning 'format' which now points to the [0/0] format
> and might be reused later to update controls ?

I could use fmt->width and fmt->height instead of mode->width and
mode->height, so I can reassign format.

> > +
> >  	if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
> >  		int exposure_max;
> >  		int exposure_def;
> > @@ -947,6 +1051,13 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
> >  {
> >  	struct v4l2_rect *compose;
> >
> > +	/*
> > +	 * The embedded data stream doesn't support selection rectangles,
> > +	 * neither on the embedded data pad nor on the source pad.
> > +	 */
> > +	if (sel->pad == IMX219_PAD_EDATA || sel->stream != 0)
> > +		return -EINVAL;
> > +
> >  	switch (sel->target) {
> >  	case V4L2_SEL_TGT_NATIVE_SIZE:
> >  		if (sel->pad != IMX219_PAD_IMAGE)
> > @@ -999,12 +1110,18 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
> >  static int imx219_init_state(struct v4l2_subdev *sd,
> >  			     struct v4l2_subdev_state *state)
> >  {
> > -	struct v4l2_subdev_route routes[1] = {
> > +	struct v4l2_subdev_route routes[2] = {
> >  		{
> >  			.sink_pad = IMX219_PAD_IMAGE,
> >  			.sink_stream = 0,
> >  			.source_pad = IMX219_PAD_SOURCE,
> > -			.source_stream = 0,
> > +			.source_stream = IMX219_STREAM_IMAGE,
> > +			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
> > +		}, {
> > +			.sink_pad = IMX219_PAD_EDATA,
> > +			.sink_stream = 0,
> > +			.source_pad = IMX219_PAD_SOURCE,
> > +			.source_stream = IMX219_STREAM_EDATA,
> >  			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
> >  		},
> >  	};
> > @@ -1016,7 +1133,7 @@ static int imx219_init_state(struct v4l2_subdev *sd,
> >  	struct v4l2_subdev_format fmt = {
> >  		.which = V4L2_SUBDEV_FORMAT_TRY,
> >  		.pad = IMX219_PAD_SOURCE,
> > -		.stream = 0,
> > +		.stream = IMX219_STREAM_IMAGE,
> >  		.format = {
> >  			.code = MEDIA_BUS_FMT_SRGGB10_1X10,
> >  			.width = supported_modes[0].width,
> > @@ -1029,6 +1146,10 @@ static int imx219_init_state(struct v4l2_subdev *sd,
> >  	if (ret)
> >  		return ret;
> >
> > +	/*
> > +	 * Set the image stream format on the source pad. This will be
> > +	 * propagated to all formats and selection rectangles internally.
> > +	 */
> >  	imx219_set_pad_format(sd, state, &fmt);
> >
> >  	return 0;
> > @@ -1045,21 +1166,27 @@ static int imx219_get_frame_desc(struct v4l2_subdev *sd, unsigned int pad,
> >  		return -EINVAL;
> >
> >  	state = v4l2_subdev_lock_and_get_active_state(sd);
> > -	fmt = v4l2_subdev_state_get_stream_format(state, IMX219_PAD_SOURCE, 0);
> > +	fmt = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
> > +					   IMX219_STREAM_IMAGE);
> >  	code = fmt->code;
> >  	v4l2_subdev_unlock_state(state);
> >
> >  	fd->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
> > -	fd->num_entries = 1;
> > +	fd->num_entries = 2;
> >
> >  	memset(fd->entry, 0, sizeof(fd->entry));
> >
> >  	fd->entry[0].pixelcode = code;
> > -	fd->entry[0].stream = 0;
> > +	fd->entry[0].stream = IMX219_STREAM_IMAGE;
> >  	fd->entry[0].bus.csi2.vc = 0;
> >  	fd->entry[0].bus.csi2.dt = imx219_format_bpp(code) == 8
> >  				 ? MIPI_CSI2_DT_RAW8 : MIPI_CSI2_DT_RAW10;
> >
> > +	fd->entry[1].pixelcode = code;
> 
> Is the mbus_code on this second entry the same as the first one ?

That's not right. I'll fix it.

> > +	fd->entry[1].stream = IMX219_STREAM_EDATA;
> > +	fd->entry[1].bus.csi2.vc = 0;
> > +	fd->entry[1].bus.csi2.dt = MIPI_CSI2_DT_EMBEDDED_8B;
> > +
> >  	return 0;
> >  }
> >
> > @@ -1323,12 +1450,14 @@ static int imx219_probe(struct i2c_client *client)
> >  	/*
> >  	 * Initialize the pads. To preserve backward compatibility with
> >  	 * userspace that used the sensor before the introduction of the
> > -	 * internal image pad, the external source pad is numbered 0 and the
> > -	 * internal image pad numbered 1.
> > +	 * internal pads, the external source pad is numbered 0 and the internal
> > +	 * image and embedded data pads numbered 1 and 2 respectively.
> >  	 */
> >  	imx219->pads[IMX219_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
> >  	imx219->pads[IMX219_PAD_IMAGE].flags = MEDIA_PAD_FL_SINK
> >  					     | MEDIA_PAD_FL_INTERNAL;
> > +	imx219->pads[IMX219_PAD_EDATA].flags = MEDIA_PAD_FL_SINK
> > +					     | MEDIA_PAD_FL_INTERNAL;
> >
> >  	ret = media_entity_pads_init(&imx219->sd.entity,
> >  				     ARRAY_SIZE(imx219->pads), imx219->pads);

-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 06/15] media: v4l: Add V4L2-PIX-FMT-Y12P format

[Laurent Pinchart]

On Mon, Mar 04, 2024 at 12:11:44PM +0100, Jacopo Mondi wrote:
> Hi Laurent
> 
> Ah
> https://patchwork.linuxtv.org/project/linux-media/patch/20240223163012.300763-2-jacopo.mondi@ideasonboard.com/
> 
> same for the Y14 one
> 
> I guess we have upported the same patch twice ?

Indeed :-)

> Whichever series gets in first wins

I'm fine with that.

> On Fri, Mar 01, 2024 at 11:32:21PM +0200, Laurent Pinchart wrote:
> > From: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> >
> > This is a packed grey-scale image format with a depth of 12 bits per
> > pixel. Two consecutive pixels are packed into 3 bytes. The first 2 bytes
> > contain the 8 high order bits of the pixels, and the 3rd byte contains
> > the 4 least significants bits of each pixel, in the same order.
> >
> > Add the entry in userspace API, and document it.
> >
> > Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > ---
> >  .../userspace-api/media/v4l/pixfmt-yuv-luma.rst       | 11 +++++++++++
> >  drivers/media/v4l2-core/v4l2-ioctl.c                  |  1 +
> >  include/uapi/linux/videodev2.h                        |  1 +
> >  3 files changed, 13 insertions(+)
> >
> > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst b/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst
> > index cf8e4dfbfbd4..a650ceda7cdb 100644
> > --- a/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst
> > +++ b/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst
> > @@ -114,6 +114,17 @@ are often referred to as greyscale formats.
> >        - ...
> >        - ...
> >
> > +    * .. _V4L2-PIX-FMT-Y12P:
> > +
> > +      - ``V4L2_PIX_FMT_Y12P``
> > +      - 'Y12P'
> > +
> > +      - Y'\ :sub:`0`\ [11:4]
> > +      - Y'\ :sub:`1`\ [11:4]
> > +      - Y'\ :sub:`1`\ [3:0] Y'\ :sub:`0`\ [3:0]
> > +      - ...
> > +      - ...
> > +
> >      * .. _V4L2-PIX-FMT-Y14:
> >
> >        - ``V4L2_PIX_FMT_Y14``
> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> > index f3697ff65817..69ac4482d844 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1313,6 +1313,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> >  	case V4L2_PIX_FMT_Y10BPACK:	descr = "10-bit Greyscale (Packed)"; break;
> >  	case V4L2_PIX_FMT_Y10P:		descr = "10-bit Greyscale (MIPI Packed)"; break;
> >  	case V4L2_PIX_FMT_IPU3_Y10:	descr = "10-bit greyscale (IPU3 Packed)"; break;
> > +	case V4L2_PIX_FMT_Y12P:		descr = "12-bit Greyscale (MIPI Packed)"; break;
> >  	case V4L2_PIX_FMT_Y8I:		descr = "Interleaved 8-bit Greyscale"; break;
> >  	case V4L2_PIX_FMT_Y12I:		descr = "Interleaved 12-bit Greyscale"; break;
> >  	case V4L2_PIX_FMT_Z16:		descr = "16-bit Depth"; break;
> > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > index 117a081c9db9..5122ac0c6765 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -599,6 +599,7 @@ struct v4l2_pix_format {
> >  #define V4L2_PIX_FMT_Y10BPACK    v4l2_fourcc('Y', '1', '0', 'B') /* 10  Greyscale bit-packed */
> >  #define V4L2_PIX_FMT_Y10P    v4l2_fourcc('Y', '1', '0', 'P') /* 10  Greyscale, MIPI RAW10 packed */
> >  #define V4L2_PIX_FMT_IPU3_Y10		v4l2_fourcc('i', 'p', '3', 'y') /* IPU3 packed 10-bit greyscale */
> > +#define V4L2_PIX_FMT_Y12P    v4l2_fourcc('Y', '1', '2', 'P') /* 12  Greyscale, MIPI RAW12 packed */
> >
> >  /* Palette formats */
> >  #define V4L2_PIX_FMT_PAL8    v4l2_fourcc('P', 'A', 'L', '8') /*  8  8-bit palette */

-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 09/15] media: bcm2835-unicam: Add support for CCP2/CSI2 camera interface

[Laurent Pinchart]

Hi Jacopo,

On Mon, Mar 04, 2024 at 06:12:21PM +0100, Jacopo Mondi wrote:
> On Fri, Mar 01, 2024 at 11:32:24PM +0200, Laurent Pinchart wrote:
> > From: Dave Stevenson <dave.stevenson@raspberrypi.com>
> >
> > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > It is represented as two video device nodes: unicam-image and
> > unicam-embedded which are connected to an internal subdev (named
> > unicam-subdev) in order to manage streams routing.
> >
> > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > Co-developed-by: Naushir Patuck <naush@raspberrypi.com>
> > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > Co-developed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > ---
> > Changes since v5:
> >
> > - Move to drivers/media/platform/broadcom/
> > - Port to the upstream V4L2 streams API
> > - Rebase on latest metadata API proposal
> > - Add missing error message
> > - Drop unneeded documentation block for unicam_isr()
> > - Drop unneeded dev_dbg() and dev_err() messages
> > - Drop unneeded streams_mask and fmt checks
> > - Drop unused unicam_sd_pad_is_sink()
> > - Drop unneeded includes
> > - Drop v4l2_ctrl_subscribe_event() call
> > - Use pm_runtime_resume_and_get()
> > - Indentation and line wrap fixes
> > - Let the framework set bus_info
> > - Use v4l2_fwnode_endpoint_parse()
> > - Fix media device cleanup
> > - Drop lane reordering checks
> > - Fix subdev state locking
> > - Drop extra debug messages
> > - Move clock handling to runtime PM handlers
> > - Reorder functions
> > - Rename init functions for more clarity
> > - Initialize runtime PM earlier
> > - Clarify error messages
> > - Simplify subdev init with local variable
> > - Fix subdev cleanup
> > - Fix typos and indentation
> > - Don't initialize local variables needlessly
> > - Simplify num lanes check
> > - Fix metadata handling in subdev set_fmt
> > - Drop manual fallback to .s_stream()
> > - Pass v4l2_pix_format to unicam_calc_format_size_bpl()
> > - Simplify unicam_set_default_format()
> > - Fix default format settings
> > - Add busy check in unicam_s_fmt_meta()
> > - Add missing \n at end of format strings
> > - Fix metadata handling in subdev set_fmt
> > - Fix locking when starting streaming
> > - Return buffers from start streaming fails
> > - Fix format validation for metadata node
> > - Use video_device_pipeline_{start,stop}() helpers
> > - Simplify format enumeration
> > - Drop unset variable
> > - Update MAINTAINERS entry
> > - Update to the upstream v4l2_async_nf API
> > - Update to the latest subdev routing API
> > - Update to the latest subdev state API
> > - Move from subdev .init_cfg() to .init_state()
> > - Update to the latest videobuf2 API
> > - Fix v4l2_subdev_enable_streams() error check
> > - Use correct pad for the connected subdev
> > - Return buffers to vb2 when start streaming fails
> > - Improve debugging in start streaming handler
> > - Simplify DMA address management
> > - Drop comment about bcm2835-camera driver
> > - Clarify comments that explain min/max sizes
> > - Pass v4l2_pix_format to unicam_try_fmt()
> > - Drop unneeded local variables
> > - Rename image-related constants and functions
> > - Turn unicam_fmt.metadata_fmt into bool
> > - Rename unicam_fmt to unicam_format_info
> > - Rename unicam_format_info variables to fmtinfo
> > - Rename unicam_node.v_fmt to fmt
> > - Add metadata formats for RAW10, RAW12 and RAW14
> > - Make metadata formats line-based
> > - Validate format on metadata video device
> > - Add Co-devlopped-by tags
> >
> > Changes since v3:
> >
> > - Add the vendor prefix for DT name
> > - Use the reg-names in DT parsing
> > - Remove MAINTAINERS entry
> >
> > Changes since v2:
> >
> > - Change code organization
> > - Remove unused variables
> > - Correct the fmt_meta functions
> > - Rewrite the start/stop streaming
> >   - You can now start the image node alone, but not the metadata one
> >   - The buffers are allocated per-node
> >   - only the required stream is started, if the route exists and is
> >     enabled
> > - Prefix the macros with UNICAM_ to not have too generic names
> > - Drop colorspace support
> >
> > Changes since v1:
> >
> > - Replace the unicam_{info,debug,error} macros with dev_*()
> > ---
> >  MAINTAINERS                                   |    1 +
> >  drivers/media/platform/Kconfig                |    1 +
> >  drivers/media/platform/Makefile               |    1 +
> >  drivers/media/platform/broadcom/Kconfig       |   23 +
> >  drivers/media/platform/broadcom/Makefile      |    3 +
> >  .../platform/broadcom/bcm2835-unicam-regs.h   |  255 ++
> >  .../media/platform/broadcom/bcm2835-unicam.c  | 2607 +++++++++++++++++
> >  7 files changed, 2891 insertions(+)
> >  create mode 100644 drivers/media/platform/broadcom/Kconfig
> >  create mode 100644 drivers/media/platform/broadcom/Makefile
> >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam.c

[snip]

> > diff --git a/drivers/media/platform/broadcom/bcm2835-unicam.c b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > new file mode 100644
> > index 000000000000..716c89b8a217
> > --- /dev/null
> > +++ b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > @@ -0,0 +1,2607 @@
> > +// SPDX-License-Identifier: GPL-2.0-only
> > +/*
> > + * BCM283x / BCM271x Unicam Capture Driver
> > + *
> > + * Copyright (C) 2017-2020 - Raspberry Pi (Trading) Ltd.
> > + *
> > + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> > + *
> > + * Based on TI am437x driver by
> > + *   Benoit Parrot <bparrot@ti.com>
> > + *   Lad, Prabhakar <prabhakar.csengg@gmail.com>
> > + *
> > + * and TI CAL camera interface driver by
> > + *    Benoit Parrot <bparrot@ti.com>
> > + *
> > + *
> > + * There are two camera drivers in the kernel for BCM283x - this one and
> > + * bcm2835-camera (currently in staging).
> > + *
> > + * This driver directly controls the Unicam peripheral - there is no
> > + * involvement with the VideoCore firmware. Unicam receives CSI-2 or CCP2 data
> > + * and writes it into SDRAM. The only potential processing options are to
> > + * repack Bayer data into an alternate format, and applying windowing. The
> > + * repacking does not shift the data, so can repack V4L2_PIX_FMT_Sxxxx10P to
> > + * V4L2_PIX_FMT_Sxxxx10, or V4L2_PIX_FMT_Sxxxx12P to V4L2_PIX_FMT_Sxxxx12, but
> > + * not generically up to V4L2_PIX_FMT_Sxxxx16. The driver will add both formats
> > + * where the relevant formats are defined, and will automatically configure the
> > + * repacking as required. Support for windowing may be added later.
> 
> Does this last paragraph applies ? I see all the supported packed raw
> format having an 'unpacked_fourcc' associated (all but the 8 bit ones
> ofc)

Maybe I'm tired, but I'm not sure to understand why you think it may not
apply.

> > + *
> > + * It should be possible to connect this driver to any sensor with a suitable
> > + * output interface and V4L2 subdevice driver.
> > + */
> > +
> > +#include <linux/clk.h>
> > +#include <linux/delay.h>
> > +#include <linux/device.h>
> > +#include <linux/dma-mapping.h>
> > +#include <linux/err.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/io.h>
> > +#include <linux/module.h>
> > +#include <linux/of.h>
> > +#include <linux/of_device.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/slab.h>
> > +#include <linux/videodev2.h>
> > +
> > +#include <media/v4l2-async.h>
> > +#include <media/v4l2-common.h>
> > +#include <media/v4l2-dev.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/v4l2-fwnode.h>
> > +#include <media/v4l2-mc.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "bcm2835-unicam-regs.h"
> > +
> > +#define UNICAM_MODULE_NAME		"unicam"
> > +
> > +/*
> > + * Unicam must request a minimum of 250Mhz from the VPU clock.
> > + * Otherwise the input FIFOs overrun and cause image corruption.
> > + */
> > +#define UNICAM_MIN_VPU_CLOCK_RATE	(250 * 1000 * 1000)
> 
> Shouldn't this be set in DT ?

With assigned-clock-rates ? Then the clock would always run at 250MHz
(or higher, depending on the rate in the DT), while the driver ensures
it runs at at least 250MHz only when the device is being used.

> > +
> > +/* Unicam has an internal DMA alignment constraint of 16 bytes for each line. */
> > +#define UNICAM_DMA_BPL_ALIGNMENT	16
> > +
> > +/*
> > + * The image stride is stored in a 16 bit register, and needs to be aligned to
> > + * the DMA constraint. As the ISP in the same SoC has a 32 bytes alignment
> > + * constraint on its input, set the image stride alignment to 32 bytes here as
> > + * well to avoid incompatible configurations.
> > + */
> > +#define UNICAM_IMAGE_BPL_ALIGNMENT	32
> > +#define UNICAM_IMAGE_MAX_BPL		((1 << 16) - UNICAM_IMAGE_BPL_ALIGNMENT)
> > +
> > +/*
> > + * Max width is therefore determined by the max stride divided by the number of
> > + * bits per pixel. Take 32bpp as a worst case. No imposed limit on the height,
> > + * so adopt a square image for want of anything better.
> > + */
> > +#define UNICAM_IMAGE_MIN_WIDTH		16
> > +#define UNICAM_IMAGE_MIN_HEIGHT		16
> > +#define UNICAM_IMAGE_MAX_WIDTH		(UNICAM_IMAGE_MAX_BPL / 4)
> > +#define UNICAM_IMAGE_MAX_HEIGHT		UNICAM_IMAGE_MAX_WIDTH
> > +
> > +/*
> > + * There's no intrinsic limits on the width and height for embedded dat. Use
> > + * the same maximum values as for the image, to avoid overflows in the image
> > + * size computation.
> > + */
> > +#define UNICAM_META_MIN_WIDTH		1
> > +#define UNICAM_META_MIN_HEIGHT		1
> > +#define UNICAM_META_MAX_WIDTH		UNICAM_IMAGE_MAX_WIDTH
> > +#define UNICAM_META_MAX_HEIGHT		UNICAM_IMAGE_MAX_HEIGHT
> > +
> > +/*
> > + * Size of the dummy buffer. Can be any size really, but the DMA
> > + * allocation works in units of page sizes.
> > + */
> > +#define UNICAM_DUMMY_BUF_SIZE		PAGE_SIZE
> > +
> > +#define UNICAM_SD_PAD_SINK		0
> > +#define UNICAM_SD_PAD_SOURCE_IMAGE	1
> > +#define UNICAM_SD_PAD_SOURCE_METADATA	2
> > +#define UNICAM_SD_NUM_PADS		(1 + UNICAM_SD_PAD_SOURCE_METADATA)
> 
> How about an enum then ?

OK.

> > +
> > +enum unicam_node_type {
> > +	UNICAM_IMAGE_NODE,
> > +	UNICAM_METADATA_NODE,
> > +	UNICAM_MAX_NODES
> > +};
> > +
> > +/*
> > + * struct unicam_format_info - Unicam media bus format information
> > + * @fourcc: V4L2 pixel format FCC identifier. 0 if n/a.
> > + * @unpacked_fourcc: V4L2 pixel format FCC identifier if the data is expanded
> > + * out to 16bpp. 0 if n/a.
> > + * @code: V4L2 media bus format code.
> > + * @depth: Bits per pixel as delivered from the source.
> > + * @csi_dt: CSI data type.
> > + * @metadata_fmt: This format only applies to the metadata pad.
> > + */
> > +struct unicam_format_info {
> > +	u32	fourcc;
> > +	u32	unpacked_fourcc;
> > +	u32	code;
> > +	u8	depth;
> > +	u8	csi_dt;
> > +	bool	metadata_fmt;
> > +};
> > +
> > +struct unicam_buffer {
> > +	struct vb2_v4l2_buffer vb;
> > +	struct list_head list;
> > +	dma_addr_t dma_addr;
> > +	unsigned int size;
> > +};
> > +
> > +static inline struct unicam_buffer *to_unicam_buffer(struct vb2_buffer *vb)
> > +{
> > +	return container_of(vb, struct unicam_buffer, vb.vb2_buf);
> > +}
> > +
> > +struct unicam_node {
> > +	bool registered;
> > +	bool streaming;
> > +	unsigned int id;
> > +
> > +	/* Pointer to the current v4l2_buffer */
> > +	struct unicam_buffer *cur_frm;
> > +	/* Pointer to the next v4l2_buffer */
> > +	struct unicam_buffer *next_frm;
> > +	/* video capture */
> > +	const struct unicam_format_info *fmtinfo;
> > +	/* Used to store current pixel format */
> > +	struct v4l2_format fmt;
> > +	/* Buffer queue used in video-buf */
> > +	struct vb2_queue buffer_queue;
> > +	/* Queue of filled frames */
> > +	struct list_head dma_queue;
> > +	/* IRQ lock for DMA queue */
> > +	spinlock_t dma_queue_lock;
> > +	/* lock used to access this structure */
> > +	struct mutex lock;
> > +	/* Identifies video device for this channel */
> > +	struct video_device video_dev;
> > +	/* Pointer to the parent handle */
> > +	struct unicam_device *dev;
> > +	struct media_pad pad;
> > +	/*
> > +	 * Dummy buffer intended to be used by unicam
> > +	 * if we have no other queued buffers to swap to.
> > +	 */
> > +	struct unicam_buffer dummy_buf;
> > +	void *dummy_buf_cpu_addr;
> > +};
> > +
> > +struct unicam_device {
> > +	struct kref kref;
> > +
> > +	/* peripheral base address */
> > +	void __iomem *base;
> > +	/* clock gating base address */
> > +	void __iomem *clk_gate_base;
> 
> Is the clock gating part of unicam or should this be expressed as a
> clock provide and be referenced in DT through a phandle ?

No idea :-) Dave, Naush ?

> > +	/* lp clock handle */
> > +	struct clk *clock;
> > +	/* vpu clock handle */
> > +	struct clk *vpu_clock;
> > +	/* V4l2 device */
> > +	struct v4l2_device v4l2_dev;
> > +	struct media_device mdev;
> > +
> > +	/* parent device */
> > +	struct device *dev;
> > +	/* subdevice async Notifier */
> 
> s/Notifier/notifier
> 
> > +	struct v4l2_async_notifier notifier;
> > +	unsigned int sequence;
> > +
> > +	/* Sensor node */
> > +	struct {
> > +		struct v4l2_subdev *subdev;
> > +		struct media_pad *pad;
> > +	} sensor;
> > +
> > +	/* Internal subdev */
> > +	struct {
> > +		struct v4l2_subdev sd;
> > +		struct media_pad pads[UNICAM_SD_NUM_PADS];
> > +		bool streaming;
> > +	} subdev;
> > +
> > +	enum v4l2_mbus_type bus_type;
> > +	/*
> > +	 * Stores bus.mipi_csi2.flags for CSI2 sensors, or
> > +	 * bus.mipi_csi1.strobe for CCP2.
> > +	 */
> > +	unsigned int bus_flags;
> > +	unsigned int max_data_lanes;
> > +	unsigned int active_data_lanes;
> > +
> > +	struct media_pipeline pipe;
> > +
> > +	struct unicam_node node[UNICAM_MAX_NODES];
> > +};
> > +
> > +static inline struct unicam_device *
> > +notifier_to_unicam_device(struct v4l2_async_notifier *notifier)
> > +{
> > +	return container_of(notifier, struct unicam_device, notifier);
> > +}
> > +
> > +static inline struct unicam_device *
> 
> I thought using inline is disouraged as the compiler knows what to do

For this kind of wrapper around container_of() the kernel usually has an
explicit inline keyword. I can drop it if needed.

> > +sd_to_unicam_device(struct v4l2_subdev *sd)
> > +{
> > +	return container_of(sd, struct unicam_device, subdev.sd);
> 
> You can also use container_of_const() if you need it

There's no const subdev anywhere.

> > +}
> > +
> > +static void unicam_release(struct kref *kref)
> > +{
> > +	struct unicam_device *unicam =
> > +		container_of(kref, struct unicam_device, kref);
> > +
> > +	if (unicam->mdev.dev)
> > +		media_device_cleanup(&unicam->mdev);
> > +
> > +	kfree(unicam);
> > +}
> > +
> > +static struct unicam_device *unicam_get(struct unicam_device *unicam)
> > +{
> > +	kref_get(&unicam->kref);
> > +	return unicam;
> > +}
> > +
> > +static void unicam_put(struct unicam_device *unicam)
> > +{
> > +	kref_put(&unicam->kref, unicam_release);
> > +}
> > +
> 
> Does this handle a bit more gracefully the pesky lifetime management
> issue in v4l2 ?

That's supposed to be the idea :-)

> > +/* -----------------------------------------------------------------------------
> > + * Misc helper functions
> > + */
> > +
> > +static inline bool unicam_sd_pad_is_source(u32 pad)
> > +{
> > +	/* Camera RX has 1 sink pad, and N source pads */
> > +	return pad != UNICAM_SD_PAD_SINK;
> > +}
> > +
> > +static inline bool is_metadata_node(struct unicam_node *node)
> > +{
> > +	return node->video_dev.device_caps & V4L2_CAP_META_CAPTURE;
> > +}
> > +
> > +static inline bool is_image_node(struct unicam_node *node)
> > +{
> > +	return node->video_dev.device_caps & V4L2_CAP_VIDEO_CAPTURE;
> > +}

[snip]

> > +/* -----------------------------------------------------------------------------
> > + * Videobuf2 queue operations
> > + */
> > +
> > +static int unicam_queue_setup(struct vb2_queue *vq,
> > +			      unsigned int *nbuffers,
> > +			      unsigned int *nplanes,
> > +			      unsigned int sizes[],
> > +			      struct device *alloc_devs[])
> > +{
> > +	struct unicam_node *node = vb2_get_drv_priv(vq);
> > +	u32 size = is_image_node(node) ? node->fmt.fmt.pix.sizeimage
> > +		 : node->fmt.fmt.meta.buffersize;
> > +
> > +	if (vq->num_buffers + *nbuffers < 3)
> 
> Why 3 ? Are these dummies ?

This may be a remnant of old code. Dave, Naush, any comment ?

> > +		*nbuffers = 3 - vq->num_buffers;
> > +
> > +	if (*nplanes) {
> > +		if (sizes[0] < size) {
> > +			dev_dbg(node->dev->dev, "sizes[0] %i < size %u\n",
> > +				sizes[0], size);
> > +			return -EINVAL;
> > +		}
> > +		size = sizes[0];
> > +	}
> > +
> > +	*nplanes = 1;
> > +	sizes[0] = size;
> > +
> > +	return 0;
> > +}
> > +
> > +static int unicam_buffer_prepare(struct vb2_buffer *vb)
> > +{
> > +	struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct unicam_buffer *buf = to_unicam_buffer(vb);
> > +	u32 size = is_image_node(node) ? node->fmt.fmt.pix.sizeimage
> > +		 : node->fmt.fmt.meta.buffersize;
> > +
> > +	if (vb2_plane_size(vb, 0) < size) {
> > +		dev_dbg(node->dev->dev,
> > +			"data will not fit into plane (%lu < %u)\n",
> > +			vb2_plane_size(vb, 0), size);
> > +		return -EINVAL;
> > +	}
> > +
> > +	buf->dma_addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
> > +	buf->size = size;
> > +
> > +	vb2_set_plane_payload(&buf->vb.vb2_buf, 0, size);
> > +
> > +	return 0;
> > +}
> > +
> > +static void unicam_return_buffers(struct unicam_node *node,
> > +				  enum vb2_buffer_state state)
> > +{
> > +	struct unicam_buffer *buf, *tmp;
> > +
> > +	list_for_each_entry_safe(buf, tmp, &node->dma_queue, list) {
> > +		list_del(&buf->list);
> > +		vb2_buffer_done(&buf->vb.vb2_buf, state);
> > +	}
> > +
> > +	if (node->cur_frm)
> > +		vb2_buffer_done(&node->cur_frm->vb.vb2_buf,
> > +				state);
> > +	if (node->next_frm && node->cur_frm != node->next_frm)
> > +		vb2_buffer_done(&node->next_frm->vb.vb2_buf,
> > +				state);
> > +
> > +	node->cur_frm = NULL;
> > +	node->next_frm = NULL;
> > +}
> > +
> > +static int unicam_start_streaming(struct vb2_queue *vq, unsigned int count)
> > +{
> > +	struct unicam_node *node = vb2_get_drv_priv(vq);
> > +	struct unicam_device *unicam = node->dev;
> > +	struct v4l2_subdev_state *state;
> > +	struct unicam_buffer *buf;
> > +	unsigned long flags;
> > +	int ret;
> > +	u32 pad, stream;
> > +	u32 remote_pad = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
> > +					     : UNICAM_SD_PAD_SOURCE_METADATA;
> > +
> > +	/* Look for the route for the given pad and stream. */
> > +	state = v4l2_subdev_lock_and_get_active_state(&unicam->subdev.sd);
> > +	ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
> > +						    remote_pad, 0,
> > +						    &pad, &stream);
> > +	v4l2_subdev_unlock_state(state);
> > +
> > +	if (ret)
> > +		goto err_return_buffers;
> > +
> > +	dev_dbg(unicam->dev, "Starting stream on %s: %u/%u -> %u/%u (%s)\n",
> > +		unicam->subdev.sd.name, pad, stream, remote_pad, 0,
> > +		is_metadata_node(node) ? "metadata" : "image");
> > +
> > +	/* The metadata node can't be started alone. */
> > +	if (is_metadata_node(node)) {
> > +		if (!unicam->node[UNICAM_IMAGE_NODE].streaming) {
> 
> Does it mean the metadata node has to be started second, or should
> this be made a nop and the metadata node gets started once the image
> node is started too ? I'm fine with the constraint of having the
> metadata node being started second fwiw

I think it would be nice to change this indeed. Dave, Naush, any
objection ?

> > +			dev_err(unicam->dev,
> > +				"Can't start metadata without image\n");
> > +			ret = -EINVAL;
> > +			goto err_return_buffers;
> > +		}
> > +
> > +		spin_lock_irqsave(&node->dma_queue_lock, flags);
> > +		buf = list_first_entry(&node->dma_queue,
> > +				       struct unicam_buffer, list);
> > +		dev_dbg(unicam->dev, "buffer %p\n", buf);
> 
> Is this useful ?

Probably not much. I'll drop it.

> > +		node->cur_frm = buf;
> > +		node->next_frm = buf;
> > +		list_del(&buf->list);
> > +		spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> > +
> > +		unicam_start_metadata(unicam, buf);
> > +		node->streaming = true;
> > +		return 0;
> > +	}
> > +
> > +	ret = pm_runtime_resume_and_get(unicam->dev);
> > +	if (ret < 0) {
> > +		dev_err(unicam->dev, "PM runtime resume failed: %d\n", ret);
> > +		goto err_return_buffers;
> > +	}
> > +
> > +	ret = video_device_pipeline_start(&node->video_dev, &unicam->pipe);
> > +	if (ret < 0) {
> > +		dev_dbg(unicam->dev, "Failed to start media pipeline: %d\n", ret);
> 
> Isn't this an err ?

The main cause of failure is a pipeline validation error, triggered by
userspace, hence the debug level.

> > +		goto err_pm_put;
> > +	}
> > +
> > +	spin_lock_irqsave(&node->dma_queue_lock, flags);
> > +	buf = list_first_entry(&node->dma_queue,
> > +			       struct unicam_buffer, list);
> > +	dev_dbg(unicam->dev, "buffer %p\n", buf);
> > +	node->cur_frm = buf;
> > +	node->next_frm = buf;
> > +	list_del(&buf->list);
> > +	spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> > +
> > +	unicam_start_rx(unicam, buf);
> > +
> > +	ret = v4l2_subdev_enable_streams(&unicam->subdev.sd, remote_pad, BIT(0));
> 
> A bit confused by the API here, shouldn't we also do this for embedded
> data ?

Not here, as the two streams go over different pads, but likely above,
as part of the change you proposed regarding stream start on the
metadata device. I'll wait for Dave and Naush to reply, and I'll rework
this function.

> > +	if (ret < 0) {
> > +		dev_err(unicam->dev, "stream on failed in subdev\n");
> > +		goto error_pipeline;
> > +	}
> > +
> > +	node->streaming = true;
> > +
> > +	return 0;
> > +
> > +error_pipeline:
> > +	video_device_pipeline_stop(&node->video_dev);
> > +err_pm_put:
> > +	pm_runtime_put_sync(unicam->dev);
> > +err_return_buffers:
> > +	unicam_return_buffers(node, VB2_BUF_STATE_QUEUED);
> > +	return ret;
> > +}
> > +
> > +static void unicam_stop_streaming(struct vb2_queue *vq)
> > +{
> > +	struct unicam_node *node = vb2_get_drv_priv(vq);
> > +	struct unicam_device *unicam = node->dev;
> > +	u32 remote_pad_index = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
> > +						   : UNICAM_SD_PAD_SOURCE_METADATA;
> > +
> > +	node->streaming = false;
> > +
> > +	v4l2_subdev_disable_streams(&unicam->subdev.sd, remote_pad_index,
> > +				    BIT(0));
> > +
> > +	/* We can stream only with the image node. */
> > +	if (is_metadata_node(node)) {
> > +		/*
> > +		 * Allow the hardware to spin in the dummy buffer.
> > +		 * This is only really needed if the embedded data pad is
> > +		 * disabled before the image pad.
> > +		 */
> > +		unicam_wr_dma_addr(node, &node->dummy_buf);
> > +		goto dequeue_buffers;
> > +	}
> > +
> > +	unicam_disable(unicam);
> > +
> > +	video_device_pipeline_stop(&node->video_dev);
> > +	pm_runtime_put(unicam->dev);
> > +
> > +dequeue_buffers:
> > +	/* Clear all queued buffers for the node */
> > +	unicam_return_buffers(node, VB2_BUF_STATE_ERROR);
> > +}
> > +
> > +static void unicam_buffer_queue(struct vb2_buffer *vb)
> > +{
> > +	struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
> > +	struct unicam_buffer *buf = to_unicam_buffer(vb);
> > +
> > +	spin_lock_irq(&node->dma_queue_lock);
> > +	list_add_tail(&buf->list, &node->dma_queue);
> > +	spin_unlock_irq(&node->dma_queue_lock);
> > +}
> > +
> > +static const struct vb2_ops unicam_video_qops = {
> > +	.queue_setup		= unicam_queue_setup,
> > +	.wait_prepare		= vb2_ops_wait_prepare,
> > +	.wait_finish		= vb2_ops_wait_finish,
> > +	.buf_prepare		= unicam_buffer_prepare,
> > +	.start_streaming	= unicam_start_streaming,
> > +	.stop_streaming		= unicam_stop_streaming,
> > +	.buf_queue		= unicam_buffer_queue,
> > +};
> > +
> > +/* -----------------------------------------------------------------------------
> > + *  V4L2 video device operations
> > + */
> > +
> > +static int unicam_querycap(struct file *file, void *priv,
> > +			   struct v4l2_capability *cap)
> > +{
> > +	strscpy(cap->driver, UNICAM_MODULE_NAME, sizeof(cap->driver));
> > +	strscpy(cap->card, UNICAM_MODULE_NAME, sizeof(cap->card));
> > +
> > +	cap->capabilities |= V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_META_CAPTURE;
> > +
> > +	return 0;
> > +}
> > +
> > +static int unicam_enum_fmt_vid(struct file *file, void  *priv,
> > +			       struct v4l2_fmtdesc *f)
> > +{
> > +	unsigned int index;
> > +	unsigned int i;
> > +
> > +	for (i = 0, index = 0; i < ARRAY_SIZE(unicam_image_formats); i++) {
> > +		if (f->mbus_code && unicam_image_formats[i].code != f->mbus_code)
> > +			continue;
> > +
> > +		if (index == f->index) {
> > +			f->pixelformat = unicam_image_formats[i].fourcc;
> > +			return 0;
> > +		}
> > +
> > +		index++;
> > +
> > +		if (!unicam_image_formats[i].unpacked_fourcc)
> > +			continue;
> > +
> > +		if (index == f->index) {
> > +			f->pixelformat = unicam_image_formats[i].unpacked_fourcc;
> > +			return 0;
> > +		}
> > +
> > +		index++;
> > +	}
> > +
> > +	return -EINVAL;
> > +}
> > +
> > +static int unicam_g_fmt_vid(struct file *file, void *priv,
> > +			    struct v4l2_format *f)
> > +{
> > +	struct unicam_node *node = video_drvdata(file);
> > +
> > +	*f = node->fmt;
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct unicam_format_info *
> > +__unicam_try_fmt_vid(struct unicam_node *node, struct v4l2_pix_format *pix)
> > +{
> > +	const struct unicam_format_info *fmtinfo;
> > +
> > +	/*
> > +	 * Default to the first format if the requested pixel format code isn't
> > +	 * supported.
> > +	 */
> > +	fmtinfo = unicam_find_format_by_fourcc(pix->pixelformat,
> > +					       UNICAM_SD_PAD_SOURCE_IMAGE);
> > +	if (!fmtinfo) {
> > +		fmtinfo = &unicam_image_formats[0];
> > +		pix->pixelformat = fmtinfo->fourcc;
> > +	}
> > +
> > +	unicam_calc_image_size_bpl(node->dev, fmtinfo, pix);
> > +
> > +	if (pix->field == V4L2_FIELD_ANY)
> > +		pix->field = V4L2_FIELD_NONE;
> > +
> > +	return fmtinfo;
> > +}
> > +
> > +static int unicam_try_fmt_vid(struct file *file, void *priv,
> > +			      struct v4l2_format *f)
> > +{
> > +	struct unicam_node *node = video_drvdata(file);
> > +
> > +	__unicam_try_fmt_vid(node, &f->fmt.pix);
> > +	return 0;
> > +}
> > +
> > +static int unicam_s_fmt_vid(struct file *file, void *priv,
> > +			    struct v4l2_format *f)
> > +{
> > +	struct unicam_node *node = video_drvdata(file);
> > +
> > +	if (vb2_is_busy(&node->buffer_queue))
> > +		return -EBUSY;
> > +
> > +	node->fmtinfo = __unicam_try_fmt_vid(node, &f->fmt.pix);
> > +	node->fmt = *f;
> > +
> > +	return 0;
> > +}
> > +
> > +static int unicam_enum_fmt_meta(struct file *file, void *priv,
> > +				struct v4l2_fmtdesc *f)
> > +{
> > +	unsigned int i, index;
> > +
> > +	for (i = 0, index = 0; i < ARRAY_SIZE(unicam_meta_formats); i++) {
> > +		if (f->mbus_code && unicam_meta_formats[i].code != f->mbus_code)
> 
> Do we want to allow mbus_code filtering for metadata ? There's a
> 1-to-1 relationship between mbus codes and pixel formats

Is there a reason not to allow it ? I think it's good for the API to be
consistent. Generally speaking, a CSI-2 receiver could receive
META_CSI2_10 and convert it to the META_8 pixel format. Filtering as a
concept thus make sense I think.

> > +			continue;
> > +		if (!unicam_meta_formats[i].metadata_fmt)
> > +			continue;
> 
> How can this be false if we're iterating on unicam_meta_formats[] ?

True. I'll drop the metadata_fmt field.

> > +
> > +		if (index == f->index) {
> > +			f->pixelformat = unicam_meta_formats[i].fourcc;
> > +			f->type = V4L2_BUF_TYPE_META_CAPTURE;
> > +			return 0;
> > +		}
> > +		index++;
> > +	}
> > +
> > +	return -EINVAL;
> > +}
> > +
> > +static int unicam_g_fmt_meta(struct file *file, void *priv,
> > +			     struct v4l2_format *f)
> > +{
> > +	struct unicam_node *node = video_drvdata(file);
> > +
> > +	f->fmt.meta = node->fmt.fmt.meta;
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct unicam_format_info *
> > +__unicam_try_fmt_meta(struct unicam_node *node, struct v4l2_meta_format *meta)
> > +{
> > +	const struct unicam_format_info *fmtinfo;
> > +
> > +	/*
> > +	 * Default to the first format if the requested pixel format code isn't
> > +	 * supported.
> > +	 */
> > +	fmtinfo = unicam_find_format_by_fourcc(meta->dataformat,
> > +					       UNICAM_SD_PAD_SOURCE_METADATA);
> > +	if (!fmtinfo) {
> > +		fmtinfo = &unicam_meta_formats[0];
> > +		meta->dataformat = fmtinfo->fourcc;
> > +	}
> > +
> > +	unicam_calc_meta_size_bpl(node->dev, fmtinfo, meta);
> > +
> > +	return fmtinfo;
> > +}
> > +
> > +static int unicam_try_fmt_meta(struct file *file, void *priv,
> > +			       struct v4l2_format *f)
> > +{
> > +	struct unicam_node *node = video_drvdata(file);
> > +
> > +	__unicam_try_fmt_vid(node, &f->fmt.pix);
> > +	return 0;
> > +}
> > +
> > +static int unicam_s_fmt_meta(struct file *file, void *priv,
> > +			     struct v4l2_format *f)
> > +{
> > +	struct unicam_node *node = video_drvdata(file);
> > +
> > +	if (vb2_is_busy(&node->buffer_queue))
> > +		return -EBUSY;
> > +
> > +	node->fmtinfo = __unicam_try_fmt_meta(node, &f->fmt.meta);
> > +	node->fmt = *f;
> > +
> > +	return 0;
> > +}
> > +
> > +static int unicam_enum_framesizes(struct file *file, void *fh,
> > +				  struct v4l2_frmsizeenum *fsize)
> > +{
> > +	struct unicam_node *node = video_drvdata(file);
> > +	int ret = -EINVAL;
> > +
> > +	if (fsize->index > 0)
> > +		return ret;
> > +
> > +	if (is_image_node(node)) {
> > +		if (!unicam_find_format_by_fourcc(fsize->pixel_format,
> > +						  UNICAM_SD_PAD_SOURCE_IMAGE))
> > +			return ret;
> > +
> > +		fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> > +		fsize->stepwise.min_width = UNICAM_IMAGE_MIN_WIDTH;
> > +		fsize->stepwise.max_width = UNICAM_IMAGE_MAX_WIDTH;
> > +		fsize->stepwise.step_width = 1;
> > +		fsize->stepwise.min_height = UNICAM_IMAGE_MIN_HEIGHT;
> > +		fsize->stepwise.max_height = UNICAM_IMAGE_MAX_HEIGHT;
> > +		fsize->stepwise.step_height = 1;
> > +	} else {
> > +		if (!unicam_find_format_by_fourcc(fsize->pixel_format,
> > +						  UNICAM_SD_PAD_SOURCE_METADATA))
> > +			return ret;
> > +
>
> Isn't this V4L2_FRMSIZE_TYPE_STEPWISE as well ?

Indeed, I'll fix that.

> > +		fsize->stepwise.min_width = UNICAM_META_MIN_WIDTH;
> > +		fsize->stepwise.max_width = UNICAM_META_MAX_WIDTH;
> > +		fsize->stepwise.step_width = 1;
> > +		fsize->stepwise.min_height = UNICAM_META_MIN_HEIGHT;
> > +		fsize->stepwise.max_height = UNICAM_META_MAX_HEIGHT;
> > +		fsize->stepwise.step_height = 1;
> > +	}
> > +
> > +	return 0;
> > +}

[snip]

-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 09/15] media: bcm2835-unicam: Add support for CCP2/CSI2 camera interface

[Dave Stevenson]

Hi Laurent and Jacopo.

I'd started this reply before Laurent's follow up - I'll respond to
that in a moment.

On Mon, 4 Mar 2024 at 17:12, Jacopo Mondi <jacopo.mondi@ideasonboard.com> wrote:
>
> Hi Laurent
>
> On Fri, Mar 01, 2024 at 11:32:24PM +0200, Laurent Pinchart wrote:
> > From: Dave Stevenson <dave.stevenson@raspberrypi.com>
> >
> > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > It is represented as two video device nodes: unicam-image and
> > unicam-embedded which are connected to an internal subdev (named
> > unicam-subdev) in order to manage streams routing.
> >
> > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > Co-developed-by: Naushir Patuck <naush@raspberrypi.com>
> > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > Co-developed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > ---
> > Changes since v5:
> >
> > - Move to drivers/media/platform/broadcom/
> > - Port to the upstream V4L2 streams API
> > - Rebase on latest metadata API proposal
> > - Add missing error message
> > - Drop unneeded documentation block for unicam_isr()
> > - Drop unneeded dev_dbg() and dev_err() messages
> > - Drop unneeded streams_mask and fmt checks
> > - Drop unused unicam_sd_pad_is_sink()
> > - Drop unneeded includes
> > - Drop v4l2_ctrl_subscribe_event() call
> > - Use pm_runtime_resume_and_get()
> > - Indentation and line wrap fixes
> > - Let the framework set bus_info
> > - Use v4l2_fwnode_endpoint_parse()
> > - Fix media device cleanup
> > - Drop lane reordering checks
> > - Fix subdev state locking
> > - Drop extra debug messages
> > - Move clock handling to runtime PM handlers
> > - Reorder functions
> > - Rename init functions for more clarity
> > - Initialize runtime PM earlier
> > - Clarify error messages
> > - Simplify subdev init with local variable
> > - Fix subdev cleanup
> > - Fix typos and indentation
> > - Don't initialize local variables needlessly
> > - Simplify num lanes check
> > - Fix metadata handling in subdev set_fmt
> > - Drop manual fallback to .s_stream()
> > - Pass v4l2_pix_format to unicam_calc_format_size_bpl()
> > - Simplify unicam_set_default_format()
> > - Fix default format settings
> > - Add busy check in unicam_s_fmt_meta()
> > - Add missing \n at end of format strings
> > - Fix metadata handling in subdev set_fmt
> > - Fix locking when starting streaming
> > - Return buffers from start streaming fails
> > - Fix format validation for metadata node
> > - Use video_device_pipeline_{start,stop}() helpers
> > - Simplify format enumeration
> > - Drop unset variable
> > - Update MAINTAINERS entry
> > - Update to the upstream v4l2_async_nf API
> > - Update to the latest subdev routing API
> > - Update to the latest subdev state API
> > - Move from subdev .init_cfg() to .init_state()
> > - Update to the latest videobuf2 API
> > - Fix v4l2_subdev_enable_streams() error check
> > - Use correct pad for the connected subdev
> > - Return buffers to vb2 when start streaming fails
> > - Improve debugging in start streaming handler
> > - Simplify DMA address management
> > - Drop comment about bcm2835-camera driver
> > - Clarify comments that explain min/max sizes
> > - Pass v4l2_pix_format to unicam_try_fmt()
> > - Drop unneeded local variables
> > - Rename image-related constants and functions
> > - Turn unicam_fmt.metadata_fmt into bool
> > - Rename unicam_fmt to unicam_format_info
> > - Rename unicam_format_info variables to fmtinfo
> > - Rename unicam_node.v_fmt to fmt
> > - Add metadata formats for RAW10, RAW12 and RAW14
> > - Make metadata formats line-based
> > - Validate format on metadata video device
> > - Add Co-devlopped-by tags
> >
> > Changes since v3:
> >
> > - Add the vendor prefix for DT name
> > - Use the reg-names in DT parsing
> > - Remove MAINTAINERS entry
> >
> > Changes since v2:
> >
> > - Change code organization
> > - Remove unused variables
> > - Correct the fmt_meta functions
> > - Rewrite the start/stop streaming
> >   - You can now start the image node alone, but not the metadata one
> >   - The buffers are allocated per-node
> >   - only the required stream is started, if the route exists and is
> >     enabled
> > - Prefix the macros with UNICAM_ to not have too generic names
> > - Drop colorspace support
> >
> > Changes since v1:
> >
> > - Replace the unicam_{info,debug,error} macros with dev_*()
> > ---
> >  MAINTAINERS                                   |    1 +
> >  drivers/media/platform/Kconfig                |    1 +
> >  drivers/media/platform/Makefile               |    1 +
> >  drivers/media/platform/broadcom/Kconfig       |   23 +
> >  drivers/media/platform/broadcom/Makefile      |    3 +
> >  .../platform/broadcom/bcm2835-unicam-regs.h   |  255 ++
> >  .../media/platform/broadcom/bcm2835-unicam.c  | 2607 +++++++++++++++++
> >  7 files changed, 2891 insertions(+)
> >  create mode 100644 drivers/media/platform/broadcom/Kconfig
> >  create mode 100644 drivers/media/platform/broadcom/Makefile
> >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam.c
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index e50a59654e6e..cc350729f467 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -4002,6 +4002,7 @@ M:      Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
> >  L:   linux-media@vger.kernel.org
> >  S:   Maintained
> >  F:   Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> > +F:   drivers/media/platform/bcm2835/
> >
> >  BROADCOM BCM47XX MIPS ARCHITECTURE
> >  M:   Hauke Mehrtens <hauke@hauke-m.de>
> > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > index 91e54215de3a..2d79bfc68c15 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -67,6 +67,7 @@ source "drivers/media/platform/amlogic/Kconfig"
> >  source "drivers/media/platform/amphion/Kconfig"
> >  source "drivers/media/platform/aspeed/Kconfig"
> >  source "drivers/media/platform/atmel/Kconfig"
> > +source "drivers/media/platform/broadcom/Kconfig"
> >  source "drivers/media/platform/cadence/Kconfig"
> >  source "drivers/media/platform/chips-media/Kconfig"
> >  source "drivers/media/platform/intel/Kconfig"
> > diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> > index 3296ec1ebe16..da17301f7439 100644
> > --- a/drivers/media/platform/Makefile
> > +++ b/drivers/media/platform/Makefile
> > @@ -10,6 +10,7 @@ obj-y += amlogic/
> >  obj-y += amphion/
> >  obj-y += aspeed/
> >  obj-y += atmel/
> > +obj-y += broadcom/
> >  obj-y += cadence/
> >  obj-y += chips-media/
> >  obj-y += intel/
> > diff --git a/drivers/media/platform/broadcom/Kconfig b/drivers/media/platform/broadcom/Kconfig
> > new file mode 100644
> > index 000000000000..cc2c9afcc948
> > --- /dev/null
> > +++ b/drivers/media/platform/broadcom/Kconfig
> > @@ -0,0 +1,23 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +config VIDEO_BCM2835_UNICAM
> > +     tristate "Broadcom BCM283x/BCM271x Unicam video capture driver"
> > +     depends on ARCH_BCM2835 || COMPILE_TEST
> > +     depends on PM
> > +     depends on VIDEO_DEV
> > +     select MEDIA_CONTROLLER
> > +     select V4L2_FWNODE
> > +     select VIDEO_V4L2_SUBDEV_API
> > +     select VIDEOBUF2_DMA_CONTIG
> > +     help
> > +       Say Y here to enable support for the BCM283x/BCM271x CSI-2 receiver.
> > +       This is a V4L2 driver that controls the CSI-2 receiver directly,
> > +       independently from the VC4 firmware.
> > +
> > +       This driver is mutually exclusive with the use of bcm2835-camera. The
> > +       firmware will disable all access to the peripheral from within the
> > +       firmware if it finds a DT node using it, and bcm2835-camera will
> > +       therefore fail to probe.
> > +
> > +       To compile this driver as a module, choose M here. The module will be
> > +       called bcm2835-unicam.
> > diff --git a/drivers/media/platform/broadcom/Makefile b/drivers/media/platform/broadcom/Makefile
> > new file mode 100644
> > index 000000000000..03d2045aba2e
> > --- /dev/null
> > +++ b/drivers/media/platform/broadcom/Makefile
> > @@ -0,0 +1,3 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +obj-$(CONFIG_VIDEO_BCM2835_UNICAM) += bcm2835-unicam.o
> > diff --git a/drivers/media/platform/broadcom/bcm2835-unicam-regs.h b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > new file mode 100644
> > index 000000000000..84775fd2fac5
> > --- /dev/null
> > +++ b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > @@ -0,0 +1,255 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +
> > +/*
> > + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> > + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> > + */
> > +
> > +#ifndef VC4_REGS_UNICAM_H
> > +#define VC4_REGS_UNICAM_H
> > +
> > +#include <linux/bits.h>
> > +
> > +/*
> > + * The following values are taken from files found within the code drop
> > + * made by Broadcom for the BCM21553 Graphics Driver, predominantly in
> > + * brcm_usrlib/dag/vmcsx/vcinclude/hardware_vc4.h.
> > + * They have been modified to be only the register offset.
> > + */
> > +#define UNICAM_CTRL          0x000
> > +#define UNICAM_STA           0x004
> > +#define UNICAM_ANA           0x008
> > +#define UNICAM_PRI           0x00c
> > +#define UNICAM_CLK           0x010
> > +#define UNICAM_CLT           0x014
> > +#define UNICAM_DAT0          0x018
> > +#define UNICAM_DAT1          0x01c
> > +#define UNICAM_DAT2          0x020
> > +#define UNICAM_DAT3          0x024
> > +#define UNICAM_DLT           0x028
> > +#define UNICAM_CMP0          0x02c
> > +#define UNICAM_CMP1          0x030
> > +#define UNICAM_CAP0          0x034
> > +#define UNICAM_CAP1          0x038
> > +#define UNICAM_ICTL          0x100
> > +#define UNICAM_ISTA          0x104
> > +#define UNICAM_IDI0          0x108
> > +#define UNICAM_IPIPE         0x10c
> > +#define UNICAM_IBSA0         0x110
> > +#define UNICAM_IBEA0         0x114
> > +#define UNICAM_IBLS          0x118
> > +#define UNICAM_IBWP          0x11c
> > +#define UNICAM_IHWIN         0x120
> > +#define UNICAM_IHSTA         0x124
> > +#define UNICAM_IVWIN         0x128
> > +#define UNICAM_IVSTA         0x12c
> > +#define UNICAM_ICC           0x130
> > +#define UNICAM_ICS           0x134
> > +#define UNICAM_IDC           0x138
> > +#define UNICAM_IDPO          0x13c
> > +#define UNICAM_IDCA          0x140
> > +#define UNICAM_IDCD          0x144
> > +#define UNICAM_IDS           0x148
> > +#define UNICAM_DCS           0x200
> > +#define UNICAM_DBSA0         0x204
> > +#define UNICAM_DBEA0         0x208
> > +#define UNICAM_DBWP          0x20c
> > +#define UNICAM_DBCTL         0x300
> > +#define UNICAM_IBSA1         0x304
> > +#define UNICAM_IBEA1         0x308
> > +#define UNICAM_IDI1          0x30c
> > +#define UNICAM_DBSA1         0x310
> > +#define UNICAM_DBEA1         0x314
> > +#define UNICAM_MISC          0x400
> > +
> > +/*
> > + * The following bitmasks are from the kernel released by Broadcom
> > + * for Android - https://android.googlesource.com/kernel/bcm/
> > + * The Rhea, Hawaii, and Java chips all contain the same VideoCore4
> > + * Unicam block as BCM2835, as defined in eg
> > + * arch/arm/mach-rhea/include/mach/rdb_A0/brcm_rdb_cam.h and similar.
> > + * Values reworked to use the kernel BIT and GENMASK macros.
> > + *
> > + * Some of the bit mnenomics have been amended to match the datasheet.
> > + */
> > +/* UNICAM_CTRL Register */
> > +#define UNICAM_CPE           BIT(0)
> > +#define UNICAM_MEM           BIT(1)
> > +#define UNICAM_CPR           BIT(2)
> > +#define UNICAM_CPM_MASK              GENMASK(3, 3)
> > +#define UNICAM_CPM_CSI2              0
> > +#define UNICAM_CPM_CCP2              1
> > +#define UNICAM_SOE           BIT(4)
> > +#define UNICAM_DCM_MASK              GENMASK(5, 5)
> > +#define UNICAM_DCM_STROBE    0
> > +#define UNICAM_DCM_DATA              1
> > +#define UNICAM_SLS           BIT(6)
> > +#define UNICAM_PFT_MASK              GENMASK(11, 8)
> > +#define UNICAM_OET_MASK              GENMASK(20, 12)
> > +
> > +/* UNICAM_STA Register */
> > +#define UNICAM_SYN           BIT(0)
> > +#define UNICAM_CS            BIT(1)
> > +#define UNICAM_SBE           BIT(2)
> > +#define UNICAM_PBE           BIT(3)
> > +#define UNICAM_HOE           BIT(4)
> > +#define UNICAM_PLE           BIT(5)
> > +#define UNICAM_SSC           BIT(6)
> > +#define UNICAM_CRCE          BIT(7)
> > +#define UNICAM_OES           BIT(8)
> > +#define UNICAM_IFO           BIT(9)
> > +#define UNICAM_OFO           BIT(10)
> > +#define UNICAM_BFO           BIT(11)
> > +#define UNICAM_DL            BIT(12)
> > +#define UNICAM_PS            BIT(13)
> > +#define UNICAM_IS            BIT(14)
> > +#define UNICAM_PI0           BIT(15)
> > +#define UNICAM_PI1           BIT(16)
> > +#define UNICAM_FSI_S         BIT(17)
> > +#define UNICAM_FEI_S         BIT(18)
> > +#define UNICAM_LCI_S         BIT(19)
> > +#define UNICAM_BUF0_RDY              BIT(20)
> > +#define UNICAM_BUF0_NO               BIT(21)
> > +#define UNICAM_BUF1_RDY              BIT(22)
> > +#define UNICAM_BUF1_NO               BIT(23)
> > +#define UNICAM_DI            BIT(24)
> > +
> > +#define UNICAM_STA_MASK_ALL \
> > +             (UNICAM_DL | \
> > +             UNICAM_SBE | \
> > +             UNICAM_PBE | \
> > +             UNICAM_HOE | \
> > +             UNICAM_PLE | \
> > +             UNICAM_SSC | \
> > +             UNICAM_CRCE | \
> > +             UNICAM_IFO | \
> > +             UNICAM_OFO | \
> > +             UNICAM_PS | \
> > +             UNICAM_PI0 | \
> > +             UNICAM_PI1)
> > +
> > +/* UNICAM_ANA Register */
> > +#define UNICAM_APD           BIT(0)
> > +#define UNICAM_BPD           BIT(1)
> > +#define UNICAM_AR            BIT(2)
> > +#define UNICAM_DDL           BIT(3)
> > +#define UNICAM_CTATADJ_MASK  GENMASK(7, 4)
> > +#define UNICAM_PTATADJ_MASK  GENMASK(11, 8)
> > +
> > +/* UNICAM_PRI Register */
> > +#define UNICAM_PE            BIT(0)
> > +#define UNICAM_PT_MASK               GENMASK(2, 1)
> > +#define UNICAM_NP_MASK               GENMASK(7, 4)
> > +#define UNICAM_PP_MASK               GENMASK(11, 8)
> > +#define UNICAM_BS_MASK               GENMASK(15, 12)
> > +#define UNICAM_BL_MASK               GENMASK(17, 16)
> > +
> > +/* UNICAM_CLK Register */
> > +#define UNICAM_CLE           BIT(0)
> > +#define UNICAM_CLPD          BIT(1)
> > +#define UNICAM_CLLPE         BIT(2)
> > +#define UNICAM_CLHSE         BIT(3)
> > +#define UNICAM_CLTRE         BIT(4)
> > +#define UNICAM_CLAC_MASK     GENMASK(8, 5)
> > +#define UNICAM_CLSTE         BIT(29)
> > +
> > +/* UNICAM_CLT Register */
> > +#define UNICAM_CLT1_MASK     GENMASK(7, 0)
> > +#define UNICAM_CLT2_MASK     GENMASK(15, 8)
> > +
> > +/* UNICAM_DATn Registers */
> > +#define UNICAM_DLE           BIT(0)
> > +#define UNICAM_DLPD          BIT(1)
> > +#define UNICAM_DLLPE         BIT(2)
> > +#define UNICAM_DLHSE         BIT(3)
> > +#define UNICAM_DLTRE         BIT(4)
> > +#define UNICAM_DLSM          BIT(5)
> > +#define UNICAM_DLFO          BIT(28)
> > +#define UNICAM_DLSTE         BIT(29)
> > +
> > +#define UNICAM_DAT_MASK_ALL  (UNICAM_DLSTE | UNICAM_DLFO)
> > +
> > +/* UNICAM_DLT Register */
> > +#define UNICAM_DLT1_MASK     GENMASK(7, 0)
> > +#define UNICAM_DLT2_MASK     GENMASK(15, 8)
> > +#define UNICAM_DLT3_MASK     GENMASK(23, 16)
> > +
> > +/* UNICAM_ICTL Register */
> > +#define UNICAM_FSIE          BIT(0)
> > +#define UNICAM_FEIE          BIT(1)
> > +#define UNICAM_IBOB          BIT(2)
> > +#define UNICAM_FCM           BIT(3)
> > +#define UNICAM_TFC           BIT(4)
> > +#define UNICAM_LIP_MASK              GENMASK(6, 5)
> > +#define UNICAM_LCIE_MASK     GENMASK(28, 16)
> > +
> > +/* UNICAM_IDI0/1 Register */
> > +#define UNICAM_ID0_MASK              GENMASK(7, 0)
> > +#define UNICAM_ID1_MASK              GENMASK(15, 8)
> > +#define UNICAM_ID2_MASK              GENMASK(23, 16)
> > +#define UNICAM_ID3_MASK              GENMASK(31, 24)
> > +
> > +/* UNICAM_ISTA Register */
> > +#define UNICAM_FSI           BIT(0)
> > +#define UNICAM_FEI           BIT(1)
> > +#define UNICAM_LCI           BIT(2)
> > +
> > +#define UNICAM_ISTA_MASK_ALL (UNICAM_FSI | UNICAM_FEI | UNICAM_LCI)
> > +
> > +/* UNICAM_IPIPE Register */
> > +#define UNICAM_PUM_MASK              GENMASK(2, 0)
> > +/* Unpacking modes */
> > +#define UNICAM_PUM_NONE              0
> > +#define UNICAM_PUM_UNPACK6   1
> > +#define UNICAM_PUM_UNPACK7   2
> > +#define UNICAM_PUM_UNPACK8   3
> > +#define UNICAM_PUM_UNPACK10  4
> > +#define UNICAM_PUM_UNPACK12  5
> > +#define UNICAM_PUM_UNPACK14  6
> > +#define UNICAM_PUM_UNPACK16  7
> > +#define UNICAM_DDM_MASK              GENMASK(6, 3)
> > +#define UNICAM_PPM_MASK              GENMASK(9, 7)
> > +/* Packing modes */
> > +#define UNICAM_PPM_NONE              0
> > +#define UNICAM_PPM_PACK8     1
> > +#define UNICAM_PPM_PACK10    2
> > +#define UNICAM_PPM_PACK12    3
> > +#define UNICAM_PPM_PACK14    4
> > +#define UNICAM_PPM_PACK16    5
> > +#define UNICAM_DEM_MASK              GENMASK(11, 10)
> > +#define UNICAM_DEBL_MASK     GENMASK(14, 12)
> > +#define UNICAM_ICM_MASK              GENMASK(16, 15)
> > +#define UNICAM_IDM_MASK              GENMASK(17, 17)
> > +
> > +/* UNICAM_ICC Register */
> > +#define UNICAM_ICFL_MASK     GENMASK(4, 0)
> > +#define UNICAM_ICFH_MASK     GENMASK(9, 5)
> > +#define UNICAM_ICST_MASK     GENMASK(12, 10)
> > +#define UNICAM_ICLT_MASK     GENMASK(15, 13)
> > +#define UNICAM_ICLL_MASK     GENMASK(31, 16)
> > +
> > +/* UNICAM_DCS Register */
> > +#define UNICAM_DIE           BIT(0)
> > +#define UNICAM_DIM           BIT(1)
> > +#define UNICAM_DBOB          BIT(3)
> > +#define UNICAM_FDE           BIT(4)
> > +#define UNICAM_LDP           BIT(5)
> > +#define UNICAM_EDL_MASK              GENMASK(15, 8)
> > +
> > +/* UNICAM_DBCTL Register */
> > +#define UNICAM_DBEN          BIT(0)
> > +#define UNICAM_BUF0_IE               BIT(1)
> > +#define UNICAM_BUF1_IE               BIT(2)
> > +
> > +/* UNICAM_CMP[0,1] register */
> > +#define UNICAM_PCE           BIT(31)
> > +#define UNICAM_GI            BIT(9)
> > +#define UNICAM_CPH           BIT(8)
> > +#define UNICAM_PCVC_MASK     GENMASK(7, 6)
> > +#define UNICAM_PCDT_MASK     GENMASK(5, 0)
> > +
> > +/* UNICAM_MISC register */
> > +#define UNICAM_FL0           BIT(6)
> > +#define UNICAM_FL1           BIT(9)
> > +
> > +#endif
> > diff --git a/drivers/media/platform/broadcom/bcm2835-unicam.c b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > new file mode 100644
> > index 000000000000..716c89b8a217
> > --- /dev/null
> > +++ b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > @@ -0,0 +1,2607 @@
> > +// SPDX-License-Identifier: GPL-2.0-only
> > +/*
> > + * BCM283x / BCM271x Unicam Capture Driver
> > + *
> > + * Copyright (C) 2017-2020 - Raspberry Pi (Trading) Ltd.
> > + *
> > + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> > + *
> > + * Based on TI am437x driver by
> > + *   Benoit Parrot <bparrot@ti.com>
> > + *   Lad, Prabhakar <prabhakar.csengg@gmail.com>
> > + *
> > + * and TI CAL camera interface driver by
> > + *    Benoit Parrot <bparrot@ti.com>
> > + *
> > + *
> > + * There are two camera drivers in the kernel for BCM283x - this one and
> > + * bcm2835-camera (currently in staging).
> > + *
> > + * This driver directly controls the Unicam peripheral - there is no
> > + * involvement with the VideoCore firmware. Unicam receives CSI-2 or CCP2 data
> > + * and writes it into SDRAM. The only potential processing options are to
> > + * repack Bayer data into an alternate format, and applying windowing. The
> > + * repacking does not shift the data, so can repack V4L2_PIX_FMT_Sxxxx10P to
> > + * V4L2_PIX_FMT_Sxxxx10, or V4L2_PIX_FMT_Sxxxx12P to V4L2_PIX_FMT_Sxxxx12, but
> > + * not generically up to V4L2_PIX_FMT_Sxxxx16. The driver will add both formats
> > + * where the relevant formats are defined, and will automatically configure the
> > + * repacking as required. Support for windowing may be added later.
>
> Does this last paragraph applies ? I see all the supported packed raw
> format having an 'unpacked_fourcc' associated (all but the 8 bit ones
> ofc)

The penultimate sentence ("The driver will add both formats where the
relevant formats are defined") can be dropped as you've dropped the
video-node centric mode of operation, therefore all formats (whether
packed or unpacked) are always advertised.

The rest of the paragraph still applies - if you request a
V4L2_PIX_FMT_SxxxxBB (not packed) format, then it the incoming data
will be unpacked for you.

> > + *
> > + * It should be possible to connect this driver to any sensor with a suitable
> > + * output interface and V4L2 subdevice driver.
> > + */
> > +
> > +#include <linux/clk.h>
> > +#include <linux/delay.h>
> > +#include <linux/device.h>
> > +#include <linux/dma-mapping.h>
> > +#include <linux/err.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/io.h>
> > +#include <linux/module.h>
> > +#include <linux/of.h>
> > +#include <linux/of_device.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/slab.h>
> > +#include <linux/videodev2.h>
> > +
> > +#include <media/v4l2-async.h>
> > +#include <media/v4l2-common.h>
> > +#include <media/v4l2-dev.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/v4l2-fwnode.h>
> > +#include <media/v4l2-mc.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "bcm2835-unicam-regs.h"
> > +
> > +#define UNICAM_MODULE_NAME           "unicam"
> > +
> > +/*
> > + * Unicam must request a minimum of 250Mhz from the VPU clock.
> > + * Otherwise the input FIFOs overrun and cause image corruption.
> > + */
> > +#define UNICAM_MIN_VPU_CLOCK_RATE    (250 * 1000 * 1000)
>
> Shouldn't this be set in DT ?
>
> > +
> > +/* Unicam has an internal DMA alignment constraint of 16 bytes for each line. */
> > +#define UNICAM_DMA_BPL_ALIGNMENT     16
> > +
> > +/*
> > + * The image stride is stored in a 16 bit register, and needs to be aligned to
> > + * the DMA constraint. As the ISP in the same SoC has a 32 bytes alignment
> > + * constraint on its input, set the image stride alignment to 32 bytes here as
> > + * well to avoid incompatible configurations.
> > + */
> > +#define UNICAM_IMAGE_BPL_ALIGNMENT   32
> > +#define UNICAM_IMAGE_MAX_BPL         ((1 << 16) - UNICAM_IMAGE_BPL_ALIGNMENT)
> > +
> > +/*
> > + * Max width is therefore determined by the max stride divided by the number of
> > + * bits per pixel. Take 32bpp as a worst case. No imposed limit on the height,
> > + * so adopt a square image for want of anything better.
> > + */
> > +#define UNICAM_IMAGE_MIN_WIDTH               16
> > +#define UNICAM_IMAGE_MIN_HEIGHT              16
> > +#define UNICAM_IMAGE_MAX_WIDTH               (UNICAM_IMAGE_MAX_BPL / 4)
> > +#define UNICAM_IMAGE_MAX_HEIGHT              UNICAM_IMAGE_MAX_WIDTH
> > +
> > +/*
> > + * There's no intrinsic limits on the width and height for embedded dat. Use
> > + * the same maximum values as for the image, to avoid overflows in the image
> > + * size computation.
> > + */
> > +#define UNICAM_META_MIN_WIDTH                1
> > +#define UNICAM_META_MIN_HEIGHT               1
> > +#define UNICAM_META_MAX_WIDTH                UNICAM_IMAGE_MAX_WIDTH
> > +#define UNICAM_META_MAX_HEIGHT               UNICAM_IMAGE_MAX_HEIGHT
> > +
> > +/*
> > + * Size of the dummy buffer. Can be any size really, but the DMA
> > + * allocation works in units of page sizes.
> > + */
> > +#define UNICAM_DUMMY_BUF_SIZE                PAGE_SIZE
> > +
> > +#define UNICAM_SD_PAD_SINK           0
> > +#define UNICAM_SD_PAD_SOURCE_IMAGE   1
> > +#define UNICAM_SD_PAD_SOURCE_METADATA        2
> > +#define UNICAM_SD_NUM_PADS           (1 + UNICAM_SD_PAD_SOURCE_METADATA)
>
> How about an enum then ?
>
> > +
> > +enum unicam_node_type {
> > +     UNICAM_IMAGE_NODE,
> > +     UNICAM_METADATA_NODE,
> > +     UNICAM_MAX_NODES
> > +};
> > +
> > +/*
> > + * struct unicam_format_info - Unicam media bus format information
> > + * @fourcc: V4L2 pixel format FCC identifier. 0 if n/a.
> > + * @unpacked_fourcc: V4L2 pixel format FCC identifier if the data is expanded
> > + * out to 16bpp. 0 if n/a.
> > + * @code: V4L2 media bus format code.
> > + * @depth: Bits per pixel as delivered from the source.
> > + * @csi_dt: CSI data type.
> > + * @metadata_fmt: This format only applies to the metadata pad.
> > + */
> > +struct unicam_format_info {
> > +     u32     fourcc;
> > +     u32     unpacked_fourcc;
> > +     u32     code;
> > +     u8      depth;
> > +     u8      csi_dt;
> > +     bool    metadata_fmt;
> > +};
> > +
> > +struct unicam_buffer {
> > +     struct vb2_v4l2_buffer vb;
> > +     struct list_head list;
> > +     dma_addr_t dma_addr;
> > +     unsigned int size;
> > +};
> > +
> > +static inline struct unicam_buffer *to_unicam_buffer(struct vb2_buffer *vb)
> > +{
> > +     return container_of(vb, struct unicam_buffer, vb.vb2_buf);
> > +}
> > +
> > +struct unicam_node {
> > +     bool registered;
> > +     bool streaming;
> > +     unsigned int id;
> > +
> > +     /* Pointer to the current v4l2_buffer */
> > +     struct unicam_buffer *cur_frm;
> > +     /* Pointer to the next v4l2_buffer */
> > +     struct unicam_buffer *next_frm;
> > +     /* video capture */
> > +     const struct unicam_format_info *fmtinfo;
> > +     /* Used to store current pixel format */
> > +     struct v4l2_format fmt;
> > +     /* Buffer queue used in video-buf */
> > +     struct vb2_queue buffer_queue;
> > +     /* Queue of filled frames */
> > +     struct list_head dma_queue;
> > +     /* IRQ lock for DMA queue */
> > +     spinlock_t dma_queue_lock;
> > +     /* lock used to access this structure */
> > +     struct mutex lock;
> > +     /* Identifies video device for this channel */
> > +     struct video_device video_dev;
> > +     /* Pointer to the parent handle */
> > +     struct unicam_device *dev;
> > +     struct media_pad pad;
> > +     /*
> > +      * Dummy buffer intended to be used by unicam
> > +      * if we have no other queued buffers to swap to.
> > +      */
> > +     struct unicam_buffer dummy_buf;
> > +     void *dummy_buf_cpu_addr;
> > +};
> > +
> > +struct unicam_device {
> > +     struct kref kref;
> > +
> > +     /* peripheral base address */
> > +     void __iomem *base;
> > +     /* clock gating base address */
> > +     void __iomem *clk_gate_base;
>
> Is the clock gating part of unicam or should this be expressed as a
> clock provide and be referenced in DT through a phandle ?

Covered in previous reviews and accepted as part of unicam.
https://patchwork.kernel.org/project/linux-media/patch/fae3d29bba67825030c0077dd9c79534b6888512.1505916622.git.dave.stevenson@raspberrypi.org/

> > +     /* lp clock handle */
> > +     struct clk *clock;
> > +     /* vpu clock handle */
> > +     struct clk *vpu_clock;
> > +     /* V4l2 device */
> > +     struct v4l2_device v4l2_dev;
> > +     struct media_device mdev;
> > +
> > +     /* parent device */
> > +     struct device *dev;
> > +     /* subdevice async Notifier */
>
> s/Notifier/notifier
>
> > +     struct v4l2_async_notifier notifier;
> > +     unsigned int sequence;
> > +
> > +     /* Sensor node */
> > +     struct {
> > +             struct v4l2_subdev *subdev;
> > +             struct media_pad *pad;
> > +     } sensor;
> > +
> > +     /* Internal subdev */
> > +     struct {
> > +             struct v4l2_subdev sd;
> > +             struct media_pad pads[UNICAM_SD_NUM_PADS];
> > +             bool streaming;
> > +     } subdev;
> > +
> > +     enum v4l2_mbus_type bus_type;
> > +     /*
> > +      * Stores bus.mipi_csi2.flags for CSI2 sensors, or
> > +      * bus.mipi_csi1.strobe for CCP2.
> > +      */
> > +     unsigned int bus_flags;
> > +     unsigned int max_data_lanes;
> > +     unsigned int active_data_lanes;
> > +
> > +     struct media_pipeline pipe;
> > +
> > +     struct unicam_node node[UNICAM_MAX_NODES];
> > +};
> > +
> > +static inline struct unicam_device *
> > +notifier_to_unicam_device(struct v4l2_async_notifier *notifier)
> > +{
> > +     return container_of(notifier, struct unicam_device, notifier);
> > +}
> > +
> > +static inline struct unicam_device *
>
> I thought using inline is disouraged as the compiler knows what to do
>
> > +sd_to_unicam_device(struct v4l2_subdev *sd)
> > +{
> > +     return container_of(sd, struct unicam_device, subdev.sd);
>
> You can also use container_of_const() if you need it
>
> > +}
> > +
> > +static void unicam_release(struct kref *kref)
> > +{
> > +     struct unicam_device *unicam =
> > +             container_of(kref, struct unicam_device, kref);
> > +
> > +     if (unicam->mdev.dev)
> > +             media_device_cleanup(&unicam->mdev);
> > +
> > +     kfree(unicam);
> > +}
> > +
> > +static struct unicam_device *unicam_get(struct unicam_device *unicam)
> > +{
> > +     kref_get(&unicam->kref);
> > +     return unicam;
> > +}
> > +
> > +static void unicam_put(struct unicam_device *unicam)
> > +{
> > +     kref_put(&unicam->kref, unicam_release);
> > +}
> > +
>
> Does this handle a bit more gracefully the pesky lifetime management
> issue in v4l2 ?
>
> > +/* -----------------------------------------------------------------------------
> > + * Misc helper functions
> > + */
> > +
> > +static inline bool unicam_sd_pad_is_source(u32 pad)
> > +{
> > +     /* Camera RX has 1 sink pad, and N source pads */
> > +     return pad != UNICAM_SD_PAD_SINK;
> > +}
> > +
> > +static inline bool is_metadata_node(struct unicam_node *node)
> > +{
> > +     return node->video_dev.device_caps & V4L2_CAP_META_CAPTURE;
> > +}
> > +
> > +static inline bool is_image_node(struct unicam_node *node)
> > +{
> > +     return node->video_dev.device_caps & V4L2_CAP_VIDEO_CAPTURE;
> > +}
> > +
> > +/* -----------------------------------------------------------------------------
> > + * Format data table and helper functions
> > + */
> > +
> > +static const struct v4l2_mbus_framefmt unicam_default_image_format = {
> > +     .width = 640,
> > +     .height = 480,
> > +     .code = MEDIA_BUS_FMT_UYVY8_1X16,
> > +     .field = V4L2_FIELD_NONE,
> > +     .colorspace = V4L2_COLORSPACE_SRGB,
> > +     .ycbcr_enc = V4L2_YCBCR_ENC_601,
> > +     .quantization = V4L2_QUANTIZATION_LIM_RANGE,
> > +     .xfer_func = V4L2_XFER_FUNC_SRGB,
> > +     .flags = 0,
> > +};
> > +
> > +static const struct v4l2_mbus_framefmt unicam_default_meta_format = {
> > +     .width = 640,
> > +     .height = 2,
> > +     .code = MEDIA_BUS_FMT_META_8,
> > +     .field = V4L2_FIELD_NONE,
> > +};
> > +
> > +static const struct unicam_format_info unicam_image_formats[] = {
> > +     /* YUV Formats */
> > +     {
> > +             .fourcc         = V4L2_PIX_FMT_YUYV,
> > +             .code           = MEDIA_BUS_FMT_YUYV8_1X16,
> > +             .depth          = 16,
> > +             .csi_dt         = 0x1e,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_UYVY,
> > +             .code           = MEDIA_BUS_FMT_UYVY8_1X16,
> > +             .depth          = 16,
> > +             .csi_dt         = 0x1e,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_YVYU,
> > +             .code           = MEDIA_BUS_FMT_YVYU8_1X16,
> > +             .depth          = 16,
> > +             .csi_dt         = 0x1e,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_VYUY,
> > +             .code           = MEDIA_BUS_FMT_VYUY8_1X16,
> > +             .depth          = 16,
> > +             .csi_dt         = 0x1e,
> > +     }, {
> > +     /* RGB Formats */
> > +             .fourcc         = V4L2_PIX_FMT_RGB565, /* gggbbbbb rrrrrggg */
> > +             .code           = MEDIA_BUS_FMT_RGB565_1X16,
> > +             .depth          = 16,
> > +             .csi_dt         = 0x22,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_RGB24, /* rgb */
> > +             .code           = MEDIA_BUS_FMT_RGB888_1X24,
> > +             .depth          = 24,
> > +             .csi_dt         = 0x24,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_BGR24, /* bgr */
> > +             .code           = MEDIA_BUS_FMT_BGR888_1X24,
> > +             .depth          = 24,
> > +             .csi_dt         = 0x24,
> > +     }, {
> > +     /* Bayer Formats */
> > +             .fourcc         = V4L2_PIX_FMT_SBGGR8,
> > +             .code           = MEDIA_BUS_FMT_SBGGR8_1X8,
> > +             .depth          = 8,
> > +             .csi_dt         = 0x2a,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_SGBRG8,
> > +             .code           = MEDIA_BUS_FMT_SGBRG8_1X8,
> > +             .depth          = 8,
> > +             .csi_dt         = 0x2a,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_SGRBG8,
> > +             .code           = MEDIA_BUS_FMT_SGRBG8_1X8,
> > +             .depth          = 8,
> > +             .csi_dt         = 0x2a,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_SRGGB8,
> > +             .code           = MEDIA_BUS_FMT_SRGGB8_1X8,
> > +             .depth          = 8,
> > +             .csi_dt         = 0x2a,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_SBGGR10P,
> > +             .unpacked_fourcc = V4L2_PIX_FMT_SBGGR10,
> > +             .code           = MEDIA_BUS_FMT_SBGGR10_1X10,
> > +             .depth          = 10,
> > +             .csi_dt         = 0x2b,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_SGBRG10P,
> > +             .unpacked_fourcc = V4L2_PIX_FMT_SGBRG10,
> > +             .code           = MEDIA_BUS_FMT_SGBRG10_1X10,
> > +             .depth          = 10,
> > +             .csi_dt         = 0x2b,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_SGRBG10P,
> > +             .unpacked_fourcc = V4L2_PIX_FMT_SGRBG10,
> > +             .code           = MEDIA_BUS_FMT_SGRBG10_1X10,
> > +             .depth          = 10,
> > +             .csi_dt         = 0x2b,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_SRGGB10P,
> > +             .unpacked_fourcc = V4L2_PIX_FMT_SRGGB10,
> > +             .code           = MEDIA_BUS_FMT_SRGGB10_1X10,
> > +             .depth          = 10,
> > +             .csi_dt         = 0x2b,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_SBGGR12P,
> > +             .unpacked_fourcc = V4L2_PIX_FMT_SBGGR12,
> > +             .code           = MEDIA_BUS_FMT_SBGGR12_1X12,
> > +             .depth          = 12,
> > +             .csi_dt         = 0x2c,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_SGBRG12P,
> > +             .unpacked_fourcc = V4L2_PIX_FMT_SGBRG12,
> > +             .code           = MEDIA_BUS_FMT_SGBRG12_1X12,
> > +             .depth          = 12,
> > +             .csi_dt         = 0x2c,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_SGRBG12P,
> > +             .unpacked_fourcc = V4L2_PIX_FMT_SGRBG12,
> > +             .code           = MEDIA_BUS_FMT_SGRBG12_1X12,
> > +             .depth          = 12,
> > +             .csi_dt         = 0x2c,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_SRGGB12P,
> > +             .unpacked_fourcc = V4L2_PIX_FMT_SRGGB12,
> > +             .code           = MEDIA_BUS_FMT_SRGGB12_1X12,
> > +             .depth          = 12,
> > +             .csi_dt         = 0x2c,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_SBGGR14P,
> > +             .unpacked_fourcc = V4L2_PIX_FMT_SBGGR14,
> > +             .code           = MEDIA_BUS_FMT_SBGGR14_1X14,
> > +             .depth          = 14,
> > +             .csi_dt         = 0x2d,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_SGBRG14P,
> > +             .unpacked_fourcc = V4L2_PIX_FMT_SGBRG14,
> > +             .code           = MEDIA_BUS_FMT_SGBRG14_1X14,
> > +             .depth          = 14,
> > +             .csi_dt         = 0x2d,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_SGRBG14P,
> > +             .unpacked_fourcc = V4L2_PIX_FMT_SGRBG14,
> > +             .code           = MEDIA_BUS_FMT_SGRBG14_1X14,
> > +             .depth          = 14,
> > +             .csi_dt         = 0x2d,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_SRGGB14P,
> > +             .unpacked_fourcc = V4L2_PIX_FMT_SRGGB14,
> > +             .code           = MEDIA_BUS_FMT_SRGGB14_1X14,
> > +             .depth          = 14,
> > +             .csi_dt         = 0x2d,
> > +     }, {
> > +     /* 16 bit Bayer formats could be supported. */
> > +
> > +     /* Greyscale formats */
> > +             .fourcc         = V4L2_PIX_FMT_GREY,
> > +             .code           = MEDIA_BUS_FMT_Y8_1X8,
> > +             .depth          = 8,
> > +             .csi_dt         = 0x2a,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_Y10P,
> > +             .unpacked_fourcc = V4L2_PIX_FMT_Y10,
> > +             .code           = MEDIA_BUS_FMT_Y10_1X10,
> > +             .depth          = 10,
> > +             .csi_dt         = 0x2b,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_Y12P,
> > +             .unpacked_fourcc = V4L2_PIX_FMT_Y12,
> > +             .code           = MEDIA_BUS_FMT_Y12_1X12,
> > +             .depth          = 12,
> > +             .csi_dt         = 0x2c,
> > +     }, {
> > +             .fourcc         = V4L2_PIX_FMT_Y14P,
> > +             .unpacked_fourcc = V4L2_PIX_FMT_Y14,
> > +             .code           = MEDIA_BUS_FMT_Y14_1X14,
> > +             .depth          = 14,
> > +             .csi_dt         = 0x2d,
> > +     },
> > +};
> > +
> > +static const struct unicam_format_info unicam_meta_formats[] = {
> > +     {
> > +             .fourcc         = V4L2_META_FMT_GENERIC_8,
> > +             .code           = MEDIA_BUS_FMT_META_8,
> > +             .depth          = 8,
> > +             .metadata_fmt   = true,
> > +     }, {
> > +             .fourcc         = V4L2_META_FMT_GENERIC_CSI2_10,
> > +             .code           = MEDIA_BUS_FMT_META_10,
> > +             .depth          = 10,
> > +             .metadata_fmt   = true,
> > +     }, {
> > +             .fourcc         = V4L2_META_FMT_GENERIC_CSI2_12,
> > +             .code           = MEDIA_BUS_FMT_META_12,
> > +             .depth          = 12,
> > +             .metadata_fmt   = true,
> > +     }, {
> > +             .fourcc         = V4L2_META_FMT_GENERIC_CSI2_14,
> > +             .code           = MEDIA_BUS_FMT_META_14,
> > +             .depth          = 14,
> > +             .metadata_fmt   = true,
> > +     },
> > +};
> > +
> > +/* Format setup functions */
> > +static const struct unicam_format_info *
> > +unicam_find_format_by_code(u32 code, u32 pad)
> > +{
> > +     const struct unicam_format_info *formats;
> > +     unsigned int num_formats;
> > +     unsigned int i;
> > +
> > +     if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> > +             formats = unicam_image_formats;
> > +             num_formats = ARRAY_SIZE(unicam_image_formats);
> > +     } else {
> > +             formats = unicam_meta_formats;
> > +             num_formats = ARRAY_SIZE(unicam_meta_formats);
> > +     }
> > +
> > +     for (i = 0; i < num_formats; i++) {
> > +             if (formats[i].code == code)
> > +                     return &formats[i];
> > +     }
> > +
> > +     return NULL;
> > +}
> > +
> > +static const struct unicam_format_info *
> > +unicam_find_format_by_fourcc(u32 fourcc, u32 pad)
> > +{
> > +     const struct unicam_format_info *formats;
> > +     unsigned int num_formats;
> > +     unsigned int i;
> > +
> > +     if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> > +             formats = unicam_image_formats;
> > +             num_formats = ARRAY_SIZE(unicam_image_formats);
> > +     } else {
> > +             formats = unicam_meta_formats;
> > +             num_formats = ARRAY_SIZE(unicam_meta_formats);
> > +     }
> > +
> > +     for (i = 0; i < num_formats; ++i) {
> > +             if (formats[i].fourcc == fourcc)
> > +                     return &formats[i];
> > +     }
> > +
> > +     return NULL;
> > +}
> > +
> > +static void unicam_calc_image_size_bpl(struct unicam_device *unicam,
> > +                                    const struct unicam_format_info *fmtinfo,
> > +                                    struct v4l2_pix_format *pix)
> > +{
> > +     u32 min_bpl;
> > +
> > +     v4l_bound_align_image(&pix->width, UNICAM_IMAGE_MIN_WIDTH,
> > +                           UNICAM_IMAGE_MAX_WIDTH, 2,
> > +                           &pix->height, UNICAM_IMAGE_MIN_HEIGHT,
> > +                           UNICAM_IMAGE_MAX_HEIGHT, 0, 0);
> > +
> > +     /* Unpacking always goes to 16bpp */
> > +     if (pix->pixelformat == fmtinfo->unpacked_fourcc)
> > +             min_bpl = pix->width * 2;
> > +     else
> > +             min_bpl = pix->width * fmtinfo->depth / 8;
> > +     min_bpl = ALIGN(min_bpl, UNICAM_IMAGE_BPL_ALIGNMENT);
> > +
> > +     pix->bytesperline = ALIGN(pix->bytesperline, UNICAM_IMAGE_BPL_ALIGNMENT);
> > +     pix->bytesperline = clamp_t(unsigned int, pix->bytesperline, min_bpl,
> > +                                 UNICAM_IMAGE_MAX_BPL);
> > +
> > +     pix->sizeimage = pix->height * pix->bytesperline;
> > +}
> > +
> > +static void unicam_calc_meta_size_bpl(struct unicam_device *unicam,
> > +                                   const struct unicam_format_info *fmtinfo,
> > +                                   struct v4l2_meta_format *meta)
> > +{
> > +     v4l_bound_align_image(&meta->width, UNICAM_META_MIN_WIDTH,
> > +                           UNICAM_META_MAX_WIDTH, 0,
> > +                           &meta->height, UNICAM_META_MIN_HEIGHT,
> > +                           UNICAM_META_MAX_HEIGHT, 0, 0);
> > +
> > +     meta->bytesperline = ALIGN(meta->width * fmtinfo->depth / 8,
> > +                                UNICAM_DMA_BPL_ALIGNMENT);
> > +     meta->buffersize = meta->height * meta->bytesperline;
> > +}
> > +
> > +/* -----------------------------------------------------------------------------
> > + * Hardware handling
> > + */
> > +
>
> As this has been used for years in BSP driver, I won't go in detail in
> the HW handling functions
>
> > +static inline void unicam_clk_write(struct unicam_device *unicam, u32 val)
> > +{
> > +     /* Pass the CM_PASSWORD along with the value. */
> > +     writel(val | 0x5a000000, unicam->clk_gate_base);
> > +}
> > +
> > +static inline u32 unicam_reg_read(struct unicam_device *unicam, u32 offset)
> > +{
> > +     return readl(unicam->base + offset);
> > +}
> > +
> > +static inline void unicam_reg_write(struct unicam_device *unicam, u32 offset, u32 val)
> > +{
> > +     writel(val, unicam->base + offset);
> > +}
> > +
> > +static inline int unicam_get_field(u32 value, u32 mask)
> > +{
> > +     return (value & mask) >> __ffs(mask);
> > +}
> > +
> > +static inline void unicam_set_field(u32 *valp, u32 field, u32 mask)
> > +{
> > +     u32 val = *valp;
> > +
> > +     val &= ~mask;
> > +     val |= (field << __ffs(mask)) & mask;
> > +     *valp = val;
> > +}
> > +
> > +static inline void unicam_reg_write_field(struct unicam_device *unicam, u32 offset,
> > +                                       u32 field, u32 mask)
> > +{
> > +     u32 val = unicam_reg_read(unicam, offset);
> > +
> > +     unicam_set_field(&val, field, mask);
> > +     unicam_reg_write(unicam, offset, val);
> > +}
> > +
> > +static void unicam_wr_dma_addr(struct unicam_node *node,
> > +                            struct unicam_buffer *buf)
> > +{
> > +     dma_addr_t endaddr = buf->dma_addr + buf->size;
> > +
> > +     if (node->id == UNICAM_IMAGE_NODE) {
> > +             unicam_reg_write(node->dev, UNICAM_IBSA0, buf->dma_addr);
> > +             unicam_reg_write(node->dev, UNICAM_IBEA0, endaddr);
> > +     } else {
> > +             unicam_reg_write(node->dev, UNICAM_DBSA0, buf->dma_addr);
> > +             unicam_reg_write(node->dev, UNICAM_DBEA0, endaddr);
> > +     }
> > +}
> > +
> > +static unsigned int unicam_get_lines_done(struct unicam_device *unicam)
> > +{
> > +     struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > +     unsigned int stride = node->fmt.fmt.pix.bytesperline;
> > +     struct unicam_buffer *frm = node->cur_frm;
> > +     dma_addr_t cur_addr;
> > +
> > +     if (!frm)
> > +             return 0;
> > +
> > +     cur_addr = unicam_reg_read(unicam, UNICAM_IBWP);
> > +     return (unsigned int)(cur_addr - frm->dma_addr) / stride;
> > +}
> > +
> > +static void unicam_schedule_next_buffer(struct unicam_node *node)
> > +{
> > +     struct unicam_buffer *buf;
> > +
> > +     buf = list_first_entry(&node->dma_queue, struct unicam_buffer, list);
> > +     node->next_frm = buf;
> > +     list_del(&buf->list);
> > +
> > +     unicam_wr_dma_addr(node, buf);
> > +}
> > +
> > +static void unicam_schedule_dummy_buffer(struct unicam_node *node)
> > +{
> > +     int node_id = is_image_node(node) ? UNICAM_IMAGE_NODE : UNICAM_METADATA_NODE;
> > +
> > +     dev_dbg(node->dev->dev, "Scheduling dummy buffer for node %d\n", node_id);
> > +
> > +     unicam_wr_dma_addr(node, &node->dummy_buf);
> > +
> > +     node->next_frm = NULL;
> > +}
> > +
> > +static void unicam_process_buffer_complete(struct unicam_node *node,
> > +                                        unsigned int sequence)
> > +{
> > +     node->cur_frm->vb.field = node->fmt.fmt.pix.field;
> > +     node->cur_frm->vb.sequence = sequence;
> > +
> > +     vb2_buffer_done(&node->cur_frm->vb.vb2_buf, VB2_BUF_STATE_DONE);
> > +}
> > +
> > +static void unicam_queue_event_sof(struct unicam_device *unicam)
> > +{
> > +     struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > +
> > +     struct v4l2_event event = {
> > +             .type = V4L2_EVENT_FRAME_SYNC,
> > +             .u.frame_sync.frame_sequence = unicam->sequence,
> > +     };
> > +
> > +     v4l2_event_queue(&node->video_dev, &event);
> > +}
> > +
> > +static irqreturn_t unicam_isr(int irq, void *dev)
> > +{
> > +     struct unicam_device *unicam = dev;
> > +     unsigned int lines_done = unicam_get_lines_done(dev);
> > +     unsigned int sequence = unicam->sequence;
> > +     unsigned int i;
> > +     u32 ista, sta;
> > +     bool fe;
> > +     u64 ts;
> > +
> > +     sta = unicam_reg_read(unicam, UNICAM_STA);
> > +     /* Write value back to clear the interrupts */
> > +     unicam_reg_write(unicam, UNICAM_STA, sta);
> > +
> > +     ista = unicam_reg_read(unicam, UNICAM_ISTA);
> > +     /* Write value back to clear the interrupts */
> > +     unicam_reg_write(unicam, UNICAM_ISTA, ista);
> > +
> > +     dev_dbg(unicam->dev, "ISR: ISTA: 0x%X, STA: 0x%X, sequence %d, lines done %d\n",
> > +             ista, sta, sequence, lines_done);
> > +
> > +     if (!(sta & (UNICAM_IS | UNICAM_PI0)))
> > +             return IRQ_HANDLED;
> > +
> > +     /*
> > +      * Look for either the Frame End interrupt or the Packet Capture status
> > +      * to signal a frame end.
> > +      */
> > +     fe = ista & UNICAM_FEI || sta & UNICAM_PI0;
> > +
> > +     /*
> > +      * We must run the frame end handler first. If we have a valid next_frm
> > +      * and we get a simultaneout FE + FS interrupt, running the FS handler
> > +      * first would null out the next_frm ptr and we would have lost the
> > +      * buffer forever.
> > +      */
> > +     if (fe) {
> > +             /*
> > +              * Ensure we have swapped buffers already as we can't
> > +              * stop the peripheral. If no buffer is available, use a
> > +              * dummy buffer to dump out frames until we get a new buffer
> > +              * to use.
> > +              */
> > +             for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > +                     if (!unicam->node[i].streaming)
> > +                             continue;
> > +
> > +                     /*
> > +                      * If cur_frm == next_frm, it means we have not had
> > +                      * a chance to swap buffers, likely due to having
> > +                      * multiple interrupts occurring simultaneously (like FE
> > +                      * + FS + LS). In this case, we cannot signal the buffer
> > +                      * as complete, as the HW will reuse that buffer.
> > +                      */
> > +                     if (unicam->node[i].cur_frm &&
> > +                         unicam->node[i].cur_frm != unicam->node[i].next_frm)
> > +                             unicam_process_buffer_complete(&unicam->node[i],
> > +                                                            sequence);
> > +                     unicam->node[i].cur_frm = unicam->node[i].next_frm;
> > +             }
> > +             unicam->sequence++;
> > +     }
> > +
> > +     if (ista & UNICAM_FSI) {
> > +             /*
> > +              * Timestamp is to be when the first data byte was captured,
> > +              * aka frame start.
> > +              */
> > +             ts = ktime_get_ns();
> > +             for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > +                     if (!unicam->node[i].streaming)
> > +                             continue;
> > +
> > +                     if (unicam->node[i].cur_frm)
> > +                             unicam->node[i].cur_frm->vb.vb2_buf.timestamp =
> > +                                                             ts;
> > +                     else
> > +                             dev_dbg(unicam->v4l2_dev.dev,
> > +                                     "ISR: [%d] Dropping frame, buffer not available at FS\n",
> > +                                     i);
> > +                     /*
> > +                      * Set the next frame output to go to a dummy frame
> > +                      * if we have not managed to obtain another frame
> > +                      * from the queue.
> > +                      */
> > +                     unicam_schedule_dummy_buffer(&unicam->node[i]);
> > +             }
> > +
> > +             unicam_queue_event_sof(unicam);
> > +     }
> > +
> > +     /*
> > +      * Cannot swap buffer at frame end, there may be a race condition
> > +      * where the HW does not actually swap it if the new frame has
> > +      * already started.
> > +      */
> > +     if (ista & (UNICAM_FSI | UNICAM_LCI) && !fe) {
> > +             for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > +                     if (!unicam->node[i].streaming)
> > +                             continue;
> > +
> > +                     spin_lock(&unicam->node[i].dma_queue_lock);
> > +                     if (!list_empty(&unicam->node[i].dma_queue) &&
> > +                         !unicam->node[i].next_frm)
> > +                             unicam_schedule_next_buffer(&unicam->node[i]);
> > +                     spin_unlock(&unicam->node[i].dma_queue_lock);
> > +             }
> > +     }
> > +
> > +     if (unicam_reg_read(unicam, UNICAM_ICTL) & UNICAM_FCM) {
> > +             /* Switch out of trigger mode if selected */
> > +             unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> > +             unicam_reg_write_field(unicam, UNICAM_ICTL, 0, UNICAM_FCM);
> > +     }
> > +     return IRQ_HANDLED;
> > +}
> > +
> > +static void unicam_set_packing_config(struct unicam_device *unicam)
> > +{
> > +     struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > +     u32 pack, unpack;
> > +     u32 val;
> > +
> > +     if (node->fmt.fmt.pix.pixelformat == node->fmtinfo->fourcc) {
> > +             unpack = UNICAM_PUM_NONE;
> > +             pack = UNICAM_PPM_NONE;
> > +     } else {
> > +             switch (node->fmtinfo->depth) {
> > +             case 8:
> > +                     unpack = UNICAM_PUM_UNPACK8;
> > +                     break;
> > +             case 10:
> > +                     unpack = UNICAM_PUM_UNPACK10;
> > +                     break;
> > +             case 12:
> > +                     unpack = UNICAM_PUM_UNPACK12;
> > +                     break;
> > +             case 14:
> > +                     unpack = UNICAM_PUM_UNPACK14;
> > +                     break;
> > +             case 16:
> > +                     unpack = UNICAM_PUM_UNPACK16;
> > +                     break;
> > +             default:
> > +                     unpack = UNICAM_PUM_NONE;
> > +                     break;
> > +             }
> > +
> > +             /* Repacking is always to 16bpp */
> > +             pack = UNICAM_PPM_PACK16;
> > +     }
> > +
> > +     val = 0;
> > +     unicam_set_field(&val, unpack, UNICAM_PUM_MASK);
> > +     unicam_set_field(&val, pack, UNICAM_PPM_MASK);
> > +     unicam_reg_write(unicam, UNICAM_IPIPE, val);
> > +}
> > +
> > +static void unicam_cfg_image_id(struct unicam_device *unicam)
> > +{
> > +     struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > +
> > +     if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> > +             /* CSI2 mode, hardcode VC 0 for now. */
> > +             unicam_reg_write(unicam, UNICAM_IDI0,
> > +                              (0 << 6) | node->fmtinfo->csi_dt);
> > +     } else {
> > +             /* CCP2 mode */
> > +             unicam_reg_write(unicam, UNICAM_IDI0,
> > +                              0x80 | node->fmtinfo->csi_dt);
> > +     }
> > +}
> > +
> > +static void unicam_enable_ed(struct unicam_device *unicam)
> > +{
> > +     u32 val = unicam_reg_read(unicam, UNICAM_DCS);
> > +
> > +     unicam_set_field(&val, 2, UNICAM_EDL_MASK);
> > +     /* Do not wrap at the end of the embedded data buffer */
> > +     unicam_set_field(&val, 0, UNICAM_DBOB);
> > +
> > +     unicam_reg_write(unicam, UNICAM_DCS, val);
> > +}
> > +
> > +static void unicam_start_rx(struct unicam_device *unicam,
> > +                         struct unicam_buffer *buf)
> > +{
> > +     struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > +     int line_int_freq = node->fmt.fmt.pix.height >> 2;
> > +     unsigned int i;
> > +     u32 val;
> > +
> > +     if (line_int_freq < 128)
> > +             line_int_freq = 128;
> > +
> > +     /* Enable lane clocks */
> > +     val = 1;
> > +     for (i = 0; i < unicam->active_data_lanes; i++)
> > +             val = val << 2 | 1;
> > +     unicam_clk_write(unicam, val);
> > +
> > +     /* Basic init */
> > +     unicam_reg_write(unicam, UNICAM_CTRL, UNICAM_MEM);
> > +
> > +     /* Enable analogue control, and leave in reset. */
> > +     val = UNICAM_AR;
> > +     unicam_set_field(&val, 7, UNICAM_CTATADJ_MASK);
> > +     unicam_set_field(&val, 7, UNICAM_PTATADJ_MASK);
> > +     unicam_reg_write(unicam, UNICAM_ANA, val);
> > +     usleep_range(1000, 2000);
> > +
> > +     /* Come out of reset */
> > +     unicam_reg_write_field(unicam, UNICAM_ANA, 0, UNICAM_AR);
> > +
> > +     /* Peripheral reset */
> > +     unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPR);
> > +     unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPR);
> > +
> > +     unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPE);
> > +
> > +     /* Enable Rx control. */
> > +     val = unicam_reg_read(unicam, UNICAM_CTRL);
> > +     if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> > +             unicam_set_field(&val, UNICAM_CPM_CSI2, UNICAM_CPM_MASK);
> > +             unicam_set_field(&val, UNICAM_DCM_STROBE, UNICAM_DCM_MASK);
> > +     } else {
> > +             unicam_set_field(&val, UNICAM_CPM_CCP2, UNICAM_CPM_MASK);
> > +             unicam_set_field(&val, unicam->bus_flags, UNICAM_DCM_MASK);
> > +     }
> > +     /* Packet framer timeout */
> > +     unicam_set_field(&val, 0xf, UNICAM_PFT_MASK);
> > +     unicam_set_field(&val, 128, UNICAM_OET_MASK);
> > +     unicam_reg_write(unicam, UNICAM_CTRL, val);
> > +
> > +     unicam_reg_write(unicam, UNICAM_IHWIN, 0);
> > +     unicam_reg_write(unicam, UNICAM_IVWIN, 0);
> > +
> > +     /* AXI bus access QoS setup */
> > +     val = unicam_reg_read(unicam, UNICAM_PRI);
> > +     unicam_set_field(&val, 0, UNICAM_BL_MASK);
> > +     unicam_set_field(&val, 0, UNICAM_BS_MASK);
> > +     unicam_set_field(&val, 0xe, UNICAM_PP_MASK);
> > +     unicam_set_field(&val, 8, UNICAM_NP_MASK);
> > +     unicam_set_field(&val, 2, UNICAM_PT_MASK);
> > +     unicam_set_field(&val, 1, UNICAM_PE);
> > +     unicam_reg_write(unicam, UNICAM_PRI, val);
> > +
> > +     unicam_reg_write_field(unicam, UNICAM_ANA, 0, UNICAM_DDL);
> > +
> > +     /* Always start in trigger frame capture mode (UNICAM_FCM set) */
> > +     val = UNICAM_FSIE | UNICAM_FEIE | UNICAM_FCM | UNICAM_IBOB;
> > +     unicam_set_field(&val, line_int_freq, UNICAM_LCIE_MASK);
> > +     unicam_reg_write(unicam, UNICAM_ICTL, val);
> > +     unicam_reg_write(unicam, UNICAM_STA, UNICAM_STA_MASK_ALL);
> > +     unicam_reg_write(unicam, UNICAM_ISTA, UNICAM_ISTA_MASK_ALL);
> > +
> > +     /* tclk_term_en */
> > +     unicam_reg_write_field(unicam, UNICAM_CLT, 2, UNICAM_CLT1_MASK);
> > +     /* tclk_settle */
> > +     unicam_reg_write_field(unicam, UNICAM_CLT, 6, UNICAM_CLT2_MASK);
> > +     /* td_term_en */
> > +     unicam_reg_write_field(unicam, UNICAM_DLT, 2, UNICAM_DLT1_MASK);
> > +     /* ths_settle */
> > +     unicam_reg_write_field(unicam, UNICAM_DLT, 6, UNICAM_DLT2_MASK);
> > +     /* trx_enable */
> > +     unicam_reg_write_field(unicam, UNICAM_DLT, 0, UNICAM_DLT3_MASK);
> > +
> > +     unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_SOE);
> > +
> > +     /* Packet compare setup - required to avoid missing frame ends */
> > +     val = 0;
> > +     unicam_set_field(&val, 1, UNICAM_PCE);
> > +     unicam_set_field(&val, 1, UNICAM_GI);
> > +     unicam_set_field(&val, 1, UNICAM_CPH);
> > +     unicam_set_field(&val, 0, UNICAM_PCVC_MASK);
> > +     unicam_set_field(&val, 1, UNICAM_PCDT_MASK);
> > +     unicam_reg_write(unicam, UNICAM_CMP0, val);
> > +
> > +     /* Enable clock lane and set up terminations */
> > +     val = 0;
> > +     if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> > +             /* CSI2 */
> > +             unicam_set_field(&val, 1, UNICAM_CLE);
> > +             unicam_set_field(&val, 1, UNICAM_CLLPE);
> > +             if (!(unicam->bus_flags & V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK)) {
> > +                     unicam_set_field(&val, 1, UNICAM_CLTRE);
> > +                     unicam_set_field(&val, 1, UNICAM_CLHSE);
> > +             }
> > +     } else {
> > +             /* CCP2 */
> > +             unicam_set_field(&val, 1, UNICAM_CLE);
> > +             unicam_set_field(&val, 1, UNICAM_CLHSE);
> > +             unicam_set_field(&val, 1, UNICAM_CLTRE);
> > +     }
> > +     unicam_reg_write(unicam, UNICAM_CLK, val);
> > +
> > +     /*
> > +      * Enable required data lanes with appropriate terminations.
> > +      * The same value needs to be written to UNICAM_DATn registers for
> > +      * the active lanes, and 0 for inactive ones.
> > +      */
> > +     val = 0;
> > +     if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> > +             /* CSI2 */
> > +             unicam_set_field(&val, 1, UNICAM_DLE);
> > +             unicam_set_field(&val, 1, UNICAM_DLLPE);
> > +             if (!(unicam->bus_flags & V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK)) {
> > +                     unicam_set_field(&val, 1, UNICAM_DLTRE);
> > +                     unicam_set_field(&val, 1, UNICAM_DLHSE);
> > +             }
> > +     } else {
> > +             /* CCP2 */
> > +             unicam_set_field(&val, 1, UNICAM_DLE);
> > +             unicam_set_field(&val, 1, UNICAM_DLHSE);
> > +             unicam_set_field(&val, 1, UNICAM_DLTRE);
> > +     }
> > +     unicam_reg_write(unicam, UNICAM_DAT0, val);
> > +
> > +     if (unicam->active_data_lanes == 1)
> > +             val = 0;
> > +     unicam_reg_write(unicam, UNICAM_DAT1, val);
> > +
> > +     if (unicam->max_data_lanes > 2) {
> > +             /*
> > +              * Registers UNICAM_DAT2 and UNICAM_DAT3 only valid if the
> > +              * instance supports more than 2 data lanes.
> > +              */
> > +             if (unicam->active_data_lanes == 2)
> > +                     val = 0;
> > +             unicam_reg_write(unicam, UNICAM_DAT2, val);
> > +
> > +             if (unicam->active_data_lanes == 3)
> > +                     val = 0;
> > +             unicam_reg_write(unicam, UNICAM_DAT3, val);
> > +     }
> > +
> > +     unicam_reg_write(unicam, UNICAM_IBLS,
> > +                      node->fmt.fmt.pix.bytesperline);
> > +     unicam_wr_dma_addr(&unicam->node[UNICAM_IMAGE_NODE], buf);
> > +     unicam_set_packing_config(unicam);
> > +     unicam_cfg_image_id(unicam);
> > +
> > +     val = unicam_reg_read(unicam, UNICAM_MISC);
> > +     unicam_set_field(&val, 1, UNICAM_FL0);
> > +     unicam_set_field(&val, 1, UNICAM_FL1);
> > +     unicam_reg_write(unicam, UNICAM_MISC, val);
> > +
> > +     /* Enable peripheral */
> > +     unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPE);
> > +
> > +     /* Load image pointers */
> > +     unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_LIP_MASK);
> > +
> > +     /*
> > +      * Enable trigger only for the first frame to
> > +      * sync correctly to the FS from the source.
> > +      */
> > +     unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> > +}
> > +
> > +static void unicam_start_metadata(struct unicam_device *unicam,
> > +                               struct unicam_buffer *buf)
> > +{
> > +     struct unicam_node *node = &unicam->node[UNICAM_METADATA_NODE];
> > +
> > +     unicam_enable_ed(unicam);
> > +     unicam_wr_dma_addr(node, buf);
> > +     unicam_reg_write_field(unicam, UNICAM_DCS, 1, UNICAM_LDP);
> > +}
> > +
> > +static void unicam_disable(struct unicam_device *unicam)
> > +{
> > +     /* Analogue lane control disable */
> > +     unicam_reg_write_field(unicam, UNICAM_ANA, 1, UNICAM_DDL);
> > +
> > +     /* Stop the output engine */
> > +     unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_SOE);
> > +
> > +     /* Disable the data lanes. */
> > +     unicam_reg_write(unicam, UNICAM_DAT0, 0);
> > +     unicam_reg_write(unicam, UNICAM_DAT1, 0);
> > +
> > +     if (unicam->max_data_lanes > 2) {
> > +             unicam_reg_write(unicam, UNICAM_DAT2, 0);
> > +             unicam_reg_write(unicam, UNICAM_DAT3, 0);
> > +     }
> > +
> > +     /* Peripheral reset */
> > +     unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPR);
> > +     usleep_range(50, 100);
> > +     unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPR);
> > +
> > +     /* Disable peripheral */
> > +     unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPE);
> > +
> > +     /* Clear ED setup */
> > +     unicam_reg_write(unicam, UNICAM_DCS, 0);
> > +
> > +     /* Disable all lane clocks */
> > +     unicam_clk_write(unicam, 0);
> > +}
> > +
> > +/* -----------------------------------------------------------------------------
> > + * V4L2 subdev operations
> > + */
> > +
> > +static int __unicam_subdev_set_routing(struct v4l2_subdev *sd,
> > +                                    struct v4l2_subdev_state *state,
> > +                                    struct v4l2_subdev_krouting *routing)
> > +{
> > +     struct v4l2_subdev_route *route;
> > +     int ret;
> > +
> > +     ret = v4l2_subdev_routing_validate(sd, routing,
> > +                                        V4L2_SUBDEV_ROUTING_ONLY_1_TO_1);
> > +     if (ret)
> > +             return ret;
> > +
> > +     ret = v4l2_subdev_set_routing(sd, state, routing);
> > +     if (ret)
> > +             return ret;
> > +
> > +     for_each_active_route(&state->routing, route) {
> > +             const struct v4l2_mbus_framefmt *def_fmt;
> > +             struct v4l2_mbus_framefmt *fmt;
> > +
> > +             if (route->source_pad == UNICAM_SD_PAD_SOURCE_IMAGE)
> > +                     def_fmt = &unicam_default_image_format;
> > +             else
> > +                     def_fmt = &unicam_default_meta_format;
> > +
> > +             fmt = v4l2_subdev_state_get_format(state, route->sink_pad,
> > +                                                route->sink_stream);
> > +             *fmt = *def_fmt;
> > +             fmt = v4l2_subdev_state_get_format(state, route->source_pad,
> > +                                                route->source_stream);
> > +             *fmt = *def_fmt;
> > +     }
> > +
> > +     return 0;
> > +}
> > +
> > +static int unicam_subdev_init_state(struct v4l2_subdev *sd,
> > +                                 struct v4l2_subdev_state *state)
> > +{
> > +     struct v4l2_subdev_route routes[] = {
> > +             {
> > +                     .sink_pad = UNICAM_SD_PAD_SINK,
> > +                     .sink_stream = 0,
> > +                     .source_pad = UNICAM_SD_PAD_SOURCE_IMAGE,
> > +                     .source_stream = 0,
> > +                     .flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
> > +             },
> > +     };
> > +
> > +     struct v4l2_subdev_krouting routing = {
> > +             .len_routes = ARRAY_SIZE(routes),
> > +             .num_routes = ARRAY_SIZE(routes),
> > +             .routes = routes,
> > +     };
> > +
> > +     /* Initialize routing to single route to the fist source pad. */
> > +     return __unicam_subdev_set_routing(sd, state, &routing);
> > +}
> > +
> > +static int unicam_subdev_enum_mbus_code(struct v4l2_subdev *sd,
> > +                                     struct v4l2_subdev_state *state,
> > +                                     struct v4l2_subdev_mbus_code_enum *code)
> > +{
> > +     u32 pad, stream;
> > +     int ret;
> > +
> > +     ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
> > +                                                 code->pad, code->stream,
> > +                                                 &pad, &stream);
> > +     if (ret)
> > +             return ret;
> > +
> > +     if (unicam_sd_pad_is_source(code->pad)) {
> > +             /* No transcoding, source and sink codes must match. */
> > +             const struct v4l2_mbus_framefmt *fmt;
> > +
> > +             fmt = v4l2_subdev_state_get_format(state, pad, stream);
> > +             if (!fmt)
> > +                     return -EINVAL;
> > +
> > +             if (code->index > 0)
> > +                     return -EINVAL;
> > +
> > +             code->code = fmt->code;
> > +     } else {
> > +             const struct unicam_format_info *formats;
> > +             unsigned int num_formats;
> > +
> > +             if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> > +                     formats = unicam_image_formats;
> > +                     num_formats = ARRAY_SIZE(unicam_image_formats);
> > +             } else {
> > +                     formats = unicam_meta_formats;
> > +                     num_formats = ARRAY_SIZE(unicam_meta_formats);
> > +             }
> > +
> > +             if (code->index >= num_formats)
> > +                     return -EINVAL;
> > +
> > +             code->code = formats[code->index].code;
> > +     }
> > +
> > +     return 0;
> > +}
> > +
> > +static int unicam_subdev_enum_frame_size(struct v4l2_subdev *sd,
> > +                                      struct v4l2_subdev_state *state,
> > +                                      struct v4l2_subdev_frame_size_enum *fse)
> > +{
> > +     u32 pad, stream;
> > +     int ret;
> > +
> > +     if (fse->index > 0)
> > +             return -EINVAL;
> > +
> > +     ret = v4l2_subdev_routing_find_opposite_end(&state->routing, fse->pad,
> > +                                                 fse->stream, &pad,
> > +                                                 &stream);
> > +     if (ret)
> > +             return ret;
> > +
> > +     if (unicam_sd_pad_is_source(fse->pad)) {
> > +             /* No transcoding, source and sink formats must match. */
> > +             const struct v4l2_mbus_framefmt *fmt;
> > +
> > +             fmt = v4l2_subdev_state_get_format(state, pad, stream);
> > +             if (!fmt)
> > +                     return -EINVAL;
> > +
> > +             if (fse->code != fmt->code)
> > +                     return -EINVAL;
> > +
> > +             fse->min_width = fmt->width;
> > +             fse->max_width = fmt->width;
> > +             fse->min_height = fmt->height;
> > +             fse->max_height = fmt->height;
> > +     } else {
> > +             const struct unicam_format_info *fmtinfo;
> > +
> > +             fmtinfo = unicam_find_format_by_code(fse->code, pad);
> > +             if (!fmtinfo)
> > +                     return -EINVAL;
> > +
> > +             if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> > +                     fse->min_width = UNICAM_IMAGE_MIN_WIDTH;
> > +                     fse->max_width = UNICAM_IMAGE_MAX_WIDTH;
> > +                     fse->min_height = UNICAM_IMAGE_MIN_HEIGHT;
> > +                     fse->max_height = UNICAM_IMAGE_MAX_HEIGHT;
> > +             } else {
> > +                     fse->min_width = UNICAM_META_MIN_WIDTH;
> > +                     fse->max_width = UNICAM_META_MAX_WIDTH;
> > +                     fse->min_height = UNICAM_META_MIN_HEIGHT;
> > +                     fse->max_height = UNICAM_META_MAX_HEIGHT;
> > +             }
> > +     }
> > +
> > +     return 0;
> > +}
> > +
> > +static int unicam_subdev_set_format(struct v4l2_subdev *sd,
> > +                                 struct v4l2_subdev_state *state,
> > +                                 struct v4l2_subdev_format *format)
> > +{
> > +     struct unicam_device *unicam = sd_to_unicam_device(sd);
> > +     struct v4l2_mbus_framefmt *sink_format, *source_format;
> > +     const struct unicam_format_info *fmtinfo;
> > +     u32 source_pad, source_stream;
> > +     int ret;
> > +
> > +     if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE &&
> > +         unicam->subdev.streaming)
> > +             return -EBUSY;
> > +
> > +     /* No transcoding, source and sink formats must match. */
> > +     if (unicam_sd_pad_is_source(format->pad))
> > +             return v4l2_subdev_get_fmt(sd, state, format);
> > +
> > +     /*
> > +      * Allowed formats for the stream on the sink pad depend on what source
> > +      * pad the stream is routed to. Find the corresponding source pad and
> > +      * use it to validate the media bus code.
> > +      */
> > +     ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
> > +                                                 format->pad, format->stream,
> > +                                                 &source_pad, &source_stream);
> > +     if (ret)
> > +             return ret;
> > +
> > +     fmtinfo = unicam_find_format_by_code(format->format.code, source_pad);
> > +     if (!fmtinfo) {
> > +             fmtinfo = source_pad == UNICAM_SD_PAD_SOURCE_IMAGE
> > +                     ? &unicam_image_formats[0] : &unicam_meta_formats[0];
> > +             format->format.code = fmtinfo->code;
> > +     }
> > +
> > +     if (source_pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> > +             format->format.width = clamp_t(unsigned int,
> > +                                            format->format.width,
> > +                                            UNICAM_IMAGE_MIN_WIDTH,
> > +                                            UNICAM_IMAGE_MAX_WIDTH);
> > +             format->format.height = clamp_t(unsigned int,
> > +                                             format->format.height,
> > +                                             UNICAM_IMAGE_MIN_HEIGHT,
> > +                                             UNICAM_IMAGE_MAX_HEIGHT);
> > +             format->format.field = V4L2_FIELD_NONE;
> > +     } else {
> > +             format->format.width = clamp_t(unsigned int,
> > +                                            format->format.width,
> > +                                            UNICAM_META_MIN_WIDTH,
> > +                                            UNICAM_META_MAX_WIDTH);
> > +             format->format.height = clamp_t(unsigned int,
> > +                                             format->format.height,
> > +                                             UNICAM_META_MIN_HEIGHT,
> > +                                             UNICAM_META_MAX_HEIGHT);
> > +             format->format.field = V4L2_FIELD_NONE;
> > +
> > +             /* Colorspace don't apply to metadata. */
> > +             format->format.colorspace = 0;
> > +             format->format.ycbcr_enc = 0;
> > +             format->format.quantization = 0;
> > +             format->format.xfer_func = 0;
> > +     }
> > +
> > +     sink_format = v4l2_subdev_state_get_format(state, format->pad,
> > +                                                format->stream);
> > +     source_format = v4l2_subdev_state_get_format(state, source_pad,
> > +                                                  source_stream);
> > +     *sink_format = format->format;
> > +     *source_format = format->format;
> > +
> > +     return 0;
> > +}
> > +
> > +static int unicam_subdev_set_routing(struct v4l2_subdev *sd,
> > +                                  struct v4l2_subdev_state *state,
> > +                                  enum v4l2_subdev_format_whence which,
> > +                                  struct v4l2_subdev_krouting *routing)
> > +{
> > +     struct unicam_device *unicam = sd_to_unicam_device(sd);
> > +
> > +     if (which == V4L2_SUBDEV_FORMAT_ACTIVE && unicam->subdev.streaming)
> > +             return -EBUSY;
> > +
> > +     return __unicam_subdev_set_routing(sd, state, routing);
> > +}
> > +
> > +static int unicam_sd_enable_streams(struct v4l2_subdev *sd,
> > +                                 struct v4l2_subdev_state *state, u32 pad,
> > +                                 u64 streams_mask)
> > +{
> > +     struct unicam_device *unicam = sd_to_unicam_device(sd);
> > +     u32 other_pad, other_stream;
> > +     int ret;
> > +
> > +     ret = v4l2_subdev_routing_find_opposite_end(&state->routing, pad, 0,
> > +                                                 &other_pad, &other_stream);
> > +     if (ret)
> > +             return ret;
> > +
> > +     unicam->sequence = 0;
> > +
> > +     ret = v4l2_subdev_enable_streams(unicam->sensor.subdev,
> > +                                      unicam->sensor.pad->index,
> > +                                      BIT(other_stream));
> > +     if (ret) {
> > +             dev_err(unicam->dev, "stream on failed in subdev\n");
> > +             return ret;
> > +     }
> > +
> > +     unicam->subdev.streaming = true;
> > +
> > +     return 0;
> > +}
> > +
> > +static int unicam_sd_disable_streams(struct v4l2_subdev *sd,
> > +                                  struct v4l2_subdev_state *state, u32 pad,
> > +                                  u64 streams_mask)
> > +{
> > +     struct unicam_device *unicam = sd_to_unicam_device(sd);
> > +     u32 other_pad, other_stream;
> > +     int ret;
> > +
> > +     ret = v4l2_subdev_routing_find_opposite_end(&state->routing, pad, 0,
> > +                                                 &other_pad, &other_stream);
> > +     if (ret)
> > +             return ret;
> > +
> > +     v4l2_subdev_disable_streams(unicam->sensor.subdev,
> > +                                 unicam->sensor.pad->index,
> > +                                 BIT(other_stream));
> > +
> > +     unicam->subdev.streaming = false;
> > +
> > +     return 0;
> > +}
> > +
> > +static const struct v4l2_subdev_pad_ops unicam_subdev_pad_ops = {
> > +     .enum_mbus_code         = unicam_subdev_enum_mbus_code,
> > +     .enum_frame_size        = unicam_subdev_enum_frame_size,
> > +     .get_fmt                = v4l2_subdev_get_fmt,
> > +     .set_fmt                = unicam_subdev_set_format,
> > +     .set_routing            = unicam_subdev_set_routing,
> > +     .enable_streams         = unicam_sd_enable_streams,
> > +     .disable_streams        = unicam_sd_disable_streams,
> > +};
> > +
> > +static const struct v4l2_subdev_ops unicam_subdev_ops = {
> > +     .pad                    = &unicam_subdev_pad_ops,
> > +};
> > +
> > +static const struct v4l2_subdev_internal_ops unicam_subdev_internal_ops = {
> > +     .init_state             = unicam_subdev_init_state,
> > +};
> > +
> > +static const struct media_entity_operations unicam_subdev_media_ops = {
> > +     .link_validate          = v4l2_subdev_link_validate,
> > +     .has_pad_interdep       = v4l2_subdev_has_pad_interdep,
> > +};
> > +
> > +static int unicam_subdev_init(struct unicam_device *unicam)
> > +{
> > +     struct v4l2_subdev *sd = &unicam->subdev.sd;
> > +     int ret;
> > +
> > +     v4l2_subdev_init(sd, &unicam_subdev_ops);
> > +     sd->internal_ops = &unicam_subdev_internal_ops;
> > +     v4l2_set_subdevdata(sd, unicam);
> > +
> > +     sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> > +     sd->entity.ops = &unicam_subdev_media_ops;
> > +     sd->dev = unicam->dev;
> > +     sd->owner = THIS_MODULE;
> > +     sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_STREAMS;
> > +
> > +     strscpy(sd->name, "unicam", sizeof(sd->name));
> > +
> > +     unicam->subdev.pads[UNICAM_SD_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> > +     unicam->subdev.pads[UNICAM_SD_PAD_SOURCE_IMAGE].flags = MEDIA_PAD_FL_SOURCE;
> > +     unicam->subdev.pads[UNICAM_SD_PAD_SOURCE_METADATA].flags = MEDIA_PAD_FL_SOURCE;
> > +
> > +     ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(unicam->subdev.pads),
> > +                                  unicam->subdev.pads);
> > +     if (ret) {
> > +             dev_err(unicam->dev, "Failed to initialize media entity: %d\n",
> > +                     ret);
> > +             return ret;
> > +     }
> > +
> > +     ret = v4l2_subdev_init_finalize(sd);
> > +     if (ret) {
> > +             dev_err(unicam->dev, "Failed to initialize subdev: %d\n", ret);
> > +             goto err_entity;
> > +     }
> > +
> > +     ret = v4l2_device_register_subdev(&unicam->v4l2_dev, sd);
> > +     if (ret) {
> > +             dev_err(unicam->dev, "Failed to register subdev: %d\n", ret);
> > +             goto err_subdev;
> > +     }
> > +
> > +     return 0;
> > +
> > +err_subdev:
> > +     v4l2_subdev_cleanup(sd);
> > +err_entity:
> > +     media_entity_cleanup(&sd->entity);
> > +     return ret;
> > +}
> > +
> > +static void unicam_subdev_cleanup(struct unicam_device *unicam)
> > +{
> > +     v4l2_subdev_cleanup(&unicam->subdev.sd);
> > +     media_entity_cleanup(&unicam->subdev.sd.entity);
> > +}
> > +
> > +/* -----------------------------------------------------------------------------
> > + * Videobuf2 queue operations
> > + */
> > +
> > +static int unicam_queue_setup(struct vb2_queue *vq,
> > +                           unsigned int *nbuffers,
> > +                           unsigned int *nplanes,
> > +                           unsigned int sizes[],
> > +                           struct device *alloc_devs[])
> > +{
> > +     struct unicam_node *node = vb2_get_drv_priv(vq);
> > +     u32 size = is_image_node(node) ? node->fmt.fmt.pix.sizeimage
> > +              : node->fmt.fmt.meta.buffersize;
> > +
> > +     if (vq->num_buffers + *nbuffers < 3)
>
> Why 3 ? Are these dummies ?
>
> > +             *nbuffers = 3 - vq->num_buffers;
> > +
> > +     if (*nplanes) {
> > +             if (sizes[0] < size) {
> > +                     dev_dbg(node->dev->dev, "sizes[0] %i < size %u\n",
> > +                             sizes[0], size);
> > +                     return -EINVAL;
> > +             }
> > +             size = sizes[0];
> > +     }
> > +
> > +     *nplanes = 1;
> > +     sizes[0] = size;
> > +
> > +     return 0;
> > +}
> > +
> > +static int unicam_buffer_prepare(struct vb2_buffer *vb)
> > +{
> > +     struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
> > +     struct unicam_buffer *buf = to_unicam_buffer(vb);
> > +     u32 size = is_image_node(node) ? node->fmt.fmt.pix.sizeimage
> > +              : node->fmt.fmt.meta.buffersize;
> > +
> > +     if (vb2_plane_size(vb, 0) < size) {
> > +             dev_dbg(node->dev->dev,
> > +                     "data will not fit into plane (%lu < %u)\n",
> > +                     vb2_plane_size(vb, 0), size);
> > +             return -EINVAL;
> > +     }
> > +
> > +     buf->dma_addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
> > +     buf->size = size;
> > +
> > +     vb2_set_plane_payload(&buf->vb.vb2_buf, 0, size);
> > +
> > +     return 0;
> > +}
> > +
> > +static void unicam_return_buffers(struct unicam_node *node,
> > +                               enum vb2_buffer_state state)
> > +{
> > +     struct unicam_buffer *buf, *tmp;
> > +
> > +     list_for_each_entry_safe(buf, tmp, &node->dma_queue, list) {
> > +             list_del(&buf->list);
> > +             vb2_buffer_done(&buf->vb.vb2_buf, state);
> > +     }
> > +
> > +     if (node->cur_frm)
> > +             vb2_buffer_done(&node->cur_frm->vb.vb2_buf,
> > +                             state);
> > +     if (node->next_frm && node->cur_frm != node->next_frm)
> > +             vb2_buffer_done(&node->next_frm->vb.vb2_buf,
> > +                             state);
> > +
> > +     node->cur_frm = NULL;
> > +     node->next_frm = NULL;
> > +}
> > +
> > +static int unicam_start_streaming(struct vb2_queue *vq, unsigned int count)
> > +{
> > +     struct unicam_node *node = vb2_get_drv_priv(vq);
> > +     struct unicam_device *unicam = node->dev;
> > +     struct v4l2_subdev_state *state;
> > +     struct unicam_buffer *buf;
> > +     unsigned long flags;
> > +     int ret;
> > +     u32 pad, stream;
> > +     u32 remote_pad = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
> > +                                          : UNICAM_SD_PAD_SOURCE_METADATA;
> > +
> > +     /* Look for the route for the given pad and stream. */
> > +     state = v4l2_subdev_lock_and_get_active_state(&unicam->subdev.sd);
> > +     ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
> > +                                                 remote_pad, 0,
> > +                                                 &pad, &stream);
> > +     v4l2_subdev_unlock_state(state);
> > +
> > +     if (ret)
> > +             goto err_return_buffers;

You've dropped calling get_mbus_config from in or around this area,
therefore any device that supports dynamic switching on number of data
lanes (eg tc358743) is broken.
Docs guidance is
"Callers should make sure they get the most up-to-date as possible
configuration from the remote end, likely calling this operation as
close as possible to stream on time"
https://www.kernel.org/doc/html/latest/driver-api/media/v4l2-subdev.html?highlight=get_mbus_config

> > +
> > +     dev_dbg(unicam->dev, "Starting stream on %s: %u/%u -> %u/%u (%s)\n",
> > +             unicam->subdev.sd.name, pad, stream, remote_pad, 0,
> > +             is_metadata_node(node) ? "metadata" : "image");
> > +
> > +     /* The metadata node can't be started alone. */
> > +     if (is_metadata_node(node)) {
> > +             if (!unicam->node[UNICAM_IMAGE_NODE].streaming) {
>
> Does it mean the metadata node has to be started second, or should
> this be made a nop and the metadata node gets started once the image
> node is started too ? I'm fine with the constraint of having the
> metadata node being started second fwiw

This has been flipped from the downstream implementation.
Downstream we expected the metadata node to be enabled first, but it
wouldn't get started until the image node was enabled.

Things like sequence numbers come to mind now that this is enabling
the image node first. If the image node has produced the first frame
before the metadata node has been enabled, then as I read it the
sequence numbers for the two nodes should be different - there haven't
really been any dropped frames as you hadn't enabled the node.

I'd also want to confirm what the behaviour is if the frame has
already started before the metadata node is enabled. Embedded data is
almost always sent before the image data, so have we potentially
missed it for the current frame, but at the end of frame are going to
return both buffers?

> > +                     dev_err(unicam->dev,
> > +                             "Can't start metadata without image\n");
> > +                     ret = -EINVAL;
> > +                     goto err_return_buffers;
> > +             }
> > +
> > +             spin_lock_irqsave(&node->dma_queue_lock, flags);
> > +             buf = list_first_entry(&node->dma_queue,
> > +                                    struct unicam_buffer, list);
> > +             dev_dbg(unicam->dev, "buffer %p\n", buf);
>
> Is this useful ?
>
> > +             node->cur_frm = buf;
> > +             node->next_frm = buf;
> > +             list_del(&buf->list);
> > +             spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> > +
> > +             unicam_start_metadata(unicam, buf);
> > +             node->streaming = true;
> > +             return 0;
> > +     }
> > +
> > +     ret = pm_runtime_resume_and_get(unicam->dev);
> > +     if (ret < 0) {
> > +             dev_err(unicam->dev, "PM runtime resume failed: %d\n", ret);
> > +             goto err_return_buffers;
> > +     }
> > +
> > +     ret = video_device_pipeline_start(&node->video_dev, &unicam->pipe);
> > +     if (ret < 0) {
> > +             dev_dbg(unicam->dev, "Failed to start media pipeline: %d\n", ret);
>
> Isn't this an err ?
>
> > +             goto err_pm_put;
> > +     }
> > +
> > +     spin_lock_irqsave(&node->dma_queue_lock, flags);
> > +     buf = list_first_entry(&node->dma_queue,
> > +                            struct unicam_buffer, list);
> > +     dev_dbg(unicam->dev, "buffer %p\n", buf);
> > +     node->cur_frm = buf;
> > +     node->next_frm = buf;
> > +     list_del(&buf->list);
> > +     spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> > +
> > +     unicam_start_rx(unicam, buf);
> > +
> > +     ret = v4l2_subdev_enable_streams(&unicam->subdev.sd, remote_pad, BIT(0));
>
> A bit confused by the API here, shouldn't we also do this for embedded
> data ?
>
> > +     if (ret < 0) {
> > +             dev_err(unicam->dev, "stream on failed in subdev\n");
> > +             goto error_pipeline;
> > +     }
> > +
> > +     node->streaming = true;
> > +
> > +     return 0;
> > +
> > +error_pipeline:
> > +     video_device_pipeline_stop(&node->video_dev);
> > +err_pm_put:
> > +     pm_runtime_put_sync(unicam->dev);
> > +err_return_buffers:
> > +     unicam_return_buffers(node, VB2_BUF_STATE_QUEUED);
> > +     return ret;
> > +}
> > +
> > +static void unicam_stop_streaming(struct vb2_queue *vq)
> > +{
> > +     struct unicam_node *node = vb2_get_drv_priv(vq);
> > +     struct unicam_device *unicam = node->dev;
> > +     u32 remote_pad_index = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
> > +                                                : UNICAM_SD_PAD_SOURCE_METADATA;
> > +
> > +     node->streaming = false;
> > +
> > +     v4l2_subdev_disable_streams(&unicam->subdev.sd, remote_pad_index,
> > +                                 BIT(0));
> > +
> > +     /* We can stream only with the image node. */
> > +     if (is_metadata_node(node)) {
> > +             /*
> > +              * Allow the hardware to spin in the dummy buffer.
> > +              * This is only really needed if the embedded data pad is
> > +              * disabled before the image pad.
> > +              */
> > +             unicam_wr_dma_addr(node, &node->dummy_buf);
> > +             goto dequeue_buffers;
> > +     }
> > +
> > +     unicam_disable(unicam);
> > +
> > +     video_device_pipeline_stop(&node->video_dev);
> > +     pm_runtime_put(unicam->dev);
> > +
> > +dequeue_buffers:
> > +     /* Clear all queued buffers for the node */
> > +     unicam_return_buffers(node, VB2_BUF_STATE_ERROR);
> > +}
> > +
> > +static void unicam_buffer_queue(struct vb2_buffer *vb)
> > +{
> > +     struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
> > +     struct unicam_buffer *buf = to_unicam_buffer(vb);
> > +
> > +     spin_lock_irq(&node->dma_queue_lock);
> > +     list_add_tail(&buf->list, &node->dma_queue);
> > +     spin_unlock_irq(&node->dma_queue_lock);
> > +}
> > +
> > +static const struct vb2_ops unicam_video_qops = {
> > +     .queue_setup            = unicam_queue_setup,
> > +     .wait_prepare           = vb2_ops_wait_prepare,
> > +     .wait_finish            = vb2_ops_wait_finish,
> > +     .buf_prepare            = unicam_buffer_prepare,
> > +     .start_streaming        = unicam_start_streaming,
> > +     .stop_streaming         = unicam_stop_streaming,
> > +     .buf_queue              = unicam_buffer_queue,
> > +};
> > +
> > +/* -----------------------------------------------------------------------------
> > + *  V4L2 video device operations
> > + */
> > +
> > +static int unicam_querycap(struct file *file, void *priv,
> > +                        struct v4l2_capability *cap)
> > +{
> > +     strscpy(cap->driver, UNICAM_MODULE_NAME, sizeof(cap->driver));
> > +     strscpy(cap->card, UNICAM_MODULE_NAME, sizeof(cap->card));
> > +
> > +     cap->capabilities |= V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_META_CAPTURE;
> > +
> > +     return 0;
> > +}
> > +
> > +static int unicam_enum_fmt_vid(struct file *file, void  *priv,
> > +                            struct v4l2_fmtdesc *f)
> > +{
> > +     unsigned int index;
> > +     unsigned int i;
> > +
> > +     for (i = 0, index = 0; i < ARRAY_SIZE(unicam_image_formats); i++) {
> > +             if (f->mbus_code && unicam_image_formats[i].code != f->mbus_code)
> > +                     continue;
> > +
> > +             if (index == f->index) {
> > +                     f->pixelformat = unicam_image_formats[i].fourcc;
> > +                     return 0;
> > +             }
> > +
> > +             index++;
> > +
> > +             if (!unicam_image_formats[i].unpacked_fourcc)
> > +                     continue;
> > +
> > +             if (index == f->index) {
> > +                     f->pixelformat = unicam_image_formats[i].unpacked_fourcc;
> > +                     return 0;
> > +             }
> > +
> > +             index++;
> > +     }
> > +
> > +     return -EINVAL;
> > +}
> > +
> > +static int unicam_g_fmt_vid(struct file *file, void *priv,
> > +                         struct v4l2_format *f)
> > +{
> > +     struct unicam_node *node = video_drvdata(file);
> > +
> > +     *f = node->fmt;
> > +
> > +     return 0;
> > +}
> > +
> > +static const struct unicam_format_info *
> > +__unicam_try_fmt_vid(struct unicam_node *node, struct v4l2_pix_format *pix)
> > +{
> > +     const struct unicam_format_info *fmtinfo;
> > +
> > +     /*
> > +      * Default to the first format if the requested pixel format code isn't
> > +      * supported.
> > +      */
> > +     fmtinfo = unicam_find_format_by_fourcc(pix->pixelformat,
> > +                                            UNICAM_SD_PAD_SOURCE_IMAGE);
> > +     if (!fmtinfo) {
> > +             fmtinfo = &unicam_image_formats[0];
> > +             pix->pixelformat = fmtinfo->fourcc;
> > +     }
> > +
> > +     unicam_calc_image_size_bpl(node->dev, fmtinfo, pix);
> > +
> > +     if (pix->field == V4L2_FIELD_ANY)
> > +             pix->field = V4L2_FIELD_NONE;
> > +
> > +     return fmtinfo;
> > +}
> > +
> > +static int unicam_try_fmt_vid(struct file *file, void *priv,
> > +                           struct v4l2_format *f)
> > +{
> > +     struct unicam_node *node = video_drvdata(file);
> > +
> > +     __unicam_try_fmt_vid(node, &f->fmt.pix);
> > +     return 0;
> > +}
> > +
> > +static int unicam_s_fmt_vid(struct file *file, void *priv,
> > +                         struct v4l2_format *f)
> > +{
> > +     struct unicam_node *node = video_drvdata(file);
> > +
> > +     if (vb2_is_busy(&node->buffer_queue))
> > +             return -EBUSY;
> > +
> > +     node->fmtinfo = __unicam_try_fmt_vid(node, &f->fmt.pix);
> > +     node->fmt = *f;
> > +
> > +     return 0;
> > +}
> > +
> > +static int unicam_enum_fmt_meta(struct file *file, void *priv,
> > +                             struct v4l2_fmtdesc *f)
> > +{
> > +     unsigned int i, index;
> > +
> > +     for (i = 0, index = 0; i < ARRAY_SIZE(unicam_meta_formats); i++) {
> > +             if (f->mbus_code && unicam_meta_formats[i].code != f->mbus_code)
>
> Do we want to allow mbus_code filtering for metadata ? There's a
> 1-to-1 relationship between mbus codes and pixel formats
>
> > +                     continue;
> > +             if (!unicam_meta_formats[i].metadata_fmt)
> > +                     continue;
>
> How can this be false if we're iterating on unicam_meta_formats[] ?
>
> > +
> > +             if (index == f->index) {
> > +                     f->pixelformat = unicam_meta_formats[i].fourcc;
> > +                     f->type = V4L2_BUF_TYPE_META_CAPTURE;
> > +                     return 0;
> > +             }
> > +             index++;
> > +     }
> > +
> > +     return -EINVAL;
> > +}
> > +
> > +static int unicam_g_fmt_meta(struct file *file, void *priv,
> > +                          struct v4l2_format *f)
> > +{
> > +     struct unicam_node *node = video_drvdata(file);
> > +
> > +     f->fmt.meta = node->fmt.fmt.meta;
> > +
> > +     return 0;
> > +}
> > +
> > +static const struct unicam_format_info *
> > +__unicam_try_fmt_meta(struct unicam_node *node, struct v4l2_meta_format *meta)
> > +{
> > +     const struct unicam_format_info *fmtinfo;
> > +
> > +     /*
> > +      * Default to the first format if the requested pixel format code isn't
> > +      * supported.
> > +      */
> > +     fmtinfo = unicam_find_format_by_fourcc(meta->dataformat,
> > +                                            UNICAM_SD_PAD_SOURCE_METADATA);
> > +     if (!fmtinfo) {
> > +             fmtinfo = &unicam_meta_formats[0];
> > +             meta->dataformat = fmtinfo->fourcc;
> > +     }
> > +
> > +     unicam_calc_meta_size_bpl(node->dev, fmtinfo, meta);
> > +
> > +     return fmtinfo;
> > +}
> > +
> > +static int unicam_try_fmt_meta(struct file *file, void *priv,
> > +                            struct v4l2_format *f)
> > +{
> > +     struct unicam_node *node = video_drvdata(file);
> > +
> > +     __unicam_try_fmt_vid(node, &f->fmt.pix);
> > +     return 0;
> > +}
> > +
> > +static int unicam_s_fmt_meta(struct file *file, void *priv,
> > +                          struct v4l2_format *f)
> > +{
> > +     struct unicam_node *node = video_drvdata(file);
> > +
> > +     if (vb2_is_busy(&node->buffer_queue))
> > +             return -EBUSY;
> > +
> > +     node->fmtinfo = __unicam_try_fmt_meta(node, &f->fmt.meta);
> > +     node->fmt = *f;
> > +
> > +     return 0;
> > +}
> > +
> > +static int unicam_enum_framesizes(struct file *file, void *fh,
> > +                               struct v4l2_frmsizeenum *fsize)
> > +{
> > +     struct unicam_node *node = video_drvdata(file);
> > +     int ret = -EINVAL;
> > +
> > +     if (fsize->index > 0)
> > +             return ret;
> > +
> > +     if (is_image_node(node)) {
> > +             if (!unicam_find_format_by_fourcc(fsize->pixel_format,
> > +                                               UNICAM_SD_PAD_SOURCE_IMAGE))
> > +                     return ret;
> > +
> > +             fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> > +             fsize->stepwise.min_width = UNICAM_IMAGE_MIN_WIDTH;
> > +             fsize->stepwise.max_width = UNICAM_IMAGE_MAX_WIDTH;
> > +             fsize->stepwise.step_width = 1;
> > +             fsize->stepwise.min_height = UNICAM_IMAGE_MIN_HEIGHT;
> > +             fsize->stepwise.max_height = UNICAM_IMAGE_MAX_HEIGHT;
> > +             fsize->stepwise.step_height = 1;
> > +     } else {
> > +             if (!unicam_find_format_by_fourcc(fsize->pixel_format,
> > +                                               UNICAM_SD_PAD_SOURCE_METADATA))
> > +                     return ret;
> > +
> Isn't this V4L2_FRMSIZE_TYPE_STEPWISE as well ?
>
> > +             fsize->stepwise.min_width = UNICAM_META_MIN_WIDTH;
> > +             fsize->stepwise.max_width = UNICAM_META_MAX_WIDTH;
> > +             fsize->stepwise.step_width = 1;
> > +             fsize->stepwise.min_height = UNICAM_META_MIN_HEIGHT;
> > +             fsize->stepwise.max_height = UNICAM_META_MAX_HEIGHT;
> > +             fsize->stepwise.step_height = 1;
> > +     }
> > +
> > +     return 0;
> > +}
> > +
> > +static int unicam_log_status(struct file *file, void *fh)
> > +{
> > +     struct unicam_node *node = video_drvdata(file);
> > +     struct unicam_device *unicam = node->dev;
> > +     u32 reg;
> > +
> > +     /* status for sub devices */
> > +     v4l2_device_call_all(&unicam->v4l2_dev, 0, core, log_status);
> > +
> > +     dev_info(unicam->dev, "-----Receiver status-----\n");
> > +     dev_info(unicam->dev, "V4L2 width/height:   %ux%u\n",
> > +              node->fmt.fmt.pix.width, node->fmt.fmt.pix.height);
> > +     dev_info(unicam->dev, "Mediabus format:     %08x\n",
> > +              node->fmtinfo->code);
> > +     dev_info(unicam->dev, "V4L2 format:         %08x\n",
> > +              node->fmt.fmt.pix.pixelformat);
> > +     reg = unicam_reg_read(unicam, UNICAM_IPIPE);
> > +     dev_info(unicam->dev, "Unpacking/packing:   %u / %u\n",
> > +              unicam_get_field(reg, UNICAM_PUM_MASK),
> > +              unicam_get_field(reg, UNICAM_PPM_MASK));
> > +     dev_info(unicam->dev, "----Live data----\n");
> > +     dev_info(unicam->dev, "Programmed stride:   %4u\n",
> > +              unicam_reg_read(unicam, UNICAM_IBLS));
> > +     dev_info(unicam->dev, "Detected resolution: %ux%u\n",
> > +              unicam_reg_read(unicam, UNICAM_IHSTA),
> > +              unicam_reg_read(unicam, UNICAM_IVSTA));
> > +     dev_info(unicam->dev, "Write pointer:       %08x\n",
> > +              unicam_reg_read(unicam, UNICAM_IBWP));
> > +
> > +     return 0;
> > +}
> > +
> > +static int unicam_subscribe_event(struct v4l2_fh *fh,
> > +                               const struct v4l2_event_subscription *sub)
> > +{
> > +     switch (sub->type) {
> > +     case V4L2_EVENT_FRAME_SYNC:
> > +             return v4l2_event_subscribe(fh, sub, 2, NULL);
> > +     default:
> > +             return -EINVAL;
> > +     }
> > +}
> > +
> > +static const struct v4l2_ioctl_ops unicam_ioctl_ops = {
> > +     .vidioc_querycap                = unicam_querycap,
> > +
> > +     .vidioc_enum_fmt_vid_cap        = unicam_enum_fmt_vid,
> > +     .vidioc_g_fmt_vid_cap           = unicam_g_fmt_vid,
> > +     .vidioc_try_fmt_vid_cap         = unicam_try_fmt_vid,
> > +     .vidioc_s_fmt_vid_cap           = unicam_s_fmt_vid,
> > +
> > +     .vidioc_enum_fmt_meta_cap       = unicam_enum_fmt_meta,
> > +     .vidioc_g_fmt_meta_cap          = unicam_g_fmt_meta,
> > +     .vidioc_try_fmt_meta_cap        = unicam_try_fmt_meta,
> > +     .vidioc_s_fmt_meta_cap          = unicam_s_fmt_meta,
> > +
> > +     .vidioc_enum_framesizes         = unicam_enum_framesizes,
> > +
> > +     .vidioc_reqbufs                 = vb2_ioctl_reqbufs,
> > +     .vidioc_create_bufs             = vb2_ioctl_create_bufs,
> > +     .vidioc_prepare_buf             = vb2_ioctl_prepare_buf,
> > +     .vidioc_querybuf                = vb2_ioctl_querybuf,
> > +     .vidioc_qbuf                    = vb2_ioctl_qbuf,
> > +     .vidioc_dqbuf                   = vb2_ioctl_dqbuf,
> > +     .vidioc_expbuf                  = vb2_ioctl_expbuf,
> > +     .vidioc_streamon                = vb2_ioctl_streamon,
> > +     .vidioc_streamoff               = vb2_ioctl_streamoff,
> > +
> > +     .vidioc_log_status              = unicam_log_status,
> > +     .vidioc_subscribe_event         = unicam_subscribe_event,
> > +     .vidioc_unsubscribe_event       = v4l2_event_unsubscribe,
> > +};
> > +
> > +/* unicam capture driver file operations */
> > +static const struct v4l2_file_operations unicam_fops = {
> > +     .owner          = THIS_MODULE,
> > +     .open           = v4l2_fh_open,
> > +     .release        = vb2_fop_release,
> > +     .poll           = vb2_fop_poll,
> > +     .unlocked_ioctl = video_ioctl2,
> > +     .mmap           = vb2_fop_mmap,
> > +};
> > +
> > +static int unicam_video_link_validate(struct media_link *link)
> > +{
> > +     struct video_device *vdev =
> > +             media_entity_to_video_device(link->sink->entity);
> > +     struct v4l2_subdev *sd =
> > +             media_entity_to_v4l2_subdev(link->source->entity);
> > +     struct unicam_node *node = video_get_drvdata(vdev);
> > +     const u32 pad = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
> > +                   : UNICAM_SD_PAD_SOURCE_METADATA;
> > +     const struct v4l2_mbus_framefmt *format;
> > +     struct v4l2_subdev_state *state;
> > +     int ret = 0;
> > +
> > +     state = v4l2_subdev_lock_and_get_active_state(sd);
> > +
> > +     format = v4l2_subdev_state_get_format(state, pad, 0);
> > +     if (!format) {
> > +             ret = -EINVAL;
> > +             goto out;
> > +     }
> > +
> > +     if (is_image_node(node)) {
> > +             const struct v4l2_pix_format *fmt = &node->fmt.fmt.pix;
> > +
> > +             if (node->fmtinfo->code != format->code ||
> > +                 fmt->height != format->height ||
> > +                 fmt->width != format->width ||
> > +                 fmt->field != format->field) {
> > +                     dev_dbg(node->dev->dev,
> > +                             "image: (%u x %u) 0x%08x %s != (%u x %u) 0x%08x %s\n",
> > +                             fmt->width, fmt->height, node->fmtinfo->code,
> > +                             v4l2_field_names[fmt->field],
> > +                             format->width, format->height, format->code,
> > +                             v4l2_field_names[format->field]);
> > +                     ret = -EPIPE;
> > +             };
>
> No need for ; here and in the next loop
>
> > +     } else {
> > +             const struct v4l2_meta_format *fmt = &node->fmt.fmt.meta;
> > +
> > +             if (node->fmtinfo->code != format->code ||
> > +                 fmt->height != format->height ||
> > +                 fmt->width != format->width) {
> > +                     dev_dbg(node->dev->dev,
> > +                             "meta: (%u x %u) 0x%04x != (%u x %u) 0x%04x\n",
> > +                             fmt->width, fmt->height, node->fmtinfo->code,
> > +                             format->width, format->height, format->code);
> > +                     ret = -EPIPE;
> > +             };
> > +     }
> > +
> > +out:
> > +     v4l2_subdev_unlock_state(state);
> > +     return ret;
> > +}
> > +
> > +static const struct media_entity_operations unicam_video_media_ops = {
> > +     .link_validate = unicam_video_link_validate,
> > +};
> > +
> > +static void unicam_node_release(struct video_device *vdev)
> > +{
> > +     struct unicam_node *node = video_get_drvdata(vdev);
> > +
> > +     unicam_put(node->dev);
> > +}
> > +
> > +static void unicam_set_default_format(struct unicam_node *node)
> > +{
> > +     if (is_image_node(node)) {
> > +             struct v4l2_pix_format *fmt = &node->fmt.fmt.pix;
> > +
> > +             node->fmtinfo = &unicam_image_formats[0];
> > +             node->fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> > +
> > +             v4l2_fill_pix_format(fmt, &unicam_default_image_format);
> > +             fmt->pixelformat = node->fmtinfo->fourcc;
> > +             unicam_calc_image_size_bpl(node->dev, node->fmtinfo, fmt);
> > +     } else {
> > +             struct v4l2_meta_format *fmt = &node->fmt.fmt.meta;
> > +
> > +             node->fmtinfo = &unicam_meta_formats[0];
> > +             node->fmt.type = V4L2_BUF_TYPE_META_CAPTURE;
> > +
> > +             fmt->dataformat = node->fmtinfo->fourcc;
> > +             fmt->width = unicam_default_meta_format.width;
> > +             fmt->height = unicam_default_meta_format.height;
> > +             unicam_calc_meta_size_bpl(node->dev, node->fmtinfo, fmt);
> > +     }
> > +}
> > +
> > +static int unicam_register_node(struct unicam_device *unicam,
> > +                             enum unicam_node_type type)
> > +{
> > +     const u32 pad_index = type == UNICAM_IMAGE_NODE
> > +                         ? UNICAM_SD_PAD_SOURCE_IMAGE
> > +                         : UNICAM_SD_PAD_SOURCE_METADATA;
> > +     struct unicam_node *node = &unicam->node[type];
> > +     struct video_device *vdev = &node->video_dev;
> > +     struct vb2_queue *q = &node->buffer_queue;
> > +     int ret;
> > +
> > +     node->dev = unicam_get(unicam);
> > +     node->id = type;
> > +
> > +     spin_lock_init(&node->dma_queue_lock);
> > +     mutex_init(&node->lock);
> > +
> > +     INIT_LIST_HEAD(&node->dma_queue);
> > +
> > +     /* Initialize the videobuf2 queue. */
> > +     q->type = type == UNICAM_IMAGE_NODE ? V4L2_BUF_TYPE_VIDEO_CAPTURE
> > +                                         : V4L2_BUF_TYPE_META_CAPTURE;
> > +     q->io_modes = VB2_MMAP | VB2_DMABUF;
> > +     q->drv_priv = node;
> > +     q->ops = &unicam_video_qops;
> > +     q->mem_ops = &vb2_dma_contig_memops;
> > +     q->buf_struct_size = sizeof(struct unicam_buffer);
> > +     q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> > +     q->lock = &node->lock;
> > +     q->min_queued_buffers = 1;
> > +     q->dev = unicam->dev;
> > +
> > +     ret = vb2_queue_init(q);
> > +     if (ret) {
> > +             dev_err(unicam->dev, "vb2_queue_init() failed\n");
> > +             goto err_unicam_put;
> > +     }
> > +
> > +     /* Initialize the video device. */
> > +     vdev->release = unicam_node_release;
> > +     vdev->fops = &unicam_fops;
> > +     vdev->ioctl_ops = &unicam_ioctl_ops;
> > +     vdev->v4l2_dev = &unicam->v4l2_dev;
> > +     vdev->vfl_dir = VFL_DIR_RX;
> > +     vdev->queue = q;
> > +     vdev->lock = &node->lock;
> > +     vdev->device_caps = type == UNICAM_IMAGE_NODE
> > +                       ? V4L2_CAP_VIDEO_CAPTURE : V4L2_CAP_META_CAPTURE;
> > +     vdev->device_caps |= V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;
> > +     vdev->entity.ops = &unicam_video_media_ops;
> > +
> > +     snprintf(vdev->name, sizeof(vdev->name), "%s-%s", UNICAM_MODULE_NAME,
> > +              type == UNICAM_IMAGE_NODE ? "image" : "embedded");
> > +
> > +     video_set_drvdata(vdev, node);
> > +
> > +     if (type == UNICAM_IMAGE_NODE)
> > +             vdev->entity.flags |= MEDIA_ENT_FL_DEFAULT;
> > +
> > +     node->pad.flags = MEDIA_PAD_FL_SINK;
> > +
> > +     ret = media_entity_pads_init(&vdev->entity, 1, &node->pad);
> > +     if (ret)
> > +             goto err_unicam_put;
> > +
> > +     node->dummy_buf.size = UNICAM_DUMMY_BUF_SIZE;
> > +     node->dummy_buf_cpu_addr = dma_alloc_coherent(unicam->dev,
> > +                                                   node->dummy_buf.size,
> > +                                                   &node->dummy_buf.dma_addr,
> > +                                                   GFP_KERNEL);
> > +     if (!node->dummy_buf_cpu_addr) {
> > +             dev_err(unicam->dev, "Unable to allocate dummy buffer.\n");
> > +             ret = -ENOMEM;
> > +             goto err_entity_cleanup;
> > +     }
> > +
> > +     unicam_set_default_format(node);
> > +
> > +     ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> > +     if (ret) {
> > +             dev_err(unicam->dev, "Unable to register video device %s\n",
> > +                     vdev->name);
> > +             goto err_dma_free;
> > +     }
> > +
> > +     node->registered = true;
> > +
> > +     ret = media_create_pad_link(&unicam->subdev.sd.entity,
> > +                                 pad_index,
> > +                                 &node->video_dev.entity,
> > +                                 0,
> > +                                 MEDIA_LNK_FL_ENABLED |
> > +                                 MEDIA_LNK_FL_IMMUTABLE);
> > +     if (ret) {
> > +             /*
> > +              * No need for cleanup, the caller will unregister the
> > +              * video device, which will drop the reference on the
> > +              * device and trigger the cleanup.
> > +              */
> > +             dev_err(unicam->dev, "Unable to create pad link for %s\n",
> > +                     unicam->sensor.subdev->name);
> > +             return ret;
> > +     }
> > +
> > +     return 0;
> > +
> > +err_dma_free:
> > +     dma_free_coherent(unicam->dev, node->dummy_buf.size,
> > +                       node->dummy_buf_cpu_addr,
> > +                       node->dummy_buf.dma_addr);
> > +err_entity_cleanup:
> > +     media_entity_cleanup(&vdev->entity);
> > +err_unicam_put:
> > +     unicam_put(unicam);
> > +     return ret;
> > +}
> > +
> > +static void unicam_unregister_nodes(struct unicam_device *unicam)
> > +{
> > +     unsigned int i;
> > +
> > +     for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > +             struct unicam_node *node = &unicam->node[i];
> > +
> > +             if (node->dummy_buf_cpu_addr)
> > +                     dma_free_coherent(unicam->dev, node->dummy_buf.size,
> > +                                       node->dummy_buf_cpu_addr,
> > +                                       node->dummy_buf.dma_addr);
> > +
> > +             if (node->registered) {
> > +                     video_unregister_device(&node->video_dev);
> > +                     node->registered = false;
> > +             }
> > +     }
> > +}
> > +
> > +/* -----------------------------------------------------------------------------
> > + * Power management
> > + */
> > +
> > +static int unicam_runtime_resume(struct device *dev)
> > +{
> > +     struct unicam_device *unicam = dev_get_drvdata(dev);
> > +     int ret;
> > +
> > +     ret = clk_set_min_rate(unicam->vpu_clock, UNICAM_MIN_VPU_CLOCK_RATE);
> > +     if (ret) {
> > +             dev_err(unicam->dev, "failed to set up VPU clock\n");
> > +             return ret;
> > +     }
>
> Same question as above, shouldn't this be an 'assigned-clock-rates in
> dts ?
>
> > +
> > +     ret = clk_prepare_enable(unicam->vpu_clock);
> > +     if (ret) {
> > +             dev_err(unicam->dev, "Failed to enable VPU clock: %d\n", ret);
> > +             goto err_vpu_clock;
> > +     }
> > +
> > +     ret = clk_set_rate(unicam->clock, 100 * 1000 * 1000);
> > +     if (ret) {
> > +             dev_err(unicam->dev, "failed to set up CSI clock\n");
> > +             goto err_vpu_prepare;
> > +     }
>
> same question
>
> > +
> > +     ret = clk_prepare_enable(unicam->clock);
> > +     if (ret) {
> > +             dev_err(unicam->dev, "Failed to enable CSI clock: %d\n", ret);
> > +             goto err_vpu_prepare;
>
> Do you need to 'unset' the rate as done for the VPU clock ?
>
> > +     }
> > +
> > +     return 0;
> > +
> > +err_vpu_prepare:
> > +     clk_disable_unprepare(unicam->vpu_clock);
> > +err_vpu_clock:
> > +     if (clk_set_min_rate(unicam->vpu_clock, 0))
> > +             dev_err(unicam->dev, "failed to reset the VPU clock\n");
> > +
> > +     return ret;
> > +}
> > +
> > +static int unicam_runtime_suspend(struct device *dev)
> > +{
> > +     struct unicam_device *unicam = dev_get_drvdata(dev);
> > +
> > +     clk_disable_unprepare(unicam->clock);
>
> (continuing  with the above question: probably not, as the clock is
> not 'unset' neither here)
>
> The rest looks good, nice to see this progressing!
>
> Thanks
>   j
>
<snip>

Re: [PATCH v6 09/15] media: bcm2835-unicam: Add support for CCP2/CSI2 camera interface

[Dave Stevenson]

On Mon, 4 Mar 2024 at 19:51, Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> Hi Jacopo,
>
> On Mon, Mar 04, 2024 at 06:12:21PM +0100, Jacopo Mondi wrote:
> > On Fri, Mar 01, 2024 at 11:32:24PM +0200, Laurent Pinchart wrote:
> > > From: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > >
> > > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > > It is represented as two video device nodes: unicam-image and
> > > unicam-embedded which are connected to an internal subdev (named
> > > unicam-subdev) in order to manage streams routing.
> > >
> > > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > Co-developed-by: Naushir Patuck <naush@raspberrypi.com>
> > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > > Co-developed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > > Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > > Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > ---
> > > Changes since v5:
> > >
> > > - Move to drivers/media/platform/broadcom/
> > > - Port to the upstream V4L2 streams API
> > > - Rebase on latest metadata API proposal
> > > - Add missing error message
> > > - Drop unneeded documentation block for unicam_isr()
> > > - Drop unneeded dev_dbg() and dev_err() messages
> > > - Drop unneeded streams_mask and fmt checks
> > > - Drop unused unicam_sd_pad_is_sink()
> > > - Drop unneeded includes
> > > - Drop v4l2_ctrl_subscribe_event() call
> > > - Use pm_runtime_resume_and_get()
> > > - Indentation and line wrap fixes
> > > - Let the framework set bus_info
> > > - Use v4l2_fwnode_endpoint_parse()
> > > - Fix media device cleanup
> > > - Drop lane reordering checks
> > > - Fix subdev state locking
> > > - Drop extra debug messages
> > > - Move clock handling to runtime PM handlers
> > > - Reorder functions
> > > - Rename init functions for more clarity
> > > - Initialize runtime PM earlier
> > > - Clarify error messages
> > > - Simplify subdev init with local variable
> > > - Fix subdev cleanup
> > > - Fix typos and indentation
> > > - Don't initialize local variables needlessly
> > > - Simplify num lanes check
> > > - Fix metadata handling in subdev set_fmt
> > > - Drop manual fallback to .s_stream()
> > > - Pass v4l2_pix_format to unicam_calc_format_size_bpl()
> > > - Simplify unicam_set_default_format()
> > > - Fix default format settings
> > > - Add busy check in unicam_s_fmt_meta()
> > > - Add missing \n at end of format strings
> > > - Fix metadata handling in subdev set_fmt
> > > - Fix locking when starting streaming
> > > - Return buffers from start streaming fails
> > > - Fix format validation for metadata node
> > > - Use video_device_pipeline_{start,stop}() helpers
> > > - Simplify format enumeration
> > > - Drop unset variable
> > > - Update MAINTAINERS entry
> > > - Update to the upstream v4l2_async_nf API
> > > - Update to the latest subdev routing API
> > > - Update to the latest subdev state API
> > > - Move from subdev .init_cfg() to .init_state()
> > > - Update to the latest videobuf2 API
> > > - Fix v4l2_subdev_enable_streams() error check
> > > - Use correct pad for the connected subdev
> > > - Return buffers to vb2 when start streaming fails
> > > - Improve debugging in start streaming handler
> > > - Simplify DMA address management
> > > - Drop comment about bcm2835-camera driver
> > > - Clarify comments that explain min/max sizes
> > > - Pass v4l2_pix_format to unicam_try_fmt()
> > > - Drop unneeded local variables
> > > - Rename image-related constants and functions
> > > - Turn unicam_fmt.metadata_fmt into bool
> > > - Rename unicam_fmt to unicam_format_info
> > > - Rename unicam_format_info variables to fmtinfo
> > > - Rename unicam_node.v_fmt to fmt
> > > - Add metadata formats for RAW10, RAW12 and RAW14
> > > - Make metadata formats line-based
> > > - Validate format on metadata video device
> > > - Add Co-devlopped-by tags
> > >
> > > Changes since v3:
> > >
> > > - Add the vendor prefix for DT name
> > > - Use the reg-names in DT parsing
> > > - Remove MAINTAINERS entry
> > >
> > > Changes since v2:
> > >
> > > - Change code organization
> > > - Remove unused variables
> > > - Correct the fmt_meta functions
> > > - Rewrite the start/stop streaming
> > >   - You can now start the image node alone, but not the metadata one
> > >   - The buffers are allocated per-node
> > >   - only the required stream is started, if the route exists and is
> > >     enabled
> > > - Prefix the macros with UNICAM_ to not have too generic names
> > > - Drop colorspace support
> > >
> > > Changes since v1:
> > >
> > > - Replace the unicam_{info,debug,error} macros with dev_*()
> > > ---
> > >  MAINTAINERS                                   |    1 +
> > >  drivers/media/platform/Kconfig                |    1 +
> > >  drivers/media/platform/Makefile               |    1 +
> > >  drivers/media/platform/broadcom/Kconfig       |   23 +
> > >  drivers/media/platform/broadcom/Makefile      |    3 +
> > >  .../platform/broadcom/bcm2835-unicam-regs.h   |  255 ++
> > >  .../media/platform/broadcom/bcm2835-unicam.c  | 2607 +++++++++++++++++
> > >  7 files changed, 2891 insertions(+)
> > >  create mode 100644 drivers/media/platform/broadcom/Kconfig
> > >  create mode 100644 drivers/media/platform/broadcom/Makefile
> > >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam.c
>
> [snip]
>
> > > diff --git a/drivers/media/platform/broadcom/bcm2835-unicam.c b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > > new file mode 100644
> > > index 000000000000..716c89b8a217
> > > --- /dev/null
> > > +++ b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > > @@ -0,0 +1,2607 @@
> > > +// SPDX-License-Identifier: GPL-2.0-only
> > > +/*
> > > + * BCM283x / BCM271x Unicam Capture Driver
> > > + *
> > > + * Copyright (C) 2017-2020 - Raspberry Pi (Trading) Ltd.
> > > + *
> > > + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > + *
> > > + * Based on TI am437x driver by
> > > + *   Benoit Parrot <bparrot@ti.com>
> > > + *   Lad, Prabhakar <prabhakar.csengg@gmail.com>
> > > + *
> > > + * and TI CAL camera interface driver by
> > > + *    Benoit Parrot <bparrot@ti.com>
> > > + *
> > > + *
> > > + * There are two camera drivers in the kernel for BCM283x - this one and
> > > + * bcm2835-camera (currently in staging).
> > > + *
> > > + * This driver directly controls the Unicam peripheral - there is no
> > > + * involvement with the VideoCore firmware. Unicam receives CSI-2 or CCP2 data
> > > + * and writes it into SDRAM. The only potential processing options are to
> > > + * repack Bayer data into an alternate format, and applying windowing. The
> > > + * repacking does not shift the data, so can repack V4L2_PIX_FMT_Sxxxx10P to
> > > + * V4L2_PIX_FMT_Sxxxx10, or V4L2_PIX_FMT_Sxxxx12P to V4L2_PIX_FMT_Sxxxx12, but
> > > + * not generically up to V4L2_PIX_FMT_Sxxxx16. The driver will add both formats
> > > + * where the relevant formats are defined, and will automatically configure the
> > > + * repacking as required. Support for windowing may be added later.
> >
> > Does this last paragraph applies ? I see all the supported packed raw
> > format having an 'unpacked_fourcc' associated (all but the 8 bit ones
> > ofc)
>
> Maybe I'm tired, but I'm not sure to understand why you think it may not
> apply.
>
> > > + *
> > > + * It should be possible to connect this driver to any sensor with a suitable
> > > + * output interface and V4L2 subdevice driver.
> > > + */
> > > +
> > > +#include <linux/clk.h>
> > > +#include <linux/delay.h>
> > > +#include <linux/device.h>
> > > +#include <linux/dma-mapping.h>
> > > +#include <linux/err.h>
> > > +#include <linux/interrupt.h>
> > > +#include <linux/io.h>
> > > +#include <linux/module.h>
> > > +#include <linux/of.h>
> > > +#include <linux/of_device.h>
> > > +#include <linux/platform_device.h>
> > > +#include <linux/pm_runtime.h>
> > > +#include <linux/slab.h>
> > > +#include <linux/videodev2.h>
> > > +
> > > +#include <media/v4l2-async.h>
> > > +#include <media/v4l2-common.h>
> > > +#include <media/v4l2-dev.h>
> > > +#include <media/v4l2-device.h>
> > > +#include <media/v4l2-event.h>
> > > +#include <media/v4l2-ioctl.h>
> > > +#include <media/v4l2-fwnode.h>
> > > +#include <media/v4l2-mc.h>
> > > +#include <media/videobuf2-dma-contig.h>
> > > +
> > > +#include "bcm2835-unicam-regs.h"
> > > +
> > > +#define UNICAM_MODULE_NAME         "unicam"
> > > +
> > > +/*
> > > + * Unicam must request a minimum of 250Mhz from the VPU clock.
> > > + * Otherwise the input FIFOs overrun and cause image corruption.
> > > + */
> > > +#define UNICAM_MIN_VPU_CLOCK_RATE  (250 * 1000 * 1000)
> >
> > Shouldn't this be set in DT ?
>
> With assigned-clock-rates ? Then the clock would always run at 250MHz
> (or higher, depending on the rate in the DT), while the driver ensures
> it runs at at least 250MHz only when the device is being used.
>
> > > +
> > > +/* Unicam has an internal DMA alignment constraint of 16 bytes for each line. */
> > > +#define UNICAM_DMA_BPL_ALIGNMENT   16
> > > +
> > > +/*
> > > + * The image stride is stored in a 16 bit register, and needs to be aligned to
> > > + * the DMA constraint. As the ISP in the same SoC has a 32 bytes alignment
> > > + * constraint on its input, set the image stride alignment to 32 bytes here as
> > > + * well to avoid incompatible configurations.
> > > + */
> > > +#define UNICAM_IMAGE_BPL_ALIGNMENT 32
> > > +#define UNICAM_IMAGE_MAX_BPL               ((1 << 16) - UNICAM_IMAGE_BPL_ALIGNMENT)
> > > +
> > > +/*
> > > + * Max width is therefore determined by the max stride divided by the number of
> > > + * bits per pixel. Take 32bpp as a worst case. No imposed limit on the height,
> > > + * so adopt a square image for want of anything better.
> > > + */
> > > +#define UNICAM_IMAGE_MIN_WIDTH             16
> > > +#define UNICAM_IMAGE_MIN_HEIGHT            16
> > > +#define UNICAM_IMAGE_MAX_WIDTH             (UNICAM_IMAGE_MAX_BPL / 4)
> > > +#define UNICAM_IMAGE_MAX_HEIGHT            UNICAM_IMAGE_MAX_WIDTH
> > > +
> > > +/*
> > > + * There's no intrinsic limits on the width and height for embedded dat. Use

s/dat/data

> > > + * the same maximum values as for the image, to avoid overflows in the image
> > > + * size computation.
> > > + */
> > > +#define UNICAM_META_MIN_WIDTH              1
> > > +#define UNICAM_META_MIN_HEIGHT             1
> > > +#define UNICAM_META_MAX_WIDTH              UNICAM_IMAGE_MAX_WIDTH
> > > +#define UNICAM_META_MAX_HEIGHT             UNICAM_IMAGE_MAX_HEIGHT
> > > +
> > > +/*
> > > + * Size of the dummy buffer. Can be any size really, but the DMA
> > > + * allocation works in units of page sizes.
> > > + */
> > > +#define UNICAM_DUMMY_BUF_SIZE              PAGE_SIZE
> > > +
> > > +#define UNICAM_SD_PAD_SINK         0
> > > +#define UNICAM_SD_PAD_SOURCE_IMAGE 1
> > > +#define UNICAM_SD_PAD_SOURCE_METADATA      2
> > > +#define UNICAM_SD_NUM_PADS         (1 + UNICAM_SD_PAD_SOURCE_METADATA)
> >
> > How about an enum then ?
>
> OK.
>
> > > +
> > > +enum unicam_node_type {
> > > +   UNICAM_IMAGE_NODE,
> > > +   UNICAM_METADATA_NODE,
> > > +   UNICAM_MAX_NODES
> > > +};
> > > +
> > > +/*
> > > + * struct unicam_format_info - Unicam media bus format information
> > > + * @fourcc: V4L2 pixel format FCC identifier. 0 if n/a.
> > > + * @unpacked_fourcc: V4L2 pixel format FCC identifier if the data is expanded
> > > + * out to 16bpp. 0 if n/a.
> > > + * @code: V4L2 media bus format code.
> > > + * @depth: Bits per pixel as delivered from the source.
> > > + * @csi_dt: CSI data type.
> > > + * @metadata_fmt: This format only applies to the metadata pad.
> > > + */
> > > +struct unicam_format_info {
> > > +   u32     fourcc;
> > > +   u32     unpacked_fourcc;
> > > +   u32     code;
> > > +   u8      depth;
> > > +   u8      csi_dt;
> > > +   bool    metadata_fmt;
> > > +};
> > > +
> > > +struct unicam_buffer {
> > > +   struct vb2_v4l2_buffer vb;
> > > +   struct list_head list;
> > > +   dma_addr_t dma_addr;
> > > +   unsigned int size;
> > > +};
> > > +
> > > +static inline struct unicam_buffer *to_unicam_buffer(struct vb2_buffer *vb)
> > > +{
> > > +   return container_of(vb, struct unicam_buffer, vb.vb2_buf);
> > > +}
> > > +
> > > +struct unicam_node {
> > > +   bool registered;
> > > +   bool streaming;
> > > +   unsigned int id;
> > > +
> > > +   /* Pointer to the current v4l2_buffer */
> > > +   struct unicam_buffer *cur_frm;
> > > +   /* Pointer to the next v4l2_buffer */
> > > +   struct unicam_buffer *next_frm;
> > > +   /* video capture */
> > > +   const struct unicam_format_info *fmtinfo;
> > > +   /* Used to store current pixel format */
> > > +   struct v4l2_format fmt;
> > > +   /* Buffer queue used in video-buf */
> > > +   struct vb2_queue buffer_queue;
> > > +   /* Queue of filled frames */
> > > +   struct list_head dma_queue;
> > > +   /* IRQ lock for DMA queue */
> > > +   spinlock_t dma_queue_lock;
> > > +   /* lock used to access this structure */
> > > +   struct mutex lock;
> > > +   /* Identifies video device for this channel */
> > > +   struct video_device video_dev;
> > > +   /* Pointer to the parent handle */
> > > +   struct unicam_device *dev;
> > > +   struct media_pad pad;
> > > +   /*
> > > +    * Dummy buffer intended to be used by unicam
> > > +    * if we have no other queued buffers to swap to.
> > > +    */
> > > +   struct unicam_buffer dummy_buf;
> > > +   void *dummy_buf_cpu_addr;
> > > +};
> > > +
> > > +struct unicam_device {
> > > +   struct kref kref;
> > > +
> > > +   /* peripheral base address */
> > > +   void __iomem *base;
> > > +   /* clock gating base address */
> > > +   void __iomem *clk_gate_base;
> >
> > Is the clock gating part of unicam or should this be expressed as a
> > clock provide and be referenced in DT through a phandle ?
>
> No idea :-) Dave, Naush ?

Answered in my previous reply.

> > > +   /* lp clock handle */
> > > +   struct clk *clock;
> > > +   /* vpu clock handle */
> > > +   struct clk *vpu_clock;
> > > +   /* V4l2 device */
> > > +   struct v4l2_device v4l2_dev;
> > > +   struct media_device mdev;
> > > +
> > > +   /* parent device */
> > > +   struct device *dev;
> > > +   /* subdevice async Notifier */
> >
> > s/Notifier/notifier
> >
> > > +   struct v4l2_async_notifier notifier;
> > > +   unsigned int sequence;
> > > +
> > > +   /* Sensor node */
> > > +   struct {
> > > +           struct v4l2_subdev *subdev;
> > > +           struct media_pad *pad;
> > > +   } sensor;
> > > +
> > > +   /* Internal subdev */
> > > +   struct {
> > > +           struct v4l2_subdev sd;
> > > +           struct media_pad pads[UNICAM_SD_NUM_PADS];
> > > +           bool streaming;
> > > +   } subdev;
> > > +
> > > +   enum v4l2_mbus_type bus_type;
> > > +   /*
> > > +    * Stores bus.mipi_csi2.flags for CSI2 sensors, or
> > > +    * bus.mipi_csi1.strobe for CCP2.
> > > +    */
> > > +   unsigned int bus_flags;
> > > +   unsigned int max_data_lanes;
> > > +   unsigned int active_data_lanes;
> > > +
> > > +   struct media_pipeline pipe;
> > > +
> > > +   struct unicam_node node[UNICAM_MAX_NODES];
> > > +};
> > > +
> > > +static inline struct unicam_device *
> > > +notifier_to_unicam_device(struct v4l2_async_notifier *notifier)
> > > +{
> > > +   return container_of(notifier, struct unicam_device, notifier);
> > > +}
> > > +
> > > +static inline struct unicam_device *
> >
> > I thought using inline is disouraged as the compiler knows what to do
>
> For this kind of wrapper around container_of() the kernel usually has an
> explicit inline keyword. I can drop it if needed.
>
> > > +sd_to_unicam_device(struct v4l2_subdev *sd)
> > > +{
> > > +   return container_of(sd, struct unicam_device, subdev.sd);
> >
> > You can also use container_of_const() if you need it
>
> There's no const subdev anywhere.
>
> > > +}
> > > +
> > > +static void unicam_release(struct kref *kref)
> > > +{
> > > +   struct unicam_device *unicam =
> > > +           container_of(kref, struct unicam_device, kref);
> > > +
> > > +   if (unicam->mdev.dev)
> > > +           media_device_cleanup(&unicam->mdev);
> > > +
> > > +   kfree(unicam);
> > > +}
> > > +
> > > +static struct unicam_device *unicam_get(struct unicam_device *unicam)
> > > +{
> > > +   kref_get(&unicam->kref);
> > > +   return unicam;
> > > +}
> > > +
> > > +static void unicam_put(struct unicam_device *unicam)
> > > +{
> > > +   kref_put(&unicam->kref, unicam_release);
> > > +}
> > > +
> >
> > Does this handle a bit more gracefully the pesky lifetime management
> > issue in v4l2 ?
>
> That's supposed to be the idea :-)
>
> > > +/* -----------------------------------------------------------------------------
> > > + * Misc helper functions
> > > + */
> > > +
> > > +static inline bool unicam_sd_pad_is_source(u32 pad)
> > > +{
> > > +   /* Camera RX has 1 sink pad, and N source pads */
> > > +   return pad != UNICAM_SD_PAD_SINK;
> > > +}
> > > +
> > > +static inline bool is_metadata_node(struct unicam_node *node)
> > > +{
> > > +   return node->video_dev.device_caps & V4L2_CAP_META_CAPTURE;
> > > +}
> > > +
> > > +static inline bool is_image_node(struct unicam_node *node)
> > > +{
> > > +   return node->video_dev.device_caps & V4L2_CAP_VIDEO_CAPTURE;
> > > +}
>
> [snip]
>
> > > +/* -----------------------------------------------------------------------------
> > > + * Videobuf2 queue operations
> > > + */
> > > +
> > > +static int unicam_queue_setup(struct vb2_queue *vq,
> > > +                         unsigned int *nbuffers,
> > > +                         unsigned int *nplanes,
> > > +                         unsigned int sizes[],
> > > +                         struct device *alloc_devs[])
> > > +{
> > > +   struct unicam_node *node = vb2_get_drv_priv(vq);
> > > +   u32 size = is_image_node(node) ? node->fmt.fmt.pix.sizeimage
> > > +            : node->fmt.fmt.meta.buffersize;
> > > +
> > > +   if (vq->num_buffers + *nbuffers < 3)
> >
> > Why 3 ? Are these dummies ?
>
> This may be a remnant of old code. Dave, Naush, any comment ?

I suspect this is legacy.
Originally the driver would only release the buffer at the frame end
when it had a new one to switch to. Naush updated with the dummy
buffer so in theory you can run with 1 buffer, but this min number of
buffers probably didn't get reduced.
Then again it may have been a misunderstanding of the framework, as
struct vb2_queue min_buffers_needed should set the minimum.

> > > +           *nbuffers = 3 - vq->num_buffers;
> > > +
> > > +   if (*nplanes) {
> > > +           if (sizes[0] < size) {
> > > +                   dev_dbg(node->dev->dev, "sizes[0] %i < size %u\n",
> > > +                           sizes[0], size);
> > > +                   return -EINVAL;
> > > +           }
> > > +           size = sizes[0];
> > > +   }
> > > +
> > > +   *nplanes = 1;
> > > +   sizes[0] = size;
> > > +
> > > +   return 0;
> > > +}
> > > +
> > > +static int unicam_buffer_prepare(struct vb2_buffer *vb)
> > > +{
> > > +   struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
> > > +   struct unicam_buffer *buf = to_unicam_buffer(vb);
> > > +   u32 size = is_image_node(node) ? node->fmt.fmt.pix.sizeimage
> > > +            : node->fmt.fmt.meta.buffersize;
> > > +
> > > +   if (vb2_plane_size(vb, 0) < size) {
> > > +           dev_dbg(node->dev->dev,
> > > +                   "data will not fit into plane (%lu < %u)\n",
> > > +                   vb2_plane_size(vb, 0), size);
> > > +           return -EINVAL;
> > > +   }
> > > +
> > > +   buf->dma_addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
> > > +   buf->size = size;
> > > +
> > > +   vb2_set_plane_payload(&buf->vb.vb2_buf, 0, size);
> > > +
> > > +   return 0;
> > > +}
> > > +
> > > +static void unicam_return_buffers(struct unicam_node *node,
> > > +                             enum vb2_buffer_state state)
> > > +{
> > > +   struct unicam_buffer *buf, *tmp;
> > > +
> > > +   list_for_each_entry_safe(buf, tmp, &node->dma_queue, list) {
> > > +           list_del(&buf->list);
> > > +           vb2_buffer_done(&buf->vb.vb2_buf, state);
> > > +   }
> > > +
> > > +   if (node->cur_frm)
> > > +           vb2_buffer_done(&node->cur_frm->vb.vb2_buf,
> > > +                           state);
> > > +   if (node->next_frm && node->cur_frm != node->next_frm)
> > > +           vb2_buffer_done(&node->next_frm->vb.vb2_buf,
> > > +                           state);
> > > +
> > > +   node->cur_frm = NULL;
> > > +   node->next_frm = NULL;
> > > +}
> > > +
> > > +static int unicam_start_streaming(struct vb2_queue *vq, unsigned int count)
> > > +{
> > > +   struct unicam_node *node = vb2_get_drv_priv(vq);
> > > +   struct unicam_device *unicam = node->dev;
> > > +   struct v4l2_subdev_state *state;
> > > +   struct unicam_buffer *buf;
> > > +   unsigned long flags;
> > > +   int ret;
> > > +   u32 pad, stream;
> > > +   u32 remote_pad = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
> > > +                                        : UNICAM_SD_PAD_SOURCE_METADATA;
> > > +
> > > +   /* Look for the route for the given pad and stream. */
> > > +   state = v4l2_subdev_lock_and_get_active_state(&unicam->subdev.sd);
> > > +   ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
> > > +                                               remote_pad, 0,
> > > +                                               &pad, &stream);
> > > +   v4l2_subdev_unlock_state(state);
> > > +
> > > +   if (ret)
> > > +           goto err_return_buffers;
> > > +
> > > +   dev_dbg(unicam->dev, "Starting stream on %s: %u/%u -> %u/%u (%s)\n",
> > > +           unicam->subdev.sd.name, pad, stream, remote_pad, 0,
> > > +           is_metadata_node(node) ? "metadata" : "image");
> > > +
> > > +   /* The metadata node can't be started alone. */
> > > +   if (is_metadata_node(node)) {
> > > +           if (!unicam->node[UNICAM_IMAGE_NODE].streaming) {
> >
> > Does it mean the metadata node has to be started second, or should
> > this be made a nop and the metadata node gets started once the image
> > node is started too ? I'm fine with the constraint of having the
> > metadata node being started second fwiw
>
> I think it would be nice to change this indeed. Dave, Naush, any
> objection ?

See previous email.

> > > +                   dev_err(unicam->dev,
> > > +                           "Can't start metadata without image\n");
> > > +                   ret = -EINVAL;
> > > +                   goto err_return_buffers;
> > > +           }
> > > +
> > > +           spin_lock_irqsave(&node->dma_queue_lock, flags);
> > > +           buf = list_first_entry(&node->dma_queue,
> > > +                                  struct unicam_buffer, list);
> > > +           dev_dbg(unicam->dev, "buffer %p\n", buf);
> >
> > Is this useful ?
>
> Probably not much. I'll drop it.
>
> > > +           node->cur_frm = buf;
> > > +           node->next_frm = buf;
> > > +           list_del(&buf->list);
> > > +           spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> > > +
> > > +           unicam_start_metadata(unicam, buf);
> > > +           node->streaming = true;
> > > +           return 0;
> > > +   }
> > > +
> > > +   ret = pm_runtime_resume_and_get(unicam->dev);
> > > +   if (ret < 0) {
> > > +           dev_err(unicam->dev, "PM runtime resume failed: %d\n", ret);
> > > +           goto err_return_buffers;
> > > +   }
> > > +
> > > +   ret = video_device_pipeline_start(&node->video_dev, &unicam->pipe);
> > > +   if (ret < 0) {
> > > +           dev_dbg(unicam->dev, "Failed to start media pipeline: %d\n", ret);
> >
> > Isn't this an err ?
>
> The main cause of failure is a pipeline validation error, triggered by
> userspace, hence the debug level.
>
> > > +           goto err_pm_put;
> > > +   }
> > > +
> > > +   spin_lock_irqsave(&node->dma_queue_lock, flags);
> > > +   buf = list_first_entry(&node->dma_queue,
> > > +                          struct unicam_buffer, list);
> > > +   dev_dbg(unicam->dev, "buffer %p\n", buf);
> > > +   node->cur_frm = buf;
> > > +   node->next_frm = buf;
> > > +   list_del(&buf->list);
> > > +   spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> > > +
> > > +   unicam_start_rx(unicam, buf);
> > > +
> > > +   ret = v4l2_subdev_enable_streams(&unicam->subdev.sd, remote_pad, BIT(0));
> >
> > A bit confused by the API here, shouldn't we also do this for embedded
> > data ?
>
> Not here, as the two streams go over different pads, but likely above,
> as part of the change you proposed regarding stream start on the
> metadata device. I'll wait for Dave and Naush to reply, and I'll rework
> this function.

I haven't read enough on the streams API, or what this implementation
looks like.

There's no sensible way for a sensor to start embedded or other
metadata without image data, so starting the image node would seem to
be the main trigger for anything. I'm also assuming we don't support
enabling additional multiplexed streams once the pipeline is already
running, so that would seem to determine some of the sequencing.

  Dave

> > > +   if (ret < 0) {
> > > +           dev_err(unicam->dev, "stream on failed in subdev\n");
> > > +           goto error_pipeline;
> > > +   }
> > > +
> > > +   node->streaming = true;
> > > +
> > > +   return 0;
> > > +
> > > +error_pipeline:
> > > +   video_device_pipeline_stop(&node->video_dev);
> > > +err_pm_put:
> > > +   pm_runtime_put_sync(unicam->dev);
> > > +err_return_buffers:
> > > +   unicam_return_buffers(node, VB2_BUF_STATE_QUEUED);
> > > +   return ret;
> > > +}
> > > +
> > > +static void unicam_stop_streaming(struct vb2_queue *vq)
> > > +{
> > > +   struct unicam_node *node = vb2_get_drv_priv(vq);
> > > +   struct unicam_device *unicam = node->dev;
> > > +   u32 remote_pad_index = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
> > > +                                              : UNICAM_SD_PAD_SOURCE_METADATA;
> > > +
> > > +   node->streaming = false;
> > > +
> > > +   v4l2_subdev_disable_streams(&unicam->subdev.sd, remote_pad_index,
> > > +                               BIT(0));
> > > +
> > > +   /* We can stream only with the image node. */
> > > +   if (is_metadata_node(node)) {
> > > +           /*
> > > +            * Allow the hardware to spin in the dummy buffer.
> > > +            * This is only really needed if the embedded data pad is
> > > +            * disabled before the image pad.
> > > +            */
> > > +           unicam_wr_dma_addr(node, &node->dummy_buf);
> > > +           goto dequeue_buffers;
> > > +   }
> > > +
> > > +   unicam_disable(unicam);
> > > +
> > > +   video_device_pipeline_stop(&node->video_dev);
> > > +   pm_runtime_put(unicam->dev);
> > > +
> > > +dequeue_buffers:
> > > +   /* Clear all queued buffers for the node */
> > > +   unicam_return_buffers(node, VB2_BUF_STATE_ERROR);
> > > +}
> > > +
> > > +static void unicam_buffer_queue(struct vb2_buffer *vb)
> > > +{
> > > +   struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
> > > +   struct unicam_buffer *buf = to_unicam_buffer(vb);
> > > +
> > > +   spin_lock_irq(&node->dma_queue_lock);
> > > +   list_add_tail(&buf->list, &node->dma_queue);
> > > +   spin_unlock_irq(&node->dma_queue_lock);
> > > +}
> > > +
> > > +static const struct vb2_ops unicam_video_qops = {
> > > +   .queue_setup            = unicam_queue_setup,
> > > +   .wait_prepare           = vb2_ops_wait_prepare,
> > > +   .wait_finish            = vb2_ops_wait_finish,
> > > +   .buf_prepare            = unicam_buffer_prepare,
> > > +   .start_streaming        = unicam_start_streaming,
> > > +   .stop_streaming         = unicam_stop_streaming,
> > > +   .buf_queue              = unicam_buffer_queue,
> > > +};
> > > +
> > > +/* -----------------------------------------------------------------------------
> > > + *  V4L2 video device operations
> > > + */
> > > +
> > > +static int unicam_querycap(struct file *file, void *priv,
> > > +                      struct v4l2_capability *cap)
> > > +{
> > > +   strscpy(cap->driver, UNICAM_MODULE_NAME, sizeof(cap->driver));
> > > +   strscpy(cap->card, UNICAM_MODULE_NAME, sizeof(cap->card));
> > > +
> > > +   cap->capabilities |= V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_META_CAPTURE;
> > > +
> > > +   return 0;
> > > +}
> > > +
> > > +static int unicam_enum_fmt_vid(struct file *file, void  *priv,
> > > +                          struct v4l2_fmtdesc *f)
> > > +{
> > > +   unsigned int index;
> > > +   unsigned int i;
> > > +
> > > +   for (i = 0, index = 0; i < ARRAY_SIZE(unicam_image_formats); i++) {
> > > +           if (f->mbus_code && unicam_image_formats[i].code != f->mbus_code)
> > > +                   continue;
> > > +
> > > +           if (index == f->index) {
> > > +                   f->pixelformat = unicam_image_formats[i].fourcc;
> > > +                   return 0;
> > > +           }
> > > +
> > > +           index++;
> > > +
> > > +           if (!unicam_image_formats[i].unpacked_fourcc)
> > > +                   continue;
> > > +
> > > +           if (index == f->index) {
> > > +                   f->pixelformat = unicam_image_formats[i].unpacked_fourcc;
> > > +                   return 0;
> > > +           }
> > > +
> > > +           index++;
> > > +   }
> > > +
> > > +   return -EINVAL;
> > > +}
> > > +
> > > +static int unicam_g_fmt_vid(struct file *file, void *priv,
> > > +                       struct v4l2_format *f)
> > > +{
> > > +   struct unicam_node *node = video_drvdata(file);
> > > +
> > > +   *f = node->fmt;
> > > +
> > > +   return 0;
> > > +}
> > > +
> > > +static const struct unicam_format_info *
> > > +__unicam_try_fmt_vid(struct unicam_node *node, struct v4l2_pix_format *pix)
> > > +{
> > > +   const struct unicam_format_info *fmtinfo;
> > > +
> > > +   /*
> > > +    * Default to the first format if the requested pixel format code isn't
> > > +    * supported.
> > > +    */
> > > +   fmtinfo = unicam_find_format_by_fourcc(pix->pixelformat,
> > > +                                          UNICAM_SD_PAD_SOURCE_IMAGE);
> > > +   if (!fmtinfo) {
> > > +           fmtinfo = &unicam_image_formats[0];
> > > +           pix->pixelformat = fmtinfo->fourcc;
> > > +   }
> > > +
> > > +   unicam_calc_image_size_bpl(node->dev, fmtinfo, pix);
> > > +
> > > +   if (pix->field == V4L2_FIELD_ANY)
> > > +           pix->field = V4L2_FIELD_NONE;
> > > +
> > > +   return fmtinfo;
> > > +}
> > > +
> > > +static int unicam_try_fmt_vid(struct file *file, void *priv,
> > > +                         struct v4l2_format *f)
> > > +{
> > > +   struct unicam_node *node = video_drvdata(file);
> > > +
> > > +   __unicam_try_fmt_vid(node, &f->fmt.pix);
> > > +   return 0;
> > > +}
> > > +
> > > +static int unicam_s_fmt_vid(struct file *file, void *priv,
> > > +                       struct v4l2_format *f)
> > > +{
> > > +   struct unicam_node *node = video_drvdata(file);
> > > +
> > > +   if (vb2_is_busy(&node->buffer_queue))
> > > +           return -EBUSY;
> > > +
> > > +   node->fmtinfo = __unicam_try_fmt_vid(node, &f->fmt.pix);
> > > +   node->fmt = *f;
> > > +
> > > +   return 0;
> > > +}
> > > +
> > > +static int unicam_enum_fmt_meta(struct file *file, void *priv,
> > > +                           struct v4l2_fmtdesc *f)
> > > +{
> > > +   unsigned int i, index;
> > > +
> > > +   for (i = 0, index = 0; i < ARRAY_SIZE(unicam_meta_formats); i++) {
> > > +           if (f->mbus_code && unicam_meta_formats[i].code != f->mbus_code)
> >
> > Do we want to allow mbus_code filtering for metadata ? There's a
> > 1-to-1 relationship between mbus codes and pixel formats
>
> Is there a reason not to allow it ? I think it's good for the API to be
> consistent. Generally speaking, a CSI-2 receiver could receive
> META_CSI2_10 and convert it to the META_8 pixel format. Filtering as a
> concept thus make sense I think.
>
> > > +                   continue;
> > > +           if (!unicam_meta_formats[i].metadata_fmt)
> > > +                   continue;
> >
> > How can this be false if we're iterating on unicam_meta_formats[] ?
>
> True. I'll drop the metadata_fmt field.
>
> > > +
> > > +           if (index == f->index) {
> > > +                   f->pixelformat = unicam_meta_formats[i].fourcc;
> > > +                   f->type = V4L2_BUF_TYPE_META_CAPTURE;
> > > +                   return 0;
> > > +           }
> > > +           index++;
> > > +   }
> > > +
> > > +   return -EINVAL;
> > > +}
> > > +
> > > +static int unicam_g_fmt_meta(struct file *file, void *priv,
> > > +                        struct v4l2_format *f)
> > > +{
> > > +   struct unicam_node *node = video_drvdata(file);
> > > +
> > > +   f->fmt.meta = node->fmt.fmt.meta;
> > > +
> > > +   return 0;
> > > +}
> > > +
> > > +static const struct unicam_format_info *
> > > +__unicam_try_fmt_meta(struct unicam_node *node, struct v4l2_meta_format *meta)
> > > +{
> > > +   const struct unicam_format_info *fmtinfo;
> > > +
> > > +   /*
> > > +    * Default to the first format if the requested pixel format code isn't
> > > +    * supported.
> > > +    */
> > > +   fmtinfo = unicam_find_format_by_fourcc(meta->dataformat,
> > > +                                          UNICAM_SD_PAD_SOURCE_METADATA);
> > > +   if (!fmtinfo) {
> > > +           fmtinfo = &unicam_meta_formats[0];
> > > +           meta->dataformat = fmtinfo->fourcc;
> > > +   }
> > > +
> > > +   unicam_calc_meta_size_bpl(node->dev, fmtinfo, meta);
> > > +
> > > +   return fmtinfo;
> > > +}
> > > +
> > > +static int unicam_try_fmt_meta(struct file *file, void *priv,
> > > +                          struct v4l2_format *f)
> > > +{
> > > +   struct unicam_node *node = video_drvdata(file);
> > > +
> > > +   __unicam_try_fmt_vid(node, &f->fmt.pix);
> > > +   return 0;
> > > +}
> > > +
> > > +static int unicam_s_fmt_meta(struct file *file, void *priv,
> > > +                        struct v4l2_format *f)
> > > +{
> > > +   struct unicam_node *node = video_drvdata(file);
> > > +
> > > +   if (vb2_is_busy(&node->buffer_queue))
> > > +           return -EBUSY;
> > > +
> > > +   node->fmtinfo = __unicam_try_fmt_meta(node, &f->fmt.meta);
> > > +   node->fmt = *f;
> > > +
> > > +   return 0;
> > > +}
> > > +
> > > +static int unicam_enum_framesizes(struct file *file, void *fh,
> > > +                             struct v4l2_frmsizeenum *fsize)
> > > +{
> > > +   struct unicam_node *node = video_drvdata(file);
> > > +   int ret = -EINVAL;
> > > +
> > > +   if (fsize->index > 0)
> > > +           return ret;
> > > +
> > > +   if (is_image_node(node)) {
> > > +           if (!unicam_find_format_by_fourcc(fsize->pixel_format,
> > > +                                             UNICAM_SD_PAD_SOURCE_IMAGE))
> > > +                   return ret;
> > > +
> > > +           fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> > > +           fsize->stepwise.min_width = UNICAM_IMAGE_MIN_WIDTH;
> > > +           fsize->stepwise.max_width = UNICAM_IMAGE_MAX_WIDTH;
> > > +           fsize->stepwise.step_width = 1;
> > > +           fsize->stepwise.min_height = UNICAM_IMAGE_MIN_HEIGHT;
> > > +           fsize->stepwise.max_height = UNICAM_IMAGE_MAX_HEIGHT;
> > > +           fsize->stepwise.step_height = 1;
> > > +   } else {
> > > +           if (!unicam_find_format_by_fourcc(fsize->pixel_format,
> > > +                                             UNICAM_SD_PAD_SOURCE_METADATA))
> > > +                   return ret;
> > > +
> >
> > Isn't this V4L2_FRMSIZE_TYPE_STEPWISE as well ?
>
> Indeed, I'll fix that.
>
> > > +           fsize->stepwise.min_width = UNICAM_META_MIN_WIDTH;
> > > +           fsize->stepwise.max_width = UNICAM_META_MAX_WIDTH;
> > > +           fsize->stepwise.step_width = 1;
> > > +           fsize->stepwise.min_height = UNICAM_META_MIN_HEIGHT;
> > > +           fsize->stepwise.max_height = UNICAM_META_MAX_HEIGHT;
> > > +           fsize->stepwise.step_height = 1;
> > > +   }
> > > +
> > > +   return 0;
> > > +}
>
> [snip]
>
> --
> Regards,
>
> Laurent Pinchart

Re: [PATCH v6 01/15] media: i2c: imx219: Inline imx219_update_pad_format() in its caller

[Dave Stevenson]

Hi Laurent

On Fri, 1 Mar 2024 at 21:32, Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> The imx219_update_pad_format() is short and called from a single place,
> in imx219_set_pad_format(). Inline the code in the caller to keep all
> format adjustements grouped in a single place and improve readability.

s/adjustements/adjustments

>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Otherwise
Reviewed-by: Dave Stevenson <dave.stevenson@raspberrypi.com>

> ---
>  drivers/media/i2c/imx219.c | 28 ++++++++++++----------------
>  1 file changed, 12 insertions(+), 16 deletions(-)
>
> diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> index e17ef2e9d9d0..3878da50860e 100644
> --- a/drivers/media/i2c/imx219.c
> +++ b/drivers/media/i2c/imx219.c
> @@ -758,21 +758,6 @@ static int imx219_set_stream(struct v4l2_subdev *sd, int enable)
>         return ret;
>  }
>
> -static void imx219_update_pad_format(struct imx219 *imx219,
> -                                    const struct imx219_mode *mode,
> -                                    struct v4l2_mbus_framefmt *fmt, u32 code)
> -{
> -       /* Bayer order varies with flips */
> -       fmt->code = imx219_get_format_code(imx219, code);
> -       fmt->width = mode->width;
> -       fmt->height = mode->height;
> -       fmt->field = V4L2_FIELD_NONE;
> -       fmt->colorspace = V4L2_COLORSPACE_RAW;
> -       fmt->ycbcr_enc = V4L2_YCBCR_ENC_601;
> -       fmt->quantization = V4L2_QUANTIZATION_FULL_RANGE;
> -       fmt->xfer_func = V4L2_XFER_FUNC_NONE;
> -}
> -
>  static int imx219_enum_mbus_code(struct v4l2_subdev *sd,
>                                  struct v4l2_subdev_state *state,
>                                  struct v4l2_subdev_mbus_code_enum *code)
> @@ -819,12 +804,23 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
>         struct v4l2_rect *crop;
>         unsigned int bin_h, bin_v;
>
> +       /*
> +        * Adjust the requested format to match the closest mode. The Bayer
> +        * order varies with flips.
> +        */
>         mode = v4l2_find_nearest_size(supported_modes,
>                                       ARRAY_SIZE(supported_modes),
>                                       width, height,
>                                       fmt->format.width, fmt->format.height);
>
> -       imx219_update_pad_format(imx219, mode, &fmt->format, fmt->format.code);
> +       fmt->format.code = imx219_get_format_code(imx219, fmt->format.code);
> +       fmt->format.width = mode->width;
> +       fmt->format.height = mode->height;
> +       fmt->format.field = V4L2_FIELD_NONE;
> +       fmt->format.colorspace = V4L2_COLORSPACE_RAW;
> +       fmt->format.ycbcr_enc = V4L2_YCBCR_ENC_601;
> +       fmt->format.quantization = V4L2_QUANTIZATION_FULL_RANGE;
> +       fmt->format.xfer_func = V4L2_XFER_FUNC_NONE;
>
>         format = v4l2_subdev_state_get_format(state, 0);
>         *format = fmt->format;
> --
> Regards,
>
> Laurent Pinchart
>

Re: [PATCH 00/15] media: Add driver for the Raspberry Pi <5 CSI-2 receiver

[Florian Fainelli]

[-- Attachment #1: Type: text/plain, Size: 2188 bytes --]

Hi Laurent,

On 3/1/2024 1:38 PM, Laurent Pinchart wrote:
> The subject line should obviously have read 'PATCH v6', and I should
> have updated Jean-Michel's e-mail address instead of blindly relying on
> get-maintainer.pl. Maybe sending patches on a Friday evening isn't he
> best idea after all. Sorry about that.
> 
> On Fri, Mar 01, 2024 at 11:32:15PM +0200, Laurent Pinchart wrote:
>> Hello everybody,
>>
>> This patch series adds a new driver for the BCM2835 (and derivative)
>> CCP2/CSI2 camera interface named Unicam. This IP core is found in the
>> VC4-based Raspberry Pi, namely the Pi Zero, Pi 3 and Pi 4.
>>
>> Camera support for Raspberry Pi 4 currently relies on a downstream
>> Unicam driver that live in the Raspberry Pi kernel tree ([1]). The
>> driver uses the V4L2 API, but works around the lack of features in V4L2
>> to properly support sensor embedded data. Since the Unicam driver
>> development by Raspberry Pi, some of those features have been merged in
>> the kernel (namely the V4L2 streams API) or are being developed (namely
>> generic metadata formats and subdev internal pads), with patches posted
>> for review on the linux-media mailing list ([2]).
>>
>> This new upstream driver is based on the downstream code, extensively
>> reworked to use the new V4L2 APIs.
>>
>> The series is based on top of a merge of
>>
>> - v7 of the generic metadata and internal pads, rebased on v6.8-rc5 ([3])
>> - the downstream ISP driver ported to mainline ([4])
>>
>> For convenience, it can be found in [5]. Note that the ISP driver is
>> getting upstreamed separately.
>>
>> The series starts with five patches that add support for streams and
>> embedded data to the imx219 driver (01/15 to 05/15). Patches 06/15 to
>> 09/15 then add the Unicam driver, with new V4L2 pixel formats (06/15 and
>> 07/15) and DT bindings (08/15) The remaining patches cover DT
>> integration (10/15 to 14/15) with a sample DT overlay for the IMX219
>> camera module (15/15).

I am really keen on taking the DTS patches now so you know those are 
taken care of, make it to linux-next shortly and then we can focus on 
the drivers/media aspects. Stefan, does that work for you?
-- 
Florian

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4221 bytes --]

Re: [PATCH 00/15] media: Add driver for the Raspberry Pi <5 CSI-2 receiver

[Laurent Pinchart]

Hi Florian,

On Fri, Mar 15, 2024 at 07:02:38AM -0700, Florian Fainelli wrote:
> On 3/1/2024 1:38 PM, Laurent Pinchart wrote:
> > The subject line should obviously have read 'PATCH v6', and I should
> > have updated Jean-Michel's e-mail address instead of blindly relying on
> > get-maintainer.pl. Maybe sending patches on a Friday evening isn't he
> > best idea after all. Sorry about that.
> > 
> > On Fri, Mar 01, 2024 at 11:32:15PM +0200, Laurent Pinchart wrote:
> >> Hello everybody,
> >>
> >> This patch series adds a new driver for the BCM2835 (and derivative)
> >> CCP2/CSI2 camera interface named Unicam. This IP core is found in the
> >> VC4-based Raspberry Pi, namely the Pi Zero, Pi 3 and Pi 4.
> >>
> >> Camera support for Raspberry Pi 4 currently relies on a downstream
> >> Unicam driver that live in the Raspberry Pi kernel tree ([1]). The
> >> driver uses the V4L2 API, but works around the lack of features in V4L2
> >> to properly support sensor embedded data. Since the Unicam driver
> >> development by Raspberry Pi, some of those features have been merged in
> >> the kernel (namely the V4L2 streams API) or are being developed (namely
> >> generic metadata formats and subdev internal pads), with patches posted
> >> for review on the linux-media mailing list ([2]).
> >>
> >> This new upstream driver is based on the downstream code, extensively
> >> reworked to use the new V4L2 APIs.
> >>
> >> The series is based on top of a merge of
> >>
> >> - v7 of the generic metadata and internal pads, rebased on v6.8-rc5 ([3])
> >> - the downstream ISP driver ported to mainline ([4])
> >>
> >> For convenience, it can be found in [5]. Note that the ISP driver is
> >> getting upstreamed separately.
> >>
> >> The series starts with five patches that add support for streams and
> >> embedded data to the imx219 driver (01/15 to 05/15). Patches 06/15 to
> >> 09/15 then add the Unicam driver, with new V4L2 pixel formats (06/15 and
> >> 07/15) and DT bindings (08/15) The remaining patches cover DT
> >> integration (10/15 to 14/15) with a sample DT overlay for the IMX219
> >> camera module (15/15).
> 
> I am really keen on taking the DTS patches now so you know those are 
> taken care of, make it to linux-next shortly and then we can focus on 
> the drivers/media aspects. Stefan, does that work for you?

10/15 and 11/15 should be fine to merge already, although 11/15 depends
on the DT bindings in 08/15. Even if the unicam driver needs a respin, I
think the bindings are safe to merge already. Do you plan to take them
in your tree with the corresponding DTS patches ?

For 12/15 (the pinctrl-based I2C multiplexing), I'd really like an ack
from Dave. 13/15 is pretty straightforward but depends on 12/15. 14/15
is independent from all the other patches and can safely be merged I
think.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 09/15] media: bcm2835-unicam: Add support for CCP2/CSI2 camera interface

[Laurent Pinchart]

On Tue, Mar 05, 2024 at 02:15:51PM +0000, Dave Stevenson wrote:
> Hi Laurent and Jacopo.
> 
> I'd started this reply before Laurent's follow up - I'll respond to
> that in a moment.
> 
> On Mon, 4 Mar 2024 at 17:12, Jacopo Mondi <jacopo.mondi@ideasonboard.com> wrote:
> >
> > Hi Laurent
> >
> > On Fri, Mar 01, 2024 at 11:32:24PM +0200, Laurent Pinchart wrote:
> > > From: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > >
> > > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > > It is represented as two video device nodes: unicam-image and
> > > unicam-embedded which are connected to an internal subdev (named
> > > unicam-subdev) in order to manage streams routing.
> > >
> > > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > Co-developed-by: Naushir Patuck <naush@raspberrypi.com>
> > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > > Co-developed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > > Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > > Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > ---
> > > Changes since v5:
> > >
> > > - Move to drivers/media/platform/broadcom/
> > > - Port to the upstream V4L2 streams API
> > > - Rebase on latest metadata API proposal
> > > - Add missing error message
> > > - Drop unneeded documentation block for unicam_isr()
> > > - Drop unneeded dev_dbg() and dev_err() messages
> > > - Drop unneeded streams_mask and fmt checks
> > > - Drop unused unicam_sd_pad_is_sink()
> > > - Drop unneeded includes
> > > - Drop v4l2_ctrl_subscribe_event() call
> > > - Use pm_runtime_resume_and_get()
> > > - Indentation and line wrap fixes
> > > - Let the framework set bus_info
> > > - Use v4l2_fwnode_endpoint_parse()
> > > - Fix media device cleanup
> > > - Drop lane reordering checks
> > > - Fix subdev state locking
> > > - Drop extra debug messages
> > > - Move clock handling to runtime PM handlers
> > > - Reorder functions
> > > - Rename init functions for more clarity
> > > - Initialize runtime PM earlier
> > > - Clarify error messages
> > > - Simplify subdev init with local variable
> > > - Fix subdev cleanup
> > > - Fix typos and indentation
> > > - Don't initialize local variables needlessly
> > > - Simplify num lanes check
> > > - Fix metadata handling in subdev set_fmt
> > > - Drop manual fallback to .s_stream()
> > > - Pass v4l2_pix_format to unicam_calc_format_size_bpl()
> > > - Simplify unicam_set_default_format()
> > > - Fix default format settings
> > > - Add busy check in unicam_s_fmt_meta()
> > > - Add missing \n at end of format strings
> > > - Fix metadata handling in subdev set_fmt
> > > - Fix locking when starting streaming
> > > - Return buffers from start streaming fails
> > > - Fix format validation for metadata node
> > > - Use video_device_pipeline_{start,stop}() helpers
> > > - Simplify format enumeration
> > > - Drop unset variable
> > > - Update MAINTAINERS entry
> > > - Update to the upstream v4l2_async_nf API
> > > - Update to the latest subdev routing API
> > > - Update to the latest subdev state API
> > > - Move from subdev .init_cfg() to .init_state()
> > > - Update to the latest videobuf2 API
> > > - Fix v4l2_subdev_enable_streams() error check
> > > - Use correct pad for the connected subdev
> > > - Return buffers to vb2 when start streaming fails
> > > - Improve debugging in start streaming handler
> > > - Simplify DMA address management
> > > - Drop comment about bcm2835-camera driver
> > > - Clarify comments that explain min/max sizes
> > > - Pass v4l2_pix_format to unicam_try_fmt()
> > > - Drop unneeded local variables
> > > - Rename image-related constants and functions
> > > - Turn unicam_fmt.metadata_fmt into bool
> > > - Rename unicam_fmt to unicam_format_info
> > > - Rename unicam_format_info variables to fmtinfo
> > > - Rename unicam_node.v_fmt to fmt
> > > - Add metadata formats for RAW10, RAW12 and RAW14
> > > - Make metadata formats line-based
> > > - Validate format on metadata video device
> > > - Add Co-devlopped-by tags
> > >
> > > Changes since v3:
> > >
> > > - Add the vendor prefix for DT name
> > > - Use the reg-names in DT parsing
> > > - Remove MAINTAINERS entry
> > >
> > > Changes since v2:
> > >
> > > - Change code organization
> > > - Remove unused variables
> > > - Correct the fmt_meta functions
> > > - Rewrite the start/stop streaming
> > >   - You can now start the image node alone, but not the metadata one
> > >   - The buffers are allocated per-node
> > >   - only the required stream is started, if the route exists and is
> > >     enabled
> > > - Prefix the macros with UNICAM_ to not have too generic names
> > > - Drop colorspace support
> > >
> > > Changes since v1:
> > >
> > > - Replace the unicam_{info,debug,error} macros with dev_*()
> > > ---
> > >  MAINTAINERS                                   |    1 +
> > >  drivers/media/platform/Kconfig                |    1 +
> > >  drivers/media/platform/Makefile               |    1 +
> > >  drivers/media/platform/broadcom/Kconfig       |   23 +
> > >  drivers/media/platform/broadcom/Makefile      |    3 +
> > >  .../platform/broadcom/bcm2835-unicam-regs.h   |  255 ++
> > >  .../media/platform/broadcom/bcm2835-unicam.c  | 2607 +++++++++++++++++
> > >  7 files changed, 2891 insertions(+)
> > >  create mode 100644 drivers/media/platform/broadcom/Kconfig
> > >  create mode 100644 drivers/media/platform/broadcom/Makefile
> > >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam.c
> > >
> > > diff --git a/MAINTAINERS b/MAINTAINERS
> > > index e50a59654e6e..cc350729f467 100644
> > > --- a/MAINTAINERS
> > > +++ b/MAINTAINERS
> > > @@ -4002,6 +4002,7 @@ M:      Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
> > >  L:   linux-media@vger.kernel.org
> > >  S:   Maintained
> > >  F:   Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> > > +F:   drivers/media/platform/bcm2835/
> > >
> > >  BROADCOM BCM47XX MIPS ARCHITECTURE
> > >  M:   Hauke Mehrtens <hauke@hauke-m.de>
> > > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > > index 91e54215de3a..2d79bfc68c15 100644
> > > --- a/drivers/media/platform/Kconfig
> > > +++ b/drivers/media/platform/Kconfig
> > > @@ -67,6 +67,7 @@ source "drivers/media/platform/amlogic/Kconfig"
> > >  source "drivers/media/platform/amphion/Kconfig"
> > >  source "drivers/media/platform/aspeed/Kconfig"
> > >  source "drivers/media/platform/atmel/Kconfig"
> > > +source "drivers/media/platform/broadcom/Kconfig"
> > >  source "drivers/media/platform/cadence/Kconfig"
> > >  source "drivers/media/platform/chips-media/Kconfig"
> > >  source "drivers/media/platform/intel/Kconfig"
> > > diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> > > index 3296ec1ebe16..da17301f7439 100644
> > > --- a/drivers/media/platform/Makefile
> > > +++ b/drivers/media/platform/Makefile
> > > @@ -10,6 +10,7 @@ obj-y += amlogic/
> > >  obj-y += amphion/
> > >  obj-y += aspeed/
> > >  obj-y += atmel/
> > > +obj-y += broadcom/
> > >  obj-y += cadence/
> > >  obj-y += chips-media/
> > >  obj-y += intel/
> > > diff --git a/drivers/media/platform/broadcom/Kconfig b/drivers/media/platform/broadcom/Kconfig
> > > new file mode 100644
> > > index 000000000000..cc2c9afcc948
> > > --- /dev/null
> > > +++ b/drivers/media/platform/broadcom/Kconfig
> > > @@ -0,0 +1,23 @@
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +
> > > +config VIDEO_BCM2835_UNICAM
> > > +     tristate "Broadcom BCM283x/BCM271x Unicam video capture driver"
> > > +     depends on ARCH_BCM2835 || COMPILE_TEST
> > > +     depends on PM
> > > +     depends on VIDEO_DEV
> > > +     select MEDIA_CONTROLLER
> > > +     select V4L2_FWNODE
> > > +     select VIDEO_V4L2_SUBDEV_API
> > > +     select VIDEOBUF2_DMA_CONTIG
> > > +     help
> > > +       Say Y here to enable support for the BCM283x/BCM271x CSI-2 receiver.
> > > +       This is a V4L2 driver that controls the CSI-2 receiver directly,
> > > +       independently from the VC4 firmware.
> > > +
> > > +       This driver is mutually exclusive with the use of bcm2835-camera. The
> > > +       firmware will disable all access to the peripheral from within the
> > > +       firmware if it finds a DT node using it, and bcm2835-camera will
> > > +       therefore fail to probe.
> > > +
> > > +       To compile this driver as a module, choose M here. The module will be
> > > +       called bcm2835-unicam.
> > > diff --git a/drivers/media/platform/broadcom/Makefile b/drivers/media/platform/broadcom/Makefile
> > > new file mode 100644
> > > index 000000000000..03d2045aba2e
> > > --- /dev/null
> > > +++ b/drivers/media/platform/broadcom/Makefile
> > > @@ -0,0 +1,3 @@
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +
> > > +obj-$(CONFIG_VIDEO_BCM2835_UNICAM) += bcm2835-unicam.o
> > > diff --git a/drivers/media/platform/broadcom/bcm2835-unicam-regs.h b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > > new file mode 100644
> > > index 000000000000..84775fd2fac5
> > > --- /dev/null
> > > +++ b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > > @@ -0,0 +1,255 @@
> > > +/* SPDX-License-Identifier: GPL-2.0-only */
> > > +
> > > +/*
> > > + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> > > + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > + */
> > > +
> > > +#ifndef VC4_REGS_UNICAM_H
> > > +#define VC4_REGS_UNICAM_H
> > > +
> > > +#include <linux/bits.h>
> > > +
> > > +/*
> > > + * The following values are taken from files found within the code drop
> > > + * made by Broadcom for the BCM21553 Graphics Driver, predominantly in
> > > + * brcm_usrlib/dag/vmcsx/vcinclude/hardware_vc4.h.
> > > + * They have been modified to be only the register offset.
> > > + */
> > > +#define UNICAM_CTRL          0x000
> > > +#define UNICAM_STA           0x004
> > > +#define UNICAM_ANA           0x008
> > > +#define UNICAM_PRI           0x00c
> > > +#define UNICAM_CLK           0x010
> > > +#define UNICAM_CLT           0x014
> > > +#define UNICAM_DAT0          0x018
> > > +#define UNICAM_DAT1          0x01c
> > > +#define UNICAM_DAT2          0x020
> > > +#define UNICAM_DAT3          0x024
> > > +#define UNICAM_DLT           0x028
> > > +#define UNICAM_CMP0          0x02c
> > > +#define UNICAM_CMP1          0x030
> > > +#define UNICAM_CAP0          0x034
> > > +#define UNICAM_CAP1          0x038
> > > +#define UNICAM_ICTL          0x100
> > > +#define UNICAM_ISTA          0x104
> > > +#define UNICAM_IDI0          0x108
> > > +#define UNICAM_IPIPE         0x10c
> > > +#define UNICAM_IBSA0         0x110
> > > +#define UNICAM_IBEA0         0x114
> > > +#define UNICAM_IBLS          0x118
> > > +#define UNICAM_IBWP          0x11c
> > > +#define UNICAM_IHWIN         0x120
> > > +#define UNICAM_IHSTA         0x124
> > > +#define UNICAM_IVWIN         0x128
> > > +#define UNICAM_IVSTA         0x12c
> > > +#define UNICAM_ICC           0x130
> > > +#define UNICAM_ICS           0x134
> > > +#define UNICAM_IDC           0x138
> > > +#define UNICAM_IDPO          0x13c
> > > +#define UNICAM_IDCA          0x140
> > > +#define UNICAM_IDCD          0x144
> > > +#define UNICAM_IDS           0x148
> > > +#define UNICAM_DCS           0x200
> > > +#define UNICAM_DBSA0         0x204
> > > +#define UNICAM_DBEA0         0x208
> > > +#define UNICAM_DBWP          0x20c
> > > +#define UNICAM_DBCTL         0x300
> > > +#define UNICAM_IBSA1         0x304
> > > +#define UNICAM_IBEA1         0x308
> > > +#define UNICAM_IDI1          0x30c
> > > +#define UNICAM_DBSA1         0x310
> > > +#define UNICAM_DBEA1         0x314
> > > +#define UNICAM_MISC          0x400
> > > +
> > > +/*
> > > + * The following bitmasks are from the kernel released by Broadcom
> > > + * for Android - https://android.googlesource.com/kernel/bcm/
> > > + * The Rhea, Hawaii, and Java chips all contain the same VideoCore4
> > > + * Unicam block as BCM2835, as defined in eg
> > > + * arch/arm/mach-rhea/include/mach/rdb_A0/brcm_rdb_cam.h and similar.
> > > + * Values reworked to use the kernel BIT and GENMASK macros.
> > > + *
> > > + * Some of the bit mnenomics have been amended to match the datasheet.
> > > + */
> > > +/* UNICAM_CTRL Register */
> > > +#define UNICAM_CPE           BIT(0)
> > > +#define UNICAM_MEM           BIT(1)
> > > +#define UNICAM_CPR           BIT(2)
> > > +#define UNICAM_CPM_MASK              GENMASK(3, 3)
> > > +#define UNICAM_CPM_CSI2              0
> > > +#define UNICAM_CPM_CCP2              1
> > > +#define UNICAM_SOE           BIT(4)
> > > +#define UNICAM_DCM_MASK              GENMASK(5, 5)
> > > +#define UNICAM_DCM_STROBE    0
> > > +#define UNICAM_DCM_DATA              1
> > > +#define UNICAM_SLS           BIT(6)
> > > +#define UNICAM_PFT_MASK              GENMASK(11, 8)
> > > +#define UNICAM_OET_MASK              GENMASK(20, 12)
> > > +
> > > +/* UNICAM_STA Register */
> > > +#define UNICAM_SYN           BIT(0)
> > > +#define UNICAM_CS            BIT(1)
> > > +#define UNICAM_SBE           BIT(2)
> > > +#define UNICAM_PBE           BIT(3)
> > > +#define UNICAM_HOE           BIT(4)
> > > +#define UNICAM_PLE           BIT(5)
> > > +#define UNICAM_SSC           BIT(6)
> > > +#define UNICAM_CRCE          BIT(7)
> > > +#define UNICAM_OES           BIT(8)
> > > +#define UNICAM_IFO           BIT(9)
> > > +#define UNICAM_OFO           BIT(10)
> > > +#define UNICAM_BFO           BIT(11)
> > > +#define UNICAM_DL            BIT(12)
> > > +#define UNICAM_PS            BIT(13)
> > > +#define UNICAM_IS            BIT(14)
> > > +#define UNICAM_PI0           BIT(15)
> > > +#define UNICAM_PI1           BIT(16)
> > > +#define UNICAM_FSI_S         BIT(17)
> > > +#define UNICAM_FEI_S         BIT(18)
> > > +#define UNICAM_LCI_S         BIT(19)
> > > +#define UNICAM_BUF0_RDY              BIT(20)
> > > +#define UNICAM_BUF0_NO               BIT(21)
> > > +#define UNICAM_BUF1_RDY              BIT(22)
> > > +#define UNICAM_BUF1_NO               BIT(23)
> > > +#define UNICAM_DI            BIT(24)
> > > +
> > > +#define UNICAM_STA_MASK_ALL \
> > > +             (UNICAM_DL | \
> > > +             UNICAM_SBE | \
> > > +             UNICAM_PBE | \
> > > +             UNICAM_HOE | \
> > > +             UNICAM_PLE | \
> > > +             UNICAM_SSC | \
> > > +             UNICAM_CRCE | \
> > > +             UNICAM_IFO | \
> > > +             UNICAM_OFO | \
> > > +             UNICAM_PS | \
> > > +             UNICAM_PI0 | \
> > > +             UNICAM_PI1)
> > > +
> > > +/* UNICAM_ANA Register */
> > > +#define UNICAM_APD           BIT(0)
> > > +#define UNICAM_BPD           BIT(1)
> > > +#define UNICAM_AR            BIT(2)
> > > +#define UNICAM_DDL           BIT(3)
> > > +#define UNICAM_CTATADJ_MASK  GENMASK(7, 4)
> > > +#define UNICAM_PTATADJ_MASK  GENMASK(11, 8)
> > > +
> > > +/* UNICAM_PRI Register */
> > > +#define UNICAM_PE            BIT(0)
> > > +#define UNICAM_PT_MASK               GENMASK(2, 1)
> > > +#define UNICAM_NP_MASK               GENMASK(7, 4)
> > > +#define UNICAM_PP_MASK               GENMASK(11, 8)
> > > +#define UNICAM_BS_MASK               GENMASK(15, 12)
> > > +#define UNICAM_BL_MASK               GENMASK(17, 16)
> > > +
> > > +/* UNICAM_CLK Register */
> > > +#define UNICAM_CLE           BIT(0)
> > > +#define UNICAM_CLPD          BIT(1)
> > > +#define UNICAM_CLLPE         BIT(2)
> > > +#define UNICAM_CLHSE         BIT(3)
> > > +#define UNICAM_CLTRE         BIT(4)
> > > +#define UNICAM_CLAC_MASK     GENMASK(8, 5)
> > > +#define UNICAM_CLSTE         BIT(29)
> > > +
> > > +/* UNICAM_CLT Register */
> > > +#define UNICAM_CLT1_MASK     GENMASK(7, 0)
> > > +#define UNICAM_CLT2_MASK     GENMASK(15, 8)
> > > +
> > > +/* UNICAM_DATn Registers */
> > > +#define UNICAM_DLE           BIT(0)
> > > +#define UNICAM_DLPD          BIT(1)
> > > +#define UNICAM_DLLPE         BIT(2)
> > > +#define UNICAM_DLHSE         BIT(3)
> > > +#define UNICAM_DLTRE         BIT(4)
> > > +#define UNICAM_DLSM          BIT(5)
> > > +#define UNICAM_DLFO          BIT(28)
> > > +#define UNICAM_DLSTE         BIT(29)
> > > +
> > > +#define UNICAM_DAT_MASK_ALL  (UNICAM_DLSTE | UNICAM_DLFO)
> > > +
> > > +/* UNICAM_DLT Register */
> > > +#define UNICAM_DLT1_MASK     GENMASK(7, 0)
> > > +#define UNICAM_DLT2_MASK     GENMASK(15, 8)
> > > +#define UNICAM_DLT3_MASK     GENMASK(23, 16)
> > > +
> > > +/* UNICAM_ICTL Register */
> > > +#define UNICAM_FSIE          BIT(0)
> > > +#define UNICAM_FEIE          BIT(1)
> > > +#define UNICAM_IBOB          BIT(2)
> > > +#define UNICAM_FCM           BIT(3)
> > > +#define UNICAM_TFC           BIT(4)
> > > +#define UNICAM_LIP_MASK              GENMASK(6, 5)
> > > +#define UNICAM_LCIE_MASK     GENMASK(28, 16)
> > > +
> > > +/* UNICAM_IDI0/1 Register */
> > > +#define UNICAM_ID0_MASK              GENMASK(7, 0)
> > > +#define UNICAM_ID1_MASK              GENMASK(15, 8)
> > > +#define UNICAM_ID2_MASK              GENMASK(23, 16)
> > > +#define UNICAM_ID3_MASK              GENMASK(31, 24)
> > > +
> > > +/* UNICAM_ISTA Register */
> > > +#define UNICAM_FSI           BIT(0)
> > > +#define UNICAM_FEI           BIT(1)
> > > +#define UNICAM_LCI           BIT(2)
> > > +
> > > +#define UNICAM_ISTA_MASK_ALL (UNICAM_FSI | UNICAM_FEI | UNICAM_LCI)
> > > +
> > > +/* UNICAM_IPIPE Register */
> > > +#define UNICAM_PUM_MASK              GENMASK(2, 0)
> > > +/* Unpacking modes */
> > > +#define UNICAM_PUM_NONE              0
> > > +#define UNICAM_PUM_UNPACK6   1
> > > +#define UNICAM_PUM_UNPACK7   2
> > > +#define UNICAM_PUM_UNPACK8   3
> > > +#define UNICAM_PUM_UNPACK10  4
> > > +#define UNICAM_PUM_UNPACK12  5
> > > +#define UNICAM_PUM_UNPACK14  6
> > > +#define UNICAM_PUM_UNPACK16  7
> > > +#define UNICAM_DDM_MASK              GENMASK(6, 3)
> > > +#define UNICAM_PPM_MASK              GENMASK(9, 7)
> > > +/* Packing modes */
> > > +#define UNICAM_PPM_NONE              0
> > > +#define UNICAM_PPM_PACK8     1
> > > +#define UNICAM_PPM_PACK10    2
> > > +#define UNICAM_PPM_PACK12    3
> > > +#define UNICAM_PPM_PACK14    4
> > > +#define UNICAM_PPM_PACK16    5
> > > +#define UNICAM_DEM_MASK              GENMASK(11, 10)
> > > +#define UNICAM_DEBL_MASK     GENMASK(14, 12)
> > > +#define UNICAM_ICM_MASK              GENMASK(16, 15)
> > > +#define UNICAM_IDM_MASK              GENMASK(17, 17)
> > > +
> > > +/* UNICAM_ICC Register */
> > > +#define UNICAM_ICFL_MASK     GENMASK(4, 0)
> > > +#define UNICAM_ICFH_MASK     GENMASK(9, 5)
> > > +#define UNICAM_ICST_MASK     GENMASK(12, 10)
> > > +#define UNICAM_ICLT_MASK     GENMASK(15, 13)
> > > +#define UNICAM_ICLL_MASK     GENMASK(31, 16)
> > > +
> > > +/* UNICAM_DCS Register */
> > > +#define UNICAM_DIE           BIT(0)
> > > +#define UNICAM_DIM           BIT(1)
> > > +#define UNICAM_DBOB          BIT(3)
> > > +#define UNICAM_FDE           BIT(4)
> > > +#define UNICAM_LDP           BIT(5)
> > > +#define UNICAM_EDL_MASK              GENMASK(15, 8)
> > > +
> > > +/* UNICAM_DBCTL Register */
> > > +#define UNICAM_DBEN          BIT(0)
> > > +#define UNICAM_BUF0_IE               BIT(1)
> > > +#define UNICAM_BUF1_IE               BIT(2)
> > > +
> > > +/* UNICAM_CMP[0,1] register */
> > > +#define UNICAM_PCE           BIT(31)
> > > +#define UNICAM_GI            BIT(9)
> > > +#define UNICAM_CPH           BIT(8)
> > > +#define UNICAM_PCVC_MASK     GENMASK(7, 6)
> > > +#define UNICAM_PCDT_MASK     GENMASK(5, 0)
> > > +
> > > +/* UNICAM_MISC register */
> > > +#define UNICAM_FL0           BIT(6)
> > > +#define UNICAM_FL1           BIT(9)
> > > +
> > > +#endif
> > > diff --git a/drivers/media/platform/broadcom/bcm2835-unicam.c b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > > new file mode 100644
> > > index 000000000000..716c89b8a217
> > > --- /dev/null
> > > +++ b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > > @@ -0,0 +1,2607 @@
> > > +// SPDX-License-Identifier: GPL-2.0-only
> > > +/*
> > > + * BCM283x / BCM271x Unicam Capture Driver
> > > + *
> > > + * Copyright (C) 2017-2020 - Raspberry Pi (Trading) Ltd.
> > > + *
> > > + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > + *
> > > + * Based on TI am437x driver by
> > > + *   Benoit Parrot <bparrot@ti.com>
> > > + *   Lad, Prabhakar <prabhakar.csengg@gmail.com>
> > > + *
> > > + * and TI CAL camera interface driver by
> > > + *    Benoit Parrot <bparrot@ti.com>
> > > + *
> > > + *
> > > + * There are two camera drivers in the kernel for BCM283x - this one and
> > > + * bcm2835-camera (currently in staging).
> > > + *
> > > + * This driver directly controls the Unicam peripheral - there is no
> > > + * involvement with the VideoCore firmware. Unicam receives CSI-2 or CCP2 data
> > > + * and writes it into SDRAM. The only potential processing options are to
> > > + * repack Bayer data into an alternate format, and applying windowing. The
> > > + * repacking does not shift the data, so can repack V4L2_PIX_FMT_Sxxxx10P to
> > > + * V4L2_PIX_FMT_Sxxxx10, or V4L2_PIX_FMT_Sxxxx12P to V4L2_PIX_FMT_Sxxxx12, but
> > > + * not generically up to V4L2_PIX_FMT_Sxxxx16. The driver will add both formats
> > > + * where the relevant formats are defined, and will automatically configure the
> > > + * repacking as required. Support for windowing may be added later.
> >
> > Does this last paragraph applies ? I see all the supported packed raw
> > format having an 'unpacked_fourcc' associated (all but the 8 bit ones
> > ofc)
> 
> The penultimate sentence ("The driver will add both formats where the
> relevant formats are defined") can be dropped as you've dropped the
> video-node centric mode of operation, therefore all formats (whether
> packed or unpacked) are always advertised.
> 
> The rest of the paragraph still applies - if you request a
> V4L2_PIX_FMT_SxxxxBB (not packed) format, then it the incoming data
> will be unpacked for you.
> 
> > > + *
> > > + * It should be possible to connect this driver to any sensor with a suitable
> > > + * output interface and V4L2 subdevice driver.
> > > + */
> > > +
> > > +#include <linux/clk.h>
> > > +#include <linux/delay.h>
> > > +#include <linux/device.h>
> > > +#include <linux/dma-mapping.h>
> > > +#include <linux/err.h>
> > > +#include <linux/interrupt.h>
> > > +#include <linux/io.h>
> > > +#include <linux/module.h>
> > > +#include <linux/of.h>
> > > +#include <linux/of_device.h>
> > > +#include <linux/platform_device.h>
> > > +#include <linux/pm_runtime.h>
> > > +#include <linux/slab.h>
> > > +#include <linux/videodev2.h>
> > > +
> > > +#include <media/v4l2-async.h>
> > > +#include <media/v4l2-common.h>
> > > +#include <media/v4l2-dev.h>
> > > +#include <media/v4l2-device.h>
> > > +#include <media/v4l2-event.h>
> > > +#include <media/v4l2-ioctl.h>
> > > +#include <media/v4l2-fwnode.h>
> > > +#include <media/v4l2-mc.h>
> > > +#include <media/videobuf2-dma-contig.h>
> > > +
> > > +#include "bcm2835-unicam-regs.h"
> > > +
> > > +#define UNICAM_MODULE_NAME           "unicam"
> > > +
> > > +/*
> > > + * Unicam must request a minimum of 250Mhz from the VPU clock.
> > > + * Otherwise the input FIFOs overrun and cause image corruption.
> > > + */
> > > +#define UNICAM_MIN_VPU_CLOCK_RATE    (250 * 1000 * 1000)
> >
> > Shouldn't this be set in DT ?
> >
> > > +
> > > +/* Unicam has an internal DMA alignment constraint of 16 bytes for each line. */
> > > +#define UNICAM_DMA_BPL_ALIGNMENT     16
> > > +
> > > +/*
> > > + * The image stride is stored in a 16 bit register, and needs to be aligned to
> > > + * the DMA constraint. As the ISP in the same SoC has a 32 bytes alignment
> > > + * constraint on its input, set the image stride alignment to 32 bytes here as
> > > + * well to avoid incompatible configurations.
> > > + */
> > > +#define UNICAM_IMAGE_BPL_ALIGNMENT   32
> > > +#define UNICAM_IMAGE_MAX_BPL         ((1 << 16) - UNICAM_IMAGE_BPL_ALIGNMENT)
> > > +
> > > +/*
> > > + * Max width is therefore determined by the max stride divided by the number of
> > > + * bits per pixel. Take 32bpp as a worst case. No imposed limit on the height,
> > > + * so adopt a square image for want of anything better.
> > > + */
> > > +#define UNICAM_IMAGE_MIN_WIDTH               16
> > > +#define UNICAM_IMAGE_MIN_HEIGHT              16
> > > +#define UNICAM_IMAGE_MAX_WIDTH               (UNICAM_IMAGE_MAX_BPL / 4)
> > > +#define UNICAM_IMAGE_MAX_HEIGHT              UNICAM_IMAGE_MAX_WIDTH
> > > +
> > > +/*
> > > + * There's no intrinsic limits on the width and height for embedded dat. Use
> > > + * the same maximum values as for the image, to avoid overflows in the image
> > > + * size computation.
> > > + */
> > > +#define UNICAM_META_MIN_WIDTH                1
> > > +#define UNICAM_META_MIN_HEIGHT               1
> > > +#define UNICAM_META_MAX_WIDTH                UNICAM_IMAGE_MAX_WIDTH
> > > +#define UNICAM_META_MAX_HEIGHT               UNICAM_IMAGE_MAX_HEIGHT
> > > +
> > > +/*
> > > + * Size of the dummy buffer. Can be any size really, but the DMA
> > > + * allocation works in units of page sizes.
> > > + */
> > > +#define UNICAM_DUMMY_BUF_SIZE                PAGE_SIZE
> > > +
> > > +#define UNICAM_SD_PAD_SINK           0
> > > +#define UNICAM_SD_PAD_SOURCE_IMAGE   1
> > > +#define UNICAM_SD_PAD_SOURCE_METADATA        2
> > > +#define UNICAM_SD_NUM_PADS           (1 + UNICAM_SD_PAD_SOURCE_METADATA)
> >
> > How about an enum then ?
> >
> > > +
> > > +enum unicam_node_type {
> > > +     UNICAM_IMAGE_NODE,
> > > +     UNICAM_METADATA_NODE,
> > > +     UNICAM_MAX_NODES
> > > +};
> > > +
> > > +/*
> > > + * struct unicam_format_info - Unicam media bus format information
> > > + * @fourcc: V4L2 pixel format FCC identifier. 0 if n/a.
> > > + * @unpacked_fourcc: V4L2 pixel format FCC identifier if the data is expanded
> > > + * out to 16bpp. 0 if n/a.
> > > + * @code: V4L2 media bus format code.
> > > + * @depth: Bits per pixel as delivered from the source.
> > > + * @csi_dt: CSI data type.
> > > + * @metadata_fmt: This format only applies to the metadata pad.
> > > + */
> > > +struct unicam_format_info {
> > > +     u32     fourcc;
> > > +     u32     unpacked_fourcc;
> > > +     u32     code;
> > > +     u8      depth;
> > > +     u8      csi_dt;
> > > +     bool    metadata_fmt;
> > > +};
> > > +
> > > +struct unicam_buffer {
> > > +     struct vb2_v4l2_buffer vb;
> > > +     struct list_head list;
> > > +     dma_addr_t dma_addr;
> > > +     unsigned int size;
> > > +};
> > > +
> > > +static inline struct unicam_buffer *to_unicam_buffer(struct vb2_buffer *vb)
> > > +{
> > > +     return container_of(vb, struct unicam_buffer, vb.vb2_buf);
> > > +}
> > > +
> > > +struct unicam_node {
> > > +     bool registered;
> > > +     bool streaming;
> > > +     unsigned int id;
> > > +
> > > +     /* Pointer to the current v4l2_buffer */
> > > +     struct unicam_buffer *cur_frm;
> > > +     /* Pointer to the next v4l2_buffer */
> > > +     struct unicam_buffer *next_frm;
> > > +     /* video capture */
> > > +     const struct unicam_format_info *fmtinfo;
> > > +     /* Used to store current pixel format */
> > > +     struct v4l2_format fmt;
> > > +     /* Buffer queue used in video-buf */
> > > +     struct vb2_queue buffer_queue;
> > > +     /* Queue of filled frames */
> > > +     struct list_head dma_queue;
> > > +     /* IRQ lock for DMA queue */
> > > +     spinlock_t dma_queue_lock;
> > > +     /* lock used to access this structure */
> > > +     struct mutex lock;
> > > +     /* Identifies video device for this channel */
> > > +     struct video_device video_dev;
> > > +     /* Pointer to the parent handle */
> > > +     struct unicam_device *dev;
> > > +     struct media_pad pad;
> > > +     /*
> > > +      * Dummy buffer intended to be used by unicam
> > > +      * if we have no other queued buffers to swap to.
> > > +      */
> > > +     struct unicam_buffer dummy_buf;
> > > +     void *dummy_buf_cpu_addr;
> > > +};
> > > +
> > > +struct unicam_device {
> > > +     struct kref kref;
> > > +
> > > +     /* peripheral base address */
> > > +     void __iomem *base;
> > > +     /* clock gating base address */
> > > +     void __iomem *clk_gate_base;
> >
> > Is the clock gating part of unicam or should this be expressed as a
> > clock provide and be referenced in DT through a phandle ?
> 
> Covered in previous reviews and accepted as part of unicam.
> https://patchwork.kernel.org/project/linux-media/patch/fae3d29bba67825030c0077dd9c79534b6888512.1505916622.git.dave.stevenson@raspberrypi.org/
> 
> > > +     /* lp clock handle */
> > > +     struct clk *clock;
> > > +     /* vpu clock handle */
> > > +     struct clk *vpu_clock;
> > > +     /* V4l2 device */
> > > +     struct v4l2_device v4l2_dev;
> > > +     struct media_device mdev;
> > > +
> > > +     /* parent device */
> > > +     struct device *dev;
> > > +     /* subdevice async Notifier */
> >
> > s/Notifier/notifier
> >
> > > +     struct v4l2_async_notifier notifier;
> > > +     unsigned int sequence;
> > > +
> > > +     /* Sensor node */
> > > +     struct {
> > > +             struct v4l2_subdev *subdev;
> > > +             struct media_pad *pad;
> > > +     } sensor;
> > > +
> > > +     /* Internal subdev */
> > > +     struct {
> > > +             struct v4l2_subdev sd;
> > > +             struct media_pad pads[UNICAM_SD_NUM_PADS];
> > > +             bool streaming;
> > > +     } subdev;
> > > +
> > > +     enum v4l2_mbus_type bus_type;
> > > +     /*
> > > +      * Stores bus.mipi_csi2.flags for CSI2 sensors, or
> > > +      * bus.mipi_csi1.strobe for CCP2.
> > > +      */
> > > +     unsigned int bus_flags;
> > > +     unsigned int max_data_lanes;
> > > +     unsigned int active_data_lanes;
> > > +
> > > +     struct media_pipeline pipe;
> > > +
> > > +     struct unicam_node node[UNICAM_MAX_NODES];
> > > +};
> > > +
> > > +static inline struct unicam_device *
> > > +notifier_to_unicam_device(struct v4l2_async_notifier *notifier)
> > > +{
> > > +     return container_of(notifier, struct unicam_device, notifier);
> > > +}
> > > +
> > > +static inline struct unicam_device *
> >
> > I thought using inline is disouraged as the compiler knows what to do
> >
> > > +sd_to_unicam_device(struct v4l2_subdev *sd)
> > > +{
> > > +     return container_of(sd, struct unicam_device, subdev.sd);
> >
> > You can also use container_of_const() if you need it
> >
> > > +}
> > > +
> > > +static void unicam_release(struct kref *kref)
> > > +{
> > > +     struct unicam_device *unicam =
> > > +             container_of(kref, struct unicam_device, kref);
> > > +
> > > +     if (unicam->mdev.dev)
> > > +             media_device_cleanup(&unicam->mdev);
> > > +
> > > +     kfree(unicam);
> > > +}
> > > +
> > > +static struct unicam_device *unicam_get(struct unicam_device *unicam)
> > > +{
> > > +     kref_get(&unicam->kref);
> > > +     return unicam;
> > > +}
> > > +
> > > +static void unicam_put(struct unicam_device *unicam)
> > > +{
> > > +     kref_put(&unicam->kref, unicam_release);
> > > +}
> > > +
> >
> > Does this handle a bit more gracefully the pesky lifetime management
> > issue in v4l2 ?
> >
> > > +/* -----------------------------------------------------------------------------
> > > + * Misc helper functions
> > > + */
> > > +
> > > +static inline bool unicam_sd_pad_is_source(u32 pad)
> > > +{
> > > +     /* Camera RX has 1 sink pad, and N source pads */
> > > +     return pad != UNICAM_SD_PAD_SINK;
> > > +}
> > > +
> > > +static inline bool is_metadata_node(struct unicam_node *node)
> > > +{
> > > +     return node->video_dev.device_caps & V4L2_CAP_META_CAPTURE;
> > > +}
> > > +
> > > +static inline bool is_image_node(struct unicam_node *node)
> > > +{
> > > +     return node->video_dev.device_caps & V4L2_CAP_VIDEO_CAPTURE;
> > > +}
> > > +
> > > +/* -----------------------------------------------------------------------------
> > > + * Format data table and helper functions
> > > + */
> > > +
> > > +static const struct v4l2_mbus_framefmt unicam_default_image_format = {
> > > +     .width = 640,
> > > +     .height = 480,
> > > +     .code = MEDIA_BUS_FMT_UYVY8_1X16,
> > > +     .field = V4L2_FIELD_NONE,
> > > +     .colorspace = V4L2_COLORSPACE_SRGB,
> > > +     .ycbcr_enc = V4L2_YCBCR_ENC_601,
> > > +     .quantization = V4L2_QUANTIZATION_LIM_RANGE,
> > > +     .xfer_func = V4L2_XFER_FUNC_SRGB,
> > > +     .flags = 0,
> > > +};
> > > +
> > > +static const struct v4l2_mbus_framefmt unicam_default_meta_format = {
> > > +     .width = 640,
> > > +     .height = 2,
> > > +     .code = MEDIA_BUS_FMT_META_8,
> > > +     .field = V4L2_FIELD_NONE,
> > > +};
> > > +
> > > +static const struct unicam_format_info unicam_image_formats[] = {
> > > +     /* YUV Formats */
> > > +     {
> > > +             .fourcc         = V4L2_PIX_FMT_YUYV,
> > > +             .code           = MEDIA_BUS_FMT_YUYV8_1X16,
> > > +             .depth          = 16,
> > > +             .csi_dt         = 0x1e,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_UYVY,
> > > +             .code           = MEDIA_BUS_FMT_UYVY8_1X16,
> > > +             .depth          = 16,
> > > +             .csi_dt         = 0x1e,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_YVYU,
> > > +             .code           = MEDIA_BUS_FMT_YVYU8_1X16,
> > > +             .depth          = 16,
> > > +             .csi_dt         = 0x1e,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_VYUY,
> > > +             .code           = MEDIA_BUS_FMT_VYUY8_1X16,
> > > +             .depth          = 16,
> > > +             .csi_dt         = 0x1e,
> > > +     }, {
> > > +     /* RGB Formats */
> > > +             .fourcc         = V4L2_PIX_FMT_RGB565, /* gggbbbbb rrrrrggg */
> > > +             .code           = MEDIA_BUS_FMT_RGB565_1X16,
> > > +             .depth          = 16,
> > > +             .csi_dt         = 0x22,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_RGB24, /* rgb */
> > > +             .code           = MEDIA_BUS_FMT_RGB888_1X24,
> > > +             .depth          = 24,
> > > +             .csi_dt         = 0x24,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_BGR24, /* bgr */
> > > +             .code           = MEDIA_BUS_FMT_BGR888_1X24,
> > > +             .depth          = 24,
> > > +             .csi_dt         = 0x24,
> > > +     }, {
> > > +     /* Bayer Formats */
> > > +             .fourcc         = V4L2_PIX_FMT_SBGGR8,
> > > +             .code           = MEDIA_BUS_FMT_SBGGR8_1X8,
> > > +             .depth          = 8,
> > > +             .csi_dt         = 0x2a,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_SGBRG8,
> > > +             .code           = MEDIA_BUS_FMT_SGBRG8_1X8,
> > > +             .depth          = 8,
> > > +             .csi_dt         = 0x2a,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_SGRBG8,
> > > +             .code           = MEDIA_BUS_FMT_SGRBG8_1X8,
> > > +             .depth          = 8,
> > > +             .csi_dt         = 0x2a,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_SRGGB8,
> > > +             .code           = MEDIA_BUS_FMT_SRGGB8_1X8,
> > > +             .depth          = 8,
> > > +             .csi_dt         = 0x2a,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_SBGGR10P,
> > > +             .unpacked_fourcc = V4L2_PIX_FMT_SBGGR10,
> > > +             .code           = MEDIA_BUS_FMT_SBGGR10_1X10,
> > > +             .depth          = 10,
> > > +             .csi_dt         = 0x2b,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_SGBRG10P,
> > > +             .unpacked_fourcc = V4L2_PIX_FMT_SGBRG10,
> > > +             .code           = MEDIA_BUS_FMT_SGBRG10_1X10,
> > > +             .depth          = 10,
> > > +             .csi_dt         = 0x2b,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_SGRBG10P,
> > > +             .unpacked_fourcc = V4L2_PIX_FMT_SGRBG10,
> > > +             .code           = MEDIA_BUS_FMT_SGRBG10_1X10,
> > > +             .depth          = 10,
> > > +             .csi_dt         = 0x2b,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_SRGGB10P,
> > > +             .unpacked_fourcc = V4L2_PIX_FMT_SRGGB10,
> > > +             .code           = MEDIA_BUS_FMT_SRGGB10_1X10,
> > > +             .depth          = 10,
> > > +             .csi_dt         = 0x2b,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_SBGGR12P,
> > > +             .unpacked_fourcc = V4L2_PIX_FMT_SBGGR12,
> > > +             .code           = MEDIA_BUS_FMT_SBGGR12_1X12,
> > > +             .depth          = 12,
> > > +             .csi_dt         = 0x2c,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_SGBRG12P,
> > > +             .unpacked_fourcc = V4L2_PIX_FMT_SGBRG12,
> > > +             .code           = MEDIA_BUS_FMT_SGBRG12_1X12,
> > > +             .depth          = 12,
> > > +             .csi_dt         = 0x2c,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_SGRBG12P,
> > > +             .unpacked_fourcc = V4L2_PIX_FMT_SGRBG12,
> > > +             .code           = MEDIA_BUS_FMT_SGRBG12_1X12,
> > > +             .depth          = 12,
> > > +             .csi_dt         = 0x2c,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_SRGGB12P,
> > > +             .unpacked_fourcc = V4L2_PIX_FMT_SRGGB12,
> > > +             .code           = MEDIA_BUS_FMT_SRGGB12_1X12,
> > > +             .depth          = 12,
> > > +             .csi_dt         = 0x2c,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_SBGGR14P,
> > > +             .unpacked_fourcc = V4L2_PIX_FMT_SBGGR14,
> > > +             .code           = MEDIA_BUS_FMT_SBGGR14_1X14,
> > > +             .depth          = 14,
> > > +             .csi_dt         = 0x2d,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_SGBRG14P,
> > > +             .unpacked_fourcc = V4L2_PIX_FMT_SGBRG14,
> > > +             .code           = MEDIA_BUS_FMT_SGBRG14_1X14,
> > > +             .depth          = 14,
> > > +             .csi_dt         = 0x2d,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_SGRBG14P,
> > > +             .unpacked_fourcc = V4L2_PIX_FMT_SGRBG14,
> > > +             .code           = MEDIA_BUS_FMT_SGRBG14_1X14,
> > > +             .depth          = 14,
> > > +             .csi_dt         = 0x2d,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_SRGGB14P,
> > > +             .unpacked_fourcc = V4L2_PIX_FMT_SRGGB14,
> > > +             .code           = MEDIA_BUS_FMT_SRGGB14_1X14,
> > > +             .depth          = 14,
> > > +             .csi_dt         = 0x2d,
> > > +     }, {
> > > +     /* 16 bit Bayer formats could be supported. */
> > > +
> > > +     /* Greyscale formats */
> > > +             .fourcc         = V4L2_PIX_FMT_GREY,
> > > +             .code           = MEDIA_BUS_FMT_Y8_1X8,
> > > +             .depth          = 8,
> > > +             .csi_dt         = 0x2a,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_Y10P,
> > > +             .unpacked_fourcc = V4L2_PIX_FMT_Y10,
> > > +             .code           = MEDIA_BUS_FMT_Y10_1X10,
> > > +             .depth          = 10,
> > > +             .csi_dt         = 0x2b,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_Y12P,
> > > +             .unpacked_fourcc = V4L2_PIX_FMT_Y12,
> > > +             .code           = MEDIA_BUS_FMT_Y12_1X12,
> > > +             .depth          = 12,
> > > +             .csi_dt         = 0x2c,
> > > +     }, {
> > > +             .fourcc         = V4L2_PIX_FMT_Y14P,
> > > +             .unpacked_fourcc = V4L2_PIX_FMT_Y14,
> > > +             .code           = MEDIA_BUS_FMT_Y14_1X14,
> > > +             .depth          = 14,
> > > +             .csi_dt         = 0x2d,
> > > +     },
> > > +};
> > > +
> > > +static const struct unicam_format_info unicam_meta_formats[] = {
> > > +     {
> > > +             .fourcc         = V4L2_META_FMT_GENERIC_8,
> > > +             .code           = MEDIA_BUS_FMT_META_8,
> > > +             .depth          = 8,
> > > +             .metadata_fmt   = true,
> > > +     }, {
> > > +             .fourcc         = V4L2_META_FMT_GENERIC_CSI2_10,
> > > +             .code           = MEDIA_BUS_FMT_META_10,
> > > +             .depth          = 10,
> > > +             .metadata_fmt   = true,
> > > +     }, {
> > > +             .fourcc         = V4L2_META_FMT_GENERIC_CSI2_12,
> > > +             .code           = MEDIA_BUS_FMT_META_12,
> > > +             .depth          = 12,
> > > +             .metadata_fmt   = true,
> > > +     }, {
> > > +             .fourcc         = V4L2_META_FMT_GENERIC_CSI2_14,
> > > +             .code           = MEDIA_BUS_FMT_META_14,
> > > +             .depth          = 14,
> > > +             .metadata_fmt   = true,
> > > +     },
> > > +};
> > > +
> > > +/* Format setup functions */
> > > +static const struct unicam_format_info *
> > > +unicam_find_format_by_code(u32 code, u32 pad)
> > > +{
> > > +     const struct unicam_format_info *formats;
> > > +     unsigned int num_formats;
> > > +     unsigned int i;
> > > +
> > > +     if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> > > +             formats = unicam_image_formats;
> > > +             num_formats = ARRAY_SIZE(unicam_image_formats);
> > > +     } else {
> > > +             formats = unicam_meta_formats;
> > > +             num_formats = ARRAY_SIZE(unicam_meta_formats);
> > > +     }
> > > +
> > > +     for (i = 0; i < num_formats; i++) {
> > > +             if (formats[i].code == code)
> > > +                     return &formats[i];
> > > +     }
> > > +
> > > +     return NULL;
> > > +}
> > > +
> > > +static const struct unicam_format_info *
> > > +unicam_find_format_by_fourcc(u32 fourcc, u32 pad)
> > > +{
> > > +     const struct unicam_format_info *formats;
> > > +     unsigned int num_formats;
> > > +     unsigned int i;
> > > +
> > > +     if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> > > +             formats = unicam_image_formats;
> > > +             num_formats = ARRAY_SIZE(unicam_image_formats);
> > > +     } else {
> > > +             formats = unicam_meta_formats;
> > > +             num_formats = ARRAY_SIZE(unicam_meta_formats);
> > > +     }
> > > +
> > > +     for (i = 0; i < num_formats; ++i) {
> > > +             if (formats[i].fourcc == fourcc)
> > > +                     return &formats[i];
> > > +     }
> > > +
> > > +     return NULL;
> > > +}
> > > +
> > > +static void unicam_calc_image_size_bpl(struct unicam_device *unicam,
> > > +                                    const struct unicam_format_info *fmtinfo,
> > > +                                    struct v4l2_pix_format *pix)
> > > +{
> > > +     u32 min_bpl;
> > > +
> > > +     v4l_bound_align_image(&pix->width, UNICAM_IMAGE_MIN_WIDTH,
> > > +                           UNICAM_IMAGE_MAX_WIDTH, 2,
> > > +                           &pix->height, UNICAM_IMAGE_MIN_HEIGHT,
> > > +                           UNICAM_IMAGE_MAX_HEIGHT, 0, 0);
> > > +
> > > +     /* Unpacking always goes to 16bpp */
> > > +     if (pix->pixelformat == fmtinfo->unpacked_fourcc)
> > > +             min_bpl = pix->width * 2;
> > > +     else
> > > +             min_bpl = pix->width * fmtinfo->depth / 8;
> > > +     min_bpl = ALIGN(min_bpl, UNICAM_IMAGE_BPL_ALIGNMENT);
> > > +
> > > +     pix->bytesperline = ALIGN(pix->bytesperline, UNICAM_IMAGE_BPL_ALIGNMENT);
> > > +     pix->bytesperline = clamp_t(unsigned int, pix->bytesperline, min_bpl,
> > > +                                 UNICAM_IMAGE_MAX_BPL);
> > > +
> > > +     pix->sizeimage = pix->height * pix->bytesperline;
> > > +}
> > > +
> > > +static void unicam_calc_meta_size_bpl(struct unicam_device *unicam,
> > > +                                   const struct unicam_format_info *fmtinfo,
> > > +                                   struct v4l2_meta_format *meta)
> > > +{
> > > +     v4l_bound_align_image(&meta->width, UNICAM_META_MIN_WIDTH,
> > > +                           UNICAM_META_MAX_WIDTH, 0,
> > > +                           &meta->height, UNICAM_META_MIN_HEIGHT,
> > > +                           UNICAM_META_MAX_HEIGHT, 0, 0);
> > > +
> > > +     meta->bytesperline = ALIGN(meta->width * fmtinfo->depth / 8,
> > > +                                UNICAM_DMA_BPL_ALIGNMENT);
> > > +     meta->buffersize = meta->height * meta->bytesperline;
> > > +}
> > > +
> > > +/* -----------------------------------------------------------------------------
> > > + * Hardware handling
> > > + */
> > > +
> >
> > As this has been used for years in BSP driver, I won't go in detail in
> > the HW handling functions
> >
> > > +static inline void unicam_clk_write(struct unicam_device *unicam, u32 val)
> > > +{
> > > +     /* Pass the CM_PASSWORD along with the value. */
> > > +     writel(val | 0x5a000000, unicam->clk_gate_base);
> > > +}
> > > +
> > > +static inline u32 unicam_reg_read(struct unicam_device *unicam, u32 offset)
> > > +{
> > > +     return readl(unicam->base + offset);
> > > +}
> > > +
> > > +static inline void unicam_reg_write(struct unicam_device *unicam, u32 offset, u32 val)
> > > +{
> > > +     writel(val, unicam->base + offset);
> > > +}
> > > +
> > > +static inline int unicam_get_field(u32 value, u32 mask)
> > > +{
> > > +     return (value & mask) >> __ffs(mask);
> > > +}
> > > +
> > > +static inline void unicam_set_field(u32 *valp, u32 field, u32 mask)
> > > +{
> > > +     u32 val = *valp;
> > > +
> > > +     val &= ~mask;
> > > +     val |= (field << __ffs(mask)) & mask;
> > > +     *valp = val;
> > > +}
> > > +
> > > +static inline void unicam_reg_write_field(struct unicam_device *unicam, u32 offset,
> > > +                                       u32 field, u32 mask)
> > > +{
> > > +     u32 val = unicam_reg_read(unicam, offset);
> > > +
> > > +     unicam_set_field(&val, field, mask);
> > > +     unicam_reg_write(unicam, offset, val);
> > > +}
> > > +
> > > +static void unicam_wr_dma_addr(struct unicam_node *node,
> > > +                            struct unicam_buffer *buf)
> > > +{
> > > +     dma_addr_t endaddr = buf->dma_addr + buf->size;
> > > +
> > > +     if (node->id == UNICAM_IMAGE_NODE) {
> > > +             unicam_reg_write(node->dev, UNICAM_IBSA0, buf->dma_addr);
> > > +             unicam_reg_write(node->dev, UNICAM_IBEA0, endaddr);
> > > +     } else {
> > > +             unicam_reg_write(node->dev, UNICAM_DBSA0, buf->dma_addr);
> > > +             unicam_reg_write(node->dev, UNICAM_DBEA0, endaddr);
> > > +     }
> > > +}
> > > +
> > > +static unsigned int unicam_get_lines_done(struct unicam_device *unicam)
> > > +{
> > > +     struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > > +     unsigned int stride = node->fmt.fmt.pix.bytesperline;
> > > +     struct unicam_buffer *frm = node->cur_frm;
> > > +     dma_addr_t cur_addr;
> > > +
> > > +     if (!frm)
> > > +             return 0;
> > > +
> > > +     cur_addr = unicam_reg_read(unicam, UNICAM_IBWP);
> > > +     return (unsigned int)(cur_addr - frm->dma_addr) / stride;
> > > +}
> > > +
> > > +static void unicam_schedule_next_buffer(struct unicam_node *node)
> > > +{
> > > +     struct unicam_buffer *buf;
> > > +
> > > +     buf = list_first_entry(&node->dma_queue, struct unicam_buffer, list);
> > > +     node->next_frm = buf;
> > > +     list_del(&buf->list);
> > > +
> > > +     unicam_wr_dma_addr(node, buf);
> > > +}
> > > +
> > > +static void unicam_schedule_dummy_buffer(struct unicam_node *node)
> > > +{
> > > +     int node_id = is_image_node(node) ? UNICAM_IMAGE_NODE : UNICAM_METADATA_NODE;
> > > +
> > > +     dev_dbg(node->dev->dev, "Scheduling dummy buffer for node %d\n", node_id);
> > > +
> > > +     unicam_wr_dma_addr(node, &node->dummy_buf);
> > > +
> > > +     node->next_frm = NULL;
> > > +}
> > > +
> > > +static void unicam_process_buffer_complete(struct unicam_node *node,
> > > +                                        unsigned int sequence)
> > > +{
> > > +     node->cur_frm->vb.field = node->fmt.fmt.pix.field;
> > > +     node->cur_frm->vb.sequence = sequence;
> > > +
> > > +     vb2_buffer_done(&node->cur_frm->vb.vb2_buf, VB2_BUF_STATE_DONE);
> > > +}
> > > +
> > > +static void unicam_queue_event_sof(struct unicam_device *unicam)
> > > +{
> > > +     struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > > +
> > > +     struct v4l2_event event = {
> > > +             .type = V4L2_EVENT_FRAME_SYNC,
> > > +             .u.frame_sync.frame_sequence = unicam->sequence,
> > > +     };
> > > +
> > > +     v4l2_event_queue(&node->video_dev, &event);
> > > +}
> > > +
> > > +static irqreturn_t unicam_isr(int irq, void *dev)
> > > +{
> > > +     struct unicam_device *unicam = dev;
> > > +     unsigned int lines_done = unicam_get_lines_done(dev);
> > > +     unsigned int sequence = unicam->sequence;
> > > +     unsigned int i;
> > > +     u32 ista, sta;
> > > +     bool fe;
> > > +     u64 ts;
> > > +
> > > +     sta = unicam_reg_read(unicam, UNICAM_STA);
> > > +     /* Write value back to clear the interrupts */
> > > +     unicam_reg_write(unicam, UNICAM_STA, sta);
> > > +
> > > +     ista = unicam_reg_read(unicam, UNICAM_ISTA);
> > > +     /* Write value back to clear the interrupts */
> > > +     unicam_reg_write(unicam, UNICAM_ISTA, ista);
> > > +
> > > +     dev_dbg(unicam->dev, "ISR: ISTA: 0x%X, STA: 0x%X, sequence %d, lines done %d\n",
> > > +             ista, sta, sequence, lines_done);
> > > +
> > > +     if (!(sta & (UNICAM_IS | UNICAM_PI0)))
> > > +             return IRQ_HANDLED;
> > > +
> > > +     /*
> > > +      * Look for either the Frame End interrupt or the Packet Capture status
> > > +      * to signal a frame end.
> > > +      */
> > > +     fe = ista & UNICAM_FEI || sta & UNICAM_PI0;
> > > +
> > > +     /*
> > > +      * We must run the frame end handler first. If we have a valid next_frm
> > > +      * and we get a simultaneout FE + FS interrupt, running the FS handler
> > > +      * first would null out the next_frm ptr and we would have lost the
> > > +      * buffer forever.
> > > +      */
> > > +     if (fe) {
> > > +             /*
> > > +              * Ensure we have swapped buffers already as we can't
> > > +              * stop the peripheral. If no buffer is available, use a
> > > +              * dummy buffer to dump out frames until we get a new buffer
> > > +              * to use.
> > > +              */
> > > +             for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > > +                     if (!unicam->node[i].streaming)
> > > +                             continue;
> > > +
> > > +                     /*
> > > +                      * If cur_frm == next_frm, it means we have not had
> > > +                      * a chance to swap buffers, likely due to having
> > > +                      * multiple interrupts occurring simultaneously (like FE
> > > +                      * + FS + LS). In this case, we cannot signal the buffer
> > > +                      * as complete, as the HW will reuse that buffer.
> > > +                      */
> > > +                     if (unicam->node[i].cur_frm &&
> > > +                         unicam->node[i].cur_frm != unicam->node[i].next_frm)
> > > +                             unicam_process_buffer_complete(&unicam->node[i],
> > > +                                                            sequence);
> > > +                     unicam->node[i].cur_frm = unicam->node[i].next_frm;
> > > +             }
> > > +             unicam->sequence++;
> > > +     }
> > > +
> > > +     if (ista & UNICAM_FSI) {
> > > +             /*
> > > +              * Timestamp is to be when the first data byte was captured,
> > > +              * aka frame start.
> > > +              */
> > > +             ts = ktime_get_ns();
> > > +             for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > > +                     if (!unicam->node[i].streaming)
> > > +                             continue;
> > > +
> > > +                     if (unicam->node[i].cur_frm)
> > > +                             unicam->node[i].cur_frm->vb.vb2_buf.timestamp =
> > > +                                                             ts;
> > > +                     else
> > > +                             dev_dbg(unicam->v4l2_dev.dev,
> > > +                                     "ISR: [%d] Dropping frame, buffer not available at FS\n",
> > > +                                     i);
> > > +                     /*
> > > +                      * Set the next frame output to go to a dummy frame
> > > +                      * if we have not managed to obtain another frame
> > > +                      * from the queue.
> > > +                      */
> > > +                     unicam_schedule_dummy_buffer(&unicam->node[i]);
> > > +             }
> > > +
> > > +             unicam_queue_event_sof(unicam);
> > > +     }
> > > +
> > > +     /*
> > > +      * Cannot swap buffer at frame end, there may be a race condition
> > > +      * where the HW does not actually swap it if the new frame has
> > > +      * already started.
> > > +      */
> > > +     if (ista & (UNICAM_FSI | UNICAM_LCI) && !fe) {
> > > +             for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > > +                     if (!unicam->node[i].streaming)
> > > +                             continue;
> > > +
> > > +                     spin_lock(&unicam->node[i].dma_queue_lock);
> > > +                     if (!list_empty(&unicam->node[i].dma_queue) &&
> > > +                         !unicam->node[i].next_frm)
> > > +                             unicam_schedule_next_buffer(&unicam->node[i]);
> > > +                     spin_unlock(&unicam->node[i].dma_queue_lock);
> > > +             }
> > > +     }
> > > +
> > > +     if (unicam_reg_read(unicam, UNICAM_ICTL) & UNICAM_FCM) {
> > > +             /* Switch out of trigger mode if selected */
> > > +             unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> > > +             unicam_reg_write_field(unicam, UNICAM_ICTL, 0, UNICAM_FCM);
> > > +     }
> > > +     return IRQ_HANDLED;
> > > +}
> > > +
> > > +static void unicam_set_packing_config(struct unicam_device *unicam)
> > > +{
> > > +     struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > > +     u32 pack, unpack;
> > > +     u32 val;
> > > +
> > > +     if (node->fmt.fmt.pix.pixelformat == node->fmtinfo->fourcc) {
> > > +             unpack = UNICAM_PUM_NONE;
> > > +             pack = UNICAM_PPM_NONE;
> > > +     } else {
> > > +             switch (node->fmtinfo->depth) {
> > > +             case 8:
> > > +                     unpack = UNICAM_PUM_UNPACK8;
> > > +                     break;
> > > +             case 10:
> > > +                     unpack = UNICAM_PUM_UNPACK10;
> > > +                     break;
> > > +             case 12:
> > > +                     unpack = UNICAM_PUM_UNPACK12;
> > > +                     break;
> > > +             case 14:
> > > +                     unpack = UNICAM_PUM_UNPACK14;
> > > +                     break;
> > > +             case 16:
> > > +                     unpack = UNICAM_PUM_UNPACK16;
> > > +                     break;
> > > +             default:
> > > +                     unpack = UNICAM_PUM_NONE;
> > > +                     break;
> > > +             }
> > > +
> > > +             /* Repacking is always to 16bpp */
> > > +             pack = UNICAM_PPM_PACK16;
> > > +     }
> > > +
> > > +     val = 0;
> > > +     unicam_set_field(&val, unpack, UNICAM_PUM_MASK);
> > > +     unicam_set_field(&val, pack, UNICAM_PPM_MASK);
> > > +     unicam_reg_write(unicam, UNICAM_IPIPE, val);
> > > +}
> > > +
> > > +static void unicam_cfg_image_id(struct unicam_device *unicam)
> > > +{
> > > +     struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > > +
> > > +     if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> > > +             /* CSI2 mode, hardcode VC 0 for now. */
> > > +             unicam_reg_write(unicam, UNICAM_IDI0,
> > > +                              (0 << 6) | node->fmtinfo->csi_dt);
> > > +     } else {
> > > +             /* CCP2 mode */
> > > +             unicam_reg_write(unicam, UNICAM_IDI0,
> > > +                              0x80 | node->fmtinfo->csi_dt);
> > > +     }
> > > +}
> > > +
> > > +static void unicam_enable_ed(struct unicam_device *unicam)
> > > +{
> > > +     u32 val = unicam_reg_read(unicam, UNICAM_DCS);
> > > +
> > > +     unicam_set_field(&val, 2, UNICAM_EDL_MASK);
> > > +     /* Do not wrap at the end of the embedded data buffer */
> > > +     unicam_set_field(&val, 0, UNICAM_DBOB);
> > > +
> > > +     unicam_reg_write(unicam, UNICAM_DCS, val);
> > > +}
> > > +
> > > +static void unicam_start_rx(struct unicam_device *unicam,
> > > +                         struct unicam_buffer *buf)
> > > +{
> > > +     struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > > +     int line_int_freq = node->fmt.fmt.pix.height >> 2;
> > > +     unsigned int i;
> > > +     u32 val;
> > > +
> > > +     if (line_int_freq < 128)
> > > +             line_int_freq = 128;
> > > +
> > > +     /* Enable lane clocks */
> > > +     val = 1;
> > > +     for (i = 0; i < unicam->active_data_lanes; i++)
> > > +             val = val << 2 | 1;
> > > +     unicam_clk_write(unicam, val);
> > > +
> > > +     /* Basic init */
> > > +     unicam_reg_write(unicam, UNICAM_CTRL, UNICAM_MEM);
> > > +
> > > +     /* Enable analogue control, and leave in reset. */
> > > +     val = UNICAM_AR;
> > > +     unicam_set_field(&val, 7, UNICAM_CTATADJ_MASK);
> > > +     unicam_set_field(&val, 7, UNICAM_PTATADJ_MASK);
> > > +     unicam_reg_write(unicam, UNICAM_ANA, val);
> > > +     usleep_range(1000, 2000);
> > > +
> > > +     /* Come out of reset */
> > > +     unicam_reg_write_field(unicam, UNICAM_ANA, 0, UNICAM_AR);
> > > +
> > > +     /* Peripheral reset */
> > > +     unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPR);
> > > +     unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPR);
> > > +
> > > +     unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPE);
> > > +
> > > +     /* Enable Rx control. */
> > > +     val = unicam_reg_read(unicam, UNICAM_CTRL);
> > > +     if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> > > +             unicam_set_field(&val, UNICAM_CPM_CSI2, UNICAM_CPM_MASK);
> > > +             unicam_set_field(&val, UNICAM_DCM_STROBE, UNICAM_DCM_MASK);
> > > +     } else {
> > > +             unicam_set_field(&val, UNICAM_CPM_CCP2, UNICAM_CPM_MASK);
> > > +             unicam_set_field(&val, unicam->bus_flags, UNICAM_DCM_MASK);
> > > +     }
> > > +     /* Packet framer timeout */
> > > +     unicam_set_field(&val, 0xf, UNICAM_PFT_MASK);
> > > +     unicam_set_field(&val, 128, UNICAM_OET_MASK);
> > > +     unicam_reg_write(unicam, UNICAM_CTRL, val);
> > > +
> > > +     unicam_reg_write(unicam, UNICAM_IHWIN, 0);
> > > +     unicam_reg_write(unicam, UNICAM_IVWIN, 0);
> > > +
> > > +     /* AXI bus access QoS setup */
> > > +     val = unicam_reg_read(unicam, UNICAM_PRI);
> > > +     unicam_set_field(&val, 0, UNICAM_BL_MASK);
> > > +     unicam_set_field(&val, 0, UNICAM_BS_MASK);
> > > +     unicam_set_field(&val, 0xe, UNICAM_PP_MASK);
> > > +     unicam_set_field(&val, 8, UNICAM_NP_MASK);
> > > +     unicam_set_field(&val, 2, UNICAM_PT_MASK);
> > > +     unicam_set_field(&val, 1, UNICAM_PE);
> > > +     unicam_reg_write(unicam, UNICAM_PRI, val);
> > > +
> > > +     unicam_reg_write_field(unicam, UNICAM_ANA, 0, UNICAM_DDL);
> > > +
> > > +     /* Always start in trigger frame capture mode (UNICAM_FCM set) */
> > > +     val = UNICAM_FSIE | UNICAM_FEIE | UNICAM_FCM | UNICAM_IBOB;
> > > +     unicam_set_field(&val, line_int_freq, UNICAM_LCIE_MASK);
> > > +     unicam_reg_write(unicam, UNICAM_ICTL, val);
> > > +     unicam_reg_write(unicam, UNICAM_STA, UNICAM_STA_MASK_ALL);
> > > +     unicam_reg_write(unicam, UNICAM_ISTA, UNICAM_ISTA_MASK_ALL);
> > > +
> > > +     /* tclk_term_en */
> > > +     unicam_reg_write_field(unicam, UNICAM_CLT, 2, UNICAM_CLT1_MASK);
> > > +     /* tclk_settle */
> > > +     unicam_reg_write_field(unicam, UNICAM_CLT, 6, UNICAM_CLT2_MASK);
> > > +     /* td_term_en */
> > > +     unicam_reg_write_field(unicam, UNICAM_DLT, 2, UNICAM_DLT1_MASK);
> > > +     /* ths_settle */
> > > +     unicam_reg_write_field(unicam, UNICAM_DLT, 6, UNICAM_DLT2_MASK);
> > > +     /* trx_enable */
> > > +     unicam_reg_write_field(unicam, UNICAM_DLT, 0, UNICAM_DLT3_MASK);
> > > +
> > > +     unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_SOE);
> > > +
> > > +     /* Packet compare setup - required to avoid missing frame ends */
> > > +     val = 0;
> > > +     unicam_set_field(&val, 1, UNICAM_PCE);
> > > +     unicam_set_field(&val, 1, UNICAM_GI);
> > > +     unicam_set_field(&val, 1, UNICAM_CPH);
> > > +     unicam_set_field(&val, 0, UNICAM_PCVC_MASK);
> > > +     unicam_set_field(&val, 1, UNICAM_PCDT_MASK);
> > > +     unicam_reg_write(unicam, UNICAM_CMP0, val);
> > > +
> > > +     /* Enable clock lane and set up terminations */
> > > +     val = 0;
> > > +     if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> > > +             /* CSI2 */
> > > +             unicam_set_field(&val, 1, UNICAM_CLE);
> > > +             unicam_set_field(&val, 1, UNICAM_CLLPE);
> > > +             if (!(unicam->bus_flags & V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK)) {
> > > +                     unicam_set_field(&val, 1, UNICAM_CLTRE);
> > > +                     unicam_set_field(&val, 1, UNICAM_CLHSE);
> > > +             }
> > > +     } else {
> > > +             /* CCP2 */
> > > +             unicam_set_field(&val, 1, UNICAM_CLE);
> > > +             unicam_set_field(&val, 1, UNICAM_CLHSE);
> > > +             unicam_set_field(&val, 1, UNICAM_CLTRE);
> > > +     }
> > > +     unicam_reg_write(unicam, UNICAM_CLK, val);
> > > +
> > > +     /*
> > > +      * Enable required data lanes with appropriate terminations.
> > > +      * The same value needs to be written to UNICAM_DATn registers for
> > > +      * the active lanes, and 0 for inactive ones.
> > > +      */
> > > +     val = 0;
> > > +     if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> > > +             /* CSI2 */
> > > +             unicam_set_field(&val, 1, UNICAM_DLE);
> > > +             unicam_set_field(&val, 1, UNICAM_DLLPE);
> > > +             if (!(unicam->bus_flags & V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK)) {
> > > +                     unicam_set_field(&val, 1, UNICAM_DLTRE);
> > > +                     unicam_set_field(&val, 1, UNICAM_DLHSE);
> > > +             }
> > > +     } else {
> > > +             /* CCP2 */
> > > +             unicam_set_field(&val, 1, UNICAM_DLE);
> > > +             unicam_set_field(&val, 1, UNICAM_DLHSE);
> > > +             unicam_set_field(&val, 1, UNICAM_DLTRE);
> > > +     }
> > > +     unicam_reg_write(unicam, UNICAM_DAT0, val);
> > > +
> > > +     if (unicam->active_data_lanes == 1)
> > > +             val = 0;
> > > +     unicam_reg_write(unicam, UNICAM_DAT1, val);
> > > +
> > > +     if (unicam->max_data_lanes > 2) {
> > > +             /*
> > > +              * Registers UNICAM_DAT2 and UNICAM_DAT3 only valid if the
> > > +              * instance supports more than 2 data lanes.
> > > +              */
> > > +             if (unicam->active_data_lanes == 2)
> > > +                     val = 0;
> > > +             unicam_reg_write(unicam, UNICAM_DAT2, val);
> > > +
> > > +             if (unicam->active_data_lanes == 3)
> > > +                     val = 0;
> > > +             unicam_reg_write(unicam, UNICAM_DAT3, val);
> > > +     }
> > > +
> > > +     unicam_reg_write(unicam, UNICAM_IBLS,
> > > +                      node->fmt.fmt.pix.bytesperline);
> > > +     unicam_wr_dma_addr(&unicam->node[UNICAM_IMAGE_NODE], buf);
> > > +     unicam_set_packing_config(unicam);
> > > +     unicam_cfg_image_id(unicam);
> > > +
> > > +     val = unicam_reg_read(unicam, UNICAM_MISC);
> > > +     unicam_set_field(&val, 1, UNICAM_FL0);
> > > +     unicam_set_field(&val, 1, UNICAM_FL1);
> > > +     unicam_reg_write(unicam, UNICAM_MISC, val);
> > > +
> > > +     /* Enable peripheral */
> > > +     unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPE);
> > > +
> > > +     /* Load image pointers */
> > > +     unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_LIP_MASK);
> > > +
> > > +     /*
> > > +      * Enable trigger only for the first frame to
> > > +      * sync correctly to the FS from the source.
> > > +      */
> > > +     unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> > > +}
> > > +
> > > +static void unicam_start_metadata(struct unicam_device *unicam,
> > > +                               struct unicam_buffer *buf)
> > > +{
> > > +     struct unicam_node *node = &unicam->node[UNICAM_METADATA_NODE];
> > > +
> > > +     unicam_enable_ed(unicam);
> > > +     unicam_wr_dma_addr(node, buf);
> > > +     unicam_reg_write_field(unicam, UNICAM_DCS, 1, UNICAM_LDP);
> > > +}
> > > +
> > > +static void unicam_disable(struct unicam_device *unicam)
> > > +{
> > > +     /* Analogue lane control disable */
> > > +     unicam_reg_write_field(unicam, UNICAM_ANA, 1, UNICAM_DDL);
> > > +
> > > +     /* Stop the output engine */
> > > +     unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_SOE);
> > > +
> > > +     /* Disable the data lanes. */
> > > +     unicam_reg_write(unicam, UNICAM_DAT0, 0);
> > > +     unicam_reg_write(unicam, UNICAM_DAT1, 0);
> > > +
> > > +     if (unicam->max_data_lanes > 2) {
> > > +             unicam_reg_write(unicam, UNICAM_DAT2, 0);
> > > +             unicam_reg_write(unicam, UNICAM_DAT3, 0);
> > > +     }
> > > +
> > > +     /* Peripheral reset */
> > > +     unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPR);
> > > +     usleep_range(50, 100);
> > > +     unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPR);
> > > +
> > > +     /* Disable peripheral */
> > > +     unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPE);
> > > +
> > > +     /* Clear ED setup */
> > > +     unicam_reg_write(unicam, UNICAM_DCS, 0);
> > > +
> > > +     /* Disable all lane clocks */
> > > +     unicam_clk_write(unicam, 0);
> > > +}
> > > +
> > > +/* -----------------------------------------------------------------------------
> > > + * V4L2 subdev operations
> > > + */
> > > +
> > > +static int __unicam_subdev_set_routing(struct v4l2_subdev *sd,
> > > +                                    struct v4l2_subdev_state *state,
> > > +                                    struct v4l2_subdev_krouting *routing)
> > > +{
> > > +     struct v4l2_subdev_route *route;
> > > +     int ret;
> > > +
> > > +     ret = v4l2_subdev_routing_validate(sd, routing,
> > > +                                        V4L2_SUBDEV_ROUTING_ONLY_1_TO_1);
> > > +     if (ret)
> > > +             return ret;
> > > +
> > > +     ret = v4l2_subdev_set_routing(sd, state, routing);
> > > +     if (ret)
> > > +             return ret;
> > > +
> > > +     for_each_active_route(&state->routing, route) {
> > > +             const struct v4l2_mbus_framefmt *def_fmt;
> > > +             struct v4l2_mbus_framefmt *fmt;
> > > +
> > > +             if (route->source_pad == UNICAM_SD_PAD_SOURCE_IMAGE)
> > > +                     def_fmt = &unicam_default_image_format;
> > > +             else
> > > +                     def_fmt = &unicam_default_meta_format;
> > > +
> > > +             fmt = v4l2_subdev_state_get_format(state, route->sink_pad,
> > > +                                                route->sink_stream);
> > > +             *fmt = *def_fmt;
> > > +             fmt = v4l2_subdev_state_get_format(state, route->source_pad,
> > > +                                                route->source_stream);
> > > +             *fmt = *def_fmt;
> > > +     }
> > > +
> > > +     return 0;
> > > +}
> > > +
> > > +static int unicam_subdev_init_state(struct v4l2_subdev *sd,
> > > +                                 struct v4l2_subdev_state *state)
> > > +{
> > > +     struct v4l2_subdev_route routes[] = {
> > > +             {
> > > +                     .sink_pad = UNICAM_SD_PAD_SINK,
> > > +                     .sink_stream = 0,
> > > +                     .source_pad = UNICAM_SD_PAD_SOURCE_IMAGE,
> > > +                     .source_stream = 0,
> > > +                     .flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
> > > +             },
> > > +     };
> > > +
> > > +     struct v4l2_subdev_krouting routing = {
> > > +             .len_routes = ARRAY_SIZE(routes),
> > > +             .num_routes = ARRAY_SIZE(routes),
> > > +             .routes = routes,
> > > +     };
> > > +
> > > +     /* Initialize routing to single route to the fist source pad. */
> > > +     return __unicam_subdev_set_routing(sd, state, &routing);
> > > +}
> > > +
> > > +static int unicam_subdev_enum_mbus_code(struct v4l2_subdev *sd,
> > > +                                     struct v4l2_subdev_state *state,
> > > +                                     struct v4l2_subdev_mbus_code_enum *code)
> > > +{
> > > +     u32 pad, stream;
> > > +     int ret;
> > > +
> > > +     ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
> > > +                                                 code->pad, code->stream,
> > > +                                                 &pad, &stream);
> > > +     if (ret)
> > > +             return ret;
> > > +
> > > +     if (unicam_sd_pad_is_source(code->pad)) {
> > > +             /* No transcoding, source and sink codes must match. */
> > > +             const struct v4l2_mbus_framefmt *fmt;
> > > +
> > > +             fmt = v4l2_subdev_state_get_format(state, pad, stream);
> > > +             if (!fmt)
> > > +                     return -EINVAL;
> > > +
> > > +             if (code->index > 0)
> > > +                     return -EINVAL;
> > > +
> > > +             code->code = fmt->code;
> > > +     } else {
> > > +             const struct unicam_format_info *formats;
> > > +             unsigned int num_formats;
> > > +
> > > +             if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> > > +                     formats = unicam_image_formats;
> > > +                     num_formats = ARRAY_SIZE(unicam_image_formats);
> > > +             } else {
> > > +                     formats = unicam_meta_formats;
> > > +                     num_formats = ARRAY_SIZE(unicam_meta_formats);
> > > +             }
> > > +
> > > +             if (code->index >= num_formats)
> > > +                     return -EINVAL;
> > > +
> > > +             code->code = formats[code->index].code;
> > > +     }
> > > +
> > > +     return 0;
> > > +}
> > > +
> > > +static int unicam_subdev_enum_frame_size(struct v4l2_subdev *sd,
> > > +                                      struct v4l2_subdev_state *state,
> > > +                                      struct v4l2_subdev_frame_size_enum *fse)
> > > +{
> > > +     u32 pad, stream;
> > > +     int ret;
> > > +
> > > +     if (fse->index > 0)
> > > +             return -EINVAL;
> > > +
> > > +     ret = v4l2_subdev_routing_find_opposite_end(&state->routing, fse->pad,
> > > +                                                 fse->stream, &pad,
> > > +                                                 &stream);
> > > +     if (ret)
> > > +             return ret;
> > > +
> > > +     if (unicam_sd_pad_is_source(fse->pad)) {
> > > +             /* No transcoding, source and sink formats must match. */
> > > +             const struct v4l2_mbus_framefmt *fmt;
> > > +
> > > +             fmt = v4l2_subdev_state_get_format(state, pad, stream);
> > > +             if (!fmt)
> > > +                     return -EINVAL;
> > > +
> > > +             if (fse->code != fmt->code)
> > > +                     return -EINVAL;
> > > +
> > > +             fse->min_width = fmt->width;
> > > +             fse->max_width = fmt->width;
> > > +             fse->min_height = fmt->height;
> > > +             fse->max_height = fmt->height;
> > > +     } else {
> > > +             const struct unicam_format_info *fmtinfo;
> > > +
> > > +             fmtinfo = unicam_find_format_by_code(fse->code, pad);
> > > +             if (!fmtinfo)
> > > +                     return -EINVAL;
> > > +
> > > +             if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> > > +                     fse->min_width = UNICAM_IMAGE_MIN_WIDTH;
> > > +                     fse->max_width = UNICAM_IMAGE_MAX_WIDTH;
> > > +                     fse->min_height = UNICAM_IMAGE_MIN_HEIGHT;
> > > +                     fse->max_height = UNICAM_IMAGE_MAX_HEIGHT;
> > > +             } else {
> > > +                     fse->min_width = UNICAM_META_MIN_WIDTH;
> > > +                     fse->max_width = UNICAM_META_MAX_WIDTH;
> > > +                     fse->min_height = UNICAM_META_MIN_HEIGHT;
> > > +                     fse->max_height = UNICAM_META_MAX_HEIGHT;
> > > +             }
> > > +     }
> > > +
> > > +     return 0;
> > > +}
> > > +
> > > +static int unicam_subdev_set_format(struct v4l2_subdev *sd,
> > > +                                 struct v4l2_subdev_state *state,
> > > +                                 struct v4l2_subdev_format *format)
> > > +{
> > > +     struct unicam_device *unicam = sd_to_unicam_device(sd);
> > > +     struct v4l2_mbus_framefmt *sink_format, *source_format;
> > > +     const struct unicam_format_info *fmtinfo;
> > > +     u32 source_pad, source_stream;
> > > +     int ret;
> > > +
> > > +     if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE &&
> > > +         unicam->subdev.streaming)
> > > +             return -EBUSY;
> > > +
> > > +     /* No transcoding, source and sink formats must match. */
> > > +     if (unicam_sd_pad_is_source(format->pad))
> > > +             return v4l2_subdev_get_fmt(sd, state, format);
> > > +
> > > +     /*
> > > +      * Allowed formats for the stream on the sink pad depend on what source
> > > +      * pad the stream is routed to. Find the corresponding source pad and
> > > +      * use it to validate the media bus code.
> > > +      */
> > > +     ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
> > > +                                                 format->pad, format->stream,
> > > +                                                 &source_pad, &source_stream);
> > > +     if (ret)
> > > +             return ret;
> > > +
> > > +     fmtinfo = unicam_find_format_by_code(format->format.code, source_pad);
> > > +     if (!fmtinfo) {
> > > +             fmtinfo = source_pad == UNICAM_SD_PAD_SOURCE_IMAGE
> > > +                     ? &unicam_image_formats[0] : &unicam_meta_formats[0];
> > > +             format->format.code = fmtinfo->code;
> > > +     }
> > > +
> > > +     if (source_pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> > > +             format->format.width = clamp_t(unsigned int,
> > > +                                            format->format.width,
> > > +                                            UNICAM_IMAGE_MIN_WIDTH,
> > > +                                            UNICAM_IMAGE_MAX_WIDTH);
> > > +             format->format.height = clamp_t(unsigned int,
> > > +                                             format->format.height,
> > > +                                             UNICAM_IMAGE_MIN_HEIGHT,
> > > +                                             UNICAM_IMAGE_MAX_HEIGHT);
> > > +             format->format.field = V4L2_FIELD_NONE;
> > > +     } else {
> > > +             format->format.width = clamp_t(unsigned int,
> > > +                                            format->format.width,
> > > +                                            UNICAM_META_MIN_WIDTH,
> > > +                                            UNICAM_META_MAX_WIDTH);
> > > +             format->format.height = clamp_t(unsigned int,
> > > +                                             format->format.height,
> > > +                                             UNICAM_META_MIN_HEIGHT,
> > > +                                             UNICAM_META_MAX_HEIGHT);
> > > +             format->format.field = V4L2_FIELD_NONE;
> > > +
> > > +             /* Colorspace don't apply to metadata. */
> > > +             format->format.colorspace = 0;
> > > +             format->format.ycbcr_enc = 0;
> > > +             format->format.quantization = 0;
> > > +             format->format.xfer_func = 0;
> > > +     }
> > > +
> > > +     sink_format = v4l2_subdev_state_get_format(state, format->pad,
> > > +                                                format->stream);
> > > +     source_format = v4l2_subdev_state_get_format(state, source_pad,
> > > +                                                  source_stream);
> > > +     *sink_format = format->format;
> > > +     *source_format = format->format;
> > > +
> > > +     return 0;
> > > +}
> > > +
> > > +static int unicam_subdev_set_routing(struct v4l2_subdev *sd,
> > > +                                  struct v4l2_subdev_state *state,
> > > +                                  enum v4l2_subdev_format_whence which,
> > > +                                  struct v4l2_subdev_krouting *routing)
> > > +{
> > > +     struct unicam_device *unicam = sd_to_unicam_device(sd);
> > > +
> > > +     if (which == V4L2_SUBDEV_FORMAT_ACTIVE && unicam->subdev.streaming)
> > > +             return -EBUSY;
> > > +
> > > +     return __unicam_subdev_set_routing(sd, state, routing);
> > > +}
> > > +
> > > +static int unicam_sd_enable_streams(struct v4l2_subdev *sd,
> > > +                                 struct v4l2_subdev_state *state, u32 pad,
> > > +                                 u64 streams_mask)
> > > +{
> > > +     struct unicam_device *unicam = sd_to_unicam_device(sd);
> > > +     u32 other_pad, other_stream;
> > > +     int ret;
> > > +
> > > +     ret = v4l2_subdev_routing_find_opposite_end(&state->routing, pad, 0,
> > > +                                                 &other_pad, &other_stream);
> > > +     if (ret)
> > > +             return ret;
> > > +
> > > +     unicam->sequence = 0;
> > > +
> > > +     ret = v4l2_subdev_enable_streams(unicam->sensor.subdev,
> > > +                                      unicam->sensor.pad->index,
> > > +                                      BIT(other_stream));
> > > +     if (ret) {
> > > +             dev_err(unicam->dev, "stream on failed in subdev\n");
> > > +             return ret;
> > > +     }
> > > +
> > > +     unicam->subdev.streaming = true;
> > > +
> > > +     return 0;
> > > +}
> > > +
> > > +static int unicam_sd_disable_streams(struct v4l2_subdev *sd,
> > > +                                  struct v4l2_subdev_state *state, u32 pad,
> > > +                                  u64 streams_mask)
> > > +{
> > > +     struct unicam_device *unicam = sd_to_unicam_device(sd);
> > > +     u32 other_pad, other_stream;
> > > +     int ret;
> > > +
> > > +     ret = v4l2_subdev_routing_find_opposite_end(&state->routing, pad, 0,
> > > +                                                 &other_pad, &other_stream);
> > > +     if (ret)
> > > +             return ret;
> > > +
> > > +     v4l2_subdev_disable_streams(unicam->sensor.subdev,
> > > +                                 unicam->sensor.pad->index,
> > > +                                 BIT(other_stream));
> > > +
> > > +     unicam->subdev.streaming = false;
> > > +
> > > +     return 0;
> > > +}
> > > +
> > > +static const struct v4l2_subdev_pad_ops unicam_subdev_pad_ops = {
> > > +     .enum_mbus_code         = unicam_subdev_enum_mbus_code,
> > > +     .enum_frame_size        = unicam_subdev_enum_frame_size,
> > > +     .get_fmt                = v4l2_subdev_get_fmt,
> > > +     .set_fmt                = unicam_subdev_set_format,
> > > +     .set_routing            = unicam_subdev_set_routing,
> > > +     .enable_streams         = unicam_sd_enable_streams,
> > > +     .disable_streams        = unicam_sd_disable_streams,
> > > +};
> > > +
> > > +static const struct v4l2_subdev_ops unicam_subdev_ops = {
> > > +     .pad                    = &unicam_subdev_pad_ops,
> > > +};
> > > +
> > > +static const struct v4l2_subdev_internal_ops unicam_subdev_internal_ops = {
> > > +     .init_state             = unicam_subdev_init_state,
> > > +};
> > > +
> > > +static const struct media_entity_operations unicam_subdev_media_ops = {
> > > +     .link_validate          = v4l2_subdev_link_validate,
> > > +     .has_pad_interdep       = v4l2_subdev_has_pad_interdep,
> > > +};
> > > +
> > > +static int unicam_subdev_init(struct unicam_device *unicam)
> > > +{
> > > +     struct v4l2_subdev *sd = &unicam->subdev.sd;
> > > +     int ret;
> > > +
> > > +     v4l2_subdev_init(sd, &unicam_subdev_ops);
> > > +     sd->internal_ops = &unicam_subdev_internal_ops;
> > > +     v4l2_set_subdevdata(sd, unicam);
> > > +
> > > +     sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> > > +     sd->entity.ops = &unicam_subdev_media_ops;
> > > +     sd->dev = unicam->dev;
> > > +     sd->owner = THIS_MODULE;
> > > +     sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_STREAMS;
> > > +
> > > +     strscpy(sd->name, "unicam", sizeof(sd->name));
> > > +
> > > +     unicam->subdev.pads[UNICAM_SD_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> > > +     unicam->subdev.pads[UNICAM_SD_PAD_SOURCE_IMAGE].flags = MEDIA_PAD_FL_SOURCE;
> > > +     unicam->subdev.pads[UNICAM_SD_PAD_SOURCE_METADATA].flags = MEDIA_PAD_FL_SOURCE;
> > > +
> > > +     ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(unicam->subdev.pads),
> > > +                                  unicam->subdev.pads);
> > > +     if (ret) {
> > > +             dev_err(unicam->dev, "Failed to initialize media entity: %d\n",
> > > +                     ret);
> > > +             return ret;
> > > +     }
> > > +
> > > +     ret = v4l2_subdev_init_finalize(sd);
> > > +     if (ret) {
> > > +             dev_err(unicam->dev, "Failed to initialize subdev: %d\n", ret);
> > > +             goto err_entity;
> > > +     }
> > > +
> > > +     ret = v4l2_device_register_subdev(&unicam->v4l2_dev, sd);
> > > +     if (ret) {
> > > +             dev_err(unicam->dev, "Failed to register subdev: %d\n", ret);
> > > +             goto err_subdev;
> > > +     }
> > > +
> > > +     return 0;
> > > +
> > > +err_subdev:
> > > +     v4l2_subdev_cleanup(sd);
> > > +err_entity:
> > > +     media_entity_cleanup(&sd->entity);
> > > +     return ret;
> > > +}
> > > +
> > > +static void unicam_subdev_cleanup(struct unicam_device *unicam)
> > > +{
> > > +     v4l2_subdev_cleanup(&unicam->subdev.sd);
> > > +     media_entity_cleanup(&unicam->subdev.sd.entity);
> > > +}
> > > +
> > > +/* -----------------------------------------------------------------------------
> > > + * Videobuf2 queue operations
> > > + */
> > > +
> > > +static int unicam_queue_setup(struct vb2_queue *vq,
> > > +                           unsigned int *nbuffers,
> > > +                           unsigned int *nplanes,
> > > +                           unsigned int sizes[],
> > > +                           struct device *alloc_devs[])
> > > +{
> > > +     struct unicam_node *node = vb2_get_drv_priv(vq);
> > > +     u32 size = is_image_node(node) ? node->fmt.fmt.pix.sizeimage
> > > +              : node->fmt.fmt.meta.buffersize;
> > > +
> > > +     if (vq->num_buffers + *nbuffers < 3)
> >
> > Why 3 ? Are these dummies ?
> >
> > > +             *nbuffers = 3 - vq->num_buffers;
> > > +
> > > +     if (*nplanes) {
> > > +             if (sizes[0] < size) {
> > > +                     dev_dbg(node->dev->dev, "sizes[0] %i < size %u\n",
> > > +                             sizes[0], size);
> > > +                     return -EINVAL;
> > > +             }
> > > +             size = sizes[0];
> > > +     }
> > > +
> > > +     *nplanes = 1;
> > > +     sizes[0] = size;
> > > +
> > > +     return 0;
> > > +}
> > > +
> > > +static int unicam_buffer_prepare(struct vb2_buffer *vb)
> > > +{
> > > +     struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
> > > +     struct unicam_buffer *buf = to_unicam_buffer(vb);
> > > +     u32 size = is_image_node(node) ? node->fmt.fmt.pix.sizeimage
> > > +              : node->fmt.fmt.meta.buffersize;
> > > +
> > > +     if (vb2_plane_size(vb, 0) < size) {
> > > +             dev_dbg(node->dev->dev,
> > > +                     "data will not fit into plane (%lu < %u)\n",
> > > +                     vb2_plane_size(vb, 0), size);
> > > +             return -EINVAL;
> > > +     }
> > > +
> > > +     buf->dma_addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
> > > +     buf->size = size;
> > > +
> > > +     vb2_set_plane_payload(&buf->vb.vb2_buf, 0, size);
> > > +
> > > +     return 0;
> > > +}
> > > +
> > > +static void unicam_return_buffers(struct unicam_node *node,
> > > +                               enum vb2_buffer_state state)
> > > +{
> > > +     struct unicam_buffer *buf, *tmp;
> > > +
> > > +     list_for_each_entry_safe(buf, tmp, &node->dma_queue, list) {
> > > +             list_del(&buf->list);
> > > +             vb2_buffer_done(&buf->vb.vb2_buf, state);
> > > +     }
> > > +
> > > +     if (node->cur_frm)
> > > +             vb2_buffer_done(&node->cur_frm->vb.vb2_buf,
> > > +                             state);
> > > +     if (node->next_frm && node->cur_frm != node->next_frm)
> > > +             vb2_buffer_done(&node->next_frm->vb.vb2_buf,
> > > +                             state);
> > > +
> > > +     node->cur_frm = NULL;
> > > +     node->next_frm = NULL;
> > > +}
> > > +
> > > +static int unicam_start_streaming(struct vb2_queue *vq, unsigned int count)
> > > +{
> > > +     struct unicam_node *node = vb2_get_drv_priv(vq);
> > > +     struct unicam_device *unicam = node->dev;
> > > +     struct v4l2_subdev_state *state;
> > > +     struct unicam_buffer *buf;
> > > +     unsigned long flags;
> > > +     int ret;
> > > +     u32 pad, stream;
> > > +     u32 remote_pad = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
> > > +                                          : UNICAM_SD_PAD_SOURCE_METADATA;
> > > +
> > > +     /* Look for the route for the given pad and stream. */
> > > +     state = v4l2_subdev_lock_and_get_active_state(&unicam->subdev.sd);
> > > +     ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
> > > +                                                 remote_pad, 0,
> > > +                                                 &pad, &stream);
> > > +     v4l2_subdev_unlock_state(state);
> > > +
> > > +     if (ret)
> > > +             goto err_return_buffers;
> 
> You've dropped calling get_mbus_config from in or around this area,
> therefore any device that supports dynamic switching on number of data
> lanes (eg tc358743) is broken.
> Docs guidance is
> "Callers should make sure they get the most up-to-date as possible
> configuration from the remote end, likely calling this operation as
> close as possible to stream on time"
> https://www.kernel.org/doc/html/latest/driver-api/media/v4l2-subdev.html?highlight=get_mbus_config

Indeed, my bad (actually I'm not sure who dropped it, the driver went
through multiple iterations, but that doesn't matter :-)). I'll add it
back.

> > > +
> > > +     dev_dbg(unicam->dev, "Starting stream on %s: %u/%u -> %u/%u (%s)\n",
> > > +             unicam->subdev.sd.name, pad, stream, remote_pad, 0,
> > > +             is_metadata_node(node) ? "metadata" : "image");
> > > +
> > > +     /* The metadata node can't be started alone. */
> > > +     if (is_metadata_node(node)) {
> > > +             if (!unicam->node[UNICAM_IMAGE_NODE].streaming) {
> >
> > Does it mean the metadata node has to be started second, or should
> > this be made a nop and the metadata node gets started once the image
> > node is started too ? I'm fine with the constraint of having the
> > metadata node being started second fwiw
> 
> This has been flipped from the downstream implementation.
> Downstream we expected the metadata node to be enabled first, but it
> wouldn't get started until the image node was enabled.

Unless I'm mistaken, the downstream implementation waits for both nodes
to be started before starting the device.

> Things like sequence numbers come to mind now that this is enabling
> the image node first. If the image node has produced the first frame
> before the metadata node has been enabled, then as I read it the
> sequence numbers for the two nodes should be different - there haven't
> really been any dropped frames as you hadn't enabled the node.
> 
> I'd also want to confirm what the behaviour is if the frame has
> already started before the metadata node is enabled. Embedded data is
> almost always sent before the image data, so have we potentially
> missed it for the current frame, but at the end of frame are going to
> return both buffers?

I think we have a possible race condition here. I'll think a bit more
about it.

> > > +                     dev_err(unicam->dev,
> > > +                             "Can't start metadata without image\n");
> > > +                     ret = -EINVAL;
> > > +                     goto err_return_buffers;
> > > +             }
> > > +
> > > +             spin_lock_irqsave(&node->dma_queue_lock, flags);
> > > +             buf = list_first_entry(&node->dma_queue,
> > > +                                    struct unicam_buffer, list);
> > > +             dev_dbg(unicam->dev, "buffer %p\n", buf);
> >
> > Is this useful ?
> >
> > > +             node->cur_frm = buf;
> > > +             node->next_frm = buf;
> > > +             list_del(&buf->list);
> > > +             spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> > > +
> > > +             unicam_start_metadata(unicam, buf);
> > > +             node->streaming = true;
> > > +             return 0;
> > > +     }
> > > +
> > > +     ret = pm_runtime_resume_and_get(unicam->dev);
> > > +     if (ret < 0) {
> > > +             dev_err(unicam->dev, "PM runtime resume failed: %d\n", ret);
> > > +             goto err_return_buffers;
> > > +     }
> > > +
> > > +     ret = video_device_pipeline_start(&node->video_dev, &unicam->pipe);
> > > +     if (ret < 0) {
> > > +             dev_dbg(unicam->dev, "Failed to start media pipeline: %d\n", ret);
> >
> > Isn't this an err ?
> >
> > > +             goto err_pm_put;
> > > +     }
> > > +
> > > +     spin_lock_irqsave(&node->dma_queue_lock, flags);
> > > +     buf = list_first_entry(&node->dma_queue,
> > > +                            struct unicam_buffer, list);
> > > +     dev_dbg(unicam->dev, "buffer %p\n", buf);
> > > +     node->cur_frm = buf;
> > > +     node->next_frm = buf;
> > > +     list_del(&buf->list);
> > > +     spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> > > +
> > > +     unicam_start_rx(unicam, buf);
> > > +
> > > +     ret = v4l2_subdev_enable_streams(&unicam->subdev.sd, remote_pad, BIT(0));
> >
> > A bit confused by the API here, shouldn't we also do this for embedded
> > data ?
> >
> > > +     if (ret < 0) {
> > > +             dev_err(unicam->dev, "stream on failed in subdev\n");
> > > +             goto error_pipeline;
> > > +     }
> > > +
> > > +     node->streaming = true;
> > > +
> > > +     return 0;
> > > +
> > > +error_pipeline:
> > > +     video_device_pipeline_stop(&node->video_dev);
> > > +err_pm_put:
> > > +     pm_runtime_put_sync(unicam->dev);
> > > +err_return_buffers:
> > > +     unicam_return_buffers(node, VB2_BUF_STATE_QUEUED);
> > > +     return ret;
> > > +}
> > > +
> > > +static void unicam_stop_streaming(struct vb2_queue *vq)
> > > +{
> > > +     struct unicam_node *node = vb2_get_drv_priv(vq);
> > > +     struct unicam_device *unicam = node->dev;
> > > +     u32 remote_pad_index = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
> > > +                                                : UNICAM_SD_PAD_SOURCE_METADATA;
> > > +
> > > +     node->streaming = false;
> > > +
> > > +     v4l2_subdev_disable_streams(&unicam->subdev.sd, remote_pad_index,
> > > +                                 BIT(0));
> > > +
> > > +     /* We can stream only with the image node. */
> > > +     if (is_metadata_node(node)) {
> > > +             /*
> > > +              * Allow the hardware to spin in the dummy buffer.
> > > +              * This is only really needed if the embedded data pad is
> > > +              * disabled before the image pad.
> > > +              */
> > > +             unicam_wr_dma_addr(node, &node->dummy_buf);
> > > +             goto dequeue_buffers;
> > > +     }
> > > +
> > > +     unicam_disable(unicam);
> > > +
> > > +     video_device_pipeline_stop(&node->video_dev);
> > > +     pm_runtime_put(unicam->dev);
> > > +
> > > +dequeue_buffers:
> > > +     /* Clear all queued buffers for the node */
> > > +     unicam_return_buffers(node, VB2_BUF_STATE_ERROR);
> > > +}
> > > +
> > > +static void unicam_buffer_queue(struct vb2_buffer *vb)
> > > +{
> > > +     struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
> > > +     struct unicam_buffer *buf = to_unicam_buffer(vb);
> > > +
> > > +     spin_lock_irq(&node->dma_queue_lock);
> > > +     list_add_tail(&buf->list, &node->dma_queue);
> > > +     spin_unlock_irq(&node->dma_queue_lock);
> > > +}
> > > +
> > > +static const struct vb2_ops unicam_video_qops = {
> > > +     .queue_setup            = unicam_queue_setup,
> > > +     .wait_prepare           = vb2_ops_wait_prepare,
> > > +     .wait_finish            = vb2_ops_wait_finish,
> > > +     .buf_prepare            = unicam_buffer_prepare,
> > > +     .start_streaming        = unicam_start_streaming,
> > > +     .stop_streaming         = unicam_stop_streaming,
> > > +     .buf_queue              = unicam_buffer_queue,
> > > +};
> > > +
> > > +/* -----------------------------------------------------------------------------
> > > + *  V4L2 video device operations
> > > + */
> > > +
> > > +static int unicam_querycap(struct file *file, void *priv,
> > > +                        struct v4l2_capability *cap)
> > > +{
> > > +     strscpy(cap->driver, UNICAM_MODULE_NAME, sizeof(cap->driver));
> > > +     strscpy(cap->card, UNICAM_MODULE_NAME, sizeof(cap->card));
> > > +
> > > +     cap->capabilities |= V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_META_CAPTURE;
> > > +
> > > +     return 0;
> > > +}
> > > +
> > > +static int unicam_enum_fmt_vid(struct file *file, void  *priv,
> > > +                            struct v4l2_fmtdesc *f)
> > > +{
> > > +     unsigned int index;
> > > +     unsigned int i;
> > > +
> > > +     for (i = 0, index = 0; i < ARRAY_SIZE(unicam_image_formats); i++) {
> > > +             if (f->mbus_code && unicam_image_formats[i].code != f->mbus_code)
> > > +                     continue;
> > > +
> > > +             if (index == f->index) {
> > > +                     f->pixelformat = unicam_image_formats[i].fourcc;
> > > +                     return 0;
> > > +             }
> > > +
> > > +             index++;
> > > +
> > > +             if (!unicam_image_formats[i].unpacked_fourcc)
> > > +                     continue;
> > > +
> > > +             if (index == f->index) {
> > > +                     f->pixelformat = unicam_image_formats[i].unpacked_fourcc;
> > > +                     return 0;
> > > +             }
> > > +
> > > +             index++;
> > > +     }
> > > +
> > > +     return -EINVAL;
> > > +}
> > > +
> > > +static int unicam_g_fmt_vid(struct file *file, void *priv,
> > > +                         struct v4l2_format *f)
> > > +{
> > > +     struct unicam_node *node = video_drvdata(file);
> > > +
> > > +     *f = node->fmt;
> > > +
> > > +     return 0;
> > > +}
> > > +
> > > +static const struct unicam_format_info *
> > > +__unicam_try_fmt_vid(struct unicam_node *node, struct v4l2_pix_format *pix)
> > > +{
> > > +     const struct unicam_format_info *fmtinfo;
> > > +
> > > +     /*
> > > +      * Default to the first format if the requested pixel format code isn't
> > > +      * supported.
> > > +      */
> > > +     fmtinfo = unicam_find_format_by_fourcc(pix->pixelformat,
> > > +                                            UNICAM_SD_PAD_SOURCE_IMAGE);
> > > +     if (!fmtinfo) {
> > > +             fmtinfo = &unicam_image_formats[0];
> > > +             pix->pixelformat = fmtinfo->fourcc;
> > > +     }
> > > +
> > > +     unicam_calc_image_size_bpl(node->dev, fmtinfo, pix);
> > > +
> > > +     if (pix->field == V4L2_FIELD_ANY)
> > > +             pix->field = V4L2_FIELD_NONE;
> > > +
> > > +     return fmtinfo;
> > > +}
> > > +
> > > +static int unicam_try_fmt_vid(struct file *file, void *priv,
> > > +                           struct v4l2_format *f)
> > > +{
> > > +     struct unicam_node *node = video_drvdata(file);
> > > +
> > > +     __unicam_try_fmt_vid(node, &f->fmt.pix);
> > > +     return 0;
> > > +}
> > > +
> > > +static int unicam_s_fmt_vid(struct file *file, void *priv,
> > > +                         struct v4l2_format *f)
> > > +{
> > > +     struct unicam_node *node = video_drvdata(file);
> > > +
> > > +     if (vb2_is_busy(&node->buffer_queue))
> > > +             return -EBUSY;
> > > +
> > > +     node->fmtinfo = __unicam_try_fmt_vid(node, &f->fmt.pix);
> > > +     node->fmt = *f;
> > > +
> > > +     return 0;
> > > +}
> > > +
> > > +static int unicam_enum_fmt_meta(struct file *file, void *priv,
> > > +                             struct v4l2_fmtdesc *f)
> > > +{
> > > +     unsigned int i, index;
> > > +
> > > +     for (i = 0, index = 0; i < ARRAY_SIZE(unicam_meta_formats); i++) {
> > > +             if (f->mbus_code && unicam_meta_formats[i].code != f->mbus_code)
> >
> > Do we want to allow mbus_code filtering for metadata ? There's a
> > 1-to-1 relationship between mbus codes and pixel formats
> >
> > > +                     continue;
> > > +             if (!unicam_meta_formats[i].metadata_fmt)
> > > +                     continue;
> >
> > How can this be false if we're iterating on unicam_meta_formats[] ?
> >
> > > +
> > > +             if (index == f->index) {
> > > +                     f->pixelformat = unicam_meta_formats[i].fourcc;
> > > +                     f->type = V4L2_BUF_TYPE_META_CAPTURE;
> > > +                     return 0;
> > > +             }
> > > +             index++;
> > > +     }
> > > +
> > > +     return -EINVAL;
> > > +}
> > > +
> > > +static int unicam_g_fmt_meta(struct file *file, void *priv,
> > > +                          struct v4l2_format *f)
> > > +{
> > > +     struct unicam_node *node = video_drvdata(file);
> > > +
> > > +     f->fmt.meta = node->fmt.fmt.meta;
> > > +
> > > +     return 0;
> > > +}
> > > +
> > > +static const struct unicam_format_info *
> > > +__unicam_try_fmt_meta(struct unicam_node *node, struct v4l2_meta_format *meta)
> > > +{
> > > +     const struct unicam_format_info *fmtinfo;
> > > +
> > > +     /*
> > > +      * Default to the first format if the requested pixel format code isn't
> > > +      * supported.
> > > +      */
> > > +     fmtinfo = unicam_find_format_by_fourcc(meta->dataformat,
> > > +                                            UNICAM_SD_PAD_SOURCE_METADATA);
> > > +     if (!fmtinfo) {
> > > +             fmtinfo = &unicam_meta_formats[0];
> > > +             meta->dataformat = fmtinfo->fourcc;
> > > +     }
> > > +
> > > +     unicam_calc_meta_size_bpl(node->dev, fmtinfo, meta);
> > > +
> > > +     return fmtinfo;
> > > +}
> > > +
> > > +static int unicam_try_fmt_meta(struct file *file, void *priv,
> > > +                            struct v4l2_format *f)
> > > +{
> > > +     struct unicam_node *node = video_drvdata(file);
> > > +
> > > +     __unicam_try_fmt_vid(node, &f->fmt.pix);
> > > +     return 0;
> > > +}
> > > +
> > > +static int unicam_s_fmt_meta(struct file *file, void *priv,
> > > +                          struct v4l2_format *f)
> > > +{
> > > +     struct unicam_node *node = video_drvdata(file);
> > > +
> > > +     if (vb2_is_busy(&node->buffer_queue))
> > > +             return -EBUSY;
> > > +
> > > +     node->fmtinfo = __unicam_try_fmt_meta(node, &f->fmt.meta);
> > > +     node->fmt = *f;
> > > +
> > > +     return 0;
> > > +}
> > > +
> > > +static int unicam_enum_framesizes(struct file *file, void *fh,
> > > +                               struct v4l2_frmsizeenum *fsize)
> > > +{
> > > +     struct unicam_node *node = video_drvdata(file);
> > > +     int ret = -EINVAL;
> > > +
> > > +     if (fsize->index > 0)
> > > +             return ret;
> > > +
> > > +     if (is_image_node(node)) {
> > > +             if (!unicam_find_format_by_fourcc(fsize->pixel_format,
> > > +                                               UNICAM_SD_PAD_SOURCE_IMAGE))
> > > +                     return ret;
> > > +
> > > +             fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> > > +             fsize->stepwise.min_width = UNICAM_IMAGE_MIN_WIDTH;
> > > +             fsize->stepwise.max_width = UNICAM_IMAGE_MAX_WIDTH;
> > > +             fsize->stepwise.step_width = 1;
> > > +             fsize->stepwise.min_height = UNICAM_IMAGE_MIN_HEIGHT;
> > > +             fsize->stepwise.max_height = UNICAM_IMAGE_MAX_HEIGHT;
> > > +             fsize->stepwise.step_height = 1;
> > > +     } else {
> > > +             if (!unicam_find_format_by_fourcc(fsize->pixel_format,
> > > +                                               UNICAM_SD_PAD_SOURCE_METADATA))
> > > +                     return ret;
> > > +
> > Isn't this V4L2_FRMSIZE_TYPE_STEPWISE as well ?
> >
> > > +             fsize->stepwise.min_width = UNICAM_META_MIN_WIDTH;
> > > +             fsize->stepwise.max_width = UNICAM_META_MAX_WIDTH;
> > > +             fsize->stepwise.step_width = 1;
> > > +             fsize->stepwise.min_height = UNICAM_META_MIN_HEIGHT;
> > > +             fsize->stepwise.max_height = UNICAM_META_MAX_HEIGHT;
> > > +             fsize->stepwise.step_height = 1;
> > > +     }
> > > +
> > > +     return 0;
> > > +}
> > > +
> > > +static int unicam_log_status(struct file *file, void *fh)
> > > +{
> > > +     struct unicam_node *node = video_drvdata(file);
> > > +     struct unicam_device *unicam = node->dev;
> > > +     u32 reg;
> > > +
> > > +     /* status for sub devices */
> > > +     v4l2_device_call_all(&unicam->v4l2_dev, 0, core, log_status);
> > > +
> > > +     dev_info(unicam->dev, "-----Receiver status-----\n");
> > > +     dev_info(unicam->dev, "V4L2 width/height:   %ux%u\n",
> > > +              node->fmt.fmt.pix.width, node->fmt.fmt.pix.height);
> > > +     dev_info(unicam->dev, "Mediabus format:     %08x\n",
> > > +              node->fmtinfo->code);
> > > +     dev_info(unicam->dev, "V4L2 format:         %08x\n",
> > > +              node->fmt.fmt.pix.pixelformat);
> > > +     reg = unicam_reg_read(unicam, UNICAM_IPIPE);
> > > +     dev_info(unicam->dev, "Unpacking/packing:   %u / %u\n",
> > > +              unicam_get_field(reg, UNICAM_PUM_MASK),
> > > +              unicam_get_field(reg, UNICAM_PPM_MASK));
> > > +     dev_info(unicam->dev, "----Live data----\n");
> > > +     dev_info(unicam->dev, "Programmed stride:   %4u\n",
> > > +              unicam_reg_read(unicam, UNICAM_IBLS));
> > > +     dev_info(unicam->dev, "Detected resolution: %ux%u\n",
> > > +              unicam_reg_read(unicam, UNICAM_IHSTA),
> > > +              unicam_reg_read(unicam, UNICAM_IVSTA));
> > > +     dev_info(unicam->dev, "Write pointer:       %08x\n",
> > > +              unicam_reg_read(unicam, UNICAM_IBWP));
> > > +
> > > +     return 0;
> > > +}
> > > +
> > > +static int unicam_subscribe_event(struct v4l2_fh *fh,
> > > +                               const struct v4l2_event_subscription *sub)
> > > +{
> > > +     switch (sub->type) {
> > > +     case V4L2_EVENT_FRAME_SYNC:
> > > +             return v4l2_event_subscribe(fh, sub, 2, NULL);
> > > +     default:
> > > +             return -EINVAL;
> > > +     }
> > > +}
> > > +
> > > +static const struct v4l2_ioctl_ops unicam_ioctl_ops = {
> > > +     .vidioc_querycap                = unicam_querycap,
> > > +
> > > +     .vidioc_enum_fmt_vid_cap        = unicam_enum_fmt_vid,
> > > +     .vidioc_g_fmt_vid_cap           = unicam_g_fmt_vid,
> > > +     .vidioc_try_fmt_vid_cap         = unicam_try_fmt_vid,
> > > +     .vidioc_s_fmt_vid_cap           = unicam_s_fmt_vid,
> > > +
> > > +     .vidioc_enum_fmt_meta_cap       = unicam_enum_fmt_meta,
> > > +     .vidioc_g_fmt_meta_cap          = unicam_g_fmt_meta,
> > > +     .vidioc_try_fmt_meta_cap        = unicam_try_fmt_meta,
> > > +     .vidioc_s_fmt_meta_cap          = unicam_s_fmt_meta,
> > > +
> > > +     .vidioc_enum_framesizes         = unicam_enum_framesizes,
> > > +
> > > +     .vidioc_reqbufs                 = vb2_ioctl_reqbufs,
> > > +     .vidioc_create_bufs             = vb2_ioctl_create_bufs,
> > > +     .vidioc_prepare_buf             = vb2_ioctl_prepare_buf,
> > > +     .vidioc_querybuf                = vb2_ioctl_querybuf,
> > > +     .vidioc_qbuf                    = vb2_ioctl_qbuf,
> > > +     .vidioc_dqbuf                   = vb2_ioctl_dqbuf,
> > > +     .vidioc_expbuf                  = vb2_ioctl_expbuf,
> > > +     .vidioc_streamon                = vb2_ioctl_streamon,
> > > +     .vidioc_streamoff               = vb2_ioctl_streamoff,
> > > +
> > > +     .vidioc_log_status              = unicam_log_status,
> > > +     .vidioc_subscribe_event         = unicam_subscribe_event,
> > > +     .vidioc_unsubscribe_event       = v4l2_event_unsubscribe,
> > > +};
> > > +
> > > +/* unicam capture driver file operations */
> > > +static const struct v4l2_file_operations unicam_fops = {
> > > +     .owner          = THIS_MODULE,
> > > +     .open           = v4l2_fh_open,
> > > +     .release        = vb2_fop_release,
> > > +     .poll           = vb2_fop_poll,
> > > +     .unlocked_ioctl = video_ioctl2,
> > > +     .mmap           = vb2_fop_mmap,
> > > +};
> > > +
> > > +static int unicam_video_link_validate(struct media_link *link)
> > > +{
> > > +     struct video_device *vdev =
> > > +             media_entity_to_video_device(link->sink->entity);
> > > +     struct v4l2_subdev *sd =
> > > +             media_entity_to_v4l2_subdev(link->source->entity);
> > > +     struct unicam_node *node = video_get_drvdata(vdev);
> > > +     const u32 pad = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
> > > +                   : UNICAM_SD_PAD_SOURCE_METADATA;
> > > +     const struct v4l2_mbus_framefmt *format;
> > > +     struct v4l2_subdev_state *state;
> > > +     int ret = 0;
> > > +
> > > +     state = v4l2_subdev_lock_and_get_active_state(sd);
> > > +
> > > +     format = v4l2_subdev_state_get_format(state, pad, 0);
> > > +     if (!format) {
> > > +             ret = -EINVAL;
> > > +             goto out;
> > > +     }
> > > +
> > > +     if (is_image_node(node)) {
> > > +             const struct v4l2_pix_format *fmt = &node->fmt.fmt.pix;
> > > +
> > > +             if (node->fmtinfo->code != format->code ||
> > > +                 fmt->height != format->height ||
> > > +                 fmt->width != format->width ||
> > > +                 fmt->field != format->field) {
> > > +                     dev_dbg(node->dev->dev,
> > > +                             "image: (%u x %u) 0x%08x %s != (%u x %u) 0x%08x %s\n",
> > > +                             fmt->width, fmt->height, node->fmtinfo->code,
> > > +                             v4l2_field_names[fmt->field],
> > > +                             format->width, format->height, format->code,
> > > +                             v4l2_field_names[format->field]);
> > > +                     ret = -EPIPE;
> > > +             };
> >
> > No need for ; here and in the next loop
> >
> > > +     } else {
> > > +             const struct v4l2_meta_format *fmt = &node->fmt.fmt.meta;
> > > +
> > > +             if (node->fmtinfo->code != format->code ||
> > > +                 fmt->height != format->height ||
> > > +                 fmt->width != format->width) {
> > > +                     dev_dbg(node->dev->dev,
> > > +                             "meta: (%u x %u) 0x%04x != (%u x %u) 0x%04x\n",
> > > +                             fmt->width, fmt->height, node->fmtinfo->code,
> > > +                             format->width, format->height, format->code);
> > > +                     ret = -EPIPE;
> > > +             };
> > > +     }
> > > +
> > > +out:
> > > +     v4l2_subdev_unlock_state(state);
> > > +     return ret;
> > > +}
> > > +
> > > +static const struct media_entity_operations unicam_video_media_ops = {
> > > +     .link_validate = unicam_video_link_validate,
> > > +};
> > > +
> > > +static void unicam_node_release(struct video_device *vdev)
> > > +{
> > > +     struct unicam_node *node = video_get_drvdata(vdev);
> > > +
> > > +     unicam_put(node->dev);
> > > +}
> > > +
> > > +static void unicam_set_default_format(struct unicam_node *node)
> > > +{
> > > +     if (is_image_node(node)) {
> > > +             struct v4l2_pix_format *fmt = &node->fmt.fmt.pix;
> > > +
> > > +             node->fmtinfo = &unicam_image_formats[0];
> > > +             node->fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> > > +
> > > +             v4l2_fill_pix_format(fmt, &unicam_default_image_format);
> > > +             fmt->pixelformat = node->fmtinfo->fourcc;
> > > +             unicam_calc_image_size_bpl(node->dev, node->fmtinfo, fmt);
> > > +     } else {
> > > +             struct v4l2_meta_format *fmt = &node->fmt.fmt.meta;
> > > +
> > > +             node->fmtinfo = &unicam_meta_formats[0];
> > > +             node->fmt.type = V4L2_BUF_TYPE_META_CAPTURE;
> > > +
> > > +             fmt->dataformat = node->fmtinfo->fourcc;
> > > +             fmt->width = unicam_default_meta_format.width;
> > > +             fmt->height = unicam_default_meta_format.height;
> > > +             unicam_calc_meta_size_bpl(node->dev, node->fmtinfo, fmt);
> > > +     }
> > > +}
> > > +
> > > +static int unicam_register_node(struct unicam_device *unicam,
> > > +                             enum unicam_node_type type)
> > > +{
> > > +     const u32 pad_index = type == UNICAM_IMAGE_NODE
> > > +                         ? UNICAM_SD_PAD_SOURCE_IMAGE
> > > +                         : UNICAM_SD_PAD_SOURCE_METADATA;
> > > +     struct unicam_node *node = &unicam->node[type];
> > > +     struct video_device *vdev = &node->video_dev;
> > > +     struct vb2_queue *q = &node->buffer_queue;
> > > +     int ret;
> > > +
> > > +     node->dev = unicam_get(unicam);
> > > +     node->id = type;
> > > +
> > > +     spin_lock_init(&node->dma_queue_lock);
> > > +     mutex_init(&node->lock);
> > > +
> > > +     INIT_LIST_HEAD(&node->dma_queue);
> > > +
> > > +     /* Initialize the videobuf2 queue. */
> > > +     q->type = type == UNICAM_IMAGE_NODE ? V4L2_BUF_TYPE_VIDEO_CAPTURE
> > > +                                         : V4L2_BUF_TYPE_META_CAPTURE;
> > > +     q->io_modes = VB2_MMAP | VB2_DMABUF;
> > > +     q->drv_priv = node;
> > > +     q->ops = &unicam_video_qops;
> > > +     q->mem_ops = &vb2_dma_contig_memops;
> > > +     q->buf_struct_size = sizeof(struct unicam_buffer);
> > > +     q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> > > +     q->lock = &node->lock;
> > > +     q->min_queued_buffers = 1;
> > > +     q->dev = unicam->dev;
> > > +
> > > +     ret = vb2_queue_init(q);
> > > +     if (ret) {
> > > +             dev_err(unicam->dev, "vb2_queue_init() failed\n");
> > > +             goto err_unicam_put;
> > > +     }
> > > +
> > > +     /* Initialize the video device. */
> > > +     vdev->release = unicam_node_release;
> > > +     vdev->fops = &unicam_fops;
> > > +     vdev->ioctl_ops = &unicam_ioctl_ops;
> > > +     vdev->v4l2_dev = &unicam->v4l2_dev;
> > > +     vdev->vfl_dir = VFL_DIR_RX;
> > > +     vdev->queue = q;
> > > +     vdev->lock = &node->lock;
> > > +     vdev->device_caps = type == UNICAM_IMAGE_NODE
> > > +                       ? V4L2_CAP_VIDEO_CAPTURE : V4L2_CAP_META_CAPTURE;
> > > +     vdev->device_caps |= V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;
> > > +     vdev->entity.ops = &unicam_video_media_ops;
> > > +
> > > +     snprintf(vdev->name, sizeof(vdev->name), "%s-%s", UNICAM_MODULE_NAME,
> > > +              type == UNICAM_IMAGE_NODE ? "image" : "embedded");
> > > +
> > > +     video_set_drvdata(vdev, node);
> > > +
> > > +     if (type == UNICAM_IMAGE_NODE)
> > > +             vdev->entity.flags |= MEDIA_ENT_FL_DEFAULT;
> > > +
> > > +     node->pad.flags = MEDIA_PAD_FL_SINK;
> > > +
> > > +     ret = media_entity_pads_init(&vdev->entity, 1, &node->pad);
> > > +     if (ret)
> > > +             goto err_unicam_put;
> > > +
> > > +     node->dummy_buf.size = UNICAM_DUMMY_BUF_SIZE;
> > > +     node->dummy_buf_cpu_addr = dma_alloc_coherent(unicam->dev,
> > > +                                                   node->dummy_buf.size,
> > > +                                                   &node->dummy_buf.dma_addr,
> > > +                                                   GFP_KERNEL);
> > > +     if (!node->dummy_buf_cpu_addr) {
> > > +             dev_err(unicam->dev, "Unable to allocate dummy buffer.\n");
> > > +             ret = -ENOMEM;
> > > +             goto err_entity_cleanup;
> > > +     }
> > > +
> > > +     unicam_set_default_format(node);
> > > +
> > > +     ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> > > +     if (ret) {
> > > +             dev_err(unicam->dev, "Unable to register video device %s\n",
> > > +                     vdev->name);
> > > +             goto err_dma_free;
> > > +     }
> > > +
> > > +     node->registered = true;
> > > +
> > > +     ret = media_create_pad_link(&unicam->subdev.sd.entity,
> > > +                                 pad_index,
> > > +                                 &node->video_dev.entity,
> > > +                                 0,
> > > +                                 MEDIA_LNK_FL_ENABLED |
> > > +                                 MEDIA_LNK_FL_IMMUTABLE);
> > > +     if (ret) {
> > > +             /*
> > > +              * No need for cleanup, the caller will unregister the
> > > +              * video device, which will drop the reference on the
> > > +              * device and trigger the cleanup.
> > > +              */
> > > +             dev_err(unicam->dev, "Unable to create pad link for %s\n",
> > > +                     unicam->sensor.subdev->name);
> > > +             return ret;
> > > +     }
> > > +
> > > +     return 0;
> > > +
> > > +err_dma_free:
> > > +     dma_free_coherent(unicam->dev, node->dummy_buf.size,
> > > +                       node->dummy_buf_cpu_addr,
> > > +                       node->dummy_buf.dma_addr);
> > > +err_entity_cleanup:
> > > +     media_entity_cleanup(&vdev->entity);
> > > +err_unicam_put:
> > > +     unicam_put(unicam);
> > > +     return ret;
> > > +}
> > > +
> > > +static void unicam_unregister_nodes(struct unicam_device *unicam)
> > > +{
> > > +     unsigned int i;
> > > +
> > > +     for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > > +             struct unicam_node *node = &unicam->node[i];
> > > +
> > > +             if (node->dummy_buf_cpu_addr)
> > > +                     dma_free_coherent(unicam->dev, node->dummy_buf.size,
> > > +                                       node->dummy_buf_cpu_addr,
> > > +                                       node->dummy_buf.dma_addr);
> > > +
> > > +             if (node->registered) {
> > > +                     video_unregister_device(&node->video_dev);
> > > +                     node->registered = false;
> > > +             }
> > > +     }
> > > +}
> > > +
> > > +/* -----------------------------------------------------------------------------
> > > + * Power management
> > > + */
> > > +
> > > +static int unicam_runtime_resume(struct device *dev)
> > > +{
> > > +     struct unicam_device *unicam = dev_get_drvdata(dev);
> > > +     int ret;
> > > +
> > > +     ret = clk_set_min_rate(unicam->vpu_clock, UNICAM_MIN_VPU_CLOCK_RATE);
> > > +     if (ret) {
> > > +             dev_err(unicam->dev, "failed to set up VPU clock\n");
> > > +             return ret;
> > > +     }
> >
> > Same question as above, shouldn't this be an 'assigned-clock-rates in
> > dts ?
> >
> > > +
> > > +     ret = clk_prepare_enable(unicam->vpu_clock);
> > > +     if (ret) {
> > > +             dev_err(unicam->dev, "Failed to enable VPU clock: %d\n", ret);
> > > +             goto err_vpu_clock;
> > > +     }
> > > +
> > > +     ret = clk_set_rate(unicam->clock, 100 * 1000 * 1000);
> > > +     if (ret) {
> > > +             dev_err(unicam->dev, "failed to set up CSI clock\n");
> > > +             goto err_vpu_prepare;
> > > +     }
> >
> > same question
> >
> > > +
> > > +     ret = clk_prepare_enable(unicam->clock);
> > > +     if (ret) {
> > > +             dev_err(unicam->dev, "Failed to enable CSI clock: %d\n", ret);
> > > +             goto err_vpu_prepare;
> >
> > Do you need to 'unset' the rate as done for the VPU clock ?
> >
> > > +     }
> > > +
> > > +     return 0;
> > > +
> > > +err_vpu_prepare:
> > > +     clk_disable_unprepare(unicam->vpu_clock);
> > > +err_vpu_clock:
> > > +     if (clk_set_min_rate(unicam->vpu_clock, 0))
> > > +             dev_err(unicam->dev, "failed to reset the VPU clock\n");
> > > +
> > > +     return ret;
> > > +}
> > > +
> > > +static int unicam_runtime_suspend(struct device *dev)
> > > +{
> > > +     struct unicam_device *unicam = dev_get_drvdata(dev);
> > > +
> > > +     clk_disable_unprepare(unicam->clock);
> >
> > (continuing  with the above question: probably not, as the clock is
> > not 'unset' neither here)
> >
> > The rest looks good, nice to see this progressing!
> >
> > Thanks
> >   j
> >
> <snip>

-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 09/15] media: bcm2835-unicam: Add support for CCP2/CSI2 camera interface

[Laurent Pinchart]

Hi Dave,

On Tue, Mar 05, 2024 at 02:56:29PM +0000, Dave Stevenson wrote:
> On Mon, 4 Mar 2024 at 19:51, Laurent Pinchart wrote:
> > On Mon, Mar 04, 2024 at 06:12:21PM +0100, Jacopo Mondi wrote:
> > > On Fri, Mar 01, 2024 at 11:32:24PM +0200, Laurent Pinchart wrote:
> > > > From: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > >
> > > > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > > > It is represented as two video device nodes: unicam-image and
> > > > unicam-embedded which are connected to an internal subdev (named
> > > > unicam-subdev) in order to manage streams routing.
> > > >
> > > > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > Co-developed-by: Naushir Patuck <naush@raspberrypi.com>
> > > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > > > Co-developed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > > > Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > > > Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > > ---
> > > > Changes since v5:
> > > >
> > > > - Move to drivers/media/platform/broadcom/
> > > > - Port to the upstream V4L2 streams API
> > > > - Rebase on latest metadata API proposal
> > > > - Add missing error message
> > > > - Drop unneeded documentation block for unicam_isr()
> > > > - Drop unneeded dev_dbg() and dev_err() messages
> > > > - Drop unneeded streams_mask and fmt checks
> > > > - Drop unused unicam_sd_pad_is_sink()
> > > > - Drop unneeded includes
> > > > - Drop v4l2_ctrl_subscribe_event() call
> > > > - Use pm_runtime_resume_and_get()
> > > > - Indentation and line wrap fixes
> > > > - Let the framework set bus_info
> > > > - Use v4l2_fwnode_endpoint_parse()
> > > > - Fix media device cleanup
> > > > - Drop lane reordering checks
> > > > - Fix subdev state locking
> > > > - Drop extra debug messages
> > > > - Move clock handling to runtime PM handlers
> > > > - Reorder functions
> > > > - Rename init functions for more clarity
> > > > - Initialize runtime PM earlier
> > > > - Clarify error messages
> > > > - Simplify subdev init with local variable
> > > > - Fix subdev cleanup
> > > > - Fix typos and indentation
> > > > - Don't initialize local variables needlessly
> > > > - Simplify num lanes check
> > > > - Fix metadata handling in subdev set_fmt
> > > > - Drop manual fallback to .s_stream()
> > > > - Pass v4l2_pix_format to unicam_calc_format_size_bpl()
> > > > - Simplify unicam_set_default_format()
> > > > - Fix default format settings
> > > > - Add busy check in unicam_s_fmt_meta()
> > > > - Add missing \n at end of format strings
> > > > - Fix metadata handling in subdev set_fmt
> > > > - Fix locking when starting streaming
> > > > - Return buffers from start streaming fails
> > > > - Fix format validation for metadata node
> > > > - Use video_device_pipeline_{start,stop}() helpers
> > > > - Simplify format enumeration
> > > > - Drop unset variable
> > > > - Update MAINTAINERS entry
> > > > - Update to the upstream v4l2_async_nf API
> > > > - Update to the latest subdev routing API
> > > > - Update to the latest subdev state API
> > > > - Move from subdev .init_cfg() to .init_state()
> > > > - Update to the latest videobuf2 API
> > > > - Fix v4l2_subdev_enable_streams() error check
> > > > - Use correct pad for the connected subdev
> > > > - Return buffers to vb2 when start streaming fails
> > > > - Improve debugging in start streaming handler
> > > > - Simplify DMA address management
> > > > - Drop comment about bcm2835-camera driver
> > > > - Clarify comments that explain min/max sizes
> > > > - Pass v4l2_pix_format to unicam_try_fmt()
> > > > - Drop unneeded local variables
> > > > - Rename image-related constants and functions
> > > > - Turn unicam_fmt.metadata_fmt into bool
> > > > - Rename unicam_fmt to unicam_format_info
> > > > - Rename unicam_format_info variables to fmtinfo
> > > > - Rename unicam_node.v_fmt to fmt
> > > > - Add metadata formats for RAW10, RAW12 and RAW14
> > > > - Make metadata formats line-based
> > > > - Validate format on metadata video device
> > > > - Add Co-devlopped-by tags
> > > >
> > > > Changes since v3:
> > > >
> > > > - Add the vendor prefix for DT name
> > > > - Use the reg-names in DT parsing
> > > > - Remove MAINTAINERS entry
> > > >
> > > > Changes since v2:
> > > >
> > > > - Change code organization
> > > > - Remove unused variables
> > > > - Correct the fmt_meta functions
> > > > - Rewrite the start/stop streaming
> > > >   - You can now start the image node alone, but not the metadata one
> > > >   - The buffers are allocated per-node
> > > >   - only the required stream is started, if the route exists and is
> > > >     enabled
> > > > - Prefix the macros with UNICAM_ to not have too generic names
> > > > - Drop colorspace support
> > > >
> > > > Changes since v1:
> > > >
> > > > - Replace the unicam_{info,debug,error} macros with dev_*()
> > > > ---
> > > >  MAINTAINERS                                   |    1 +
> > > >  drivers/media/platform/Kconfig                |    1 +
> > > >  drivers/media/platform/Makefile               |    1 +
> > > >  drivers/media/platform/broadcom/Kconfig       |   23 +
> > > >  drivers/media/platform/broadcom/Makefile      |    3 +
> > > >  .../platform/broadcom/bcm2835-unicam-regs.h   |  255 ++
> > > >  .../media/platform/broadcom/bcm2835-unicam.c  | 2607 +++++++++++++++++
> > > >  7 files changed, 2891 insertions(+)
> > > >  create mode 100644 drivers/media/platform/broadcom/Kconfig
> > > >  create mode 100644 drivers/media/platform/broadcom/Makefile
> > > >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > > >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam.c
> >
> > [snip]
> >
> > > > diff --git a/drivers/media/platform/broadcom/bcm2835-unicam.c b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > > > new file mode 100644
> > > > index 000000000000..716c89b8a217
> > > > --- /dev/null
> > > > +++ b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > > > @@ -0,0 +1,2607 @@

[snip]

> > > > +/* -----------------------------------------------------------------------------
> > > > + * Videobuf2 queue operations
> > > > + */
> > > > +
> > > > +static int unicam_queue_setup(struct vb2_queue *vq,
> > > > +                         unsigned int *nbuffers,
> > > > +                         unsigned int *nplanes,
> > > > +                         unsigned int sizes[],
> > > > +                         struct device *alloc_devs[])
> > > > +{
> > > > +   struct unicam_node *node = vb2_get_drv_priv(vq);
> > > > +   u32 size = is_image_node(node) ? node->fmt.fmt.pix.sizeimage
> > > > +            : node->fmt.fmt.meta.buffersize;
> > > > +
> > > > +   if (vq->num_buffers + *nbuffers < 3)
> > >
> > > Why 3 ? Are these dummies ?
> >
> > This may be a remnant of old code. Dave, Naush, any comment ?
> 
> I suspect this is legacy.
> Originally the driver would only release the buffer at the frame end
> when it had a new one to switch to. Naush updated with the dummy
> buffer so in theory you can run with 1 buffer, but this min number of
> buffers probably didn't get reduced.
> Then again it may have been a misunderstanding of the framework, as
> struct vb2_queue min_buffers_needed should set the minimum.

I'll drop this.

> > > > +           *nbuffers = 3 - vq->num_buffers;
> > > > +
> > > > +   if (*nplanes) {
> > > > +           if (sizes[0] < size) {
> > > > +                   dev_dbg(node->dev->dev, "sizes[0] %i < size %u\n",
> > > > +                           sizes[0], size);
> > > > +                   return -EINVAL;
> > > > +           }
> > > > +           size = sizes[0];
> > > > +   }
> > > > +
> > > > +   *nplanes = 1;
> > > > +   sizes[0] = size;
> > > > +
> > > > +   return 0;
> > > > +}

[snip]

> > > > +static int unicam_start_streaming(struct vb2_queue *vq, unsigned int count)
> > > > +{
> > > > +   struct unicam_node *node = vb2_get_drv_priv(vq);
> > > > +   struct unicam_device *unicam = node->dev;
> > > > +   struct v4l2_subdev_state *state;
> > > > +   struct unicam_buffer *buf;
> > > > +   unsigned long flags;
> > > > +   int ret;
> > > > +   u32 pad, stream;
> > > > +   u32 remote_pad = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
> > > > +                                        : UNICAM_SD_PAD_SOURCE_METADATA;
> > > > +
> > > > +   /* Look for the route for the given pad and stream. */
> > > > +   state = v4l2_subdev_lock_and_get_active_state(&unicam->subdev.sd);
> > > > +   ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
> > > > +                                               remote_pad, 0,
> > > > +                                               &pad, &stream);
> > > > +   v4l2_subdev_unlock_state(state);
> > > > +
> > > > +   if (ret)
> > > > +           goto err_return_buffers;
> > > > +
> > > > +   dev_dbg(unicam->dev, "Starting stream on %s: %u/%u -> %u/%u (%s)\n",
> > > > +           unicam->subdev.sd.name, pad, stream, remote_pad, 0,
> > > > +           is_metadata_node(node) ? "metadata" : "image");
> > > > +
> > > > +   /* The metadata node can't be started alone. */
> > > > +   if (is_metadata_node(node)) {
> > > > +           if (!unicam->node[UNICAM_IMAGE_NODE].streaming) {
> > >
> > > Does it mean the metadata node has to be started second, or should
> > > this be made a nop and the metadata node gets started once the image
> > > node is started too ? I'm fine with the constraint of having the
> > > metadata node being started second fwiw
> >
> > I think it would be nice to change this indeed. Dave, Naush, any
> > objection ?
> 
> See previous email.
> 
> > > > +                   dev_err(unicam->dev,
> > > > +                           "Can't start metadata without image\n");
> > > > +                   ret = -EINVAL;
> > > > +                   goto err_return_buffers;
> > > > +           }
> > > > +
> > > > +           spin_lock_irqsave(&node->dma_queue_lock, flags);
> > > > +           buf = list_first_entry(&node->dma_queue,
> > > > +                                  struct unicam_buffer, list);
> > > > +           dev_dbg(unicam->dev, "buffer %p\n", buf);
> > >
> > > Is this useful ?
> >
> > Probably not much. I'll drop it.
> >
> > > > +           node->cur_frm = buf;
> > > > +           node->next_frm = buf;
> > > > +           list_del(&buf->list);
> > > > +           spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> > > > +
> > > > +           unicam_start_metadata(unicam, buf);
> > > > +           node->streaming = true;
> > > > +           return 0;
> > > > +   }
> > > > +
> > > > +   ret = pm_runtime_resume_and_get(unicam->dev);
> > > > +   if (ret < 0) {
> > > > +           dev_err(unicam->dev, "PM runtime resume failed: %d\n", ret);
> > > > +           goto err_return_buffers;
> > > > +   }
> > > > +
> > > > +   ret = video_device_pipeline_start(&node->video_dev, &unicam->pipe);
> > > > +   if (ret < 0) {
> > > > +           dev_dbg(unicam->dev, "Failed to start media pipeline: %d\n", ret);
> > >
> > > Isn't this an err ?
> >
> > The main cause of failure is a pipeline validation error, triggered by
> > userspace, hence the debug level.
> >
> > > > +           goto err_pm_put;
> > > > +   }
> > > > +
> > > > +   spin_lock_irqsave(&node->dma_queue_lock, flags);
> > > > +   buf = list_first_entry(&node->dma_queue,
> > > > +                          struct unicam_buffer, list);
> > > > +   dev_dbg(unicam->dev, "buffer %p\n", buf);
> > > > +   node->cur_frm = buf;
> > > > +   node->next_frm = buf;
> > > > +   list_del(&buf->list);
> > > > +   spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> > > > +
> > > > +   unicam_start_rx(unicam, buf);
> > > > +
> > > > +   ret = v4l2_subdev_enable_streams(&unicam->subdev.sd, remote_pad, BIT(0));
> > >
> > > A bit confused by the API here, shouldn't we also do this for embedded
> > > data ?
> >
> > Not here, as the two streams go over different pads, but likely above,
> > as part of the change you proposed regarding stream start on the
> > metadata device. I'll wait for Dave and Naush to reply, and I'll rework
> > this function.
> 
> I haven't read enough on the streams API, or what this implementation
> looks like.
> 
> There's no sensible way for a sensor to start embedded or other
> metadata without image data, so starting the image node would seem to
> be the main trigger for anything. I'm also assuming we don't support
> enabling additional multiplexed streams once the pipeline is already
> running, so that would seem to determine some of the sequencing.

The API allows enabling and disabling streams independently, which is a
useful feature for instance when multiple cameras are multiplexed over a
single CSI-2 link with virtual channels. For image + embedded data, not
only is the sequencing fixed (as you mentioned, the sensor can't send
embedded data without image data), but synchronization is also
important.

> > > > +   if (ret < 0) {
> > > > +           dev_err(unicam->dev, "stream on failed in subdev\n");
> > > > +           goto error_pipeline;
> > > > +   }
> > > > +
> > > > +   node->streaming = true;
> > > > +
> > > > +   return 0;
> > > > +
> > > > +error_pipeline:
> > > > +   video_device_pipeline_stop(&node->video_dev);
> > > > +err_pm_put:
> > > > +   pm_runtime_put_sync(unicam->dev);
> > > > +err_return_buffers:
> > > > +   unicam_return_buffers(node, VB2_BUF_STATE_QUEUED);
> > > > +   return ret;
> > > > +}

[snip]

-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 12/15] ARM: dts: bcm2711-rpi: Add pinctrl-based multiplexing for I2C0

[Dave Stevenson]

Hi Laurent

On Fri, 1 Mar 2024 at 21:32, Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> From: Uwe Kleine-König <uwe@kleine-koenig.org>
>
> BCM2711-based Raspberry Pi boards (4B, CM4 and 400) multiplex the I2C0
> controller over two sets of pins, GPIO0+1 and GPIO44+45. The former is
> exposed on the 40-pin header, while the latter is used for the CSI and
> DSI connectors.

It's true for all Pis that I2C0 is exposed over 2 sets of gpios.
Seeing as we want to support cameras on Pi0-3, is there a reason not
to include the mux on those?

Looking back I had started this way back in [1] with all the variants.
I thought I'd posted the v2 follow up, but can't find it.
The original Pi 1 models A & B were the annoyances. The rev1 put the
camera on i2c1 GPIOs 2&3, with the rev2 on i2c0 with GPIOs 0&1.

Whilst it would be nice to have support for all platforms, this
doesn't stop us moving the mux into bcm283x-rpi.dtsi at a later date
to add support for the other devices.
I'm happy enough having the first step of getting Pi4 working, with
others being done later.

[1] https://linux-rpi-kernel.infradead.narkive.com/lmzYlT3c/rfc-arm-dts-add-i2cmux-pinctrl-config-to-raspberry-pi-i2c-0

> Add a pinctrl-based I2C bus multiplexer to bcm2711-rpi.dtsi to model
> this multiplexing. The two child buses are named i2c0_0 and i2c0_1.
>
> Note that if you modified the dts before to add devices to the i2c bus
> appearing on pins gpio0 + gpio1 (either directly in the dts or using an
> overlay), you have to put these into the i2c0_0 node introduced here
> now.
>
> Signed-off-by: Uwe Kleine-König <uwe@kleine-koenig.org>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
> Changes since v3:
>
> - Split addition of the RTC to a separate patch
> - Move the mux to bcm2711-rpi.dtsi
> ---
>  arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi | 31 +++++++++++++++++++++
>  1 file changed, 31 insertions(+)
>
> diff --git a/arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi b/arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi
> index 86188eabeb24..826ed6efa9ff 100644
> --- a/arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi
> +++ b/arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi
> @@ -17,6 +17,32 @@ aliases {
>                 pcie0 = &pcie0;
>                 blconfig = &blconfig;
>         };
> +
> +       i2c0mux: i2c0mux {
> +               compatible = "i2c-mux-pinctrl";
> +               #address-cells = <1>;
> +               #size-cells = <0>;
> +
> +               i2c-parent = <&i2c0>;
> +
> +               pinctrl-names = "i2c0", "i2c0-vc";
> +               pinctrl-0 = <&i2c0_gpio0>;
> +               pinctrl-1 = <&i2c0_gpio44>;
> +
> +               status = "disabled";

Why defaulting to disabled?

The current mainline DT defaults to i2c0 being enabled on GPIOs 0&1
(done via bcm2835-rpi.dtsi).
If the mux is disabled, then this change has left i2c0 being enabled
but with no pinctrl property, so it's not connected to the outside
world.
GPIOs 44&45 have never had any other user, therefore claiming them for
the mux isn't a regression in my view.


As long as we can enable the other platforms later, and with the minor
caveat over being enabled or not:

Acked-by: Dave Stevenson <dave.stevenson@raspberrypi.com>

Minor point that CONFIG_I2C_MUX_PINCTRL appears not to be in the arm64
defconfig. I don't know what the policy is there, but there seem to be
many other SoCs throwing modules in there for their configurations.
It is in arm/multi_v7_defconfig.

  Dave

> +
> +               i2c0_0: i2c@0 {
> +                       reg = <0>;
> +                       #address-cells = <1>;
> +                       #size-cells = <0>;
> +               };
> +
> +               i2c0_1: i2c@1 {
> +                       reg = <1>;
> +                       #address-cells = <1>;
> +                       #size-cells = <0>;
> +               };
> +       };
>  };
>
>  &firmware {
> @@ -49,6 +75,11 @@ &hvs {
>         clocks = <&firmware_clocks 4>;
>  };
>
> +&i2c0 {
> +       /delete-property/ pinctrl-names;
> +       /delete-property/ pinctrl-0;
> +};
> +
>  &rmem {
>         /*
>          * RPi4's co-processor will copy the board's bootloader configuration
> --
> Regards,
>
> Laurent Pinchart
>

Re: [PATCH v6 13/15] ARM: dts: bcm2711-rpi-cm4-io: Add RTC on I2C0

[Dave Stevenson]

On Fri, 1 Mar 2024 at 21:32, Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> From: Uwe Kleine-König <uwe@kleine-koenig.org>
>
> The cm4-io board comes with a PCF85063 on I2C0, connected to the GPIO44
> and GPIO45 pins. Add it to the device tree.
>
> Signed-off-by: Uwe Kleine-König <uwe@kleine-koenig.org>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
> Changes since v4:
>
> - Use the right part number in the compatible string
> - Add the quartz-load-femtofarads property
>
> Changes since v3:
>
> - Separate addition of the RTC to a patch of its own
> ---
>  .../boot/dts/broadcom/bcm2711-rpi-cm4-io.dts    | 17 +++++++++++++++++
>  1 file changed, 17 insertions(+)
>
> diff --git a/arch/arm/boot/dts/broadcom/bcm2711-rpi-cm4-io.dts b/arch/arm/boot/dts/broadcom/bcm2711-rpi-cm4-io.dts
> index d7ba02f586d3..d5ebb535afd2 100644
> --- a/arch/arm/boot/dts/broadcom/bcm2711-rpi-cm4-io.dts
> +++ b/arch/arm/boot/dts/broadcom/bcm2711-rpi-cm4-io.dts
> @@ -101,6 +101,23 @@ &genet {
>         status = "okay";
>  };
>
> +&i2c0 {
> +       status = "okay";
> +};

This is already the default from bcm2835-rpi.dtsi

> +
> +&i2c0_1 {
> +       rtc@51 {
> +               /* Attention: An alarm resets the machine */
> +               compatible = "nxp,pcf85063a";
> +               reg = <0x51>;
> +               quartz-load-femtofarads = <7000>;
> +       };
> +};
> +
> +&i2c0mux {
> +       status = "okay";

Default status queried on patch 12/15.

With those clarified:

Acked-by: Dave Stevenson <dave.stevenson@raspberrypi.com>

> +};
> +
>  &led_act {
>         gpios = <&gpio 42 GPIO_ACTIVE_HIGH>;
>  };
> --
> Regards,
>
> Laurent Pinchart
>

Re: [PATCH v6 13/15] ARM: dts: bcm2711-rpi-cm4-io: Add RTC on I2C0

[Laurent Pinchart]

Hi Dave,

On Mon, Mar 18, 2024 at 02:56:53PM +0000, Dave Stevenson wrote:
> On Fri, 1 Mar 2024 at 21:32, Laurent Pinchart wrote:
> >
> > From: Uwe Kleine-König <uwe@kleine-koenig.org>
> >
> > The cm4-io board comes with a PCF85063 on I2C0, connected to the GPIO44
> > and GPIO45 pins. Add it to the device tree.
> >
> > Signed-off-by: Uwe Kleine-König <uwe@kleine-koenig.org>
> > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > ---
> > Changes since v4:
> >
> > - Use the right part number in the compatible string
> > - Add the quartz-load-femtofarads property
> >
> > Changes since v3:
> >
> > - Separate addition of the RTC to a patch of its own
> > ---
> >  .../boot/dts/broadcom/bcm2711-rpi-cm4-io.dts    | 17 +++++++++++++++++
> >  1 file changed, 17 insertions(+)
> >
> > diff --git a/arch/arm/boot/dts/broadcom/bcm2711-rpi-cm4-io.dts b/arch/arm/boot/dts/broadcom/bcm2711-rpi-cm4-io.dts
> > index d7ba02f586d3..d5ebb535afd2 100644
> > --- a/arch/arm/boot/dts/broadcom/bcm2711-rpi-cm4-io.dts
> > +++ b/arch/arm/boot/dts/broadcom/bcm2711-rpi-cm4-io.dts
> > @@ -101,6 +101,23 @@ &genet {
> >         status = "okay";
> >  };
> >
> > +&i2c0 {
> > +       status = "okay";
> > +};
> 
> This is already the default from bcm2835-rpi.dtsi

Indeed. I probably got lost in the maze of .dtsi files :-) I'll drop
this.

> > +
> > +&i2c0_1 {
> > +       rtc@51 {
> > +               /* Attention: An alarm resets the machine */
> > +               compatible = "nxp,pcf85063a";
> > +               reg = <0x51>;
> > +               quartz-load-femtofarads = <7000>;
> > +       };
> > +};
> > +
> > +&i2c0mux {
> > +       status = "okay";
> 
> Default status queried on patch 12/15.
> 
> With those clarified:
> 
> Acked-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> 
> > +};
> > +
> >  &led_act {
> >         gpios = <&gpio 42 GPIO_ACTIVE_HIGH>;
> >  };

-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 12/15] ARM: dts: bcm2711-rpi: Add pinctrl-based multiplexing for I2C0

[Laurent Pinchart]

Hi Dave,

On Mon, Mar 18, 2024 at 02:56:33PM +0000, Dave Stevenson wrote:
> On Fri, 1 Mar 2024 at 21:32, Laurent Pinchart wrote:
> >
> > From: Uwe Kleine-König <uwe@kleine-koenig.org>
> >
> > BCM2711-based Raspberry Pi boards (4B, CM4 and 400) multiplex the I2C0
> > controller over two sets of pins, GPIO0+1 and GPIO44+45. The former is
> > exposed on the 40-pin header, while the latter is used for the CSI and
> > DSI connectors.
> 
> It's true for all Pis that I2C0 is exposed over 2 sets of gpios.
> Seeing as we want to support cameras on Pi0-3, is there a reason not
> to include the mux on those?

Simplicity :-) I got lost in the maze of differences in .dtsi files
between the upstream and downstream kernels. Given that not all Pi's
have device trees upstream, I decided to start simple(r).

> Looking back I had started this way back in [1] with all the variants.
> I thought I'd posted the v2 follow up, but can't find it.
> The original Pi 1 models A & B were the annoyances. The rev1 put the
> camera on i2c1 GPIOs 2&3, with the rev2 on i2c0 with GPIOs 0&1.
> 
> Whilst it would be nice to have support for all platforms, this
> doesn't stop us moving the mux into bcm283x-rpi.dtsi at a later date
> to add support for the other devices.
> I'm happy enough having the first step of getting Pi4 working, with
> others being done later.

Thanks :-) I would also be happy for other boards to get I2C0 mux
support later.

> [1] https://linux-rpi-kernel.infradead.narkive.com/lmzYlT3c/rfc-arm-dts-add-i2cmux-pinctrl-config-to-raspberry-pi-i2c-0
> 
> > Add a pinctrl-based I2C bus multiplexer to bcm2711-rpi.dtsi to model
> > this multiplexing. The two child buses are named i2c0_0 and i2c0_1.
> >
> > Note that if you modified the dts before to add devices to the i2c bus
> > appearing on pins gpio0 + gpio1 (either directly in the dts or using an
> > overlay), you have to put these into the i2c0_0 node introduced here
> > now.
> >
> > Signed-off-by: Uwe Kleine-König <uwe@kleine-koenig.org>
> > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > ---
> > Changes since v3:
> >
> > - Split addition of the RTC to a separate patch
> > - Move the mux to bcm2711-rpi.dtsi
> > ---
> >  arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi | 31 +++++++++++++++++++++
> >  1 file changed, 31 insertions(+)
> >
> > diff --git a/arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi b/arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi
> > index 86188eabeb24..826ed6efa9ff 100644
> > --- a/arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi
> > +++ b/arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi
> > @@ -17,6 +17,32 @@ aliases {
> >                 pcie0 = &pcie0;
> >                 blconfig = &blconfig;
> >         };
> > +
> > +       i2c0mux: i2c0mux {
> > +               compatible = "i2c-mux-pinctrl";
> > +               #address-cells = <1>;
> > +               #size-cells = <0>;
> > +
> > +               i2c-parent = <&i2c0>;
> > +
> > +               pinctrl-names = "i2c0", "i2c0-vc";
> > +               pinctrl-0 = <&i2c0_gpio0>;
> > +               pinctrl-1 = <&i2c0_gpio44>;
> > +
> > +               status = "disabled";
> 
> Why defaulting to disabled?
> 
> The current mainline DT defaults to i2c0 being enabled on GPIOs 0&1
> (done via bcm2835-rpi.dtsi).
> If the mux is disabled, then this change has left i2c0 being enabled
> but with no pinctrl property, so it's not connected to the outside
> world.
> GPIOs 44&45 have never had any other user, therefore claiming them for
> the mux isn't a regression in my view.

I don't recall why I disabled it. Your explanation makes sense, I'll
drop the status property.

> As long as we can enable the other platforms later, and with the minor
> caveat over being enabled or not:
> 
> Acked-by: Dave Stevenson <dave.stevenson@raspberrypi.com>

Thank you. I'll send a new version of the series soon, Florian wanted to
pick the DT integration sooner than later.

> Minor point that CONFIG_I2C_MUX_PINCTRL appears not to be in the arm64
> defconfig. I don't know what the policy is there, but there seem to be
> many other SoCs throwing modules in there for their configurations.
> It is in arm/multi_v7_defconfig.

Good question.

> > +
> > +               i2c0_0: i2c@0 {
> > +                       reg = <0>;
> > +                       #address-cells = <1>;
> > +                       #size-cells = <0>;
> > +               };
> > +
> > +               i2c0_1: i2c@1 {
> > +                       reg = <1>;
> > +                       #address-cells = <1>;
> > +                       #size-cells = <0>;
> > +               };
> > +       };
> >  };
> >
> >  &firmware {
> > @@ -49,6 +75,11 @@ &hvs {
> >         clocks = <&firmware_clocks 4>;
> >  };
> >
> > +&i2c0 {
> > +       /delete-property/ pinctrl-names;
> > +       /delete-property/ pinctrl-0;
> > +};
> > +
> >  &rmem {
> >         /*
> >          * RPi4's co-processor will copy the board's bootloader configuration

-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 09/15] media: bcm2835-unicam: Add support for CCP2/CSI2 camera interface

[Naushir Patuck]

Hi Laurent,

Thank you for submitting this!

On Fri, 1 Mar 2024 at 21:32, Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> From: Dave Stevenson <dave.stevenson@raspberrypi.com>
>
> Add a driver for the Unicam camera receiver block on BCM283x processors.
> It is represented as two video device nodes: unicam-image and
> unicam-embedded which are connected to an internal subdev (named
> unicam-subdev) in order to manage streams routing.
>
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> Co-developed-by: Naushir Patuck <naush@raspberrypi.com>
> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> Co-developed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
> Changes since v5:
>
> - Move to drivers/media/platform/broadcom/
> - Port to the upstream V4L2 streams API
> - Rebase on latest metadata API proposal
> - Add missing error message
> - Drop unneeded documentation block for unicam_isr()
> - Drop unneeded dev_dbg() and dev_err() messages
> - Drop unneeded streams_mask and fmt checks
> - Drop unused unicam_sd_pad_is_sink()
> - Drop unneeded includes
> - Drop v4l2_ctrl_subscribe_event() call
> - Use pm_runtime_resume_and_get()
> - Indentation and line wrap fixes
> - Let the framework set bus_info
> - Use v4l2_fwnode_endpoint_parse()
> - Fix media device cleanup
> - Drop lane reordering checks
> - Fix subdev state locking
> - Drop extra debug messages
> - Move clock handling to runtime PM handlers
> - Reorder functions
> - Rename init functions for more clarity
> - Initialize runtime PM earlier
> - Clarify error messages
> - Simplify subdev init with local variable
> - Fix subdev cleanup
> - Fix typos and indentation
> - Don't initialize local variables needlessly
> - Simplify num lanes check
> - Fix metadata handling in subdev set_fmt
> - Drop manual fallback to .s_stream()
> - Pass v4l2_pix_format to unicam_calc_format_size_bpl()
> - Simplify unicam_set_default_format()
> - Fix default format settings
> - Add busy check in unicam_s_fmt_meta()
> - Add missing \n at end of format strings
> - Fix metadata handling in subdev set_fmt
> - Fix locking when starting streaming
> - Return buffers from start streaming fails
> - Fix format validation for metadata node
> - Use video_device_pipeline_{start,stop}() helpers
> - Simplify format enumeration
> - Drop unset variable
> - Update MAINTAINERS entry
> - Update to the upstream v4l2_async_nf API
> - Update to the latest subdev routing API
> - Update to the latest subdev state API
> - Move from subdev .init_cfg() to .init_state()
> - Update to the latest videobuf2 API
> - Fix v4l2_subdev_enable_streams() error check
> - Use correct pad for the connected subdev
> - Return buffers to vb2 when start streaming fails
> - Improve debugging in start streaming handler
> - Simplify DMA address management
> - Drop comment about bcm2835-camera driver
> - Clarify comments that explain min/max sizes
> - Pass v4l2_pix_format to unicam_try_fmt()
> - Drop unneeded local variables
> - Rename image-related constants and functions
> - Turn unicam_fmt.metadata_fmt into bool
> - Rename unicam_fmt to unicam_format_info
> - Rename unicam_format_info variables to fmtinfo
> - Rename unicam_node.v_fmt to fmt
> - Add metadata formats for RAW10, RAW12 and RAW14
> - Make metadata formats line-based
> - Validate format on metadata video device
> - Add Co-devlopped-by tags
>
> Changes since v3:
>
> - Add the vendor prefix for DT name
> - Use the reg-names in DT parsing
> - Remove MAINTAINERS entry
>
> Changes since v2:
>
> - Change code organization
> - Remove unused variables
> - Correct the fmt_meta functions
> - Rewrite the start/stop streaming
>   - You can now start the image node alone, but not the metadata one
>   - The buffers are allocated per-node
>   - only the required stream is started, if the route exists and is
>     enabled
> - Prefix the macros with UNICAM_ to not have too generic names
> - Drop colorspace support
>
> Changes since v1:
>
> - Replace the unicam_{info,debug,error} macros with dev_*()
> ---
>  MAINTAINERS                                   |    1 +
>  drivers/media/platform/Kconfig                |    1 +
>  drivers/media/platform/Makefile               |    1 +
>  drivers/media/platform/broadcom/Kconfig       |   23 +
>  drivers/media/platform/broadcom/Makefile      |    3 +
>  .../platform/broadcom/bcm2835-unicam-regs.h   |  255 ++
>  .../media/platform/broadcom/bcm2835-unicam.c  | 2607 +++++++++++++++++
>  7 files changed, 2891 insertions(+)
>  create mode 100644 drivers/media/platform/broadcom/Kconfig
>  create mode 100644 drivers/media/platform/broadcom/Makefile
>  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam-regs.h
>  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam.c
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index e50a59654e6e..cc350729f467 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -4002,6 +4002,7 @@ M:        Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
>  L:     linux-media@vger.kernel.org
>  S:     Maintained
>  F:     Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> +F:     drivers/media/platform/bcm2835/
>
>  BROADCOM BCM47XX MIPS ARCHITECTURE
>  M:     Hauke Mehrtens <hauke@hauke-m.de>
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index 91e54215de3a..2d79bfc68c15 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -67,6 +67,7 @@ source "drivers/media/platform/amlogic/Kconfig"
>  source "drivers/media/platform/amphion/Kconfig"
>  source "drivers/media/platform/aspeed/Kconfig"
>  source "drivers/media/platform/atmel/Kconfig"
> +source "drivers/media/platform/broadcom/Kconfig"
>  source "drivers/media/platform/cadence/Kconfig"
>  source "drivers/media/platform/chips-media/Kconfig"
>  source "drivers/media/platform/intel/Kconfig"
> diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> index 3296ec1ebe16..da17301f7439 100644
> --- a/drivers/media/platform/Makefile
> +++ b/drivers/media/platform/Makefile
> @@ -10,6 +10,7 @@ obj-y += amlogic/
>  obj-y += amphion/
>  obj-y += aspeed/
>  obj-y += atmel/
> +obj-y += broadcom/
>  obj-y += cadence/
>  obj-y += chips-media/
>  obj-y += intel/
> diff --git a/drivers/media/platform/broadcom/Kconfig b/drivers/media/platform/broadcom/Kconfig
> new file mode 100644
> index 000000000000..cc2c9afcc948
> --- /dev/null
> +++ b/drivers/media/platform/broadcom/Kconfig
> @@ -0,0 +1,23 @@
> +# SPDX-License-Identifier: GPL-2.0
> +
> +config VIDEO_BCM2835_UNICAM
> +       tristate "Broadcom BCM283x/BCM271x Unicam video capture driver"
> +       depends on ARCH_BCM2835 || COMPILE_TEST
> +       depends on PM
> +       depends on VIDEO_DEV
> +       select MEDIA_CONTROLLER
> +       select V4L2_FWNODE
> +       select VIDEO_V4L2_SUBDEV_API
> +       select VIDEOBUF2_DMA_CONTIG
> +       help
> +         Say Y here to enable support for the BCM283x/BCM271x CSI-2 receiver.
> +         This is a V4L2 driver that controls the CSI-2 receiver directly,
> +         independently from the VC4 firmware.
> +
> +         This driver is mutually exclusive with the use of bcm2835-camera. The
> +         firmware will disable all access to the peripheral from within the
> +         firmware if it finds a DT node using it, and bcm2835-camera will
> +         therefore fail to probe.
> +
> +         To compile this driver as a module, choose M here. The module will be
> +         called bcm2835-unicam.
> diff --git a/drivers/media/platform/broadcom/Makefile b/drivers/media/platform/broadcom/Makefile
> new file mode 100644
> index 000000000000..03d2045aba2e
> --- /dev/null
> +++ b/drivers/media/platform/broadcom/Makefile
> @@ -0,0 +1,3 @@
> +# SPDX-License-Identifier: GPL-2.0
> +
> +obj-$(CONFIG_VIDEO_BCM2835_UNICAM) += bcm2835-unicam.o
> diff --git a/drivers/media/platform/broadcom/bcm2835-unicam-regs.h b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> new file mode 100644
> index 000000000000..84775fd2fac5
> --- /dev/null
> +++ b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> @@ -0,0 +1,255 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +
> +/*
> + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> + */
> +
> +#ifndef VC4_REGS_UNICAM_H
> +#define VC4_REGS_UNICAM_H
> +
> +#include <linux/bits.h>
> +
> +/*
> + * The following values are taken from files found within the code drop
> + * made by Broadcom for the BCM21553 Graphics Driver, predominantly in
> + * brcm_usrlib/dag/vmcsx/vcinclude/hardware_vc4.h.
> + * They have been modified to be only the register offset.
> + */
> +#define UNICAM_CTRL            0x000
> +#define UNICAM_STA             0x004
> +#define UNICAM_ANA             0x008
> +#define UNICAM_PRI             0x00c
> +#define UNICAM_CLK             0x010
> +#define UNICAM_CLT             0x014
> +#define UNICAM_DAT0            0x018
> +#define UNICAM_DAT1            0x01c
> +#define UNICAM_DAT2            0x020
> +#define UNICAM_DAT3            0x024
> +#define UNICAM_DLT             0x028
> +#define UNICAM_CMP0            0x02c
> +#define UNICAM_CMP1            0x030
> +#define UNICAM_CAP0            0x034
> +#define UNICAM_CAP1            0x038
> +#define UNICAM_ICTL            0x100
> +#define UNICAM_ISTA            0x104
> +#define UNICAM_IDI0            0x108
> +#define UNICAM_IPIPE           0x10c
> +#define UNICAM_IBSA0           0x110
> +#define UNICAM_IBEA0           0x114
> +#define UNICAM_IBLS            0x118
> +#define UNICAM_IBWP            0x11c
> +#define UNICAM_IHWIN           0x120
> +#define UNICAM_IHSTA           0x124
> +#define UNICAM_IVWIN           0x128
> +#define UNICAM_IVSTA           0x12c
> +#define UNICAM_ICC             0x130
> +#define UNICAM_ICS             0x134
> +#define UNICAM_IDC             0x138
> +#define UNICAM_IDPO            0x13c
> +#define UNICAM_IDCA            0x140
> +#define UNICAM_IDCD            0x144
> +#define UNICAM_IDS             0x148
> +#define UNICAM_DCS             0x200
> +#define UNICAM_DBSA0           0x204
> +#define UNICAM_DBEA0           0x208
> +#define UNICAM_DBWP            0x20c
> +#define UNICAM_DBCTL           0x300
> +#define UNICAM_IBSA1           0x304
> +#define UNICAM_IBEA1           0x308
> +#define UNICAM_IDI1            0x30c
> +#define UNICAM_DBSA1           0x310
> +#define UNICAM_DBEA1           0x314
> +#define UNICAM_MISC            0x400
> +
> +/*
> + * The following bitmasks are from the kernel released by Broadcom
> + * for Android - https://android.googlesource.com/kernel/bcm/
> + * The Rhea, Hawaii, and Java chips all contain the same VideoCore4
> + * Unicam block as BCM2835, as defined in eg
> + * arch/arm/mach-rhea/include/mach/rdb_A0/brcm_rdb_cam.h and similar.
> + * Values reworked to use the kernel BIT and GENMASK macros.
> + *
> + * Some of the bit mnenomics have been amended to match the datasheet.
> + */
> +/* UNICAM_CTRL Register */
> +#define UNICAM_CPE             BIT(0)
> +#define UNICAM_MEM             BIT(1)
> +#define UNICAM_CPR             BIT(2)
> +#define UNICAM_CPM_MASK                GENMASK(3, 3)
> +#define UNICAM_CPM_CSI2                0
> +#define UNICAM_CPM_CCP2                1
> +#define UNICAM_SOE             BIT(4)
> +#define UNICAM_DCM_MASK                GENMASK(5, 5)
> +#define UNICAM_DCM_STROBE      0
> +#define UNICAM_DCM_DATA                1
> +#define UNICAM_SLS             BIT(6)
> +#define UNICAM_PFT_MASK                GENMASK(11, 8)
> +#define UNICAM_OET_MASK                GENMASK(20, 12)
> +
> +/* UNICAM_STA Register */
> +#define UNICAM_SYN             BIT(0)
> +#define UNICAM_CS              BIT(1)
> +#define UNICAM_SBE             BIT(2)
> +#define UNICAM_PBE             BIT(3)
> +#define UNICAM_HOE             BIT(4)
> +#define UNICAM_PLE             BIT(5)
> +#define UNICAM_SSC             BIT(6)
> +#define UNICAM_CRCE            BIT(7)
> +#define UNICAM_OES             BIT(8)
> +#define UNICAM_IFO             BIT(9)
> +#define UNICAM_OFO             BIT(10)
> +#define UNICAM_BFO             BIT(11)
> +#define UNICAM_DL              BIT(12)
> +#define UNICAM_PS              BIT(13)
> +#define UNICAM_IS              BIT(14)
> +#define UNICAM_PI0             BIT(15)
> +#define UNICAM_PI1             BIT(16)
> +#define UNICAM_FSI_S           BIT(17)
> +#define UNICAM_FEI_S           BIT(18)
> +#define UNICAM_LCI_S           BIT(19)
> +#define UNICAM_BUF0_RDY                BIT(20)
> +#define UNICAM_BUF0_NO         BIT(21)
> +#define UNICAM_BUF1_RDY                BIT(22)
> +#define UNICAM_BUF1_NO         BIT(23)
> +#define UNICAM_DI              BIT(24)
> +
> +#define UNICAM_STA_MASK_ALL \
> +               (UNICAM_DL | \
> +               UNICAM_SBE | \
> +               UNICAM_PBE | \
> +               UNICAM_HOE | \
> +               UNICAM_PLE | \
> +               UNICAM_SSC | \
> +               UNICAM_CRCE | \
> +               UNICAM_IFO | \
> +               UNICAM_OFO | \
> +               UNICAM_PS | \
> +               UNICAM_PI0 | \
> +               UNICAM_PI1)
> +
> +/* UNICAM_ANA Register */
> +#define UNICAM_APD             BIT(0)
> +#define UNICAM_BPD             BIT(1)
> +#define UNICAM_AR              BIT(2)
> +#define UNICAM_DDL             BIT(3)
> +#define UNICAM_CTATADJ_MASK    GENMASK(7, 4)
> +#define UNICAM_PTATADJ_MASK    GENMASK(11, 8)
> +
> +/* UNICAM_PRI Register */
> +#define UNICAM_PE              BIT(0)
> +#define UNICAM_PT_MASK         GENMASK(2, 1)
> +#define UNICAM_NP_MASK         GENMASK(7, 4)
> +#define UNICAM_PP_MASK         GENMASK(11, 8)
> +#define UNICAM_BS_MASK         GENMASK(15, 12)
> +#define UNICAM_BL_MASK         GENMASK(17, 16)
> +
> +/* UNICAM_CLK Register */
> +#define UNICAM_CLE             BIT(0)
> +#define UNICAM_CLPD            BIT(1)
> +#define UNICAM_CLLPE           BIT(2)
> +#define UNICAM_CLHSE           BIT(3)
> +#define UNICAM_CLTRE           BIT(4)
> +#define UNICAM_CLAC_MASK       GENMASK(8, 5)
> +#define UNICAM_CLSTE           BIT(29)
> +
> +/* UNICAM_CLT Register */
> +#define UNICAM_CLT1_MASK       GENMASK(7, 0)
> +#define UNICAM_CLT2_MASK       GENMASK(15, 8)
> +
> +/* UNICAM_DATn Registers */
> +#define UNICAM_DLE             BIT(0)
> +#define UNICAM_DLPD            BIT(1)
> +#define UNICAM_DLLPE           BIT(2)
> +#define UNICAM_DLHSE           BIT(3)
> +#define UNICAM_DLTRE           BIT(4)
> +#define UNICAM_DLSM            BIT(5)
> +#define UNICAM_DLFO            BIT(28)
> +#define UNICAM_DLSTE           BIT(29)
> +
> +#define UNICAM_DAT_MASK_ALL    (UNICAM_DLSTE | UNICAM_DLFO)
> +
> +/* UNICAM_DLT Register */
> +#define UNICAM_DLT1_MASK       GENMASK(7, 0)
> +#define UNICAM_DLT2_MASK       GENMASK(15, 8)
> +#define UNICAM_DLT3_MASK       GENMASK(23, 16)
> +
> +/* UNICAM_ICTL Register */
> +#define UNICAM_FSIE            BIT(0)
> +#define UNICAM_FEIE            BIT(1)
> +#define UNICAM_IBOB            BIT(2)
> +#define UNICAM_FCM             BIT(3)
> +#define UNICAM_TFC             BIT(4)
> +#define UNICAM_LIP_MASK                GENMASK(6, 5)
> +#define UNICAM_LCIE_MASK       GENMASK(28, 16)
> +
> +/* UNICAM_IDI0/1 Register */
> +#define UNICAM_ID0_MASK                GENMASK(7, 0)
> +#define UNICAM_ID1_MASK                GENMASK(15, 8)
> +#define UNICAM_ID2_MASK                GENMASK(23, 16)
> +#define UNICAM_ID3_MASK                GENMASK(31, 24)
> +
> +/* UNICAM_ISTA Register */
> +#define UNICAM_FSI             BIT(0)
> +#define UNICAM_FEI             BIT(1)
> +#define UNICAM_LCI             BIT(2)
> +
> +#define UNICAM_ISTA_MASK_ALL   (UNICAM_FSI | UNICAM_FEI | UNICAM_LCI)
> +
> +/* UNICAM_IPIPE Register */
> +#define UNICAM_PUM_MASK                GENMASK(2, 0)
> +/* Unpacking modes */
> +#define UNICAM_PUM_NONE                0
> +#define UNICAM_PUM_UNPACK6     1
> +#define UNICAM_PUM_UNPACK7     2
> +#define UNICAM_PUM_UNPACK8     3
> +#define UNICAM_PUM_UNPACK10    4
> +#define UNICAM_PUM_UNPACK12    5
> +#define UNICAM_PUM_UNPACK14    6
> +#define UNICAM_PUM_UNPACK16    7
> +#define UNICAM_DDM_MASK                GENMASK(6, 3)
> +#define UNICAM_PPM_MASK                GENMASK(9, 7)
> +/* Packing modes */
> +#define UNICAM_PPM_NONE                0
> +#define UNICAM_PPM_PACK8       1
> +#define UNICAM_PPM_PACK10      2
> +#define UNICAM_PPM_PACK12      3
> +#define UNICAM_PPM_PACK14      4
> +#define UNICAM_PPM_PACK16      5
> +#define UNICAM_DEM_MASK                GENMASK(11, 10)
> +#define UNICAM_DEBL_MASK       GENMASK(14, 12)
> +#define UNICAM_ICM_MASK                GENMASK(16, 15)
> +#define UNICAM_IDM_MASK                GENMASK(17, 17)
> +
> +/* UNICAM_ICC Register */
> +#define UNICAM_ICFL_MASK       GENMASK(4, 0)
> +#define UNICAM_ICFH_MASK       GENMASK(9, 5)
> +#define UNICAM_ICST_MASK       GENMASK(12, 10)
> +#define UNICAM_ICLT_MASK       GENMASK(15, 13)
> +#define UNICAM_ICLL_MASK       GENMASK(31, 16)
> +
> +/* UNICAM_DCS Register */
> +#define UNICAM_DIE             BIT(0)
> +#define UNICAM_DIM             BIT(1)
> +#define UNICAM_DBOB            BIT(3)
> +#define UNICAM_FDE             BIT(4)
> +#define UNICAM_LDP             BIT(5)
> +#define UNICAM_EDL_MASK                GENMASK(15, 8)
> +
> +/* UNICAM_DBCTL Register */
> +#define UNICAM_DBEN            BIT(0)
> +#define UNICAM_BUF0_IE         BIT(1)
> +#define UNICAM_BUF1_IE         BIT(2)
> +
> +/* UNICAM_CMP[0,1] register */
> +#define UNICAM_PCE             BIT(31)
> +#define UNICAM_GI              BIT(9)
> +#define UNICAM_CPH             BIT(8)
> +#define UNICAM_PCVC_MASK       GENMASK(7, 6)
> +#define UNICAM_PCDT_MASK       GENMASK(5, 0)
> +
> +/* UNICAM_MISC register */
> +#define UNICAM_FL0             BIT(6)
> +#define UNICAM_FL1             BIT(9)
> +
> +#endif
> diff --git a/drivers/media/platform/broadcom/bcm2835-unicam.c b/drivers/media/platform/broadcom/bcm2835-unicam.c
> new file mode 100644
> index 000000000000..716c89b8a217
> --- /dev/null
> +++ b/drivers/media/platform/broadcom/bcm2835-unicam.c
> @@ -0,0 +1,2607 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * BCM283x / BCM271x Unicam Capture Driver
> + *
> + * Copyright (C) 2017-2020 - Raspberry Pi (Trading) Ltd.
> + *
> + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> + *
> + * Based on TI am437x driver by
> + *   Benoit Parrot <bparrot@ti.com>
> + *   Lad, Prabhakar <prabhakar.csengg@gmail.com>
> + *
> + * and TI CAL camera interface driver by
> + *    Benoit Parrot <bparrot@ti.com>
> + *
> + *
> + * There are two camera drivers in the kernel for BCM283x - this one and
> + * bcm2835-camera (currently in staging).
> + *
> + * This driver directly controls the Unicam peripheral - there is no
> + * involvement with the VideoCore firmware. Unicam receives CSI-2 or CCP2 data
> + * and writes it into SDRAM. The only potential processing options are to
> + * repack Bayer data into an alternate format, and applying windowing. The
> + * repacking does not shift the data, so can repack V4L2_PIX_FMT_Sxxxx10P to
> + * V4L2_PIX_FMT_Sxxxx10, or V4L2_PIX_FMT_Sxxxx12P to V4L2_PIX_FMT_Sxxxx12, but
> + * not generically up to V4L2_PIX_FMT_Sxxxx16. The driver will add both formats
> + * where the relevant formats are defined, and will automatically configure the
> + * repacking as required. Support for windowing may be added later.
> + *
> + * It should be possible to connect this driver to any sensor with a suitable
> + * output interface and V4L2 subdevice driver.
> + */
> +
> +#include <linux/clk.h>
> +#include <linux/delay.h>
> +#include <linux/device.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/err.h>
> +#include <linux/interrupt.h>
> +#include <linux/io.h>
> +#include <linux/module.h>
> +#include <linux/of.h>
> +#include <linux/of_device.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/slab.h>
> +#include <linux/videodev2.h>
> +
> +#include <media/v4l2-async.h>
> +#include <media/v4l2-common.h>
> +#include <media/v4l2-dev.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/v4l2-fwnode.h>
> +#include <media/v4l2-mc.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "bcm2835-unicam-regs.h"
> +
> +#define UNICAM_MODULE_NAME             "unicam"
> +
> +/*
> + * Unicam must request a minimum of 250Mhz from the VPU clock.
> + * Otherwise the input FIFOs overrun and cause image corruption.
> + */
> +#define UNICAM_MIN_VPU_CLOCK_RATE      (250 * 1000 * 1000)
> +
> +/* Unicam has an internal DMA alignment constraint of 16 bytes for each line. */
> +#define UNICAM_DMA_BPL_ALIGNMENT       16
> +
> +/*
> + * The image stride is stored in a 16 bit register, and needs to be aligned to
> + * the DMA constraint. As the ISP in the same SoC has a 32 bytes alignment
> + * constraint on its input, set the image stride alignment to 32 bytes here as
> + * well to avoid incompatible configurations.
> + */
> +#define UNICAM_IMAGE_BPL_ALIGNMENT     32
> +#define UNICAM_IMAGE_MAX_BPL           ((1 << 16) - UNICAM_IMAGE_BPL_ALIGNMENT)
> +
> +/*
> + * Max width is therefore determined by the max stride divided by the number of
> + * bits per pixel. Take 32bpp as a worst case. No imposed limit on the height,
> + * so adopt a square image for want of anything better.
> + */
> +#define UNICAM_IMAGE_MIN_WIDTH         16
> +#define UNICAM_IMAGE_MIN_HEIGHT                16
> +#define UNICAM_IMAGE_MAX_WIDTH         (UNICAM_IMAGE_MAX_BPL / 4)
> +#define UNICAM_IMAGE_MAX_HEIGHT                UNICAM_IMAGE_MAX_WIDTH
> +
> +/*
> + * There's no intrinsic limits on the width and height for embedded dat. Use
> + * the same maximum values as for the image, to avoid overflows in the image
> + * size computation.
> + */
> +#define UNICAM_META_MIN_WIDTH          1
> +#define UNICAM_META_MIN_HEIGHT         1
> +#define UNICAM_META_MAX_WIDTH          UNICAM_IMAGE_MAX_WIDTH
> +#define UNICAM_META_MAX_HEIGHT         UNICAM_IMAGE_MAX_HEIGHT
> +
> +/*
> + * Size of the dummy buffer. Can be any size really, but the DMA
> + * allocation works in units of page sizes.
> + */
> +#define UNICAM_DUMMY_BUF_SIZE          PAGE_SIZE
> +
> +#define UNICAM_SD_PAD_SINK             0
> +#define UNICAM_SD_PAD_SOURCE_IMAGE     1
> +#define UNICAM_SD_PAD_SOURCE_METADATA  2
> +#define UNICAM_SD_NUM_PADS             (1 + UNICAM_SD_PAD_SOURCE_METADATA)
> +
> +enum unicam_node_type {
> +       UNICAM_IMAGE_NODE,
> +       UNICAM_METADATA_NODE,
> +       UNICAM_MAX_NODES
> +};
> +
> +/*
> + * struct unicam_format_info - Unicam media bus format information
> + * @fourcc: V4L2 pixel format FCC identifier. 0 if n/a.
> + * @unpacked_fourcc: V4L2 pixel format FCC identifier if the data is expanded
> + * out to 16bpp. 0 if n/a.
> + * @code: V4L2 media bus format code.
> + * @depth: Bits per pixel as delivered from the source.
> + * @csi_dt: CSI data type.
> + * @metadata_fmt: This format only applies to the metadata pad.
> + */
> +struct unicam_format_info {
> +       u32     fourcc;
> +       u32     unpacked_fourcc;
> +       u32     code;
> +       u8      depth;
> +       u8      csi_dt;
> +       bool    metadata_fmt;
> +};
> +
> +struct unicam_buffer {
> +       struct vb2_v4l2_buffer vb;
> +       struct list_head list;
> +       dma_addr_t dma_addr;
> +       unsigned int size;
> +};
> +
> +static inline struct unicam_buffer *to_unicam_buffer(struct vb2_buffer *vb)
> +{
> +       return container_of(vb, struct unicam_buffer, vb.vb2_buf);
> +}
> +
> +struct unicam_node {
> +       bool registered;
> +       bool streaming;
> +       unsigned int id;
> +
> +       /* Pointer to the current v4l2_buffer */
> +       struct unicam_buffer *cur_frm;
> +       /* Pointer to the next v4l2_buffer */
> +       struct unicam_buffer *next_frm;
> +       /* video capture */
> +       const struct unicam_format_info *fmtinfo;
> +       /* Used to store current pixel format */
> +       struct v4l2_format fmt;
> +       /* Buffer queue used in video-buf */
> +       struct vb2_queue buffer_queue;
> +       /* Queue of filled frames */
> +       struct list_head dma_queue;
> +       /* IRQ lock for DMA queue */
> +       spinlock_t dma_queue_lock;
> +       /* lock used to access this structure */
> +       struct mutex lock;
> +       /* Identifies video device for this channel */
> +       struct video_device video_dev;
> +       /* Pointer to the parent handle */
> +       struct unicam_device *dev;
> +       struct media_pad pad;
> +       /*
> +        * Dummy buffer intended to be used by unicam
> +        * if we have no other queued buffers to swap to.
> +        */
> +       struct unicam_buffer dummy_buf;
> +       void *dummy_buf_cpu_addr;
> +};
> +
> +struct unicam_device {
> +       struct kref kref;
> +
> +       /* peripheral base address */
> +       void __iomem *base;
> +       /* clock gating base address */
> +       void __iomem *clk_gate_base;
> +       /* lp clock handle */
> +       struct clk *clock;
> +       /* vpu clock handle */
> +       struct clk *vpu_clock;
> +       /* V4l2 device */
> +       struct v4l2_device v4l2_dev;
> +       struct media_device mdev;
> +
> +       /* parent device */
> +       struct device *dev;
> +       /* subdevice async Notifier */
> +       struct v4l2_async_notifier notifier;
> +       unsigned int sequence;
> +
> +       /* Sensor node */
> +       struct {
> +               struct v4l2_subdev *subdev;
> +               struct media_pad *pad;
> +       } sensor;
> +
> +       /* Internal subdev */
> +       struct {
> +               struct v4l2_subdev sd;
> +               struct media_pad pads[UNICAM_SD_NUM_PADS];
> +               bool streaming;
> +       } subdev;
> +
> +       enum v4l2_mbus_type bus_type;
> +       /*
> +        * Stores bus.mipi_csi2.flags for CSI2 sensors, or
> +        * bus.mipi_csi1.strobe for CCP2.
> +        */
> +       unsigned int bus_flags;
> +       unsigned int max_data_lanes;
> +       unsigned int active_data_lanes;
> +
> +       struct media_pipeline pipe;
> +
> +       struct unicam_node node[UNICAM_MAX_NODES];
> +};
> +
> +static inline struct unicam_device *
> +notifier_to_unicam_device(struct v4l2_async_notifier *notifier)
> +{
> +       return container_of(notifier, struct unicam_device, notifier);
> +}
> +
> +static inline struct unicam_device *
> +sd_to_unicam_device(struct v4l2_subdev *sd)
> +{
> +       return container_of(sd, struct unicam_device, subdev.sd);
> +}
> +
> +static void unicam_release(struct kref *kref)
> +{
> +       struct unicam_device *unicam =
> +               container_of(kref, struct unicam_device, kref);
> +
> +       if (unicam->mdev.dev)
> +               media_device_cleanup(&unicam->mdev);
> +
> +       kfree(unicam);
> +}
> +
> +static struct unicam_device *unicam_get(struct unicam_device *unicam)
> +{
> +       kref_get(&unicam->kref);
> +       return unicam;
> +}
> +
> +static void unicam_put(struct unicam_device *unicam)
> +{
> +       kref_put(&unicam->kref, unicam_release);
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * Misc helper functions
> + */
> +
> +static inline bool unicam_sd_pad_is_source(u32 pad)
> +{
> +       /* Camera RX has 1 sink pad, and N source pads */
> +       return pad != UNICAM_SD_PAD_SINK;
> +}
> +
> +static inline bool is_metadata_node(struct unicam_node *node)
> +{
> +       return node->video_dev.device_caps & V4L2_CAP_META_CAPTURE;
> +}
> +
> +static inline bool is_image_node(struct unicam_node *node)
> +{
> +       return node->video_dev.device_caps & V4L2_CAP_VIDEO_CAPTURE;
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * Format data table and helper functions
> + */
> +
> +static const struct v4l2_mbus_framefmt unicam_default_image_format = {
> +       .width = 640,
> +       .height = 480,
> +       .code = MEDIA_BUS_FMT_UYVY8_1X16,
> +       .field = V4L2_FIELD_NONE,
> +       .colorspace = V4L2_COLORSPACE_SRGB,
> +       .ycbcr_enc = V4L2_YCBCR_ENC_601,
> +       .quantization = V4L2_QUANTIZATION_LIM_RANGE,
> +       .xfer_func = V4L2_XFER_FUNC_SRGB,
> +       .flags = 0,
> +};
> +
> +static const struct v4l2_mbus_framefmt unicam_default_meta_format = {
> +       .width = 640,
> +       .height = 2,
> +       .code = MEDIA_BUS_FMT_META_8,
> +       .field = V4L2_FIELD_NONE,
> +};
> +
> +static const struct unicam_format_info unicam_image_formats[] = {
> +       /* YUV Formats */
> +       {
> +               .fourcc         = V4L2_PIX_FMT_YUYV,
> +               .code           = MEDIA_BUS_FMT_YUYV8_1X16,
> +               .depth          = 16,
> +               .csi_dt         = 0x1e,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_UYVY,
> +               .code           = MEDIA_BUS_FMT_UYVY8_1X16,
> +               .depth          = 16,
> +               .csi_dt         = 0x1e,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_YVYU,
> +               .code           = MEDIA_BUS_FMT_YVYU8_1X16,
> +               .depth          = 16,
> +               .csi_dt         = 0x1e,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_VYUY,
> +               .code           = MEDIA_BUS_FMT_VYUY8_1X16,
> +               .depth          = 16,
> +               .csi_dt         = 0x1e,
> +       }, {
> +       /* RGB Formats */
> +               .fourcc         = V4L2_PIX_FMT_RGB565, /* gggbbbbb rrrrrggg */
> +               .code           = MEDIA_BUS_FMT_RGB565_1X16,
> +               .depth          = 16,
> +               .csi_dt         = 0x22,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_RGB24, /* rgb */
> +               .code           = MEDIA_BUS_FMT_RGB888_1X24,
> +               .depth          = 24,
> +               .csi_dt         = 0x24,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_BGR24, /* bgr */
> +               .code           = MEDIA_BUS_FMT_BGR888_1X24,
> +               .depth          = 24,
> +               .csi_dt         = 0x24,
> +       }, {
> +       /* Bayer Formats */
> +               .fourcc         = V4L2_PIX_FMT_SBGGR8,
> +               .code           = MEDIA_BUS_FMT_SBGGR8_1X8,
> +               .depth          = 8,
> +               .csi_dt         = 0x2a,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_SGBRG8,
> +               .code           = MEDIA_BUS_FMT_SGBRG8_1X8,
> +               .depth          = 8,
> +               .csi_dt         = 0x2a,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_SGRBG8,
> +               .code           = MEDIA_BUS_FMT_SGRBG8_1X8,
> +               .depth          = 8,
> +               .csi_dt         = 0x2a,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_SRGGB8,
> +               .code           = MEDIA_BUS_FMT_SRGGB8_1X8,
> +               .depth          = 8,
> +               .csi_dt         = 0x2a,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_SBGGR10P,
> +               .unpacked_fourcc = V4L2_PIX_FMT_SBGGR10,
> +               .code           = MEDIA_BUS_FMT_SBGGR10_1X10,
> +               .depth          = 10,
> +               .csi_dt         = 0x2b,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_SGBRG10P,
> +               .unpacked_fourcc = V4L2_PIX_FMT_SGBRG10,
> +               .code           = MEDIA_BUS_FMT_SGBRG10_1X10,
> +               .depth          = 10,
> +               .csi_dt         = 0x2b,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_SGRBG10P,
> +               .unpacked_fourcc = V4L2_PIX_FMT_SGRBG10,
> +               .code           = MEDIA_BUS_FMT_SGRBG10_1X10,
> +               .depth          = 10,
> +               .csi_dt         = 0x2b,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_SRGGB10P,
> +               .unpacked_fourcc = V4L2_PIX_FMT_SRGGB10,
> +               .code           = MEDIA_BUS_FMT_SRGGB10_1X10,
> +               .depth          = 10,
> +               .csi_dt         = 0x2b,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_SBGGR12P,
> +               .unpacked_fourcc = V4L2_PIX_FMT_SBGGR12,
> +               .code           = MEDIA_BUS_FMT_SBGGR12_1X12,
> +               .depth          = 12,
> +               .csi_dt         = 0x2c,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_SGBRG12P,
> +               .unpacked_fourcc = V4L2_PIX_FMT_SGBRG12,
> +               .code           = MEDIA_BUS_FMT_SGBRG12_1X12,
> +               .depth          = 12,
> +               .csi_dt         = 0x2c,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_SGRBG12P,
> +               .unpacked_fourcc = V4L2_PIX_FMT_SGRBG12,
> +               .code           = MEDIA_BUS_FMT_SGRBG12_1X12,
> +               .depth          = 12,
> +               .csi_dt         = 0x2c,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_SRGGB12P,
> +               .unpacked_fourcc = V4L2_PIX_FMT_SRGGB12,
> +               .code           = MEDIA_BUS_FMT_SRGGB12_1X12,
> +               .depth          = 12,
> +               .csi_dt         = 0x2c,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_SBGGR14P,
> +               .unpacked_fourcc = V4L2_PIX_FMT_SBGGR14,
> +               .code           = MEDIA_BUS_FMT_SBGGR14_1X14,
> +               .depth          = 14,
> +               .csi_dt         = 0x2d,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_SGBRG14P,
> +               .unpacked_fourcc = V4L2_PIX_FMT_SGBRG14,
> +               .code           = MEDIA_BUS_FMT_SGBRG14_1X14,
> +               .depth          = 14,
> +               .csi_dt         = 0x2d,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_SGRBG14P,
> +               .unpacked_fourcc = V4L2_PIX_FMT_SGRBG14,
> +               .code           = MEDIA_BUS_FMT_SGRBG14_1X14,
> +               .depth          = 14,
> +               .csi_dt         = 0x2d,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_SRGGB14P,
> +               .unpacked_fourcc = V4L2_PIX_FMT_SRGGB14,
> +               .code           = MEDIA_BUS_FMT_SRGGB14_1X14,
> +               .depth          = 14,
> +               .csi_dt         = 0x2d,
> +       }, {
> +       /* 16 bit Bayer formats could be supported. */
> +
> +       /* Greyscale formats */
> +               .fourcc         = V4L2_PIX_FMT_GREY,
> +               .code           = MEDIA_BUS_FMT_Y8_1X8,
> +               .depth          = 8,
> +               .csi_dt         = 0x2a,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_Y10P,
> +               .unpacked_fourcc = V4L2_PIX_FMT_Y10,
> +               .code           = MEDIA_BUS_FMT_Y10_1X10,
> +               .depth          = 10,
> +               .csi_dt         = 0x2b,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_Y12P,
> +               .unpacked_fourcc = V4L2_PIX_FMT_Y12,
> +               .code           = MEDIA_BUS_FMT_Y12_1X12,
> +               .depth          = 12,
> +               .csi_dt         = 0x2c,
> +       }, {
> +               .fourcc         = V4L2_PIX_FMT_Y14P,
> +               .unpacked_fourcc = V4L2_PIX_FMT_Y14,
> +               .code           = MEDIA_BUS_FMT_Y14_1X14,
> +               .depth          = 14,
> +               .csi_dt         = 0x2d,
> +       },
> +};
> +
> +static const struct unicam_format_info unicam_meta_formats[] = {
> +       {
> +               .fourcc         = V4L2_META_FMT_GENERIC_8,
> +               .code           = MEDIA_BUS_FMT_META_8,
> +               .depth          = 8,
> +               .metadata_fmt   = true,
> +       }, {
> +               .fourcc         = V4L2_META_FMT_GENERIC_CSI2_10,
> +               .code           = MEDIA_BUS_FMT_META_10,
> +               .depth          = 10,
> +               .metadata_fmt   = true,
> +       }, {
> +               .fourcc         = V4L2_META_FMT_GENERIC_CSI2_12,
> +               .code           = MEDIA_BUS_FMT_META_12,
> +               .depth          = 12,
> +               .metadata_fmt   = true,
> +       }, {
> +               .fourcc         = V4L2_META_FMT_GENERIC_CSI2_14,
> +               .code           = MEDIA_BUS_FMT_META_14,
> +               .depth          = 14,
> +               .metadata_fmt   = true,
> +       },
> +};
> +
> +/* Format setup functions */
> +static const struct unicam_format_info *
> +unicam_find_format_by_code(u32 code, u32 pad)
> +{
> +       const struct unicam_format_info *formats;
> +       unsigned int num_formats;
> +       unsigned int i;
> +
> +       if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> +               formats = unicam_image_formats;
> +               num_formats = ARRAY_SIZE(unicam_image_formats);
> +       } else {
> +               formats = unicam_meta_formats;
> +               num_formats = ARRAY_SIZE(unicam_meta_formats);
> +       }
> +
> +       for (i = 0; i < num_formats; i++) {
> +               if (formats[i].code == code)
> +                       return &formats[i];
> +       }
> +
> +       return NULL;
> +}
> +
> +static const struct unicam_format_info *
> +unicam_find_format_by_fourcc(u32 fourcc, u32 pad)
> +{
> +       const struct unicam_format_info *formats;
> +       unsigned int num_formats;
> +       unsigned int i;
> +
> +       if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> +               formats = unicam_image_formats;
> +               num_formats = ARRAY_SIZE(unicam_image_formats);
> +       } else {
> +               formats = unicam_meta_formats;
> +               num_formats = ARRAY_SIZE(unicam_meta_formats);
> +       }
> +
> +       for (i = 0; i < num_formats; ++i) {
> +               if (formats[i].fourcc == fourcc)
> +                       return &formats[i];
> +       }
> +
> +       return NULL;
> +}
> +
> +static void unicam_calc_image_size_bpl(struct unicam_device *unicam,
> +                                      const struct unicam_format_info *fmtinfo,
> +                                      struct v4l2_pix_format *pix)
> +{
> +       u32 min_bpl;
> +
> +       v4l_bound_align_image(&pix->width, UNICAM_IMAGE_MIN_WIDTH,
> +                             UNICAM_IMAGE_MAX_WIDTH, 2,
> +                             &pix->height, UNICAM_IMAGE_MIN_HEIGHT,
> +                             UNICAM_IMAGE_MAX_HEIGHT, 0, 0);
> +
> +       /* Unpacking always goes to 16bpp */
> +       if (pix->pixelformat == fmtinfo->unpacked_fourcc)
> +               min_bpl = pix->width * 2;
> +       else
> +               min_bpl = pix->width * fmtinfo->depth / 8;
> +       min_bpl = ALIGN(min_bpl, UNICAM_IMAGE_BPL_ALIGNMENT);
> +
> +       pix->bytesperline = ALIGN(pix->bytesperline, UNICAM_IMAGE_BPL_ALIGNMENT);
> +       pix->bytesperline = clamp_t(unsigned int, pix->bytesperline, min_bpl,
> +                                   UNICAM_IMAGE_MAX_BPL);
> +
> +       pix->sizeimage = pix->height * pix->bytesperline;
> +}
> +
> +static void unicam_calc_meta_size_bpl(struct unicam_device *unicam,
> +                                     const struct unicam_format_info *fmtinfo,
> +                                     struct v4l2_meta_format *meta)
> +{
> +       v4l_bound_align_image(&meta->width, UNICAM_META_MIN_WIDTH,
> +                             UNICAM_META_MAX_WIDTH, 0,
> +                             &meta->height, UNICAM_META_MIN_HEIGHT,
> +                             UNICAM_META_MAX_HEIGHT, 0, 0);
> +
> +       meta->bytesperline = ALIGN(meta->width * fmtinfo->depth / 8,
> +                                  UNICAM_DMA_BPL_ALIGNMENT);
> +       meta->buffersize = meta->height * meta->bytesperline;
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * Hardware handling
> + */
> +
> +static inline void unicam_clk_write(struct unicam_device *unicam, u32 val)
> +{
> +       /* Pass the CM_PASSWORD along with the value. */
> +       writel(val | 0x5a000000, unicam->clk_gate_base);
> +}
> +
> +static inline u32 unicam_reg_read(struct unicam_device *unicam, u32 offset)
> +{
> +       return readl(unicam->base + offset);
> +}
> +
> +static inline void unicam_reg_write(struct unicam_device *unicam, u32 offset, u32 val)
> +{
> +       writel(val, unicam->base + offset);
> +}
> +
> +static inline int unicam_get_field(u32 value, u32 mask)
> +{
> +       return (value & mask) >> __ffs(mask);
> +}
> +
> +static inline void unicam_set_field(u32 *valp, u32 field, u32 mask)
> +{
> +       u32 val = *valp;
> +
> +       val &= ~mask;
> +       val |= (field << __ffs(mask)) & mask;
> +       *valp = val;
> +}
> +
> +static inline void unicam_reg_write_field(struct unicam_device *unicam, u32 offset,
> +                                         u32 field, u32 mask)
> +{
> +       u32 val = unicam_reg_read(unicam, offset);
> +
> +       unicam_set_field(&val, field, mask);
> +       unicam_reg_write(unicam, offset, val);
> +}
> +
> +static void unicam_wr_dma_addr(struct unicam_node *node,
> +                              struct unicam_buffer *buf)
> +{
> +       dma_addr_t endaddr = buf->dma_addr + buf->size;
> +
> +       if (node->id == UNICAM_IMAGE_NODE) {
> +               unicam_reg_write(node->dev, UNICAM_IBSA0, buf->dma_addr);
> +               unicam_reg_write(node->dev, UNICAM_IBEA0, endaddr);
> +       } else {
> +               unicam_reg_write(node->dev, UNICAM_DBSA0, buf->dma_addr);
> +               unicam_reg_write(node->dev, UNICAM_DBEA0, endaddr);
> +       }
> +}
> +
> +static unsigned int unicam_get_lines_done(struct unicam_device *unicam)
> +{
> +       struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> +       unsigned int stride = node->fmt.fmt.pix.bytesperline;
> +       struct unicam_buffer *frm = node->cur_frm;
> +       dma_addr_t cur_addr;
> +
> +       if (!frm)
> +               return 0;
> +
> +       cur_addr = unicam_reg_read(unicam, UNICAM_IBWP);
> +       return (unsigned int)(cur_addr - frm->dma_addr) / stride;
> +}
> +
> +static void unicam_schedule_next_buffer(struct unicam_node *node)
> +{
> +       struct unicam_buffer *buf;
> +
> +       buf = list_first_entry(&node->dma_queue, struct unicam_buffer, list);
> +       node->next_frm = buf;
> +       list_del(&buf->list);
> +
> +       unicam_wr_dma_addr(node, buf);
> +}
> +
> +static void unicam_schedule_dummy_buffer(struct unicam_node *node)
> +{
> +       int node_id = is_image_node(node) ? UNICAM_IMAGE_NODE : UNICAM_METADATA_NODE;
> +
> +       dev_dbg(node->dev->dev, "Scheduling dummy buffer for node %d\n", node_id);
> +
> +       unicam_wr_dma_addr(node, &node->dummy_buf);
> +
> +       node->next_frm = NULL;
> +}
> +
> +static void unicam_process_buffer_complete(struct unicam_node *node,
> +                                          unsigned int sequence)
> +{
> +       node->cur_frm->vb.field = node->fmt.fmt.pix.field;
> +       node->cur_frm->vb.sequence = sequence;
> +
> +       vb2_buffer_done(&node->cur_frm->vb.vb2_buf, VB2_BUF_STATE_DONE);
> +}
> +
> +static void unicam_queue_event_sof(struct unicam_device *unicam)
> +{
> +       struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> +
> +       struct v4l2_event event = {
> +               .type = V4L2_EVENT_FRAME_SYNC,
> +               .u.frame_sync.frame_sequence = unicam->sequence,
> +       };
> +
> +       v4l2_event_queue(&node->video_dev, &event);
> +}
> +
> +static irqreturn_t unicam_isr(int irq, void *dev)
> +{
> +       struct unicam_device *unicam = dev;
> +       unsigned int lines_done = unicam_get_lines_done(dev);
> +       unsigned int sequence = unicam->sequence;
> +       unsigned int i;
> +       u32 ista, sta;
> +       bool fe;
> +       u64 ts;
> +
> +       sta = unicam_reg_read(unicam, UNICAM_STA);
> +       /* Write value back to clear the interrupts */
> +       unicam_reg_write(unicam, UNICAM_STA, sta);
> +
> +       ista = unicam_reg_read(unicam, UNICAM_ISTA);
> +       /* Write value back to clear the interrupts */
> +       unicam_reg_write(unicam, UNICAM_ISTA, ista);
> +
> +       dev_dbg(unicam->dev, "ISR: ISTA: 0x%X, STA: 0x%X, sequence %d, lines done %d\n",
> +               ista, sta, sequence, lines_done);
> +
> +       if (!(sta & (UNICAM_IS | UNICAM_PI0)))
> +               return IRQ_HANDLED;
> +
> +       /*
> +        * Look for either the Frame End interrupt or the Packet Capture status
> +        * to signal a frame end.
> +        */
> +       fe = ista & UNICAM_FEI || sta & UNICAM_PI0;
> +
> +       /*
> +        * We must run the frame end handler first. If we have a valid next_frm
> +        * and we get a simultaneout FE + FS interrupt, running the FS handler
> +        * first would null out the next_frm ptr and we would have lost the
> +        * buffer forever.
> +        */
> +       if (fe) {
> +               /*
> +                * Ensure we have swapped buffers already as we can't
> +                * stop the peripheral. If no buffer is available, use a
> +                * dummy buffer to dump out frames until we get a new buffer
> +                * to use.
> +                */
> +               for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> +                       if (!unicam->node[i].streaming)
> +                               continue;
> +
> +                       /*
> +                        * If cur_frm == next_frm, it means we have not had
> +                        * a chance to swap buffers, likely due to having
> +                        * multiple interrupts occurring simultaneously (like FE
> +                        * + FS + LS). In this case, we cannot signal the buffer
> +                        * as complete, as the HW will reuse that buffer.
> +                        */
> +                       if (unicam->node[i].cur_frm &&
> +                           unicam->node[i].cur_frm != unicam->node[i].next_frm)
> +                               unicam_process_buffer_complete(&unicam->node[i],
> +                                                              sequence);
> +                       unicam->node[i].cur_frm = unicam->node[i].next_frm;
> +               }
> +               unicam->sequence++;
> +       }
> +
> +       if (ista & UNICAM_FSI) {
> +               /*
> +                * Timestamp is to be when the first data byte was captured,
> +                * aka frame start.
> +                */
> +               ts = ktime_get_ns();
> +               for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> +                       if (!unicam->node[i].streaming)
> +                               continue;
> +
> +                       if (unicam->node[i].cur_frm)
> +                               unicam->node[i].cur_frm->vb.vb2_buf.timestamp =
> +                                                               ts;
> +                       else
> +                               dev_dbg(unicam->v4l2_dev.dev,
> +                                       "ISR: [%d] Dropping frame, buffer not available at FS\n",
> +                                       i);
> +                       /*
> +                        * Set the next frame output to go to a dummy frame
> +                        * if we have not managed to obtain another frame
> +                        * from the queue.
> +                        */
> +                       unicam_schedule_dummy_buffer(&unicam->node[i]);
> +               }
> +
> +               unicam_queue_event_sof(unicam);
> +       }
> +
> +       /*
> +        * Cannot swap buffer at frame end, there may be a race condition
> +        * where the HW does not actually swap it if the new frame has
> +        * already started.
> +        */
> +       if (ista & (UNICAM_FSI | UNICAM_LCI) && !fe) {
> +               for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> +                       if (!unicam->node[i].streaming)
> +                               continue;
> +
> +                       spin_lock(&unicam->node[i].dma_queue_lock);
> +                       if (!list_empty(&unicam->node[i].dma_queue) &&
> +                           !unicam->node[i].next_frm)
> +                               unicam_schedule_next_buffer(&unicam->node[i]);
> +                       spin_unlock(&unicam->node[i].dma_queue_lock);
> +               }
> +       }
> +
> +       if (unicam_reg_read(unicam, UNICAM_ICTL) & UNICAM_FCM) {
> +               /* Switch out of trigger mode if selected */
> +               unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> +               unicam_reg_write_field(unicam, UNICAM_ICTL, 0, UNICAM_FCM);
> +       }
> +       return IRQ_HANDLED;
> +}
> +
> +static void unicam_set_packing_config(struct unicam_device *unicam)
> +{
> +       struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> +       u32 pack, unpack;
> +       u32 val;
> +
> +       if (node->fmt.fmt.pix.pixelformat == node->fmtinfo->fourcc) {
> +               unpack = UNICAM_PUM_NONE;
> +               pack = UNICAM_PPM_NONE;
> +       } else {
> +               switch (node->fmtinfo->depth) {
> +               case 8:
> +                       unpack = UNICAM_PUM_UNPACK8;
> +                       break;
> +               case 10:
> +                       unpack = UNICAM_PUM_UNPACK10;
> +                       break;
> +               case 12:
> +                       unpack = UNICAM_PUM_UNPACK12;
> +                       break;
> +               case 14:
> +                       unpack = UNICAM_PUM_UNPACK14;
> +                       break;
> +               case 16:
> +                       unpack = UNICAM_PUM_UNPACK16;
> +                       break;
> +               default:
> +                       unpack = UNICAM_PUM_NONE;
> +                       break;
> +               }
> +
> +               /* Repacking is always to 16bpp */
> +               pack = UNICAM_PPM_PACK16;
> +       }
> +
> +       val = 0;
> +       unicam_set_field(&val, unpack, UNICAM_PUM_MASK);
> +       unicam_set_field(&val, pack, UNICAM_PPM_MASK);
> +       unicam_reg_write(unicam, UNICAM_IPIPE, val);
> +}
> +
> +static void unicam_cfg_image_id(struct unicam_device *unicam)
> +{
> +       struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> +
> +       if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> +               /* CSI2 mode, hardcode VC 0 for now. */
> +               unicam_reg_write(unicam, UNICAM_IDI0,
> +                                (0 << 6) | node->fmtinfo->csi_dt);
> +       } else {
> +               /* CCP2 mode */
> +               unicam_reg_write(unicam, UNICAM_IDI0,
> +                                0x80 | node->fmtinfo->csi_dt);
> +       }
> +}
> +
> +static void unicam_enable_ed(struct unicam_device *unicam)
> +{
> +       u32 val = unicam_reg_read(unicam, UNICAM_DCS);
> +
> +       unicam_set_field(&val, 2, UNICAM_EDL_MASK);
> +       /* Do not wrap at the end of the embedded data buffer */
> +       unicam_set_field(&val, 0, UNICAM_DBOB);
> +
> +       unicam_reg_write(unicam, UNICAM_DCS, val);
> +}
> +
> +static void unicam_start_rx(struct unicam_device *unicam,
> +                           struct unicam_buffer *buf)
> +{
> +       struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> +       int line_int_freq = node->fmt.fmt.pix.height >> 2;
> +       unsigned int i;
> +       u32 val;
> +
> +       if (line_int_freq < 128)
> +               line_int_freq = 128;
> +
> +       /* Enable lane clocks */
> +       val = 1;
> +       for (i = 0; i < unicam->active_data_lanes; i++)
> +               val = val << 2 | 1;
> +       unicam_clk_write(unicam, val);
> +
> +       /* Basic init */
> +       unicam_reg_write(unicam, UNICAM_CTRL, UNICAM_MEM);
> +
> +       /* Enable analogue control, and leave in reset. */
> +       val = UNICAM_AR;
> +       unicam_set_field(&val, 7, UNICAM_CTATADJ_MASK);
> +       unicam_set_field(&val, 7, UNICAM_PTATADJ_MASK);
> +       unicam_reg_write(unicam, UNICAM_ANA, val);
> +       usleep_range(1000, 2000);
> +
> +       /* Come out of reset */
> +       unicam_reg_write_field(unicam, UNICAM_ANA, 0, UNICAM_AR);
> +
> +       /* Peripheral reset */
> +       unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPR);
> +       unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPR);
> +
> +       unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPE);
> +
> +       /* Enable Rx control. */
> +       val = unicam_reg_read(unicam, UNICAM_CTRL);
> +       if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> +               unicam_set_field(&val, UNICAM_CPM_CSI2, UNICAM_CPM_MASK);
> +               unicam_set_field(&val, UNICAM_DCM_STROBE, UNICAM_DCM_MASK);
> +       } else {
> +               unicam_set_field(&val, UNICAM_CPM_CCP2, UNICAM_CPM_MASK);
> +               unicam_set_field(&val, unicam->bus_flags, UNICAM_DCM_MASK);
> +       }
> +       /* Packet framer timeout */
> +       unicam_set_field(&val, 0xf, UNICAM_PFT_MASK);
> +       unicam_set_field(&val, 128, UNICAM_OET_MASK);
> +       unicam_reg_write(unicam, UNICAM_CTRL, val);
> +
> +       unicam_reg_write(unicam, UNICAM_IHWIN, 0);
> +       unicam_reg_write(unicam, UNICAM_IVWIN, 0);
> +
> +       /* AXI bus access QoS setup */
> +       val = unicam_reg_read(unicam, UNICAM_PRI);
> +       unicam_set_field(&val, 0, UNICAM_BL_MASK);
> +       unicam_set_field(&val, 0, UNICAM_BS_MASK);
> +       unicam_set_field(&val, 0xe, UNICAM_PP_MASK);
> +       unicam_set_field(&val, 8, UNICAM_NP_MASK);
> +       unicam_set_field(&val, 2, UNICAM_PT_MASK);
> +       unicam_set_field(&val, 1, UNICAM_PE);
> +       unicam_reg_write(unicam, UNICAM_PRI, val);
> +
> +       unicam_reg_write_field(unicam, UNICAM_ANA, 0, UNICAM_DDL);
> +
> +       /* Always start in trigger frame capture mode (UNICAM_FCM set) */
> +       val = UNICAM_FSIE | UNICAM_FEIE | UNICAM_FCM | UNICAM_IBOB;
> +       unicam_set_field(&val, line_int_freq, UNICAM_LCIE_MASK);
> +       unicam_reg_write(unicam, UNICAM_ICTL, val);
> +       unicam_reg_write(unicam, UNICAM_STA, UNICAM_STA_MASK_ALL);
> +       unicam_reg_write(unicam, UNICAM_ISTA, UNICAM_ISTA_MASK_ALL);
> +
> +       /* tclk_term_en */
> +       unicam_reg_write_field(unicam, UNICAM_CLT, 2, UNICAM_CLT1_MASK);
> +       /* tclk_settle */
> +       unicam_reg_write_field(unicam, UNICAM_CLT, 6, UNICAM_CLT2_MASK);
> +       /* td_term_en */
> +       unicam_reg_write_field(unicam, UNICAM_DLT, 2, UNICAM_DLT1_MASK);
> +       /* ths_settle */
> +       unicam_reg_write_field(unicam, UNICAM_DLT, 6, UNICAM_DLT2_MASK);
> +       /* trx_enable */
> +       unicam_reg_write_field(unicam, UNICAM_DLT, 0, UNICAM_DLT3_MASK);
> +
> +       unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_SOE);
> +
> +       /* Packet compare setup - required to avoid missing frame ends */
> +       val = 0;
> +       unicam_set_field(&val, 1, UNICAM_PCE);
> +       unicam_set_field(&val, 1, UNICAM_GI);
> +       unicam_set_field(&val, 1, UNICAM_CPH);
> +       unicam_set_field(&val, 0, UNICAM_PCVC_MASK);
> +       unicam_set_field(&val, 1, UNICAM_PCDT_MASK);
> +       unicam_reg_write(unicam, UNICAM_CMP0, val);
> +
> +       /* Enable clock lane and set up terminations */
> +       val = 0;
> +       if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> +               /* CSI2 */
> +               unicam_set_field(&val, 1, UNICAM_CLE);
> +               unicam_set_field(&val, 1, UNICAM_CLLPE);
> +               if (!(unicam->bus_flags & V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK)) {
> +                       unicam_set_field(&val, 1, UNICAM_CLTRE);
> +                       unicam_set_field(&val, 1, UNICAM_CLHSE);
> +               }
> +       } else {
> +               /* CCP2 */
> +               unicam_set_field(&val, 1, UNICAM_CLE);
> +               unicam_set_field(&val, 1, UNICAM_CLHSE);
> +               unicam_set_field(&val, 1, UNICAM_CLTRE);
> +       }
> +       unicam_reg_write(unicam, UNICAM_CLK, val);
> +
> +       /*
> +        * Enable required data lanes with appropriate terminations.
> +        * The same value needs to be written to UNICAM_DATn registers for
> +        * the active lanes, and 0 for inactive ones.
> +        */
> +       val = 0;
> +       if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> +               /* CSI2 */
> +               unicam_set_field(&val, 1, UNICAM_DLE);
> +               unicam_set_field(&val, 1, UNICAM_DLLPE);
> +               if (!(unicam->bus_flags & V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK)) {
> +                       unicam_set_field(&val, 1, UNICAM_DLTRE);
> +                       unicam_set_field(&val, 1, UNICAM_DLHSE);
> +               }
> +       } else {
> +               /* CCP2 */
> +               unicam_set_field(&val, 1, UNICAM_DLE);
> +               unicam_set_field(&val, 1, UNICAM_DLHSE);
> +               unicam_set_field(&val, 1, UNICAM_DLTRE);
> +       }
> +       unicam_reg_write(unicam, UNICAM_DAT0, val);
> +
> +       if (unicam->active_data_lanes == 1)
> +               val = 0;
> +       unicam_reg_write(unicam, UNICAM_DAT1, val);
> +
> +       if (unicam->max_data_lanes > 2) {
> +               /*
> +                * Registers UNICAM_DAT2 and UNICAM_DAT3 only valid if the
> +                * instance supports more than 2 data lanes.
> +                */
> +               if (unicam->active_data_lanes == 2)
> +                       val = 0;
> +               unicam_reg_write(unicam, UNICAM_DAT2, val);
> +
> +               if (unicam->active_data_lanes == 3)
> +                       val = 0;
> +               unicam_reg_write(unicam, UNICAM_DAT3, val);
> +       }
> +
> +       unicam_reg_write(unicam, UNICAM_IBLS,
> +                        node->fmt.fmt.pix.bytesperline);
> +       unicam_wr_dma_addr(&unicam->node[UNICAM_IMAGE_NODE], buf);
> +       unicam_set_packing_config(unicam);
> +       unicam_cfg_image_id(unicam);
> +
> +       val = unicam_reg_read(unicam, UNICAM_MISC);
> +       unicam_set_field(&val, 1, UNICAM_FL0);
> +       unicam_set_field(&val, 1, UNICAM_FL1);
> +       unicam_reg_write(unicam, UNICAM_MISC, val);
> +
> +       /* Enable peripheral */
> +       unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPE);
> +
> +       /* Load image pointers */
> +       unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_LIP_MASK);
> +
> +       /*
> +        * Enable trigger only for the first frame to
> +        * sync correctly to the FS from the source.
> +        */
> +       unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> +}
> +
> +static void unicam_start_metadata(struct unicam_device *unicam,
> +                                 struct unicam_buffer *buf)
> +{
> +       struct unicam_node *node = &unicam->node[UNICAM_METADATA_NODE];
> +
> +       unicam_enable_ed(unicam);
> +       unicam_wr_dma_addr(node, buf);
> +       unicam_reg_write_field(unicam, UNICAM_DCS, 1, UNICAM_LDP);
> +}
> +
> +static void unicam_disable(struct unicam_device *unicam)
> +{
> +       /* Analogue lane control disable */
> +       unicam_reg_write_field(unicam, UNICAM_ANA, 1, UNICAM_DDL);
> +
> +       /* Stop the output engine */
> +       unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_SOE);
> +
> +       /* Disable the data lanes. */
> +       unicam_reg_write(unicam, UNICAM_DAT0, 0);
> +       unicam_reg_write(unicam, UNICAM_DAT1, 0);
> +
> +       if (unicam->max_data_lanes > 2) {
> +               unicam_reg_write(unicam, UNICAM_DAT2, 0);
> +               unicam_reg_write(unicam, UNICAM_DAT3, 0);
> +       }
> +
> +       /* Peripheral reset */
> +       unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPR);
> +       usleep_range(50, 100);
> +       unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPR);
> +
> +       /* Disable peripheral */
> +       unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPE);
> +
> +       /* Clear ED setup */
> +       unicam_reg_write(unicam, UNICAM_DCS, 0);
> +
> +       /* Disable all lane clocks */
> +       unicam_clk_write(unicam, 0);
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * V4L2 subdev operations
> + */
> +
> +static int __unicam_subdev_set_routing(struct v4l2_subdev *sd,
> +                                      struct v4l2_subdev_state *state,
> +                                      struct v4l2_subdev_krouting *routing)
> +{
> +       struct v4l2_subdev_route *route;
> +       int ret;
> +
> +       ret = v4l2_subdev_routing_validate(sd, routing,
> +                                          V4L2_SUBDEV_ROUTING_ONLY_1_TO_1);
> +       if (ret)
> +               return ret;
> +
> +       ret = v4l2_subdev_set_routing(sd, state, routing);
> +       if (ret)
> +               return ret;
> +
> +       for_each_active_route(&state->routing, route) {
> +               const struct v4l2_mbus_framefmt *def_fmt;
> +               struct v4l2_mbus_framefmt *fmt;
> +
> +               if (route->source_pad == UNICAM_SD_PAD_SOURCE_IMAGE)
> +                       def_fmt = &unicam_default_image_format;
> +               else
> +                       def_fmt = &unicam_default_meta_format;
> +
> +               fmt = v4l2_subdev_state_get_format(state, route->sink_pad,
> +                                                  route->sink_stream);
> +               *fmt = *def_fmt;
> +               fmt = v4l2_subdev_state_get_format(state, route->source_pad,
> +                                                  route->source_stream);
> +               *fmt = *def_fmt;
> +       }
> +
> +       return 0;
> +}
> +
> +static int unicam_subdev_init_state(struct v4l2_subdev *sd,
> +                                   struct v4l2_subdev_state *state)
> +{
> +       struct v4l2_subdev_route routes[] = {
> +               {
> +                       .sink_pad = UNICAM_SD_PAD_SINK,
> +                       .sink_stream = 0,
> +                       .source_pad = UNICAM_SD_PAD_SOURCE_IMAGE,
> +                       .source_stream = 0,
> +                       .flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
> +               },
> +       };
> +
> +       struct v4l2_subdev_krouting routing = {
> +               .len_routes = ARRAY_SIZE(routes),
> +               .num_routes = ARRAY_SIZE(routes),
> +               .routes = routes,
> +       };
> +
> +       /* Initialize routing to single route to the fist source pad. */
> +       return __unicam_subdev_set_routing(sd, state, &routing);
> +}
> +
> +static int unicam_subdev_enum_mbus_code(struct v4l2_subdev *sd,
> +                                       struct v4l2_subdev_state *state,
> +                                       struct v4l2_subdev_mbus_code_enum *code)
> +{
> +       u32 pad, stream;
> +       int ret;
> +
> +       ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
> +                                                   code->pad, code->stream,
> +                                                   &pad, &stream);
> +       if (ret)
> +               return ret;
> +
> +       if (unicam_sd_pad_is_source(code->pad)) {
> +               /* No transcoding, source and sink codes must match. */
> +               const struct v4l2_mbus_framefmt *fmt;
> +
> +               fmt = v4l2_subdev_state_get_format(state, pad, stream);
> +               if (!fmt)
> +                       return -EINVAL;
> +
> +               if (code->index > 0)
> +                       return -EINVAL;
> +
> +               code->code = fmt->code;
> +       } else {
> +               const struct unicam_format_info *formats;
> +               unsigned int num_formats;
> +
> +               if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> +                       formats = unicam_image_formats;
> +                       num_formats = ARRAY_SIZE(unicam_image_formats);
> +               } else {
> +                       formats = unicam_meta_formats;
> +                       num_formats = ARRAY_SIZE(unicam_meta_formats);
> +               }
> +
> +               if (code->index >= num_formats)
> +                       return -EINVAL;
> +
> +               code->code = formats[code->index].code;
> +       }
> +
> +       return 0;
> +}
> +
> +static int unicam_subdev_enum_frame_size(struct v4l2_subdev *sd,
> +                                        struct v4l2_subdev_state *state,
> +                                        struct v4l2_subdev_frame_size_enum *fse)
> +{
> +       u32 pad, stream;
> +       int ret;
> +
> +       if (fse->index > 0)
> +               return -EINVAL;
> +
> +       ret = v4l2_subdev_routing_find_opposite_end(&state->routing, fse->pad,
> +                                                   fse->stream, &pad,
> +                                                   &stream);
> +       if (ret)
> +               return ret;
> +
> +       if (unicam_sd_pad_is_source(fse->pad)) {
> +               /* No transcoding, source and sink formats must match. */
> +               const struct v4l2_mbus_framefmt *fmt;
> +
> +               fmt = v4l2_subdev_state_get_format(state, pad, stream);
> +               if (!fmt)
> +                       return -EINVAL;
> +
> +               if (fse->code != fmt->code)
> +                       return -EINVAL;
> +
> +               fse->min_width = fmt->width;
> +               fse->max_width = fmt->width;
> +               fse->min_height = fmt->height;
> +               fse->max_height = fmt->height;
> +       } else {
> +               const struct unicam_format_info *fmtinfo;
> +
> +               fmtinfo = unicam_find_format_by_code(fse->code, pad);
> +               if (!fmtinfo)
> +                       return -EINVAL;
> +
> +               if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> +                       fse->min_width = UNICAM_IMAGE_MIN_WIDTH;
> +                       fse->max_width = UNICAM_IMAGE_MAX_WIDTH;
> +                       fse->min_height = UNICAM_IMAGE_MIN_HEIGHT;
> +                       fse->max_height = UNICAM_IMAGE_MAX_HEIGHT;
> +               } else {
> +                       fse->min_width = UNICAM_META_MIN_WIDTH;
> +                       fse->max_width = UNICAM_META_MAX_WIDTH;
> +                       fse->min_height = UNICAM_META_MIN_HEIGHT;
> +                       fse->max_height = UNICAM_META_MAX_HEIGHT;
> +               }
> +       }
> +
> +       return 0;
> +}
> +
> +static int unicam_subdev_set_format(struct v4l2_subdev *sd,
> +                                   struct v4l2_subdev_state *state,
> +                                   struct v4l2_subdev_format *format)
> +{
> +       struct unicam_device *unicam = sd_to_unicam_device(sd);
> +       struct v4l2_mbus_framefmt *sink_format, *source_format;
> +       const struct unicam_format_info *fmtinfo;
> +       u32 source_pad, source_stream;
> +       int ret;
> +
> +       if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE &&
> +           unicam->subdev.streaming)
> +               return -EBUSY;
> +
> +       /* No transcoding, source and sink formats must match. */
> +       if (unicam_sd_pad_is_source(format->pad))
> +               return v4l2_subdev_get_fmt(sd, state, format);
> +
> +       /*
> +        * Allowed formats for the stream on the sink pad depend on what source
> +        * pad the stream is routed to. Find the corresponding source pad and
> +        * use it to validate the media bus code.
> +        */
> +       ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
> +                                                   format->pad, format->stream,
> +                                                   &source_pad, &source_stream);
> +       if (ret)
> +               return ret;
> +
> +       fmtinfo = unicam_find_format_by_code(format->format.code, source_pad);
> +       if (!fmtinfo) {
> +               fmtinfo = source_pad == UNICAM_SD_PAD_SOURCE_IMAGE
> +                       ? &unicam_image_formats[0] : &unicam_meta_formats[0];
> +               format->format.code = fmtinfo->code;
> +       }
> +
> +       if (source_pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> +               format->format.width = clamp_t(unsigned int,
> +                                              format->format.width,
> +                                              UNICAM_IMAGE_MIN_WIDTH,
> +                                              UNICAM_IMAGE_MAX_WIDTH);
> +               format->format.height = clamp_t(unsigned int,
> +                                               format->format.height,
> +                                               UNICAM_IMAGE_MIN_HEIGHT,
> +                                               UNICAM_IMAGE_MAX_HEIGHT);
> +               format->format.field = V4L2_FIELD_NONE;
> +       } else {
> +               format->format.width = clamp_t(unsigned int,
> +                                              format->format.width,
> +                                              UNICAM_META_MIN_WIDTH,
> +                                              UNICAM_META_MAX_WIDTH);
> +               format->format.height = clamp_t(unsigned int,
> +                                               format->format.height,
> +                                               UNICAM_META_MIN_HEIGHT,
> +                                               UNICAM_META_MAX_HEIGHT);
> +               format->format.field = V4L2_FIELD_NONE;
> +
> +               /* Colorspace don't apply to metadata. */
> +               format->format.colorspace = 0;
> +               format->format.ycbcr_enc = 0;
> +               format->format.quantization = 0;
> +               format->format.xfer_func = 0;
> +       }
> +
> +       sink_format = v4l2_subdev_state_get_format(state, format->pad,
> +                                                  format->stream);
> +       source_format = v4l2_subdev_state_get_format(state, source_pad,
> +                                                    source_stream);
> +       *sink_format = format->format;
> +       *source_format = format->format;
> +
> +       return 0;
> +}
> +
> +static int unicam_subdev_set_routing(struct v4l2_subdev *sd,
> +                                    struct v4l2_subdev_state *state,
> +                                    enum v4l2_subdev_format_whence which,
> +                                    struct v4l2_subdev_krouting *routing)
> +{
> +       struct unicam_device *unicam = sd_to_unicam_device(sd);
> +
> +       if (which == V4L2_SUBDEV_FORMAT_ACTIVE && unicam->subdev.streaming)
> +               return -EBUSY;
> +
> +       return __unicam_subdev_set_routing(sd, state, routing);
> +}
> +
> +static int unicam_sd_enable_streams(struct v4l2_subdev *sd,
> +                                   struct v4l2_subdev_state *state, u32 pad,
> +                                   u64 streams_mask)
> +{
> +       struct unicam_device *unicam = sd_to_unicam_device(sd);
> +       u32 other_pad, other_stream;
> +       int ret;
> +
> +       ret = v4l2_subdev_routing_find_opposite_end(&state->routing, pad, 0,
> +                                                   &other_pad, &other_stream);
> +       if (ret)
> +               return ret;
> +
> +       unicam->sequence = 0;
> +
> +       ret = v4l2_subdev_enable_streams(unicam->sensor.subdev,
> +                                        unicam->sensor.pad->index,
> +                                        BIT(other_stream));
> +       if (ret) {
> +               dev_err(unicam->dev, "stream on failed in subdev\n");
> +               return ret;
> +       }
> +
> +       unicam->subdev.streaming = true;
> +
> +       return 0;
> +}
> +
> +static int unicam_sd_disable_streams(struct v4l2_subdev *sd,
> +                                    struct v4l2_subdev_state *state, u32 pad,
> +                                    u64 streams_mask)
> +{
> +       struct unicam_device *unicam = sd_to_unicam_device(sd);
> +       u32 other_pad, other_stream;
> +       int ret;
> +
> +       ret = v4l2_subdev_routing_find_opposite_end(&state->routing, pad, 0,
> +                                                   &other_pad, &other_stream);
> +       if (ret)
> +               return ret;
> +
> +       v4l2_subdev_disable_streams(unicam->sensor.subdev,
> +                                   unicam->sensor.pad->index,
> +                                   BIT(other_stream));
> +
> +       unicam->subdev.streaming = false;
> +
> +       return 0;
> +}
> +
> +static const struct v4l2_subdev_pad_ops unicam_subdev_pad_ops = {
> +       .enum_mbus_code         = unicam_subdev_enum_mbus_code,
> +       .enum_frame_size        = unicam_subdev_enum_frame_size,
> +       .get_fmt                = v4l2_subdev_get_fmt,
> +       .set_fmt                = unicam_subdev_set_format,
> +       .set_routing            = unicam_subdev_set_routing,
> +       .enable_streams         = unicam_sd_enable_streams,
> +       .disable_streams        = unicam_sd_disable_streams,
> +};
> +
> +static const struct v4l2_subdev_ops unicam_subdev_ops = {
> +       .pad                    = &unicam_subdev_pad_ops,
> +};
> +
> +static const struct v4l2_subdev_internal_ops unicam_subdev_internal_ops = {
> +       .init_state             = unicam_subdev_init_state,
> +};
> +
> +static const struct media_entity_operations unicam_subdev_media_ops = {
> +       .link_validate          = v4l2_subdev_link_validate,
> +       .has_pad_interdep       = v4l2_subdev_has_pad_interdep,
> +};
> +
> +static int unicam_subdev_init(struct unicam_device *unicam)
> +{
> +       struct v4l2_subdev *sd = &unicam->subdev.sd;
> +       int ret;
> +
> +       v4l2_subdev_init(sd, &unicam_subdev_ops);
> +       sd->internal_ops = &unicam_subdev_internal_ops;
> +       v4l2_set_subdevdata(sd, unicam);
> +
> +       sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> +       sd->entity.ops = &unicam_subdev_media_ops;
> +       sd->dev = unicam->dev;
> +       sd->owner = THIS_MODULE;
> +       sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_STREAMS;
> +
> +       strscpy(sd->name, "unicam", sizeof(sd->name));
> +
> +       unicam->subdev.pads[UNICAM_SD_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> +       unicam->subdev.pads[UNICAM_SD_PAD_SOURCE_IMAGE].flags = MEDIA_PAD_FL_SOURCE;
> +       unicam->subdev.pads[UNICAM_SD_PAD_SOURCE_METADATA].flags = MEDIA_PAD_FL_SOURCE;
> +
> +       ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(unicam->subdev.pads),
> +                                    unicam->subdev.pads);
> +       if (ret) {
> +               dev_err(unicam->dev, "Failed to initialize media entity: %d\n",
> +                       ret);
> +               return ret;
> +       }
> +
> +       ret = v4l2_subdev_init_finalize(sd);
> +       if (ret) {
> +               dev_err(unicam->dev, "Failed to initialize subdev: %d\n", ret);
> +               goto err_entity;
> +       }
> +
> +       ret = v4l2_device_register_subdev(&unicam->v4l2_dev, sd);
> +       if (ret) {
> +               dev_err(unicam->dev, "Failed to register subdev: %d\n", ret);
> +               goto err_subdev;
> +       }
> +
> +       return 0;
> +
> +err_subdev:
> +       v4l2_subdev_cleanup(sd);
> +err_entity:
> +       media_entity_cleanup(&sd->entity);
> +       return ret;
> +}
> +
> +static void unicam_subdev_cleanup(struct unicam_device *unicam)
> +{
> +       v4l2_subdev_cleanup(&unicam->subdev.sd);
> +       media_entity_cleanup(&unicam->subdev.sd.entity);
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * Videobuf2 queue operations
> + */
> +
> +static int unicam_queue_setup(struct vb2_queue *vq,
> +                             unsigned int *nbuffers,
> +                             unsigned int *nplanes,
> +                             unsigned int sizes[],
> +                             struct device *alloc_devs[])
> +{
> +       struct unicam_node *node = vb2_get_drv_priv(vq);
> +       u32 size = is_image_node(node) ? node->fmt.fmt.pix.sizeimage
> +                : node->fmt.fmt.meta.buffersize;
> +
> +       if (vq->num_buffers + *nbuffers < 3)
> +               *nbuffers = 3 - vq->num_buffers;
> +
> +       if (*nplanes) {
> +               if (sizes[0] < size) {
> +                       dev_dbg(node->dev->dev, "sizes[0] %i < size %u\n",
> +                               sizes[0], size);
> +                       return -EINVAL;
> +               }
> +               size = sizes[0];
> +       }
> +
> +       *nplanes = 1;
> +       sizes[0] = size;
> +
> +       return 0;
> +}hgjk
> +
> +static int unicam_buffer_prepare(struct vb2_buffer *vb)


> +{
> +       struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
> +       struct unicam_buffer *buf = to_unicam_buffer(vb);
> +       u32 size = is_image_node(node) ? node->fmt.fmt.pix.sizeimage
> +                : node->fmt.fmt.meta.buffersize;
> +
> +       if (vb2_plane_size(vb, 0) < size) {
> +               dev_dbg(node->dev->dev,
> +                       "data will not fit into plane (%lu < %u)\n",
> +                       vb2_plane_size(vb, 0), size);
> +               return -EINVAL;
> +       }
> +
> +       buf->dma_addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
> +       buf->size = size;
> +
> +       vb2_set_plane_payload(&buf->vb.vb2_buf, 0, size);
> +
> +       return 0;
> +}
> +
> +static void unicam_return_buffers(struct unicam_node *node,
> +                                 enum vb2_buffer_state state)
> +{
> +       struct unicam_buffer *buf, *tmp;
> +
> +       list_for_each_entry_safe(buf, tmp, &node->dma_queue, list) {
> +               list_del(&buf->list);
> +               vb2_buffer_done(&buf->vb.vb2_buf, state);
> +       }
> +
> +       if (node->cur_frm)
> +               vb2_buffer_done(&node->cur_frm->vb.vb2_buf,
> +                               state);
> +       if (node->next_frm && node->cur_frm != node->next_frm)
> +               vb2_buffer_done(&node->next_frm->vb.vb2_buf,
> +                               state);
> +
> +       node->cur_frm = NULL;
> +       node->next_frm = NULL;
> +}
> +
> +static int unicam_start_streaming(struct vb2_queue *vq, unsigned int count)
> +{
> +       struct unicam_node *node = vb2_get_drv_priv(vq);
> +       struct unicam_device *unicam = node->dev;
> +       struct v4l2_subdev_state *state;
> +       struct unicam_buffer *buf;
> +       unsigned long flags;
> +       int ret;
> +       u32 pad, stream;
> +       u32 remote_pad = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
> +                                            : UNICAM_SD_PAD_SOURCE_METADATA;
> +
> +       /* Look for the route for the given pad and stream. */
> +       state = v4l2_subdev_lock_and_get_active_state(&unicam->subdev.sd);
> +       ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
> +                                                   remote_pad, 0,
> +                                                   &pad, &stream);
> +       v4l2_subdev_unlock_state(state);
> +
> +       if (ret)
> +               goto err_return_buffers;
> +
> +       dev_dbg(unicam->dev, "Starting stream on %s: %u/%u -> %u/%u (%s)\n",
> +               unicam->subdev.sd.name, pad, stream, remote_pad, 0,
> +               is_metadata_node(node) ? "metadata" : "image");
> +
> +       /* The metadata node can't be started alone. */
> +       if (is_metadata_node(node)) {
> +               if (!unicam->node[UNICAM_IMAGE_NODE].streaming) {
> +                       dev_err(unicam->dev,
> +                               "Can't start metadata without image\n");
> +                       ret = -EINVAL;
> +                       goto err_return_buffers;
> +               }

There's a slight change of behaviour in this function when compared to
the downstream/BSP non-streams enabled driver.

In the BSP driver, if the embedded data node has been enabled, we wait
for both image and embedded data nodes to have start_streaming()
called before starting the sensor (see
https://github.com/raspberrypi/linux/blob/c04af98514c26014a4f29ec87b3ece95626059bd/drivers/media/platform/bcm2835/bcm2835-unicam.c#L2559).
This is also the same for the Pi 5 CFE driver.

With the logic in this function, we only wait for start_streaming() on
the image node then start the sensor streaming immediately.  When
start_streaming() for the embedded data node is subsequently called,
we end up with the first N buffers missing and/or invalid as the HW
channel is enabled while the sensor is streaming.  I noticed this when
using libcamera where we start image then embedded node.  If I flip
things around (start embedded first then image), everything works as
expected.

Could we add back the test to ensure all nodes are streaming before
starting the sensor?

Regards,
Naush

> +
> +               spin_lock_irqsave(&node->dma_queue_lock, flags);
> +               buf = list_first_entry(&node->dma_queue,
> +                                      struct unicam_buffer, list);
> +               dev_dbg(unicam->dev, "buffer %p\n", buf);
> +               node->cur_frm = buf;
> +               node->next_frm = buf;
> +               list_del(&buf->list);
> +               spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> +
> +               unicam_start_metadata(unicam, buf);
> +               node->streaming = true;
> +               return 0;
> +       }
> +
> +       ret = pm_runtime_resume_and_get(unicam->dev);
> +       if (ret < 0) {
> +               dev_err(unicam->dev, "PM runtime resume failed: %d\n", ret);
> +               goto err_return_buffers;
> +       }
> +
> +       ret = video_device_pipeline_start(&node->video_dev, &unicam->pipe);
> +       if (ret < 0) {
> +               dev_dbg(unicam->dev, "Failed to start media pipeline: %d\n", ret);
> +               goto err_pm_put;
> +       }
> +
> +       spin_lock_irqsave(&node->dma_queue_lock, flags);
> +       buf = list_first_entry(&node->dma_queue,
> +                              struct unicam_buffer, list);
> +       dev_dbg(unicam->dev, "buffer %p\n", buf);
> +       node->cur_frm = buf;
> +       node->next_frm = buf;
> +       list_del(&buf->list);
> +       spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> +
> +       unicam_start_rx(unicam, buf);
> +
> +       ret = v4l2_subdev_enable_streams(&unicam->subdev.sd, remote_pad, BIT(0));
> +       if (ret < 0) {
> +               dev_err(unicam->dev, "stream on failed in subdev\n");
> +               goto error_pipeline;
> +       }
> +
> +       node->streaming = true;
> +
> +       return 0;
> +
> +error_pipeline:
> +       video_device_pipeline_stop(&node->video_dev);
> +err_pm_put:
> +       pm_runtime_put_sync(unicam->dev);
> +err_return_buffers:
> +       unicam_return_buffers(node, VB2_BUF_STATE_QUEUED);
> +       return ret;
> +}
> +
> +static void unicam_stop_streaming(struct vb2_queue *vq)
> +{
> +       struct unicam_node *node = vb2_get_drv_priv(vq);
> +       struct unicam_device *unicam = node->dev;
> +       u32 remote_pad_index = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
> +                                                  : UNICAM_SD_PAD_SOURCE_METADATA;
> +
> +       node->streaming = false;
> +
> +       v4l2_subdev_disable_streams(&unicam->subdev.sd, remote_pad_index,
> +                                   BIT(0));
> +
> +       /* We can stream only with the image node. */
> +       if (is_metadata_node(node)) {
> +               /*
> +                * Allow the hardware to spin in the dummy buffer.
> +                * This is only really needed if the embedded data pad is
> +                * disabled before the image pad.
> +                */
> +               unicam_wr_dma_addr(node, &node->dummy_buf);
> +               goto dequeue_buffers;
> +       }
> +
> +       unicam_disable(unicam);
> +
> +       video_device_pipeline_stop(&node->video_dev);
> +       pm_runtime_put(unicam->dev);
> +
> +dequeue_buffers:
> +       /* Clear all queued buffers for the node */
> +       unicam_return_buffers(node, VB2_BUF_STATE_ERROR);
> +}
> +
> +static void unicam_buffer_queue(struct vb2_buffer *vb)
> +{
> +       struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
> +       struct unicam_buffer *buf = to_unicam_buffer(vb);
> +
> +       spin_lock_irq(&node->dma_queue_lock);
> +       list_add_tail(&buf->list, &node->dma_queue);
> +       spin_unlock_irq(&node->dma_queue_lock);
> +}
> +
> +static const struct vb2_ops unicam_video_qops = {
> +       .queue_setup            = unicam_queue_setup,
> +       .wait_prepare           = vb2_ops_wait_prepare,
> +       .wait_finish            = vb2_ops_wait_finish,
> +       .buf_prepare            = unicam_buffer_prepare,
> +       .start_streaming        = unicam_start_streaming,
> +       .stop_streaming         = unicam_stop_streaming,
> +       .buf_queue              = unicam_buffer_queue,
> +};
> +
> +/* -----------------------------------------------------------------------------
> + *  V4L2 video device operations
> + */
> +
> +static int unicam_querycap(struct file *file, void *priv,
> +                          struct v4l2_capability *cap)
> +{
> +       strscpy(cap->driver, UNICAM_MODULE_NAME, sizeof(cap->driver));
> +       strscpy(cap->card, UNICAM_MODULE_NAME, sizeof(cap->card));
> +
> +       cap->capabilities |= V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_META_CAPTURE;
> +
> +       return 0;
> +}
> +
> +static int unicam_enum_fmt_vid(struct file *file, void  *priv,
> +                              struct v4l2_fmtdesc *f)
> +{
> +       unsigned int index;
> +       unsigned int i;
> +
> +       for (i = 0, index = 0; i < ARRAY_SIZE(unicam_image_formats); i++) {
> +               if (f->mbus_code && unicam_image_formats[i].code != f->mbus_code)
> +                       continue;
> +
> +               if (index == f->index) {
> +                       f->pixelformat = unicam_image_formats[i].fourcc;
> +                       return 0;
> +               }
> +
> +               index++;
> +
> +               if (!unicam_image_formats[i].unpacked_fourcc)
> +                       continue;
> +
> +               if (index == f->index) {
> +                       f->pixelformat = unicam_image_formats[i].unpacked_fourcc;
> +                       return 0;
> +               }
> +
> +               index++;
> +       }
> +
> +       return -EINVAL;
> +}
> +
> +static int unicam_g_fmt_vid(struct file *file, void *priv,
> +                           struct v4l2_format *f)
> +{
> +       struct unicam_node *node = video_drvdata(file);
> +
> +       *f = node->fmt;
> +
> +       return 0;
> +}
> +
> +static const struct unicam_format_info *
> +__unicam_try_fmt_vid(struct unicam_node *node, struct v4l2_pix_format *pix)
> +{
> +       const struct unicam_format_info *fmtinfo;
> +
> +       /*
> +        * Default to the first format if the requested pixel format code isn't
> +        * supported.
> +        */
> +       fmtinfo = unicam_find_format_by_fourcc(pix->pixelformat,
> +                                              UNICAM_SD_PAD_SOURCE_IMAGE);
> +       if (!fmtinfo) {
> +               fmtinfo = &unicam_image_formats[0];
> +               pix->pixelformat = fmtinfo->fourcc;
> +       }
> +
> +       unicam_calc_image_size_bpl(node->dev, fmtinfo, pix);
> +
> +       if (pix->field == V4L2_FIELD_ANY)
> +               pix->field = V4L2_FIELD_NONE;
> +
> +       return fmtinfo;
> +}
> +
> +static int unicam_try_fmt_vid(struct file *file, void *priv,
> +                             struct v4l2_format *f)
> +{
> +       struct unicam_node *node = video_drvdata(file);
> +
> +       __unicam_try_fmt_vid(node, &f->fmt.pix);
> +       return 0;
> +}
> +
> +static int unicam_s_fmt_vid(struct file *file, void *priv,
> +                           struct v4l2_format *f)
> +{
> +       struct unicam_node *node = video_drvdata(file);
> +
> +       if (vb2_is_busy(&node->buffer_queue))
> +               return -EBUSY;
> +
> +       node->fmtinfo = __unicam_try_fmt_vid(node, &f->fmt.pix);
> +       node->fmt = *f;
> +
> +       return 0;
> +}
> +
> +static int unicam_enum_fmt_meta(struct file *file, void *priv,
> +                               struct v4l2_fmtdesc *f)
> +{
> +       unsigned int i, index;
> +
> +       for (i = 0, index = 0; i < ARRAY_SIZE(unicam_meta_formats); i++) {
> +               if (f->mbus_code && unicam_meta_formats[i].code != f->mbus_code)
> +                       continue;
> +               if (!unicam_meta_formats[i].metadata_fmt)
> +                       continue;
> +
> +               if (index == f->index) {
> +                       f->pixelformat = unicam_meta_formats[i].fourcc;
> +                       f->type = V4L2_BUF_TYPE_META_CAPTURE;
> +                       return 0;
> +               }
> +               index++;
> +       }
> +
> +       return -EINVAL;
> +}
> +
> +static int unicam_g_fmt_meta(struct file *file, void *priv,
> +                            struct v4l2_format *f)
> +{
> +       struct unicam_node *node = video_drvdata(file);
> +
> +       f->fmt.meta = node->fmt.fmt.meta;
> +
> +       return 0;
> +}
> +
> +static const struct unicam_format_info *
> +__unicam_try_fmt_meta(struct unicam_node *node, struct v4l2_meta_format *meta)
> +{
> +       const struct unicam_format_info *fmtinfo;
> +
> +       /*
> +        * Default to the first format if the requested pixel format code isn't
> +        * supported.
> +        */
> +       fmtinfo = unicam_find_format_by_fourcc(meta->dataformat,
> +                                              UNICAM_SD_PAD_SOURCE_METADATA);
> +       if (!fmtinfo) {
> +               fmtinfo = &unicam_meta_formats[0];
> +               meta->dataformat = fmtinfo->fourcc;
> +       }
> +
> +       unicam_calc_meta_size_bpl(node->dev, fmtinfo, meta);
> +
> +       return fmtinfo;
> +}
> +
> +static int unicam_try_fmt_meta(struct file *file, void *priv,
> +                              struct v4l2_format *f)
> +{
> +       struct unicam_node *node = video_drvdata(file);
> +
> +       __unicam_try_fmt_vid(node, &f->fmt.pix);
> +       return 0;
> +}
> +
> +static int unicam_s_fmt_meta(struct file *file, void *priv,
> +                            struct v4l2_format *f)
> +{
> +       struct unicam_node *node = video_drvdata(file);
> +
> +       if (vb2_is_busy(&node->buffer_queue))
> +               return -EBUSY;
> +
> +       node->fmtinfo = __unicam_try_fmt_meta(node, &f->fmt.meta);
> +       node->fmt = *f;
> +
> +       return 0;
> +}
> +
> +static int unicam_enum_framesizes(struct file *file, void *fh,
> +                                 struct v4l2_frmsizeenum *fsize)
> +{
> +       struct unicam_node *node = video_drvdata(file);
> +       int ret = -EINVAL;
> +
> +       if (fsize->index > 0)
> +               return ret;
> +
> +       if (is_image_node(node)) {
> +               if (!unicam_find_format_by_fourcc(fsize->pixel_format,
> +                                                 UNICAM_SD_PAD_SOURCE_IMAGE))
> +                       return ret;
> +
> +               fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> +               fsize->stepwise.min_width = UNICAM_IMAGE_MIN_WIDTH;
> +               fsize->stepwise.max_width = UNICAM_IMAGE_MAX_WIDTH;
> +               fsize->stepwise.step_width = 1;
> +               fsize->stepwise.min_height = UNICAM_IMAGE_MIN_HEIGHT;
> +               fsize->stepwise.max_height = UNICAM_IMAGE_MAX_HEIGHT;
> +               fsize->stepwise.step_height = 1;
> +       } else {
> +               if (!unicam_find_format_by_fourcc(fsize->pixel_format,
> +                                                 UNICAM_SD_PAD_SOURCE_METADATA))
> +                       return ret;
> +
> +               fsize->stepwise.min_width = UNICAM_META_MIN_WIDTH;
> +               fsize->stepwise.max_width = UNICAM_META_MAX_WIDTH;
> +               fsize->stepwise.step_width = 1;
> +               fsize->stepwise.min_height = UNICAM_META_MIN_HEIGHT;
> +               fsize->stepwise.max_height = UNICAM_META_MAX_HEIGHT;
> +               fsize->stepwise.step_height = 1;
> +       }
> +
> +       return 0;
> +}
> +
> +static int unicam_log_status(struct file *file, void *fh)
> +{
> +       struct unicam_node *node = video_drvdata(file);
> +       struct unicam_device *unicam = node->dev;
> +       u32 reg;
> +
> +       /* status for sub devices */
> +       v4l2_device_call_all(&unicam->v4l2_dev, 0, core, log_status);
> +
> +       dev_info(unicam->dev, "-----Receiver status-----\n");
> +       dev_info(unicam->dev, "V4L2 width/height:   %ux%u\n",
> +                node->fmt.fmt.pix.width, node->fmt.fmt.pix.height);
> +       dev_info(unicam->dev, "Mediabus format:     %08x\n",
> +                node->fmtinfo->code);
> +       dev_info(unicam->dev, "V4L2 format:         %08x\n",
> +                node->fmt.fmt.pix.pixelformat);
> +       reg = unicam_reg_read(unicam, UNICAM_IPIPE);
> +       dev_info(unicam->dev, "Unpacking/packing:   %u / %u\n",
> +                unicam_get_field(reg, UNICAM_PUM_MASK),
> +                unicam_get_field(reg, UNICAM_PPM_MASK));
> +       dev_info(unicam->dev, "----Live data----\n");
> +       dev_info(unicam->dev, "Programmed stride:   %4u\n",
> +                unicam_reg_read(unicam, UNICAM_IBLS));
> +       dev_info(unicam->dev, "Detected resolution: %ux%u\n",
> +                unicam_reg_read(unicam, UNICAM_IHSTA),
> +                unicam_reg_read(unicam, UNICAM_IVSTA));
> +       dev_info(unicam->dev, "Write pointer:       %08x\n",
> +                unicam_reg_read(unicam, UNICAM_IBWP));
> +
> +       return 0;
> +}
> +
> +static int unicam_subscribe_event(struct v4l2_fh *fh,
> +                                 const struct v4l2_event_subscription *sub)
> +{
> +       switch (sub->type) {
> +       case V4L2_EVENT_FRAME_SYNC:
> +               return v4l2_event_subscribe(fh, sub, 2, NULL);
> +       default:
> +               return -EINVAL;
> +       }
> +}
> +
> +static const struct v4l2_ioctl_ops unicam_ioctl_ops = {
> +       .vidioc_querycap                = unicam_querycap,
> +
> +       .vidioc_enum_fmt_vid_cap        = unicam_enum_fmt_vid,
> +       .vidioc_g_fmt_vid_cap           = unicam_g_fmt_vid,
> +       .vidioc_try_fmt_vid_cap         = unicam_try_fmt_vid,
> +       .vidioc_s_fmt_vid_cap           = unicam_s_fmt_vid,
> +
> +       .vidioc_enum_fmt_meta_cap       = unicam_enum_fmt_meta,
> +       .vidioc_g_fmt_meta_cap          = unicam_g_fmt_meta,
> +       .vidioc_try_fmt_meta_cap        = unicam_try_fmt_meta,
> +       .vidioc_s_fmt_meta_cap          = unicam_s_fmt_meta,
> +
> +       .vidioc_enum_framesizes         = unicam_enum_framesizes,
> +
> +       .vidioc_reqbufs                 = vb2_ioctl_reqbufs,
> +       .vidioc_create_bufs             = vb2_ioctl_create_bufs,
> +       .vidioc_prepare_buf             = vb2_ioctl_prepare_buf,
> +       .vidioc_querybuf                = vb2_ioctl_querybuf,
> +       .vidioc_qbuf                    = vb2_ioctl_qbuf,
> +       .vidioc_dqbuf                   = vb2_ioctl_dqbuf,
> +       .vidioc_expbuf                  = vb2_ioctl_expbuf,
> +       .vidioc_streamon                = vb2_ioctl_streamon,
> +       .vidioc_streamoff               = vb2_ioctl_streamoff,
> +
> +       .vidioc_log_status              = unicam_log_status,
> +       .vidioc_subscribe_event         = unicam_subscribe_event,
> +       .vidioc_unsubscribe_event       = v4l2_event_unsubscribe,
> +};
> +
> +/* unicam capture driver file operations */
> +static const struct v4l2_file_operations unicam_fops = {
> +       .owner          = THIS_MODULE,
> +       .open           = v4l2_fh_open,
> +       .release        = vb2_fop_release,
> +       .poll           = vb2_fop_poll,
> +       .unlocked_ioctl = video_ioctl2,
> +       .mmap           = vb2_fop_mmap,
> +};
> +
> +static int unicam_video_link_validate(struct media_link *link)
> +{
> +       struct video_device *vdev =
> +               media_entity_to_video_device(link->sink->entity);
> +       struct v4l2_subdev *sd =
> +               media_entity_to_v4l2_subdev(link->source->entity);
> +       struct unicam_node *node = video_get_drvdata(vdev);
> +       const u32 pad = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
> +                     : UNICAM_SD_PAD_SOURCE_METADATA;
> +       const struct v4l2_mbus_framefmt *format;
> +       struct v4l2_subdev_state *state;
> +       int ret = 0;
> +
> +       state = v4l2_subdev_lock_and_get_active_state(sd);
> +
> +       format = v4l2_subdev_state_get_format(state, pad, 0);
> +       if (!format) {
> +               ret = -EINVAL;
> +               goto out;
> +       }
> +
> +       if (is_image_node(node)) {
> +               const struct v4l2_pix_format *fmt = &node->fmt.fmt.pix;
> +
> +               if (node->fmtinfo->code != format->code ||
> +                   fmt->height != format->height ||
> +                   fmt->width != format->width ||
> +                   fmt->field != format->field) {
> +                       dev_dbg(node->dev->dev,
> +                               "image: (%u x %u) 0x%08x %s != (%u x %u) 0x%08x %s\n",
> +                               fmt->width, fmt->height, node->fmtinfo->code,
> +                               v4l2_field_names[fmt->field],
> +                               format->width, format->height, format->code,
> +                               v4l2_field_names[format->field]);
> +                       ret = -EPIPE;
> +               };
> +       } else {
> +               const struct v4l2_meta_format *fmt = &node->fmt.fmt.meta;
> +
> +               if (node->fmtinfo->code != format->code ||
> +                   fmt->height != format->height ||
> +                   fmt->width != format->width) {
> +                       dev_dbg(node->dev->dev,
> +                               "meta: (%u x %u) 0x%04x != (%u x %u) 0x%04x\n",
> +                               fmt->width, fmt->height, node->fmtinfo->code,
> +                               format->width, format->height, format->code);
> +                       ret = -EPIPE;
> +               };
> +       }
> +
> +out:
> +       v4l2_subdev_unlock_state(state);
> +       return ret;
> +}
> +
> +static const struct media_entity_operations unicam_video_media_ops = {
> +       .link_validate = unicam_video_link_validate,
> +};
> +
> +static void unicam_node_release(struct video_device *vdev)
> +{
> +       struct unicam_node *node = video_get_drvdata(vdev);
> +
> +       unicam_put(node->dev);
> +}
> +
> +static void unicam_set_default_format(struct unicam_node *node)
> +{
> +       if (is_image_node(node)) {
> +               struct v4l2_pix_format *fmt = &node->fmt.fmt.pix;
> +
> +               node->fmtinfo = &unicam_image_formats[0];
> +               node->fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> +
> +               v4l2_fill_pix_format(fmt, &unicam_default_image_format);
> +               fmt->pixelformat = node->fmtinfo->fourcc;
> +               unicam_calc_image_size_bpl(node->dev, node->fmtinfo, fmt);
> +       } else {
> +               struct v4l2_meta_format *fmt = &node->fmt.fmt.meta;
> +
> +               node->fmtinfo = &unicam_meta_formats[0];
> +               node->fmt.type = V4L2_BUF_TYPE_META_CAPTURE;
> +
> +               fmt->dataformat = node->fmtinfo->fourcc;
> +               fmt->width = unicam_default_meta_format.width;
> +               fmt->height = unicam_default_meta_format.height;
> +               unicam_calc_meta_size_bpl(node->dev, node->fmtinfo, fmt);
> +       }
> +}
> +
> +static int unicam_register_node(struct unicam_device *unicam,
> +                               enum unicam_node_type type)
> +{
> +       const u32 pad_index = type == UNICAM_IMAGE_NODE
> +                           ? UNICAM_SD_PAD_SOURCE_IMAGE
> +                           : UNICAM_SD_PAD_SOURCE_METADATA;
> +       struct unicam_node *node = &unicam->node[type];
> +       struct video_device *vdev = &node->video_dev;
> +       struct vb2_queue *q = &node->buffer_queue;
> +       int ret;
> +
> +       node->dev = unicam_get(unicam);
> +       node->id = type;
> +
> +       spin_lock_init(&node->dma_queue_lock);
> +       mutex_init(&node->lock);
> +
> +       INIT_LIST_HEAD(&node->dma_queue);
> +
> +       /* Initialize the videobuf2 queue. */
> +       q->type = type == UNICAM_IMAGE_NODE ? V4L2_BUF_TYPE_VIDEO_CAPTURE
> +                                           : V4L2_BUF_TYPE_META_CAPTURE;
> +       q->io_modes = VB2_MMAP | VB2_DMABUF;
> +       q->drv_priv = node;
> +       q->ops = &unicam_video_qops;
> +       q->mem_ops = &vb2_dma_contig_memops;
> +       q->buf_struct_size = sizeof(struct unicam_buffer);
> +       q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> +       q->lock = &node->lock;
> +       q->min_queued_buffers = 1;
> +       q->dev = unicam->dev;
> +
> +       ret = vb2_queue_init(q);
> +       if (ret) {
> +               dev_err(unicam->dev, "vb2_queue_init() failed\n");
> +               goto err_unicam_put;
> +       }
> +
> +       /* Initialize the video device. */
> +       vdev->release = unicam_node_release;
> +       vdev->fops = &unicam_fops;
> +       vdev->ioctl_ops = &unicam_ioctl_ops;
> +       vdev->v4l2_dev = &unicam->v4l2_dev;
> +       vdev->vfl_dir = VFL_DIR_RX;
> +       vdev->queue = q;
> +       vdev->lock = &node->lock;
> +       vdev->device_caps = type == UNICAM_IMAGE_NODE
> +                         ? V4L2_CAP_VIDEO_CAPTURE : V4L2_CAP_META_CAPTURE;
> +       vdev->device_caps |= V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;
> +       vdev->entity.ops = &unicam_video_media_ops;
> +
> +       snprintf(vdev->name, sizeof(vdev->name), "%s-%s", UNICAM_MODULE_NAME,
> +                type == UNICAM_IMAGE_NODE ? "image" : "embedded");
> +
> +       video_set_drvdata(vdev, node);
> +
> +       if (type == UNICAM_IMAGE_NODE)
> +               vdev->entity.flags |= MEDIA_ENT_FL_DEFAULT;
> +
> +       node->pad.flags = MEDIA_PAD_FL_SINK;
> +
> +       ret = media_entity_pads_init(&vdev->entity, 1, &node->pad);
> +       if (ret)
> +               goto err_unicam_put;
> +
> +       node->dummy_buf.size = UNICAM_DUMMY_BUF_SIZE;
> +       node->dummy_buf_cpu_addr = dma_alloc_coherent(unicam->dev,
> +                                                     node->dummy_buf.size,
> +                                                     &node->dummy_buf.dma_addr,
> +                                                     GFP_KERNEL);
> +       if (!node->dummy_buf_cpu_addr) {
> +               dev_err(unicam->dev, "Unable to allocate dummy buffer.\n");
> +               ret = -ENOMEM;
> +               goto err_entity_cleanup;
> +       }
> +
> +       unicam_set_default_format(node);
> +
> +       ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> +       if (ret) {
> +               dev_err(unicam->dev, "Unable to register video device %s\n",
> +                       vdev->name);
> +               goto err_dma_free;
> +       }
> +
> +       node->registered = true;
> +
> +       ret = media_create_pad_link(&unicam->subdev.sd.entity,
> +                                   pad_index,
> +                                   &node->video_dev.entity,
> +                                   0,
> +                                   MEDIA_LNK_FL_ENABLED |
> +                                   MEDIA_LNK_FL_IMMUTABLE);
> +       if (ret) {
> +               /*
> +                * No need for cleanup, the caller will unregister the
> +                * video device, which will drop the reference on the
> +                * device and trigger the cleanup.
> +                */
> +               dev_err(unicam->dev, "Unable to create pad link for %s\n",
> +                       unicam->sensor.subdev->name);
> +               return ret;
> +       }
> +
> +       return 0;
> +
> +err_dma_free:
> +       dma_free_coherent(unicam->dev, node->dummy_buf.size,
> +                         node->dummy_buf_cpu_addr,
> +                         node->dummy_buf.dma_addr);
> +err_entity_cleanup:
> +       media_entity_cleanup(&vdev->entity);
> +err_unicam_put:
> +       unicam_put(unicam);
> +       return ret;
> +}
> +
> +static void unicam_unregister_nodes(struct unicam_device *unicam)
> +{
> +       unsigned int i;
> +
> +       for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> +               struct unicam_node *node = &unicam->node[i];
> +
> +               if (node->dummy_buf_cpu_addr)
> +                       dma_free_coherent(unicam->dev, node->dummy_buf.size,
> +                                         node->dummy_buf_cpu_addr,
> +                                         node->dummy_buf.dma_addr);
> +
> +               if (node->registered) {
> +                       video_unregister_device(&node->video_dev);
> +                       node->registered = false;
> +               }
> +       }
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * Power management
> + */
> +
> +static int unicam_runtime_resume(struct device *dev)
> +{
> +       struct unicam_device *unicam = dev_get_drvdata(dev);
> +       int ret;
> +
> +       ret = clk_set_min_rate(unicam->vpu_clock, UNICAM_MIN_VPU_CLOCK_RATE);
> +       if (ret) {
> +               dev_err(unicam->dev, "failed to set up VPU clock\n");
> +               return ret;
> +       }
> +
> +       ret = clk_prepare_enable(unicam->vpu_clock);
> +       if (ret) {
> +               dev_err(unicam->dev, "Failed to enable VPU clock: %d\n", ret);
> +               goto err_vpu_clock;
> +       }
> +
> +       ret = clk_set_rate(unicam->clock, 100 * 1000 * 1000);
> +       if (ret) {
> +               dev_err(unicam->dev, "failed to set up CSI clock\n");
> +               goto err_vpu_prepare;
> +       }
> +
> +       ret = clk_prepare_enable(unicam->clock);
> +       if (ret) {
> +               dev_err(unicam->dev, "Failed to enable CSI clock: %d\n", ret);
> +               goto err_vpu_prepare;
> +       }
> +
> +       return 0;
> +
> +err_vpu_prepare:
> +       clk_disable_unprepare(unicam->vpu_clock);
> +err_vpu_clock:
> +       if (clk_set_min_rate(unicam->vpu_clock, 0))
> +               dev_err(unicam->dev, "failed to reset the VPU clock\n");
> +
> +       return ret;
> +}
> +
> +static int unicam_runtime_suspend(struct device *dev)
> +{
> +       struct unicam_device *unicam = dev_get_drvdata(dev);
> +
> +       clk_disable_unprepare(unicam->clock);
> +
> +       if (clk_set_min_rate(unicam->vpu_clock, 0))
> +               dev_err(unicam->dev, "failed to reset the VPU clock\n");
> +
> +       clk_disable_unprepare(unicam->vpu_clock);
> +
> +       return 0;
> +}
> +
> +static const struct dev_pm_ops unicam_pm_ops = {
> +       RUNTIME_PM_OPS(unicam_runtime_suspend, unicam_runtime_resume, NULL)
> +};
> +
> +/* -----------------------------------------------------------------------------
> + * V4L2 async notifier
> + */
> +
> +static int unicam_async_bound(struct v4l2_async_notifier *notifier,
> +                             struct v4l2_subdev *subdev,
> +                             struct v4l2_async_connection *asc)
> +{
> +       struct unicam_device *unicam = notifier_to_unicam_device(notifier);
> +       struct media_pad *sink = &unicam->subdev.pads[UNICAM_SD_PAD_SINK];
> +       struct media_pad *source;
> +       int ret;
> +
> +       dev_dbg(unicam->dev, "Using sensor %s for capture\n",
> +               subdev->name);
> +
> +       ret = v4l2_create_fwnode_links_to_pad(subdev, sink, MEDIA_LNK_FL_ENABLED |
> +                                             MEDIA_LNK_FL_IMMUTABLE);
> +       if (ret)
> +               return ret;
> +
> +       source = media_pad_remote_pad_unique(sink);
> +       if (!source) {
> +               dev_err(unicam->dev, "No connected sensor pad\n");
> +               return -ENOTCONN;
> +       }
> +
> +       unicam->sensor.subdev = subdev;
> +       unicam->sensor.pad = source;
> +
> +       return 0;
> +}
> +
> +static int unicam_async_complete(struct v4l2_async_notifier *notifier)
> +{
> +       struct unicam_device *unicam = notifier_to_unicam_device(notifier);
> +       int ret;
> +
> +       ret = unicam_register_node(unicam, UNICAM_IMAGE_NODE);
> +       if (ret) {
> +               dev_err(unicam->dev, "Unable to register image video device.\n");
> +               goto unregister;
> +       }
> +
> +       ret = unicam_register_node(unicam, UNICAM_METADATA_NODE);
> +       if (ret) {
> +               dev_err(unicam->dev, "Unable to register metadata video device.\n");
> +               goto unregister;
> +       }
> +
> +       ret = v4l2_device_register_subdev_nodes(&unicam->v4l2_dev);
> +       if (ret) {
> +               dev_err(unicam->dev, "Unable to register subdev nodes.\n");
> +               goto unregister;
> +       }
> +
> +       return 0;
> +
> +unregister:
> +       unicam_unregister_nodes(unicam);
> +       unicam_put(unicam);
> +
> +       return ret;
> +}
> +
> +static const struct v4l2_async_notifier_operations unicam_async_ops = {
> +       .bound = unicam_async_bound,
> +       .complete = unicam_async_complete,
> +};
> +
> +static int unicam_async_nf_init(struct unicam_device *unicam)
> +{
> +       struct v4l2_fwnode_endpoint ep = { };
> +       struct fwnode_handle *ep_handle;
> +       struct v4l2_async_connection *asc;
> +       int ret;
> +
> +       ret = of_property_read_u32(unicam->dev->of_node, "brcm,num-data-lanes",
> +                                  &unicam->max_data_lanes);
> +       if (ret < 0) {
> +               dev_err(unicam->dev, "Missing %s DT property\n",
> +                       "brcm,num-data-lanes");
> +               return -EINVAL;
> +       }
> +
> +       /* Get and parse the local endpoint. */
> +       ep_handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(unicam->dev), 0, 0,
> +                                                   FWNODE_GRAPH_ENDPOINT_NEXT);
> +       if (!ep_handle) {
> +               dev_err(unicam->dev, "No endpoint found\n");
> +               return -ENODEV;
> +       }
> +
> +       ret = v4l2_fwnode_endpoint_parse(ep_handle, &ep);
> +       if (ret) {
> +               dev_err(unicam->dev, "Failed to parse endpoint: %d\n", ret);
> +               goto error;
> +       }
> +
> +       unicam->bus_type = ep.bus_type;
> +
> +       switch (ep.bus_type) {
> +       case V4L2_MBUS_CSI2_DPHY: {
> +               unsigned int num_data_lanes = ep.bus.mipi_csi2.num_data_lanes;
> +
> +               if (num_data_lanes != 1 && num_data_lanes != 2 &&
> +                   num_data_lanes != 4) {
> +                       dev_err(unicam->dev, "%u data lanes not supported\n",
> +                               num_data_lanes);
> +                       goto error;
> +               }
> +
> +               if (num_data_lanes > unicam->max_data_lanes) {
> +                       dev_err(unicam->dev,
> +                               "Endpoint uses %u data lanes when %u are supported\n",
> +                               num_data_lanes, unicam->max_data_lanes);
> +                       goto error;
> +               }
> +
> +               unicam->active_data_lanes = num_data_lanes;
> +               unicam->bus_flags = ep.bus.mipi_csi2.flags;
> +               break;
> +       }
> +
> +       case V4L2_MBUS_CCP2:
> +               unicam->max_data_lanes = 1;
> +               unicam->active_data_lanes = 1;
> +               unicam->bus_flags = ep.bus.mipi_csi1.strobe;
> +               break;
> +
> +       default:
> +               /* Unsupported bus type */
> +               dev_err(unicam->dev, "Unsupported bus type %u\n", ep.bus_type);
> +               goto error;
> +       }
> +
> +       /* Initialize and register the async notifier. */
> +       v4l2_async_nf_init(&unicam->notifier, &unicam->v4l2_dev);
> +
> +       asc = v4l2_async_nf_add_fwnode_remote(&unicam->notifier, ep_handle,
> +                                             struct v4l2_async_connection);
> +       fwnode_handle_put(ep_handle);
> +       ep_handle = NULL;
> +
> +       if (IS_ERR(asc)) {
> +               ret = PTR_ERR(asc);
> +               dev_err(unicam->dev, "Failed to add entry to notifier: %d\n",
> +                       ret);
> +               goto error;
> +       }
> +
> +       unicam->notifier.ops = &unicam_async_ops;
> +
> +       ret = v4l2_async_nf_register(&unicam->notifier);
> +       if (ret) {
> +               dev_err(unicam->dev, "Error registering device notifier: %d\n",
> +                       ret);
> +               goto error;
> +       }
> +
> +       return 0;
> +
> +error:
> +       fwnode_handle_put(ep_handle);
> +       return ret;
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * Probe & remove
> + */
> +
> +static int unicam_media_init(struct unicam_device *unicam)
> +{
> +       int ret;
> +
> +       unicam->mdev.dev = unicam->dev;
> +       strscpy(unicam->mdev.model, UNICAM_MODULE_NAME,
> +               sizeof(unicam->mdev.model));
> +       strscpy(unicam->mdev.serial, "", sizeof(unicam->mdev.serial));
> +       unicam->mdev.hw_revision = 0;
> +
> +       media_device_init(&unicam->mdev);
> +
> +       unicam->v4l2_dev.mdev = &unicam->mdev;
> +
> +       ret = v4l2_device_register(unicam->dev, &unicam->v4l2_dev);
> +       if (ret < 0) {
> +               dev_err(unicam->dev, "Unable to register v4l2 device\n");
> +               goto err_media_cleanup;
> +       }
> +
> +       ret = media_device_register(&unicam->mdev);
> +       if (ret < 0) {
> +               dev_err(unicam->dev,
> +                       "Unable to register media-controller device\n");
> +               goto err_v4l2_unregister;
> +       }
> +
> +       return 0;
> +
> +err_v4l2_unregister:
> +       v4l2_device_unregister(&unicam->v4l2_dev);
> +err_media_cleanup:
> +       media_device_cleanup(&unicam->mdev);
> +       return ret;
> +}
> +
> +static int unicam_probe(struct platform_device *pdev)
> +{
> +       struct unicam_device *unicam;
> +       int ret;
> +
> +       unicam = kzalloc(sizeof(*unicam), GFP_KERNEL);
> +       if (!unicam)
> +               return -ENOMEM;
> +
> +       kref_init(&unicam->kref);
> +       unicam->dev = &pdev->dev;
> +       platform_set_drvdata(pdev, unicam);
> +
> +       unicam->base = devm_platform_ioremap_resource_byname(pdev, "unicam");
> +       if (IS_ERR(unicam->base)) {
> +               ret = PTR_ERR(unicam->base);
> +               goto err_unicam_put;
> +       }
> +
> +       unicam->clk_gate_base = devm_platform_ioremap_resource_byname(pdev, "cmi");
> +       if (IS_ERR(unicam->clk_gate_base)) {
> +               ret = PTR_ERR(unicam->clk_gate_base);
> +               goto err_unicam_put;
> +       }
> +
> +       unicam->clock = devm_clk_get(&pdev->dev, "lp");
> +       if (IS_ERR(unicam->clock)) {
> +               dev_err(unicam->dev, "Failed to get lp clock\n");
> +               ret = PTR_ERR(unicam->clock);
> +               goto err_unicam_put;
> +       }
> +
> +       unicam->vpu_clock = devm_clk_get(&pdev->dev, "vpu");
> +       if (IS_ERR(unicam->vpu_clock)) {
> +               dev_err(unicam->dev, "Failed to get vpu clock\n");
> +               ret = PTR_ERR(unicam->vpu_clock);
> +               goto err_unicam_put;
> +       }
> +
> +       ret = platform_get_irq(pdev, 0);
> +       if (ret <= 0) {
> +               dev_err(&pdev->dev, "No IRQ resource\n");
> +               ret = -EINVAL;
> +               goto err_unicam_put;
> +       }
> +
> +       ret = devm_request_irq(&pdev->dev, ret, unicam_isr, 0,
> +                              "unicam_capture0", unicam);
> +       if (ret) {
> +               dev_err(&pdev->dev, "Unable to request interrupt\n");
> +               ret = -EINVAL;
> +               goto err_unicam_put;
> +       }
> +
> +       /* Enable the block power domain. */
> +       pm_runtime_enable(&pdev->dev);
> +
> +       ret = unicam_media_init(unicam);
> +       if (ret)
> +               goto err_pm_runtime;
> +
> +       ret = unicam_subdev_init(unicam);
> +       if (ret)
> +               goto err_media_unregister;
> +
> +       ret = unicam_async_nf_init(unicam);
> +       if (ret)
> +               goto err_subdev_unregister;
> +
> +       return 0;
> +
> +err_subdev_unregister:
> +       unicam_subdev_cleanup(unicam);
> +err_media_unregister:
> +       media_device_unregister(&unicam->mdev);
> +err_pm_runtime:
> +       pm_runtime_disable(&pdev->dev);
> +err_unicam_put:
> +       unicam_put(unicam);
> +
> +       return ret;
> +}
> +
> +static int unicam_remove(struct platform_device *pdev)
> +{
> +       struct unicam_device *unicam = platform_get_drvdata(pdev);
> +
> +       unicam_unregister_nodes(unicam);
> +       v4l2_device_unregister(&unicam->v4l2_dev);
> +       media_device_unregister(&unicam->mdev);
> +       v4l2_async_nf_unregister(&unicam->notifier);
> +
> +       unicam_subdev_cleanup(unicam);
> +
> +       unicam_put(unicam);
> +
> +       pm_runtime_disable(&pdev->dev);
> +
> +       return 0;
> +}
> +
> +static const struct of_device_id unicam_of_match[] = {
> +       { .compatible = "brcm,bcm2835-unicam", },
> +       { /* sentinel */ },
> +};
> +MODULE_DEVICE_TABLE(of, unicam_of_match);
> +
> +static struct platform_driver unicam_driver = {
> +       .probe          = unicam_probe,
> +       .remove         = unicam_remove,
> +       .driver = {
> +               .name   = UNICAM_MODULE_NAME,
> +               .pm     = pm_ptr(&unicam_pm_ops),
> +               .of_match_table = of_match_ptr(unicam_of_match),
> +       },
> +};
> +
> +module_platform_driver(unicam_driver);
> +
> +MODULE_AUTHOR("Dave Stevenson <dave.stevenson@raspberrypi.com>");
> +MODULE_DESCRIPTION("BCM2835 Unicam driver");
> +MODULE_LICENSE("GPL");

> --
> Regards,
>
> Laurent Pinchart
>

Re: [PATCH v6 09/15] media: bcm2835-unicam: Add support for CCP2/CSI2 camera interface

[Laurent Pinchart]

Hi Naush,

On Wed, Mar 20, 2024 at 12:30:36PM +0000, Naushir Patuck wrote:
> On Fri, 1 Mar 2024 at 21:32, Laurent Pinchart wrote:
> >
> > From: Dave Stevenson <dave.stevenson@raspberrypi.com>
> >
> > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > It is represented as two video device nodes: unicam-image and
> > unicam-embedded which are connected to an internal subdev (named
> > unicam-subdev) in order to manage streams routing.
> >
> > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > Co-developed-by: Naushir Patuck <naush@raspberrypi.com>
> > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > Co-developed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > ---

[snip]

> > ---
> >  MAINTAINERS                                   |    1 +
> >  drivers/media/platform/Kconfig                |    1 +
> >  drivers/media/platform/Makefile               |    1 +
> >  drivers/media/platform/broadcom/Kconfig       |   23 +
> >  drivers/media/platform/broadcom/Makefile      |    3 +
> >  .../platform/broadcom/bcm2835-unicam-regs.h   |  255 ++
> >  .../media/platform/broadcom/bcm2835-unicam.c  | 2607 +++++++++++++++++
> >  7 files changed, 2891 insertions(+)
> >  create mode 100644 drivers/media/platform/broadcom/Kconfig
> >  create mode 100644 drivers/media/platform/broadcom/Makefile
> >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam.c

[snip]

> > diff --git a/drivers/media/platform/broadcom/bcm2835-unicam.c b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > new file mode 100644
> > index 000000000000..716c89b8a217
> > --- /dev/null
> > +++ b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > @@ -0,0 +1,2607 @@

[snip]

> > +static int unicam_start_streaming(struct vb2_queue *vq, unsigned int count)
> > +{
> > +       struct unicam_node *node = vb2_get_drv_priv(vq);
> > +       struct unicam_device *unicam = node->dev;
> > +       struct v4l2_subdev_state *state;
> > +       struct unicam_buffer *buf;
> > +       unsigned long flags;
> > +       int ret;
> > +       u32 pad, stream;
> > +       u32 remote_pad = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
> > +                                            : UNICAM_SD_PAD_SOURCE_METADATA;
> > +
> > +       /* Look for the route for the given pad and stream. */
> > +       state = v4l2_subdev_lock_and_get_active_state(&unicam->subdev.sd);
> > +       ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
> > +                                                   remote_pad, 0,
> > +                                                   &pad, &stream);
> > +       v4l2_subdev_unlock_state(state);
> > +
> > +       if (ret)
> > +               goto err_return_buffers;
> > +
> > +       dev_dbg(unicam->dev, "Starting stream on %s: %u/%u -> %u/%u (%s)\n",
> > +               unicam->subdev.sd.name, pad, stream, remote_pad, 0,
> > +               is_metadata_node(node) ? "metadata" : "image");
> > +
> > +       /* The metadata node can't be started alone. */
> > +       if (is_metadata_node(node)) {
> > +               if (!unicam->node[UNICAM_IMAGE_NODE].streaming) {
> > +                       dev_err(unicam->dev,
> > +                               "Can't start metadata without image\n");
> > +                       ret = -EINVAL;
> > +                       goto err_return_buffers;
> > +               }
> 
> There's a slight change of behaviour in this function when compared to
> the downstream/BSP non-streams enabled driver.
> 
> In the BSP driver, if the embedded data node has been enabled, we wait
> for both image and embedded data nodes to have start_streaming()
> called before starting the sensor (see
> https://github.com/raspberrypi/linux/blob/c04af98514c26014a4f29ec87b3ece95626059bd/drivers/media/platform/bcm2835/bcm2835-unicam.c#L2559).
> This is also the same for the Pi 5 CFE driver.
> 
> With the logic in this function, we only wait for start_streaming() on
> the image node then start the sensor streaming immediately.  When
> start_streaming() for the embedded data node is subsequently called,
> we end up with the first N buffers missing and/or invalid as the HW
> channel is enabled while the sensor is streaming.  I noticed this when
> using libcamera where we start image then embedded node.  If I flip
> things around (start embedded first then image), everything works as
> expected.
> 
> Could we add back the test to ensure all nodes are streaming before
> starting the sensor?

Yes, I don't think the current implementation is good. I'm not sure why
the logic got changed, but I'll address it in the next version.

> > +
> > +               spin_lock_irqsave(&node->dma_queue_lock, flags);
> > +               buf = list_first_entry(&node->dma_queue,
> > +                                      struct unicam_buffer, list);
> > +               dev_dbg(unicam->dev, "buffer %p\n", buf);
> > +               node->cur_frm = buf;
> > +               node->next_frm = buf;
> > +               list_del(&buf->list);
> > +               spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> > +
> > +               unicam_start_metadata(unicam, buf);
> > +               node->streaming = true;
> > +               return 0;
> > +       }
> > +
> > +       ret = pm_runtime_resume_and_get(unicam->dev);
> > +       if (ret < 0) {
> > +               dev_err(unicam->dev, "PM runtime resume failed: %d\n", ret);
> > +               goto err_return_buffers;
> > +       }
> > +
> > +       ret = video_device_pipeline_start(&node->video_dev, &unicam->pipe);
> > +       if (ret < 0) {
> > +               dev_dbg(unicam->dev, "Failed to start media pipeline: %d\n", ret);
> > +               goto err_pm_put;
> > +       }
> > +
> > +       spin_lock_irqsave(&node->dma_queue_lock, flags);
> > +       buf = list_first_entry(&node->dma_queue,
> > +                              struct unicam_buffer, list);
> > +       dev_dbg(unicam->dev, "buffer %p\n", buf);
> > +       node->cur_frm = buf;
> > +       node->next_frm = buf;
> > +       list_del(&buf->list);
> > +       spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> > +
> > +       unicam_start_rx(unicam, buf);
> > +
> > +       ret = v4l2_subdev_enable_streams(&unicam->subdev.sd, remote_pad, BIT(0));
> > +       if (ret < 0) {
> > +               dev_err(unicam->dev, "stream on failed in subdev\n");
> > +               goto error_pipeline;
> > +       }
> > +
> > +       node->streaming = true;
> > +
> > +       return 0;
> > +
> > +error_pipeline:
> > +       video_device_pipeline_stop(&node->video_dev);
> > +err_pm_put:
> > +       pm_runtime_put_sync(unicam->dev);
> > +err_return_buffers:
> > +       unicam_return_buffers(node, VB2_BUF_STATE_QUEUED);
> > +       return ret;
> > +}

[snip]

-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 09/15] media: bcm2835-unicam: Add support for CCP2/CSI2 camera interface

[Jean-Michel Hautbois]

Hi Laurent, Naush,

On 21/03/2024 20:57, Laurent Pinchart wrote:
> Hi Naush,
> 
> On Wed, Mar 20, 2024 at 12:30:36PM +0000, Naushir Patuck wrote:
>> On Fri, 1 Mar 2024 at 21:32, Laurent Pinchart wrote:
>>>
>>> From: Dave Stevenson <dave.stevenson@raspberrypi.com>
>>>
>>> Add a driver for the Unicam camera receiver block on BCM283x processors.
>>> It is represented as two video device nodes: unicam-image and
>>> unicam-embedded which are connected to an internal subdev (named
>>> unicam-subdev) in order to manage streams routing.
>>>
>>> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
>>> Co-developed-by: Naushir Patuck <naush@raspberrypi.com>
>>> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
>>> Co-developed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
>>> Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
>>> Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>> ---
> 
> [snip]
> 
>>> ---
>>>   MAINTAINERS                                   |    1 +
>>>   drivers/media/platform/Kconfig                |    1 +
>>>   drivers/media/platform/Makefile               |    1 +
>>>   drivers/media/platform/broadcom/Kconfig       |   23 +
>>>   drivers/media/platform/broadcom/Makefile      |    3 +
>>>   .../platform/broadcom/bcm2835-unicam-regs.h   |  255 ++
>>>   .../media/platform/broadcom/bcm2835-unicam.c  | 2607 +++++++++++++++++
>>>   7 files changed, 2891 insertions(+)
>>>   create mode 100644 drivers/media/platform/broadcom/Kconfig
>>>   create mode 100644 drivers/media/platform/broadcom/Makefile
>>>   create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam-regs.h
>>>   create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam.c
> 
> [snip]
> 
>>> diff --git a/drivers/media/platform/broadcom/bcm2835-unicam.c b/drivers/media/platform/broadcom/bcm2835-unicam.c
>>> new file mode 100644
>>> index 000000000000..716c89b8a217
>>> --- /dev/null
>>> +++ b/drivers/media/platform/broadcom/bcm2835-unicam.c
>>> @@ -0,0 +1,2607 @@
> 
> [snip]
> 
>>> +static int unicam_start_streaming(struct vb2_queue *vq, unsigned int count)
>>> +{
>>> +       struct unicam_node *node = vb2_get_drv_priv(vq);
>>> +       struct unicam_device *unicam = node->dev;
>>> +       struct v4l2_subdev_state *state;
>>> +       struct unicam_buffer *buf;
>>> +       unsigned long flags;
>>> +       int ret;
>>> +       u32 pad, stream;
>>> +       u32 remote_pad = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
>>> +                                            : UNICAM_SD_PAD_SOURCE_METADATA;
>>> +
>>> +       /* Look for the route for the given pad and stream. */
>>> +       state = v4l2_subdev_lock_and_get_active_state(&unicam->subdev.sd);
>>> +       ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
>>> +                                                   remote_pad, 0,
>>> +                                                   &pad, &stream);
>>> +       v4l2_subdev_unlock_state(state);
>>> +
>>> +       if (ret)
>>> +               goto err_return_buffers;
>>> +
>>> +       dev_dbg(unicam->dev, "Starting stream on %s: %u/%u -> %u/%u (%s)\n",
>>> +               unicam->subdev.sd.name, pad, stream, remote_pad, 0,
>>> +               is_metadata_node(node) ? "metadata" : "image");
>>> +
>>> +       /* The metadata node can't be started alone. */
>>> +       if (is_metadata_node(node)) {
>>> +               if (!unicam->node[UNICAM_IMAGE_NODE].streaming) {
>>> +                       dev_err(unicam->dev,
>>> +                               "Can't start metadata without image\n");
>>> +                       ret = -EINVAL;
>>> +                       goto err_return_buffers;
>>> +               }
>>
>> There's a slight change of behaviour in this function when compared to
>> the downstream/BSP non-streams enabled driver.
>>
>> In the BSP driver, if the embedded data node has been enabled, we wait
>> for both image and embedded data nodes to have start_streaming()
>> called before starting the sensor (see
>> https://github.com/raspberrypi/linux/blob/c04af98514c26014a4f29ec87b3ece95626059bd/drivers/media/platform/bcm2835/bcm2835-unicam.c#L2559).
>> This is also the same for the Pi 5 CFE driver.
>>
>> With the logic in this function, we only wait for start_streaming() on
>> the image node then start the sensor streaming immediately.  When
>> start_streaming() for the embedded data node is subsequently called,
>> we end up with the first N buffers missing and/or invalid as the HW
>> channel is enabled while the sensor is streaming.  I noticed this when
>> using libcamera where we start image then embedded node.  If I flip
>> things around (start embedded first then image), everything works as
>> expected.
>>
>> Could we add back the test to ensure all nodes are streaming before
>> starting the sensor?
> 
> Yes, I don't think the current implementation is good. I'm not sure why
> the logic got changed, but I'll address it in the next version.
> 

I think I did it in v4. The aim of this change was to be able to start 
streaming on the image node even if the metadata node is not started. 
The opposite was not to be possible, so if the metadata node is started, 
then the image node must be started for streaming to begin. The 
downstream logic is that both nodes must always be started, but at the 
time of v4, not everything was ready for this, if I remember correctly.

JM

>>> +
>>> +               spin_lock_irqsave(&node->dma_queue_lock, flags);
>>> +               buf = list_first_entry(&node->dma_queue,
>>> +                                      struct unicam_buffer, list);
>>> +               dev_dbg(unicam->dev, "buffer %p\n", buf);
>>> +               node->cur_frm = buf;
>>> +               node->next_frm = buf;
>>> +               list_del(&buf->list);
>>> +               spin_unlock_irqrestore(&node->dma_queue_lock, flags);
>>> +
>>> +               unicam_start_metadata(unicam, buf);
>>> +               node->streaming = true;
>>> +               return 0;
>>> +       }
>>> +
>>> +       ret = pm_runtime_resume_and_get(unicam->dev);
>>> +       if (ret < 0) {
>>> +               dev_err(unicam->dev, "PM runtime resume failed: %d\n", ret);
>>> +               goto err_return_buffers;
>>> +       }
>>> +
>>> +       ret = video_device_pipeline_start(&node->video_dev, &unicam->pipe);
>>> +       if (ret < 0) {
>>> +               dev_dbg(unicam->dev, "Failed to start media pipeline: %d\n", ret);
>>> +               goto err_pm_put;
>>> +       }
>>> +
>>> +       spin_lock_irqsave(&node->dma_queue_lock, flags);
>>> +       buf = list_first_entry(&node->dma_queue,
>>> +                              struct unicam_buffer, list);
>>> +       dev_dbg(unicam->dev, "buffer %p\n", buf);
>>> +       node->cur_frm = buf;
>>> +       node->next_frm = buf;
>>> +       list_del(&buf->list);
>>> +       spin_unlock_irqrestore(&node->dma_queue_lock, flags);
>>> +
>>> +       unicam_start_rx(unicam, buf);
>>> +
>>> +       ret = v4l2_subdev_enable_streams(&unicam->subdev.sd, remote_pad, BIT(0));
>>> +       if (ret < 0) {
>>> +               dev_err(unicam->dev, "stream on failed in subdev\n");
>>> +               goto error_pipeline;
>>> +       }
>>> +
>>> +       node->streaming = true;
>>> +
>>> +       return 0;
>>> +
>>> +error_pipeline:
>>> +       video_device_pipeline_stop(&node->video_dev);
>>> +err_pm_put:
>>> +       pm_runtime_put_sync(unicam->dev);
>>> +err_return_buffers:
>>> +       unicam_return_buffers(node, VB2_BUF_STATE_QUEUED);
>>> +       return ret;
>>> +}
> 
> [snip]
> 

Re: [PATCH v6 08/15] dt-bindings: media: Add bindings for bcm2835-unicam

[Sakari Ailus]

Hi Laurent,

Thanks for the set.

On Fri, Mar 01, 2024 at 11:32:23PM +0200, Laurent Pinchart wrote:
> From: Dave Stevenson <dave.stevenson@raspberrypi.com>
> 
> Introduce the dt-bindings documentation for bcm2835 CCP2/CSI2 Unicam
> camera interface.
> 
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> Co-developed-by: Naushir Patuck <naush@raspberrypi.com>
> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> Co-developed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Reviewed-by: Rob Herring <robh@kernel.org>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
> Changes since v5:
> 
> - Squash MAINTAINERS changes in
> 
> Changes since v3:
> 
> - Make MAINTAINERS its own patch
> - Describe the reg and clocks correctly
> - Use a vendor entry for the number of data lanes
> ---
>  .../bindings/media/brcm,bcm2835-unicam.yaml   | 117 ++++++++++++++++++
>  MAINTAINERS                                   |   6 +
>  2 files changed, 123 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> 
> diff --git a/Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml b/Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> new file mode 100644
> index 000000000000..1938ace23b3d
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> @@ -0,0 +1,117 @@
> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/media/brcm,bcm2835-unicam.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Broadcom BCM283x Camera Interface (Unicam)
> +
> +maintainers:
> +  - Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
> +
> +description: |-
> +  The Unicam block on BCM283x SoCs is the receiver for either
> +  CSI-2 or CCP2 data from image sensors or similar devices.
> +
> +  The main platform using this SoC is the Raspberry Pi family of boards.  On
> +  the Pi the VideoCore firmware can also control this hardware block, and
> +  driving it from two different processors will cause issues.  To avoid this,
> +  the firmware checks the device tree configuration during boot. If it finds
> +  device tree nodes whose name starts with 'csi' then it will stop the firmware
> +  accessing the block, and it can then safely be used via the device tree
> +  binding.
> +
> +properties:
> +  compatible:
> +    const: brcm,bcm2835-unicam
> +
> +  reg:
> +    items:
> +      - description: Unicam block.
> +      - description: Clock Manager Image (CMI) block.
> +
> +  reg-names:
> +    items:
> +      - const: unicam
> +      - const: cmi
> +
> +  interrupts:
> +    maxItems: 1
> +
> +  clocks:
> +    items:
> +      - description: Clock to drive the LP state machine of Unicam.
> +      - description: Clock for the VPU (core clock).
> +
> +  clock-names:
> +    items:
> +      - const: lp
> +      - const: vpu
> +
> +  power-domains:
> +    items:
> +      - description: Unicam power domain
> +
> +  brcm,num-data-lanes:
> +    $ref: /schemas/types.yaml#/definitions/uint32
> +    enum: [ 2, 4 ]
> +    description: |
> +      Number of CSI-2 data lanes supported by this Unicam instance. The number
> +      of data lanes actively used is specified with the data-lanes endpoint
> +      property.
> +
> +  port:
> +    $ref: /schemas/graph.yaml#/$defs/port-base
> +    unevaluatedProperties: false
> +
> +    properties:
> +      endpoint:
> +        $ref: /schemas/media/video-interfaces.yaml#
> +        unevaluatedProperties: false
> +
> +        properties:
> +          data-lanes: true
> +
> +        required:
> +          - data-lanes

As the device supports multiple data interfaces (at least so it seems when
looking at the driver code), you need to list the bus-type property here,
too.

> +
> +    required:
> +      - endpoint
> +
> +required:
> +  - compatible
> +  - reg
> +  - reg-names
> +  - interrupts
> +  - clocks
> +  - clock-names
> +  - power-domains
> +  - brcm,num-data-lanes
> +  - port
> +
> +additionalProperties: False
> +
> +examples:
> +  - |
> +    #include <dt-bindings/clock/bcm2835.h>
> +    #include <dt-bindings/interrupt-controller/arm-gic.h>
> +    #include <dt-bindings/power/raspberrypi-power.h>
> +    csi1: csi@7e801000 {
> +        compatible = "brcm,bcm2835-unicam";
> +        reg = <0x7e801000 0x800>,
> +              <0x7e802004 0x4>;
> +        reg-names = "unicam", "cmi";
> +        interrupts = <GIC_SPI 103 IRQ_TYPE_LEVEL_HIGH>;
> +        clocks = <&clocks BCM2835_CLOCK_CAM1>,
> +                 <&firmware_clocks 4>;
> +        clock-names = "lp", "vpu";
> +        power-domains = <&power RPI_POWER_DOMAIN_UNICAM1>;
> +        brcm,num-data-lanes = <2>;
> +        port {
> +                csi1_ep: endpoint {
> +                        remote-endpoint = <&imx219_0>;
> +                        data-lanes = <1 2>;
> +                };
> +        };
> +    };
> +...
> diff --git a/MAINTAINERS b/MAINTAINERS
> index fada59148cb5..e50a59654e6e 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -3997,6 +3997,12 @@ N:	bcm113*
>  N:	bcm216*
>  N:	kona
>  
> +BROADCOM BCM2835 CAMERA DRIVERS
> +M:	Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
> +L:	linux-media@vger.kernel.org
> +S:	Maintained
> +F:	Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> +
>  BROADCOM BCM47XX MIPS ARCHITECTURE
>  M:	Hauke Mehrtens <hauke@hauke-m.de>
>  M:	Rafał Miłecki <zajec5@gmail.com>

-- 
Regards,

Sakari Ailus

Re: [PATCH v6 09/15] media: bcm2835-unicam: Add support for CCP2/CSI2 camera interface

[Sakari Ailus]

Hi Laurent,

On Fri, Mar 01, 2024 at 11:32:24PM +0200, Laurent Pinchart wrote:
> From: Dave Stevenson <dave.stevenson@raspberrypi.com>
> 
> Add a driver for the Unicam camera receiver block on BCM283x processors.
> It is represented as two video device nodes: unicam-image and
> unicam-embedded which are connected to an internal subdev (named
> unicam-subdev) in order to manage streams routing.
> 
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> Co-developed-by: Naushir Patuck <naush@raspberrypi.com>
> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> Co-developed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Thanks for submitting this, it's the cleanest and neatest Unicom driver
I've ever seen!

Some mostly unimportant comments below, however the bus-type issue needs to
be addressed.

> ---
> Changes since v5:
> 
> - Move to drivers/media/platform/broadcom/
> - Port to the upstream V4L2 streams API
> - Rebase on latest metadata API proposal
> - Add missing error message
> - Drop unneeded documentation block for unicam_isr()
> - Drop unneeded dev_dbg() and dev_err() messages
> - Drop unneeded streams_mask and fmt checks
> - Drop unused unicam_sd_pad_is_sink()
> - Drop unneeded includes
> - Drop v4l2_ctrl_subscribe_event() call
> - Use pm_runtime_resume_and_get()
> - Indentation and line wrap fixes
> - Let the framework set bus_info
> - Use v4l2_fwnode_endpoint_parse()
> - Fix media device cleanup
> - Drop lane reordering checks
> - Fix subdev state locking
> - Drop extra debug messages
> - Move clock handling to runtime PM handlers
> - Reorder functions
> - Rename init functions for more clarity
> - Initialize runtime PM earlier
> - Clarify error messages
> - Simplify subdev init with local variable
> - Fix subdev cleanup
> - Fix typos and indentation
> - Don't initialize local variables needlessly
> - Simplify num lanes check
> - Fix metadata handling in subdev set_fmt
> - Drop manual fallback to .s_stream()
> - Pass v4l2_pix_format to unicam_calc_format_size_bpl()
> - Simplify unicam_set_default_format()
> - Fix default format settings
> - Add busy check in unicam_s_fmt_meta()
> - Add missing \n at end of format strings
> - Fix metadata handling in subdev set_fmt
> - Fix locking when starting streaming
> - Return buffers from start streaming fails
> - Fix format validation for metadata node
> - Use video_device_pipeline_{start,stop}() helpers
> - Simplify format enumeration
> - Drop unset variable
> - Update MAINTAINERS entry
> - Update to the upstream v4l2_async_nf API
> - Update to the latest subdev routing API
> - Update to the latest subdev state API
> - Move from subdev .init_cfg() to .init_state()
> - Update to the latest videobuf2 API
> - Fix v4l2_subdev_enable_streams() error check
> - Use correct pad for the connected subdev
> - Return buffers to vb2 when start streaming fails
> - Improve debugging in start streaming handler
> - Simplify DMA address management
> - Drop comment about bcm2835-camera driver
> - Clarify comments that explain min/max sizes
> - Pass v4l2_pix_format to unicam_try_fmt()
> - Drop unneeded local variables
> - Rename image-related constants and functions
> - Turn unicam_fmt.metadata_fmt into bool
> - Rename unicam_fmt to unicam_format_info
> - Rename unicam_format_info variables to fmtinfo
> - Rename unicam_node.v_fmt to fmt
> - Add metadata formats for RAW10, RAW12 and RAW14
> - Make metadata formats line-based
> - Validate format on metadata video device
> - Add Co-devlopped-by tags
> 
> Changes since v3:
> 
> - Add the vendor prefix for DT name
> - Use the reg-names in DT parsing
> - Remove MAINTAINERS entry
> 
> Changes since v2:
> 
> - Change code organization
> - Remove unused variables
> - Correct the fmt_meta functions
> - Rewrite the start/stop streaming
>   - You can now start the image node alone, but not the metadata one
>   - The buffers are allocated per-node
>   - only the required stream is started, if the route exists and is
>     enabled
> - Prefix the macros with UNICAM_ to not have too generic names
> - Drop colorspace support
> 
> Changes since v1:
> 
> - Replace the unicam_{info,debug,error} macros with dev_*()
> ---
>  MAINTAINERS                                   |    1 +
>  drivers/media/platform/Kconfig                |    1 +
>  drivers/media/platform/Makefile               |    1 +
>  drivers/media/platform/broadcom/Kconfig       |   23 +
>  drivers/media/platform/broadcom/Makefile      |    3 +
>  .../platform/broadcom/bcm2835-unicam-regs.h   |  255 ++
>  .../media/platform/broadcom/bcm2835-unicam.c  | 2607 +++++++++++++++++
>  7 files changed, 2891 insertions(+)
>  create mode 100644 drivers/media/platform/broadcom/Kconfig
>  create mode 100644 drivers/media/platform/broadcom/Makefile
>  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam-regs.h
>  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam.c
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index e50a59654e6e..cc350729f467 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -4002,6 +4002,7 @@ M:	Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
>  L:	linux-media@vger.kernel.org
>  S:	Maintained
>  F:	Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> +F:	drivers/media/platform/bcm2835/
>  
>  BROADCOM BCM47XX MIPS ARCHITECTURE
>  M:	Hauke Mehrtens <hauke@hauke-m.de>
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index 91e54215de3a..2d79bfc68c15 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -67,6 +67,7 @@ source "drivers/media/platform/amlogic/Kconfig"
>  source "drivers/media/platform/amphion/Kconfig"
>  source "drivers/media/platform/aspeed/Kconfig"
>  source "drivers/media/platform/atmel/Kconfig"
> +source "drivers/media/platform/broadcom/Kconfig"
>  source "drivers/media/platform/cadence/Kconfig"
>  source "drivers/media/platform/chips-media/Kconfig"
>  source "drivers/media/platform/intel/Kconfig"
> diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> index 3296ec1ebe16..da17301f7439 100644
> --- a/drivers/media/platform/Makefile
> +++ b/drivers/media/platform/Makefile
> @@ -10,6 +10,7 @@ obj-y += amlogic/
>  obj-y += amphion/
>  obj-y += aspeed/
>  obj-y += atmel/
> +obj-y += broadcom/
>  obj-y += cadence/
>  obj-y += chips-media/
>  obj-y += intel/
> diff --git a/drivers/media/platform/broadcom/Kconfig b/drivers/media/platform/broadcom/Kconfig
> new file mode 100644
> index 000000000000..cc2c9afcc948
> --- /dev/null
> +++ b/drivers/media/platform/broadcom/Kconfig
> @@ -0,0 +1,23 @@
> +# SPDX-License-Identifier: GPL-2.0
> +
> +config VIDEO_BCM2835_UNICAM
> +	tristate "Broadcom BCM283x/BCM271x Unicam video capture driver"
> +	depends on ARCH_BCM2835 || COMPILE_TEST
> +	depends on PM
> +	depends on VIDEO_DEV
> +	select MEDIA_CONTROLLER
> +	select V4L2_FWNODE
> +	select VIDEO_V4L2_SUBDEV_API
> +	select VIDEOBUF2_DMA_CONTIG
> +	help
> +	  Say Y here to enable support for the BCM283x/BCM271x CSI-2 receiver.
> +	  This is a V4L2 driver that controls the CSI-2 receiver directly,
> +	  independently from the VC4 firmware.
> +
> +	  This driver is mutually exclusive with the use of bcm2835-camera. The
> +	  firmware will disable all access to the peripheral from within the
> +	  firmware if it finds a DT node using it, and bcm2835-camera will
> +	  therefore fail to probe.
> +
> +	  To compile this driver as a module, choose M here. The module will be
> +	  called bcm2835-unicam.
> diff --git a/drivers/media/platform/broadcom/Makefile b/drivers/media/platform/broadcom/Makefile
> new file mode 100644
> index 000000000000..03d2045aba2e
> --- /dev/null
> +++ b/drivers/media/platform/broadcom/Makefile
> @@ -0,0 +1,3 @@
> +# SPDX-License-Identifier: GPL-2.0
> +
> +obj-$(CONFIG_VIDEO_BCM2835_UNICAM) += bcm2835-unicam.o
> diff --git a/drivers/media/platform/broadcom/bcm2835-unicam-regs.h b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> new file mode 100644
> index 000000000000..84775fd2fac5
> --- /dev/null
> +++ b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> @@ -0,0 +1,255 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +
> +/*
> + * Copyright (C) 2017-2020 Raspberry Pi Trading.

Anything up to 2024?


> + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> + */
> +
> +#ifndef VC4_REGS_UNICAM_H
> +#define VC4_REGS_UNICAM_H
> +
> +#include <linux/bits.h>
> +
> +/*
> + * The following values are taken from files found within the code drop
> + * made by Broadcom for the BCM21553 Graphics Driver, predominantly in
> + * brcm_usrlib/dag/vmcsx/vcinclude/hardware_vc4.h.
> + * They have been modified to be only the register offset.
> + */
> +#define UNICAM_CTRL		0x000
> +#define UNICAM_STA		0x004
> +#define UNICAM_ANA		0x008
> +#define UNICAM_PRI		0x00c
> +#define UNICAM_CLK		0x010
> +#define UNICAM_CLT		0x014
> +#define UNICAM_DAT0		0x018
> +#define UNICAM_DAT1		0x01c
> +#define UNICAM_DAT2		0x020
> +#define UNICAM_DAT3		0x024
> +#define UNICAM_DLT		0x028
> +#define UNICAM_CMP0		0x02c
> +#define UNICAM_CMP1		0x030
> +#define UNICAM_CAP0		0x034
> +#define UNICAM_CAP1		0x038
> +#define UNICAM_ICTL		0x100
> +#define UNICAM_ISTA		0x104
> +#define UNICAM_IDI0		0x108
> +#define UNICAM_IPIPE		0x10c
> +#define UNICAM_IBSA0		0x110
> +#define UNICAM_IBEA0		0x114
> +#define UNICAM_IBLS		0x118
> +#define UNICAM_IBWP		0x11c
> +#define UNICAM_IHWIN		0x120
> +#define UNICAM_IHSTA		0x124
> +#define UNICAM_IVWIN		0x128
> +#define UNICAM_IVSTA		0x12c
> +#define UNICAM_ICC		0x130
> +#define UNICAM_ICS		0x134
> +#define UNICAM_IDC		0x138
> +#define UNICAM_IDPO		0x13c
> +#define UNICAM_IDCA		0x140
> +#define UNICAM_IDCD		0x144
> +#define UNICAM_IDS		0x148
> +#define UNICAM_DCS		0x200
> +#define UNICAM_DBSA0		0x204
> +#define UNICAM_DBEA0		0x208
> +#define UNICAM_DBWP		0x20c
> +#define UNICAM_DBCTL		0x300
> +#define UNICAM_IBSA1		0x304
> +#define UNICAM_IBEA1		0x308
> +#define UNICAM_IDI1		0x30c
> +#define UNICAM_DBSA1		0x310
> +#define UNICAM_DBEA1		0x314
> +#define UNICAM_MISC		0x400
> +
> +/*
> + * The following bitmasks are from the kernel released by Broadcom
> + * for Android - https://android.googlesource.com/kernel/bcm/
> + * The Rhea, Hawaii, and Java chips all contain the same VideoCore4
> + * Unicam block as BCM2835, as defined in eg
> + * arch/arm/mach-rhea/include/mach/rdb_A0/brcm_rdb_cam.h and similar.
> + * Values reworked to use the kernel BIT and GENMASK macros.
> + *
> + * Some of the bit mnenomics have been amended to match the datasheet.
> + */
> +/* UNICAM_CTRL Register */
> +#define UNICAM_CPE		BIT(0)
> +#define UNICAM_MEM		BIT(1)
> +#define UNICAM_CPR		BIT(2)
> +#define UNICAM_CPM_MASK		GENMASK(3, 3)
> +#define UNICAM_CPM_CSI2		0
> +#define UNICAM_CPM_CCP2		1
> +#define UNICAM_SOE		BIT(4)
> +#define UNICAM_DCM_MASK		GENMASK(5, 5)
> +#define UNICAM_DCM_STROBE	0
> +#define UNICAM_DCM_DATA		1
> +#define UNICAM_SLS		BIT(6)
> +#define UNICAM_PFT_MASK		GENMASK(11, 8)
> +#define UNICAM_OET_MASK		GENMASK(20, 12)
> +
> +/* UNICAM_STA Register */
> +#define UNICAM_SYN		BIT(0)
> +#define UNICAM_CS		BIT(1)
> +#define UNICAM_SBE		BIT(2)
> +#define UNICAM_PBE		BIT(3)
> +#define UNICAM_HOE		BIT(4)
> +#define UNICAM_PLE		BIT(5)
> +#define UNICAM_SSC		BIT(6)
> +#define UNICAM_CRCE		BIT(7)
> +#define UNICAM_OES		BIT(8)
> +#define UNICAM_IFO		BIT(9)
> +#define UNICAM_OFO		BIT(10)
> +#define UNICAM_BFO		BIT(11)
> +#define UNICAM_DL		BIT(12)
> +#define UNICAM_PS		BIT(13)
> +#define UNICAM_IS		BIT(14)
> +#define UNICAM_PI0		BIT(15)
> +#define UNICAM_PI1		BIT(16)
> +#define UNICAM_FSI_S		BIT(17)
> +#define UNICAM_FEI_S		BIT(18)
> +#define UNICAM_LCI_S		BIT(19)
> +#define UNICAM_BUF0_RDY		BIT(20)
> +#define UNICAM_BUF0_NO		BIT(21)
> +#define UNICAM_BUF1_RDY		BIT(22)
> +#define UNICAM_BUF1_NO		BIT(23)
> +#define UNICAM_DI		BIT(24)
> +
> +#define UNICAM_STA_MASK_ALL \
> +		(UNICAM_DL | \
> +		UNICAM_SBE | \

This and the lines below should start at right of the opening parenthesis,
not at it.

> +		UNICAM_PBE | \
> +		UNICAM_HOE | \
> +		UNICAM_PLE | \
> +		UNICAM_SSC | \
> +		UNICAM_CRCE | \
> +		UNICAM_IFO | \
> +		UNICAM_OFO | \
> +		UNICAM_PS | \
> +		UNICAM_PI0 | \
> +		UNICAM_PI1)
> +
> +/* UNICAM_ANA Register */
> +#define UNICAM_APD		BIT(0)
> +#define UNICAM_BPD		BIT(1)
> +#define UNICAM_AR		BIT(2)
> +#define UNICAM_DDL		BIT(3)
> +#define UNICAM_CTATADJ_MASK	GENMASK(7, 4)
> +#define UNICAM_PTATADJ_MASK	GENMASK(11, 8)
> +
> +/* UNICAM_PRI Register */
> +#define UNICAM_PE		BIT(0)
> +#define UNICAM_PT_MASK		GENMASK(2, 1)
> +#define UNICAM_NP_MASK		GENMASK(7, 4)
> +#define UNICAM_PP_MASK		GENMASK(11, 8)
> +#define UNICAM_BS_MASK		GENMASK(15, 12)
> +#define UNICAM_BL_MASK		GENMASK(17, 16)
> +
> +/* UNICAM_CLK Register */
> +#define UNICAM_CLE		BIT(0)
> +#define UNICAM_CLPD		BIT(1)
> +#define UNICAM_CLLPE		BIT(2)
> +#define UNICAM_CLHSE		BIT(3)
> +#define UNICAM_CLTRE		BIT(4)
> +#define UNICAM_CLAC_MASK	GENMASK(8, 5)
> +#define UNICAM_CLSTE		BIT(29)
> +
> +/* UNICAM_CLT Register */
> +#define UNICAM_CLT1_MASK	GENMASK(7, 0)
> +#define UNICAM_CLT2_MASK	GENMASK(15, 8)
> +
> +/* UNICAM_DATn Registers */
> +#define UNICAM_DLE		BIT(0)
> +#define UNICAM_DLPD		BIT(1)
> +#define UNICAM_DLLPE		BIT(2)
> +#define UNICAM_DLHSE		BIT(3)
> +#define UNICAM_DLTRE		BIT(4)
> +#define UNICAM_DLSM		BIT(5)
> +#define UNICAM_DLFO		BIT(28)
> +#define UNICAM_DLSTE		BIT(29)
> +
> +#define UNICAM_DAT_MASK_ALL	(UNICAM_DLSTE | UNICAM_DLFO)
> +
> +/* UNICAM_DLT Register */
> +#define UNICAM_DLT1_MASK	GENMASK(7, 0)
> +#define UNICAM_DLT2_MASK	GENMASK(15, 8)
> +#define UNICAM_DLT3_MASK	GENMASK(23, 16)
> +
> +/* UNICAM_ICTL Register */
> +#define UNICAM_FSIE		BIT(0)
> +#define UNICAM_FEIE		BIT(1)
> +#define UNICAM_IBOB		BIT(2)
> +#define UNICAM_FCM		BIT(3)
> +#define UNICAM_TFC		BIT(4)
> +#define UNICAM_LIP_MASK		GENMASK(6, 5)
> +#define UNICAM_LCIE_MASK	GENMASK(28, 16)
> +
> +/* UNICAM_IDI0/1 Register */
> +#define UNICAM_ID0_MASK		GENMASK(7, 0)
> +#define UNICAM_ID1_MASK		GENMASK(15, 8)
> +#define UNICAM_ID2_MASK		GENMASK(23, 16)
> +#define UNICAM_ID3_MASK		GENMASK(31, 24)
> +
> +/* UNICAM_ISTA Register */
> +#define UNICAM_FSI		BIT(0)
> +#define UNICAM_FEI		BIT(1)
> +#define UNICAM_LCI		BIT(2)
> +
> +#define UNICAM_ISTA_MASK_ALL	(UNICAM_FSI | UNICAM_FEI | UNICAM_LCI)
> +
> +/* UNICAM_IPIPE Register */
> +#define UNICAM_PUM_MASK		GENMASK(2, 0)
> +/* Unpacking modes */
> +#define UNICAM_PUM_NONE		0
> +#define UNICAM_PUM_UNPACK6	1
> +#define UNICAM_PUM_UNPACK7	2
> +#define UNICAM_PUM_UNPACK8	3
> +#define UNICAM_PUM_UNPACK10	4
> +#define UNICAM_PUM_UNPACK12	5
> +#define UNICAM_PUM_UNPACK14	6
> +#define UNICAM_PUM_UNPACK16	7
> +#define UNICAM_DDM_MASK		GENMASK(6, 3)
> +#define UNICAM_PPM_MASK		GENMASK(9, 7)
> +/* Packing modes */
> +#define UNICAM_PPM_NONE		0
> +#define UNICAM_PPM_PACK8	1
> +#define UNICAM_PPM_PACK10	2
> +#define UNICAM_PPM_PACK12	3
> +#define UNICAM_PPM_PACK14	4
> +#define UNICAM_PPM_PACK16	5
> +#define UNICAM_DEM_MASK		GENMASK(11, 10)
> +#define UNICAM_DEBL_MASK	GENMASK(14, 12)
> +#define UNICAM_ICM_MASK		GENMASK(16, 15)
> +#define UNICAM_IDM_MASK		GENMASK(17, 17)
> +
> +/* UNICAM_ICC Register */
> +#define UNICAM_ICFL_MASK	GENMASK(4, 0)
> +#define UNICAM_ICFH_MASK	GENMASK(9, 5)
> +#define UNICAM_ICST_MASK	GENMASK(12, 10)
> +#define UNICAM_ICLT_MASK	GENMASK(15, 13)
> +#define UNICAM_ICLL_MASK	GENMASK(31, 16)
> +
> +/* UNICAM_DCS Register */
> +#define UNICAM_DIE		BIT(0)
> +#define UNICAM_DIM		BIT(1)
> +#define UNICAM_DBOB		BIT(3)
> +#define UNICAM_FDE		BIT(4)
> +#define UNICAM_LDP		BIT(5)
> +#define UNICAM_EDL_MASK		GENMASK(15, 8)
> +
> +/* UNICAM_DBCTL Register */
> +#define UNICAM_DBEN		BIT(0)
> +#define UNICAM_BUF0_IE		BIT(1)
> +#define UNICAM_BUF1_IE		BIT(2)
> +
> +/* UNICAM_CMP[0,1] register */
> +#define UNICAM_PCE		BIT(31)
> +#define UNICAM_GI		BIT(9)
> +#define UNICAM_CPH		BIT(8)
> +#define UNICAM_PCVC_MASK	GENMASK(7, 6)
> +#define UNICAM_PCDT_MASK	GENMASK(5, 0)
> +
> +/* UNICAM_MISC register */
> +#define UNICAM_FL0		BIT(6)
> +#define UNICAM_FL1		BIT(9)
> +
> +#endif
> diff --git a/drivers/media/platform/broadcom/bcm2835-unicam.c b/drivers/media/platform/broadcom/bcm2835-unicam.c
> new file mode 100644
> index 000000000000..716c89b8a217
> --- /dev/null
> +++ b/drivers/media/platform/broadcom/bcm2835-unicam.c
> @@ -0,0 +1,2607 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * BCM283x / BCM271x Unicam Capture Driver
> + *
> + * Copyright (C) 2017-2020 - Raspberry Pi (Trading) Ltd.
> + *
> + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> + *
> + * Based on TI am437x driver by
> + *   Benoit Parrot <bparrot@ti.com>
> + *   Lad, Prabhakar <prabhakar.csengg@gmail.com>
> + *
> + * and TI CAL camera interface driver by
> + *    Benoit Parrot <bparrot@ti.com>
> + *
> + *
> + * There are two camera drivers in the kernel for BCM283x - this one and
> + * bcm2835-camera (currently in staging).
> + *
> + * This driver directly controls the Unicam peripheral - there is no
> + * involvement with the VideoCore firmware. Unicam receives CSI-2 or CCP2 data
> + * and writes it into SDRAM. The only potential processing options are to
> + * repack Bayer data into an alternate format, and applying windowing. The
> + * repacking does not shift the data, so can repack V4L2_PIX_FMT_Sxxxx10P to
> + * V4L2_PIX_FMT_Sxxxx10, or V4L2_PIX_FMT_Sxxxx12P to V4L2_PIX_FMT_Sxxxx12, but
> + * not generically up to V4L2_PIX_FMT_Sxxxx16. The driver will add both formats
> + * where the relevant formats are defined, and will automatically configure the
> + * repacking as required. Support for windowing may be added later.
> + *
> + * It should be possible to connect this driver to any sensor with a suitable
> + * output interface and V4L2 subdevice driver.
> + */
> +
> +#include <linux/clk.h>
> +#include <linux/delay.h>
> +#include <linux/device.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/err.h>
> +#include <linux/interrupt.h>
> +#include <linux/io.h>
> +#include <linux/module.h>
> +#include <linux/of.h>
> +#include <linux/of_device.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/slab.h>
> +#include <linux/videodev2.h>
> +
> +#include <media/v4l2-async.h>
> +#include <media/v4l2-common.h>
> +#include <media/v4l2-dev.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/v4l2-fwnode.h>
> +#include <media/v4l2-mc.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "bcm2835-unicam-regs.h"
> +
> +#define UNICAM_MODULE_NAME		"unicam"
> +
> +/*
> + * Unicam must request a minimum of 250Mhz from the VPU clock.
> + * Otherwise the input FIFOs overrun and cause image corruption.
> + */
> +#define UNICAM_MIN_VPU_CLOCK_RATE	(250 * 1000 * 1000)
> +
> +/* Unicam has an internal DMA alignment constraint of 16 bytes for each line. */
> +#define UNICAM_DMA_BPL_ALIGNMENT	16
> +
> +/*
> + * The image stride is stored in a 16 bit register, and needs to be aligned to
> + * the DMA constraint. As the ISP in the same SoC has a 32 bytes alignment
> + * constraint on its input, set the image stride alignment to 32 bytes here as
> + * well to avoid incompatible configurations.
> + */
> +#define UNICAM_IMAGE_BPL_ALIGNMENT	32
> +#define UNICAM_IMAGE_MAX_BPL		((1 << 16) - UNICAM_IMAGE_BPL_ALIGNMENT)

Bit shifted values should be unsigned.

> +
> +/*
> + * Max width is therefore determined by the max stride divided by the number of
> + * bits per pixel. Take 32bpp as a worst case. No imposed limit on the height,
> + * so adopt a square image for want of anything better.
> + */
> +#define UNICAM_IMAGE_MIN_WIDTH		16
> +#define UNICAM_IMAGE_MIN_HEIGHT		16
> +#define UNICAM_IMAGE_MAX_WIDTH		(UNICAM_IMAGE_MAX_BPL / 4)
> +#define UNICAM_IMAGE_MAX_HEIGHT		UNICAM_IMAGE_MAX_WIDTH
> +
> +/*
> + * There's no intrinsic limits on the width and height for embedded dat. Use
> + * the same maximum values as for the image, to avoid overflows in the image
> + * size computation.
> + */
> +#define UNICAM_META_MIN_WIDTH		1
> +#define UNICAM_META_MIN_HEIGHT		1
> +#define UNICAM_META_MAX_WIDTH		UNICAM_IMAGE_MAX_WIDTH
> +#define UNICAM_META_MAX_HEIGHT		UNICAM_IMAGE_MAX_HEIGHT
> +
> +/*
> + * Size of the dummy buffer. Can be any size really, but the DMA
> + * allocation works in units of page sizes.
> + */
> +#define UNICAM_DUMMY_BUF_SIZE		PAGE_SIZE
> +
> +#define UNICAM_SD_PAD_SINK		0
> +#define UNICAM_SD_PAD_SOURCE_IMAGE	1
> +#define UNICAM_SD_PAD_SOURCE_METADATA	2
> +#define UNICAM_SD_NUM_PADS		(1 + UNICAM_SD_PAD_SOURCE_METADATA)
> +
> +enum unicam_node_type {
> +	UNICAM_IMAGE_NODE,
> +	UNICAM_METADATA_NODE,
> +	UNICAM_MAX_NODES
> +};
> +
> +/*
> + * struct unicam_format_info - Unicam media bus format information
> + * @fourcc: V4L2 pixel format FCC identifier. 0 if n/a.
> + * @unpacked_fourcc: V4L2 pixel format FCC identifier if the data is expanded
> + * out to 16bpp. 0 if n/a.
> + * @code: V4L2 media bus format code.
> + * @depth: Bits per pixel as delivered from the source.
> + * @csi_dt: CSI data type.
> + * @metadata_fmt: This format only applies to the metadata pad.
> + */
> +struct unicam_format_info {
> +	u32	fourcc;
> +	u32	unpacked_fourcc;
> +	u32	code;
> +	u8	depth;
> +	u8	csi_dt;
> +	bool	metadata_fmt;
> +};
> +
> +struct unicam_buffer {
> +	struct vb2_v4l2_buffer vb;
> +	struct list_head list;
> +	dma_addr_t dma_addr;
> +	unsigned int size;
> +};
> +
> +static inline struct unicam_buffer *to_unicam_buffer(struct vb2_buffer *vb)
> +{
> +	return container_of(vb, struct unicam_buffer, vb.vb2_buf);
> +}
> +
> +struct unicam_node {
> +	bool registered;
> +	bool streaming;
> +	unsigned int id;
> +
> +	/* Pointer to the current v4l2_buffer */
> +	struct unicam_buffer *cur_frm;
> +	/* Pointer to the next v4l2_buffer */
> +	struct unicam_buffer *next_frm;
> +	/* video capture */
> +	const struct unicam_format_info *fmtinfo;
> +	/* Used to store current pixel format */
> +	struct v4l2_format fmt;
> +	/* Buffer queue used in video-buf */
> +	struct vb2_queue buffer_queue;
> +	/* Queue of filled frames */
> +	struct list_head dma_queue;
> +	/* IRQ lock for DMA queue */
> +	spinlock_t dma_queue_lock;
> +	/* lock used to access this structure */
> +	struct mutex lock;
> +	/* Identifies video device for this channel */
> +	struct video_device video_dev;
> +	/* Pointer to the parent handle */
> +	struct unicam_device *dev;
> +	struct media_pad pad;
> +	/*
> +	 * Dummy buffer intended to be used by unicam
> +	 * if we have no other queued buffers to swap to.
> +	 */
> +	struct unicam_buffer dummy_buf;
> +	void *dummy_buf_cpu_addr;
> +};
> +
> +struct unicam_device {
> +	struct kref kref;
> +
> +	/* peripheral base address */
> +	void __iomem *base;
> +	/* clock gating base address */
> +	void __iomem *clk_gate_base;
> +	/* lp clock handle */
> +	struct clk *clock;
> +	/* vpu clock handle */
> +	struct clk *vpu_clock;
> +	/* V4l2 device */
> +	struct v4l2_device v4l2_dev;
> +	struct media_device mdev;
> +
> +	/* parent device */
> +	struct device *dev;
> +	/* subdevice async Notifier */
> +	struct v4l2_async_notifier notifier;
> +	unsigned int sequence;
> +
> +	/* Sensor node */
> +	struct {
> +		struct v4l2_subdev *subdev;
> +		struct media_pad *pad;
> +	} sensor;
> +
> +	/* Internal subdev */
> +	struct {
> +		struct v4l2_subdev sd;
> +		struct media_pad pads[UNICAM_SD_NUM_PADS];
> +		bool streaming;
> +	} subdev;
> +
> +	enum v4l2_mbus_type bus_type;
> +	/*
> +	 * Stores bus.mipi_csi2.flags for CSI2 sensors, or
> +	 * bus.mipi_csi1.strobe for CCP2.
> +	 */
> +	unsigned int bus_flags;
> +	unsigned int max_data_lanes;
> +	unsigned int active_data_lanes;
> +
> +	struct media_pipeline pipe;
> +
> +	struct unicam_node node[UNICAM_MAX_NODES];
> +};
> +
> +static inline struct unicam_device *
> +notifier_to_unicam_device(struct v4l2_async_notifier *notifier)
> +{
> +	return container_of(notifier, struct unicam_device, notifier);
> +}
> +
> +static inline struct unicam_device *
> +sd_to_unicam_device(struct v4l2_subdev *sd)
> +{
> +	return container_of(sd, struct unicam_device, subdev.sd);
> +}
> +
> +static void unicam_release(struct kref *kref)
> +{
> +	struct unicam_device *unicam =
> +		container_of(kref, struct unicam_device, kref);
> +
> +	if (unicam->mdev.dev)
> +		media_device_cleanup(&unicam->mdev);
> +
> +	kfree(unicam);
> +}
> +
> +static struct unicam_device *unicam_get(struct unicam_device *unicam)
> +{
> +	kref_get(&unicam->kref);

A newline here would be nice.

> +	return unicam;
> +}
> +
> +static void unicam_put(struct unicam_device *unicam)
> +{
> +	kref_put(&unicam->kref, unicam_release);
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * Misc helper functions
> + */
> +
> +static inline bool unicam_sd_pad_is_source(u32 pad)
> +{
> +	/* Camera RX has 1 sink pad, and N source pads */
> +	return pad != UNICAM_SD_PAD_SINK;
> +}
> +
> +static inline bool is_metadata_node(struct unicam_node *node)
> +{
> +	return node->video_dev.device_caps & V4L2_CAP_META_CAPTURE;
> +}
> +
> +static inline bool is_image_node(struct unicam_node *node)
> +{
> +	return node->video_dev.device_caps & V4L2_CAP_VIDEO_CAPTURE;
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * Format data table and helper functions
> + */
> +
> +static const struct v4l2_mbus_framefmt unicam_default_image_format = {
> +	.width = 640,
> +	.height = 480,
> +	.code = MEDIA_BUS_FMT_UYVY8_1X16,
> +	.field = V4L2_FIELD_NONE,
> +	.colorspace = V4L2_COLORSPACE_SRGB,
> +	.ycbcr_enc = V4L2_YCBCR_ENC_601,
> +	.quantization = V4L2_QUANTIZATION_LIM_RANGE,
> +	.xfer_func = V4L2_XFER_FUNC_SRGB,
> +	.flags = 0,
> +};
> +
> +static const struct v4l2_mbus_framefmt unicam_default_meta_format = {
> +	.width = 640,
> +	.height = 2,
> +	.code = MEDIA_BUS_FMT_META_8,
> +	.field = V4L2_FIELD_NONE,
> +};
> +
> +static const struct unicam_format_info unicam_image_formats[] = {
> +	/* YUV Formats */
> +	{
> +		.fourcc		= V4L2_PIX_FMT_YUYV,
> +		.code		= MEDIA_BUS_FMT_YUYV8_1X16,
> +		.depth		= 16,
> +		.csi_dt		= 0x1e,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_UYVY,
> +		.code		= MEDIA_BUS_FMT_UYVY8_1X16,
> +		.depth		= 16,
> +		.csi_dt		= 0x1e,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_YVYU,
> +		.code		= MEDIA_BUS_FMT_YVYU8_1X16,
> +		.depth		= 16,
> +		.csi_dt		= 0x1e,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_VYUY,
> +		.code		= MEDIA_BUS_FMT_VYUY8_1X16,
> +		.depth		= 16,
> +		.csi_dt		= 0x1e,
> +	}, {
> +	/* RGB Formats */
> +		.fourcc		= V4L2_PIX_FMT_RGB565, /* gggbbbbb rrrrrggg */
> +		.code		= MEDIA_BUS_FMT_RGB565_1X16,
> +		.depth		= 16,
> +		.csi_dt		= 0x22,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_RGB24, /* rgb */
> +		.code		= MEDIA_BUS_FMT_RGB888_1X24,
> +		.depth		= 24,
> +		.csi_dt		= 0x24,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_BGR24, /* bgr */
> +		.code		= MEDIA_BUS_FMT_BGR888_1X24,
> +		.depth		= 24,
> +		.csi_dt		= 0x24,
> +	}, {
> +	/* Bayer Formats */
> +		.fourcc		= V4L2_PIX_FMT_SBGGR8,
> +		.code		= MEDIA_BUS_FMT_SBGGR8_1X8,
> +		.depth		= 8,
> +		.csi_dt		= 0x2a,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SGBRG8,
> +		.code		= MEDIA_BUS_FMT_SGBRG8_1X8,
> +		.depth		= 8,
> +		.csi_dt		= 0x2a,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SGRBG8,
> +		.code		= MEDIA_BUS_FMT_SGRBG8_1X8,
> +		.depth		= 8,
> +		.csi_dt		= 0x2a,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SRGGB8,
> +		.code		= MEDIA_BUS_FMT_SRGGB8_1X8,
> +		.depth		= 8,
> +		.csi_dt		= 0x2a,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SBGGR10P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SBGGR10,
> +		.code		= MEDIA_BUS_FMT_SBGGR10_1X10,
> +		.depth		= 10,
> +		.csi_dt		= 0x2b,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SGBRG10P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SGBRG10,
> +		.code		= MEDIA_BUS_FMT_SGBRG10_1X10,
> +		.depth		= 10,
> +		.csi_dt		= 0x2b,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SGRBG10P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SGRBG10,
> +		.code		= MEDIA_BUS_FMT_SGRBG10_1X10,
> +		.depth		= 10,
> +		.csi_dt		= 0x2b,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SRGGB10P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SRGGB10,
> +		.code		= MEDIA_BUS_FMT_SRGGB10_1X10,
> +		.depth		= 10,
> +		.csi_dt		= 0x2b,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SBGGR12P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SBGGR12,
> +		.code		= MEDIA_BUS_FMT_SBGGR12_1X12,
> +		.depth		= 12,
> +		.csi_dt		= 0x2c,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SGBRG12P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SGBRG12,
> +		.code		= MEDIA_BUS_FMT_SGBRG12_1X12,
> +		.depth		= 12,
> +		.csi_dt		= 0x2c,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SGRBG12P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SGRBG12,
> +		.code		= MEDIA_BUS_FMT_SGRBG12_1X12,
> +		.depth		= 12,
> +		.csi_dt		= 0x2c,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SRGGB12P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SRGGB12,
> +		.code		= MEDIA_BUS_FMT_SRGGB12_1X12,
> +		.depth		= 12,
> +		.csi_dt		= 0x2c,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SBGGR14P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SBGGR14,
> +		.code		= MEDIA_BUS_FMT_SBGGR14_1X14,
> +		.depth		= 14,
> +		.csi_dt		= 0x2d,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SGBRG14P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SGBRG14,
> +		.code		= MEDIA_BUS_FMT_SGBRG14_1X14,
> +		.depth		= 14,
> +		.csi_dt		= 0x2d,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SGRBG14P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SGRBG14,
> +		.code		= MEDIA_BUS_FMT_SGRBG14_1X14,
> +		.depth		= 14,
> +		.csi_dt		= 0x2d,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_SRGGB14P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_SRGGB14,
> +		.code		= MEDIA_BUS_FMT_SRGGB14_1X14,
> +		.depth		= 14,
> +		.csi_dt		= 0x2d,
> +	}, {
> +	/* 16 bit Bayer formats could be supported. */
> +
> +	/* Greyscale formats */
> +		.fourcc		= V4L2_PIX_FMT_GREY,
> +		.code		= MEDIA_BUS_FMT_Y8_1X8,
> +		.depth		= 8,
> +		.csi_dt		= 0x2a,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_Y10P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_Y10,
> +		.code		= MEDIA_BUS_FMT_Y10_1X10,
> +		.depth		= 10,
> +		.csi_dt		= 0x2b,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_Y12P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_Y12,
> +		.code		= MEDIA_BUS_FMT_Y12_1X12,
> +		.depth		= 12,
> +		.csi_dt		= 0x2c,
> +	}, {
> +		.fourcc		= V4L2_PIX_FMT_Y14P,
> +		.unpacked_fourcc = V4L2_PIX_FMT_Y14,
> +		.code		= MEDIA_BUS_FMT_Y14_1X14,
> +		.depth		= 14,
> +		.csi_dt		= 0x2d,
> +	},
> +};
> +
> +static const struct unicam_format_info unicam_meta_formats[] = {
> +	{
> +		.fourcc		= V4L2_META_FMT_GENERIC_8,
> +		.code		= MEDIA_BUS_FMT_META_8,
> +		.depth		= 8,
> +		.metadata_fmt	= true,
> +	}, {
> +		.fourcc		= V4L2_META_FMT_GENERIC_CSI2_10,
> +		.code		= MEDIA_BUS_FMT_META_10,
> +		.depth		= 10,
> +		.metadata_fmt	= true,
> +	}, {
> +		.fourcc		= V4L2_META_FMT_GENERIC_CSI2_12,
> +		.code		= MEDIA_BUS_FMT_META_12,
> +		.depth		= 12,
> +		.metadata_fmt	= true,
> +	}, {
> +		.fourcc		= V4L2_META_FMT_GENERIC_CSI2_14,
> +		.code		= MEDIA_BUS_FMT_META_14,
> +		.depth		= 14,
> +		.metadata_fmt	= true,
> +	},
> +};
> +
> +/* Format setup functions */
> +static const struct unicam_format_info *
> +unicam_find_format_by_code(u32 code, u32 pad)
> +{
> +	const struct unicam_format_info *formats;
> +	unsigned int num_formats;
> +	unsigned int i;
> +
> +	if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> +		formats = unicam_image_formats;
> +		num_formats = ARRAY_SIZE(unicam_image_formats);
> +	} else {
> +		formats = unicam_meta_formats;
> +		num_formats = ARRAY_SIZE(unicam_meta_formats);
> +	}
> +
> +	for (i = 0; i < num_formats; i++) {
> +		if (formats[i].code == code)
> +			return &formats[i];
> +	}
> +
> +	return NULL;
> +}
> +
> +static const struct unicam_format_info *
> +unicam_find_format_by_fourcc(u32 fourcc, u32 pad)
> +{
> +	const struct unicam_format_info *formats;
> +	unsigned int num_formats;
> +	unsigned int i;
> +
> +	if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> +		formats = unicam_image_formats;
> +		num_formats = ARRAY_SIZE(unicam_image_formats);
> +	} else {
> +		formats = unicam_meta_formats;
> +		num_formats = ARRAY_SIZE(unicam_meta_formats);
> +	}
> +
> +	for (i = 0; i < num_formats; ++i) {
> +		if (formats[i].fourcc == fourcc)
> +			return &formats[i];
> +	}
> +
> +	return NULL;
> +}
> +
> +static void unicam_calc_image_size_bpl(struct unicam_device *unicam,
> +				       const struct unicam_format_info *fmtinfo,
> +				       struct v4l2_pix_format *pix)
> +{
> +	u32 min_bpl;
> +
> +	v4l_bound_align_image(&pix->width, UNICAM_IMAGE_MIN_WIDTH,
> +			      UNICAM_IMAGE_MAX_WIDTH, 2,
> +			      &pix->height, UNICAM_IMAGE_MIN_HEIGHT,
> +			      UNICAM_IMAGE_MAX_HEIGHT, 0, 0);
> +
> +	/* Unpacking always goes to 16bpp */
> +	if (pix->pixelformat == fmtinfo->unpacked_fourcc)
> +		min_bpl = pix->width * 2;
> +	else
> +		min_bpl = pix->width * fmtinfo->depth / 8;
> +	min_bpl = ALIGN(min_bpl, UNICAM_IMAGE_BPL_ALIGNMENT);
> +
> +	pix->bytesperline = ALIGN(pix->bytesperline, UNICAM_IMAGE_BPL_ALIGNMENT);
> +	pix->bytesperline = clamp_t(unsigned int, pix->bytesperline, min_bpl,
> +				    UNICAM_IMAGE_MAX_BPL);
> +
> +	pix->sizeimage = pix->height * pix->bytesperline;
> +}
> +
> +static void unicam_calc_meta_size_bpl(struct unicam_device *unicam,
> +				      const struct unicam_format_info *fmtinfo,
> +				      struct v4l2_meta_format *meta)
> +{
> +	v4l_bound_align_image(&meta->width, UNICAM_META_MIN_WIDTH,
> +			      UNICAM_META_MAX_WIDTH, 0,
> +			      &meta->height, UNICAM_META_MIN_HEIGHT,
> +			      UNICAM_META_MAX_HEIGHT, 0, 0);
> +
> +	meta->bytesperline = ALIGN(meta->width * fmtinfo->depth / 8,
> +				   UNICAM_DMA_BPL_ALIGNMENT);
> +	meta->buffersize = meta->height * meta->bytesperline;
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * Hardware handling
> + */
> +
> +static inline void unicam_clk_write(struct unicam_device *unicam, u32 val)
> +{
> +	/* Pass the CM_PASSWORD along with the value. */
> +	writel(val | 0x5a000000, unicam->clk_gate_base);
> +}
> +
> +static inline u32 unicam_reg_read(struct unicam_device *unicam, u32 offset)
> +{
> +	return readl(unicam->base + offset);
> +}
> +
> +static inline void unicam_reg_write(struct unicam_device *unicam, u32 offset, u32 val)
> +{
> +	writel(val, unicam->base + offset);
> +}
> +
> +static inline int unicam_get_field(u32 value, u32 mask)
> +{
> +	return (value & mask) >> __ffs(mask);
> +}
> +
> +static inline void unicam_set_field(u32 *valp, u32 field, u32 mask)
> +{
> +	u32 val = *valp;
> +
> +	val &= ~mask;
> +	val |= (field << __ffs(mask)) & mask;
> +	*valp = val;
> +}
> +
> +static inline void unicam_reg_write_field(struct unicam_device *unicam, u32 offset,
> +					  u32 field, u32 mask)
> +{
> +	u32 val = unicam_reg_read(unicam, offset);
> +
> +	unicam_set_field(&val, field, mask);
> +	unicam_reg_write(unicam, offset, val);
> +}
> +
> +static void unicam_wr_dma_addr(struct unicam_node *node,
> +			       struct unicam_buffer *buf)
> +{
> +	dma_addr_t endaddr = buf->dma_addr + buf->size;
> +
> +	if (node->id == UNICAM_IMAGE_NODE) {
> +		unicam_reg_write(node->dev, UNICAM_IBSA0, buf->dma_addr);
> +		unicam_reg_write(node->dev, UNICAM_IBEA0, endaddr);
> +	} else {
> +		unicam_reg_write(node->dev, UNICAM_DBSA0, buf->dma_addr);
> +		unicam_reg_write(node->dev, UNICAM_DBEA0, endaddr);
> +	}
> +}
> +
> +static unsigned int unicam_get_lines_done(struct unicam_device *unicam)
> +{
> +	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> +	unsigned int stride = node->fmt.fmt.pix.bytesperline;
> +	struct unicam_buffer *frm = node->cur_frm;
> +	dma_addr_t cur_addr;
> +
> +	if (!frm)
> +		return 0;
> +
> +	cur_addr = unicam_reg_read(unicam, UNICAM_IBWP);
> +	return (unsigned int)(cur_addr - frm->dma_addr) / stride;
> +}
> +
> +static void unicam_schedule_next_buffer(struct unicam_node *node)
> +{
> +	struct unicam_buffer *buf;
> +
> +	buf = list_first_entry(&node->dma_queue, struct unicam_buffer, list);
> +	node->next_frm = buf;
> +	list_del(&buf->list);
> +
> +	unicam_wr_dma_addr(node, buf);
> +}
> +
> +static void unicam_schedule_dummy_buffer(struct unicam_node *node)
> +{
> +	int node_id = is_image_node(node) ? UNICAM_IMAGE_NODE : UNICAM_METADATA_NODE;
> +
> +	dev_dbg(node->dev->dev, "Scheduling dummy buffer for node %d\n", node_id);
> +
> +	unicam_wr_dma_addr(node, &node->dummy_buf);
> +
> +	node->next_frm = NULL;
> +}
> +
> +static void unicam_process_buffer_complete(struct unicam_node *node,
> +					   unsigned int sequence)
> +{
> +	node->cur_frm->vb.field = node->fmt.fmt.pix.field;
> +	node->cur_frm->vb.sequence = sequence;
> +
> +	vb2_buffer_done(&node->cur_frm->vb.vb2_buf, VB2_BUF_STATE_DONE);
> +}
> +
> +static void unicam_queue_event_sof(struct unicam_device *unicam)
> +{
> +	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> +

Extra newline.

> +	struct v4l2_event event = {
> +		.type = V4L2_EVENT_FRAME_SYNC,
> +		.u.frame_sync.frame_sequence = unicam->sequence,
> +	};
> +
> +	v4l2_event_queue(&node->video_dev, &event);
> +}
> +
> +static irqreturn_t unicam_isr(int irq, void *dev)
> +{
> +	struct unicam_device *unicam = dev;
> +	unsigned int lines_done = unicam_get_lines_done(dev);
> +	unsigned int sequence = unicam->sequence;
> +	unsigned int i;
> +	u32 ista, sta;
> +	bool fe;
> +	u64 ts;
> +
> +	sta = unicam_reg_read(unicam, UNICAM_STA);
> +	/* Write value back to clear the interrupts */
> +	unicam_reg_write(unicam, UNICAM_STA, sta);
> +
> +	ista = unicam_reg_read(unicam, UNICAM_ISTA);
> +	/* Write value back to clear the interrupts */
> +	unicam_reg_write(unicam, UNICAM_ISTA, ista);
> +
> +	dev_dbg(unicam->dev, "ISR: ISTA: 0x%X, STA: 0x%X, sequence %d, lines done %d\n",
> +		ista, sta, sequence, lines_done);
> +
> +	if (!(sta & (UNICAM_IS | UNICAM_PI0)))
> +		return IRQ_HANDLED;
> +
> +	/*
> +	 * Look for either the Frame End interrupt or the Packet Capture status
> +	 * to signal a frame end.
> +	 */
> +	fe = ista & UNICAM_FEI || sta & UNICAM_PI0;
> +
> +	/*
> +	 * We must run the frame end handler first. If we have a valid next_frm
> +	 * and we get a simultaneout FE + FS interrupt, running the FS handler
> +	 * first would null out the next_frm ptr and we would have lost the
> +	 * buffer forever.
> +	 */
> +	if (fe) {
> +		/*
> +		 * Ensure we have swapped buffers already as we can't
> +		 * stop the peripheral. If no buffer is available, use a
> +		 * dummy buffer to dump out frames until we get a new buffer
> +		 * to use.
> +		 */
> +		for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> +			if (!unicam->node[i].streaming)
> +				continue;
> +
> +			/*
> +			 * If cur_frm == next_frm, it means we have not had
> +			 * a chance to swap buffers, likely due to having
> +			 * multiple interrupts occurring simultaneously (like FE
> +			 * + FS + LS). In this case, we cannot signal the buffer
> +			 * as complete, as the HW will reuse that buffer.
> +			 */
> +			if (unicam->node[i].cur_frm &&
> +			    unicam->node[i].cur_frm != unicam->node[i].next_frm)
> +				unicam_process_buffer_complete(&unicam->node[i],
> +							       sequence);
> +			unicam->node[i].cur_frm = unicam->node[i].next_frm;
> +		}
> +		unicam->sequence++;

Does access to this data need to be serialised somehow.

> +	}
> +
> +	if (ista & UNICAM_FSI) {
> +		/*
> +		 * Timestamp is to be when the first data byte was captured,
> +		 * aka frame start.
> +		 */
> +		ts = ktime_get_ns();
> +		for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> +			if (!unicam->node[i].streaming)
> +				continue;
> +
> +			if (unicam->node[i].cur_frm)
> +				unicam->node[i].cur_frm->vb.vb2_buf.timestamp =
> +								ts;
> +			else
> +				dev_dbg(unicam->v4l2_dev.dev,
> +					"ISR: [%d] Dropping frame, buffer not available at FS\n",
> +					i);
> +			/*
> +			 * Set the next frame output to go to a dummy frame
> +			 * if we have not managed to obtain another frame
> +			 * from the queue.
> +			 */
> +			unicam_schedule_dummy_buffer(&unicam->node[i]);
> +		}
> +
> +		unicam_queue_event_sof(unicam);
> +	}
> +
> +	/*
> +	 * Cannot swap buffer at frame end, there may be a race condition
> +	 * where the HW does not actually swap it if the new frame has
> +	 * already started.
> +	 */
> +	if (ista & (UNICAM_FSI | UNICAM_LCI) && !fe) {
> +		for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> +			if (!unicam->node[i].streaming)
> +				continue;
> +
> +			spin_lock(&unicam->node[i].dma_queue_lock);
> +			if (!list_empty(&unicam->node[i].dma_queue) &&
> +			    !unicam->node[i].next_frm)
> +				unicam_schedule_next_buffer(&unicam->node[i]);
> +			spin_unlock(&unicam->node[i].dma_queue_lock);
> +		}
> +	}
> +
> +	if (unicam_reg_read(unicam, UNICAM_ICTL) & UNICAM_FCM) {
> +		/* Switch out of trigger mode if selected */
> +		unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> +		unicam_reg_write_field(unicam, UNICAM_ICTL, 0, UNICAM_FCM);
> +	}
> +	return IRQ_HANDLED;
> +}
> +
> +static void unicam_set_packing_config(struct unicam_device *unicam)
> +{
> +	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> +	u32 pack, unpack;
> +	u32 val;
> +
> +	if (node->fmt.fmt.pix.pixelformat == node->fmtinfo->fourcc) {
> +		unpack = UNICAM_PUM_NONE;
> +		pack = UNICAM_PPM_NONE;
> +	} else {
> +		switch (node->fmtinfo->depth) {
> +		case 8:
> +			unpack = UNICAM_PUM_UNPACK8;
> +			break;
> +		case 10:
> +			unpack = UNICAM_PUM_UNPACK10;
> +			break;
> +		case 12:
> +			unpack = UNICAM_PUM_UNPACK12;
> +			break;
> +		case 14:
> +			unpack = UNICAM_PUM_UNPACK14;
> +			break;
> +		case 16:
> +			unpack = UNICAM_PUM_UNPACK16;
> +			break;
> +		default:
> +			unpack = UNICAM_PUM_NONE;
> +			break;
> +		}
> +
> +		/* Repacking is always to 16bpp */
> +		pack = UNICAM_PPM_PACK16;

Also 8-bit data?

> +	}
> +
> +	val = 0;

You could do initialisation in declaration.

> +	unicam_set_field(&val, unpack, UNICAM_PUM_MASK);
> +	unicam_set_field(&val, pack, UNICAM_PPM_MASK);
> +	unicam_reg_write(unicam, UNICAM_IPIPE, val);
> +}
> +
> +static void unicam_cfg_image_id(struct unicam_device *unicam)
> +{
> +	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> +
> +	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> +		/* CSI2 mode, hardcode VC 0 for now. */
> +		unicam_reg_write(unicam, UNICAM_IDI0,
> +				 (0 << 6) | node->fmtinfo->csi_dt);
> +	} else {
> +		/* CCP2 mode */
> +		unicam_reg_write(unicam, UNICAM_IDI0,
> +				 0x80 | node->fmtinfo->csi_dt);
> +	}
> +}
> +
> +static void unicam_enable_ed(struct unicam_device *unicam)
> +{
> +	u32 val = unicam_reg_read(unicam, UNICAM_DCS);
> +
> +	unicam_set_field(&val, 2, UNICAM_EDL_MASK);
> +	/* Do not wrap at the end of the embedded data buffer */
> +	unicam_set_field(&val, 0, UNICAM_DBOB);
> +
> +	unicam_reg_write(unicam, UNICAM_DCS, val);
> +}
> +
> +static void unicam_start_rx(struct unicam_device *unicam,
> +			    struct unicam_buffer *buf)
> +{
> +	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> +	int line_int_freq = node->fmt.fmt.pix.height >> 2;
> +	unsigned int i;
> +	u32 val;
> +
> +	if (line_int_freq < 128)
> +		line_int_freq = 128;

	line_int_freq = max(line_int_freq, 128);

> +
> +	/* Enable lane clocks */
> +	val = 1;

Initialise in the loop initialisation below, I'd say.

> +	for (i = 0; i < unicam->active_data_lanes; i++)
> +		val = val << 2 | 1;
> +	unicam_clk_write(unicam, val);
> +
> +	/* Basic init */
> +	unicam_reg_write(unicam, UNICAM_CTRL, UNICAM_MEM);
> +
> +	/* Enable analogue control, and leave in reset. */
> +	val = UNICAM_AR;
> +	unicam_set_field(&val, 7, UNICAM_CTATADJ_MASK);
> +	unicam_set_field(&val, 7, UNICAM_PTATADJ_MASK);
> +	unicam_reg_write(unicam, UNICAM_ANA, val);
> +	usleep_range(1000, 2000);
> +
> +	/* Come out of reset */
> +	unicam_reg_write_field(unicam, UNICAM_ANA, 0, UNICAM_AR);
> +
> +	/* Peripheral reset */
> +	unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPR);
> +	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPR);
> +
> +	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPE);
> +
> +	/* Enable Rx control. */
> +	val = unicam_reg_read(unicam, UNICAM_CTRL);
> +	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> +		unicam_set_field(&val, UNICAM_CPM_CSI2, UNICAM_CPM_MASK);
> +		unicam_set_field(&val, UNICAM_DCM_STROBE, UNICAM_DCM_MASK);
> +	} else {
> +		unicam_set_field(&val, UNICAM_CPM_CCP2, UNICAM_CPM_MASK);
> +		unicam_set_field(&val, unicam->bus_flags, UNICAM_DCM_MASK);
> +	}
> +	/* Packet framer timeout */
> +	unicam_set_field(&val, 0xf, UNICAM_PFT_MASK);
> +	unicam_set_field(&val, 128, UNICAM_OET_MASK);
> +	unicam_reg_write(unicam, UNICAM_CTRL, val);
> +
> +	unicam_reg_write(unicam, UNICAM_IHWIN, 0);
> +	unicam_reg_write(unicam, UNICAM_IVWIN, 0);
> +
> +	/* AXI bus access QoS setup */
> +	val = unicam_reg_read(unicam, UNICAM_PRI);
> +	unicam_set_field(&val, 0, UNICAM_BL_MASK);
> +	unicam_set_field(&val, 0, UNICAM_BS_MASK);
> +	unicam_set_field(&val, 0xe, UNICAM_PP_MASK);
> +	unicam_set_field(&val, 8, UNICAM_NP_MASK);
> +	unicam_set_field(&val, 2, UNICAM_PT_MASK);
> +	unicam_set_field(&val, 1, UNICAM_PE);
> +	unicam_reg_write(unicam, UNICAM_PRI, val);
> +
> +	unicam_reg_write_field(unicam, UNICAM_ANA, 0, UNICAM_DDL);
> +
> +	/* Always start in trigger frame capture mode (UNICAM_FCM set) */
> +	val = UNICAM_FSIE | UNICAM_FEIE | UNICAM_FCM | UNICAM_IBOB;
> +	unicam_set_field(&val, line_int_freq, UNICAM_LCIE_MASK);
> +	unicam_reg_write(unicam, UNICAM_ICTL, val);
> +	unicam_reg_write(unicam, UNICAM_STA, UNICAM_STA_MASK_ALL);
> +	unicam_reg_write(unicam, UNICAM_ISTA, UNICAM_ISTA_MASK_ALL);
> +
> +	/* tclk_term_en */
> +	unicam_reg_write_field(unicam, UNICAM_CLT, 2, UNICAM_CLT1_MASK);
> +	/* tclk_settle */
> +	unicam_reg_write_field(unicam, UNICAM_CLT, 6, UNICAM_CLT2_MASK);
> +	/* td_term_en */
> +	unicam_reg_write_field(unicam, UNICAM_DLT, 2, UNICAM_DLT1_MASK);
> +	/* ths_settle */
> +	unicam_reg_write_field(unicam, UNICAM_DLT, 6, UNICAM_DLT2_MASK);
> +	/* trx_enable */
> +	unicam_reg_write_field(unicam, UNICAM_DLT, 0, UNICAM_DLT3_MASK);
> +
> +	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_SOE);
> +
> +	/* Packet compare setup - required to avoid missing frame ends */
> +	val = 0;
> +	unicam_set_field(&val, 1, UNICAM_PCE);
> +	unicam_set_field(&val, 1, UNICAM_GI);
> +	unicam_set_field(&val, 1, UNICAM_CPH);
> +	unicam_set_field(&val, 0, UNICAM_PCVC_MASK);
> +	unicam_set_field(&val, 1, UNICAM_PCDT_MASK);
> +	unicam_reg_write(unicam, UNICAM_CMP0, val);
> +
> +	/* Enable clock lane and set up terminations */
> +	val = 0;
> +	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> +		/* CSI2 */
> +		unicam_set_field(&val, 1, UNICAM_CLE);
> +		unicam_set_field(&val, 1, UNICAM_CLLPE);
> +		if (!(unicam->bus_flags & V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK)) {
> +			unicam_set_field(&val, 1, UNICAM_CLTRE);
> +			unicam_set_field(&val, 1, UNICAM_CLHSE);
> +		}
> +	} else {
> +		/* CCP2 */
> +		unicam_set_field(&val, 1, UNICAM_CLE);
> +		unicam_set_field(&val, 1, UNICAM_CLHSE);
> +		unicam_set_field(&val, 1, UNICAM_CLTRE);
> +	}
> +	unicam_reg_write(unicam, UNICAM_CLK, val);
> +
> +	/*
> +	 * Enable required data lanes with appropriate terminations.
> +	 * The same value needs to be written to UNICAM_DATn registers for
> +	 * the active lanes, and 0 for inactive ones.
> +	 */
> +	val = 0;
> +	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> +		/* CSI2 */
> +		unicam_set_field(&val, 1, UNICAM_DLE);
> +		unicam_set_field(&val, 1, UNICAM_DLLPE);
> +		if (!(unicam->bus_flags & V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK)) {
> +			unicam_set_field(&val, 1, UNICAM_DLTRE);
> +			unicam_set_field(&val, 1, UNICAM_DLHSE);
> +		}
> +	} else {
> +		/* CCP2 */
> +		unicam_set_field(&val, 1, UNICAM_DLE);
> +		unicam_set_field(&val, 1, UNICAM_DLHSE);
> +		unicam_set_field(&val, 1, UNICAM_DLTRE);
> +	}
> +	unicam_reg_write(unicam, UNICAM_DAT0, val);
> +
> +	if (unicam->active_data_lanes == 1)
> +		val = 0;
> +	unicam_reg_write(unicam, UNICAM_DAT1, val);
> +
> +	if (unicam->max_data_lanes > 2) {
> +		/*
> +		 * Registers UNICAM_DAT2 and UNICAM_DAT3 only valid if the
> +		 * instance supports more than 2 data lanes.
> +		 */
> +		if (unicam->active_data_lanes == 2)
> +			val = 0;
> +		unicam_reg_write(unicam, UNICAM_DAT2, val);
> +
> +		if (unicam->active_data_lanes == 3)
> +			val = 0;
> +		unicam_reg_write(unicam, UNICAM_DAT3, val);
> +	}
> +
> +	unicam_reg_write(unicam, UNICAM_IBLS,
> +			 node->fmt.fmt.pix.bytesperline);
> +	unicam_wr_dma_addr(&unicam->node[UNICAM_IMAGE_NODE], buf);
> +	unicam_set_packing_config(unicam);
> +	unicam_cfg_image_id(unicam);
> +
> +	val = unicam_reg_read(unicam, UNICAM_MISC);
> +	unicam_set_field(&val, 1, UNICAM_FL0);
> +	unicam_set_field(&val, 1, UNICAM_FL1);
> +	unicam_reg_write(unicam, UNICAM_MISC, val);
> +
> +	/* Enable peripheral */
> +	unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPE);
> +
> +	/* Load image pointers */
> +	unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_LIP_MASK);
> +
> +	/*
> +	 * Enable trigger only for the first frame to
> +	 * sync correctly to the FS from the source.
> +	 */
> +	unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> +}
> +
> +static void unicam_start_metadata(struct unicam_device *unicam,
> +				  struct unicam_buffer *buf)
> +{
> +	struct unicam_node *node = &unicam->node[UNICAM_METADATA_NODE];
> +
> +	unicam_enable_ed(unicam);
> +	unicam_wr_dma_addr(node, buf);
> +	unicam_reg_write_field(unicam, UNICAM_DCS, 1, UNICAM_LDP);
> +}
> +
> +static void unicam_disable(struct unicam_device *unicam)
> +{
> +	/* Analogue lane control disable */
> +	unicam_reg_write_field(unicam, UNICAM_ANA, 1, UNICAM_DDL);
> +
> +	/* Stop the output engine */
> +	unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_SOE);
> +
> +	/* Disable the data lanes. */
> +	unicam_reg_write(unicam, UNICAM_DAT0, 0);
> +	unicam_reg_write(unicam, UNICAM_DAT1, 0);
> +
> +	if (unicam->max_data_lanes > 2) {
> +		unicam_reg_write(unicam, UNICAM_DAT2, 0);
> +		unicam_reg_write(unicam, UNICAM_DAT3, 0);
> +	}
> +
> +	/* Peripheral reset */
> +	unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPR);
> +	usleep_range(50, 100);
> +	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPR);
> +
> +	/* Disable peripheral */
> +	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPE);
> +
> +	/* Clear ED setup */
> +	unicam_reg_write(unicam, UNICAM_DCS, 0);
> +
> +	/* Disable all lane clocks */
> +	unicam_clk_write(unicam, 0);
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * V4L2 subdev operations
> + */
> +
> +static int __unicam_subdev_set_routing(struct v4l2_subdev *sd,
> +				       struct v4l2_subdev_state *state,
> +				       struct v4l2_subdev_krouting *routing)
> +{
> +	struct v4l2_subdev_route *route;
> +	int ret;
> +
> +	ret = v4l2_subdev_routing_validate(sd, routing,
> +					   V4L2_SUBDEV_ROUTING_ONLY_1_TO_1);
> +	if (ret)
> +		return ret;
> +
> +	ret = v4l2_subdev_set_routing(sd, state, routing);
> +	if (ret)
> +		return ret;
> +
> +	for_each_active_route(&state->routing, route) {
> +		const struct v4l2_mbus_framefmt *def_fmt;
> +		struct v4l2_mbus_framefmt *fmt;
> +
> +		if (route->source_pad == UNICAM_SD_PAD_SOURCE_IMAGE)
> +			def_fmt = &unicam_default_image_format;
> +		else
> +			def_fmt = &unicam_default_meta_format;
> +
> +		fmt = v4l2_subdev_state_get_format(state, route->sink_pad,
> +						   route->sink_stream);
> +		*fmt = *def_fmt;
> +		fmt = v4l2_subdev_state_get_format(state, route->source_pad,
> +						   route->source_stream);
> +		*fmt = *def_fmt;
> +	}
> +
> +	return 0;
> +}
> +
> +static int unicam_subdev_init_state(struct v4l2_subdev *sd,
> +				    struct v4l2_subdev_state *state)
> +{
> +	struct v4l2_subdev_route routes[] = {
> +		{
> +			.sink_pad = UNICAM_SD_PAD_SINK,
> +			.sink_stream = 0,
> +			.source_pad = UNICAM_SD_PAD_SOURCE_IMAGE,
> +			.source_stream = 0,
> +			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
> +		},
> +	};
> +
> +	struct v4l2_subdev_krouting routing = {
> +		.len_routes = ARRAY_SIZE(routes),
> +		.num_routes = ARRAY_SIZE(routes),
> +		.routes = routes,
> +	};
> +
> +	/* Initialize routing to single route to the fist source pad. */
> +	return __unicam_subdev_set_routing(sd, state, &routing);
> +}
> +
> +static int unicam_subdev_enum_mbus_code(struct v4l2_subdev *sd,
> +					struct v4l2_subdev_state *state,
> +					struct v4l2_subdev_mbus_code_enum *code)
> +{
> +	u32 pad, stream;
> +	int ret;
> +
> +	ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
> +						    code->pad, code->stream,
> +						    &pad, &stream);
> +	if (ret)
> +		return ret;
> +
> +	if (unicam_sd_pad_is_source(code->pad)) {
> +		/* No transcoding, source and sink codes must match. */
> +		const struct v4l2_mbus_framefmt *fmt;
> +
> +		fmt = v4l2_subdev_state_get_format(state, pad, stream);
> +		if (!fmt)
> +			return -EINVAL;
> +
> +		if (code->index > 0)
> +			return -EINVAL;
> +
> +		code->code = fmt->code;
> +	} else {
> +		const struct unicam_format_info *formats;
> +		unsigned int num_formats;
> +
> +		if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> +			formats = unicam_image_formats;
> +			num_formats = ARRAY_SIZE(unicam_image_formats);
> +		} else {
> +			formats = unicam_meta_formats;
> +			num_formats = ARRAY_SIZE(unicam_meta_formats);
> +		}
> +
> +		if (code->index >= num_formats)
> +			return -EINVAL;
> +
> +		code->code = formats[code->index].code;
> +	}
> +
> +	return 0;
> +}
> +
> +static int unicam_subdev_enum_frame_size(struct v4l2_subdev *sd,
> +					 struct v4l2_subdev_state *state,
> +					 struct v4l2_subdev_frame_size_enum *fse)
> +{
> +	u32 pad, stream;
> +	int ret;
> +
> +	if (fse->index > 0)
> +		return -EINVAL;
> +
> +	ret = v4l2_subdev_routing_find_opposite_end(&state->routing, fse->pad,
> +						    fse->stream, &pad,
> +						    &stream);
> +	if (ret)
> +		return ret;
> +
> +	if (unicam_sd_pad_is_source(fse->pad)) {
> +		/* No transcoding, source and sink formats must match. */
> +		const struct v4l2_mbus_framefmt *fmt;
> +
> +		fmt = v4l2_subdev_state_get_format(state, pad, stream);
> +		if (!fmt)
> +			return -EINVAL;
> +
> +		if (fse->code != fmt->code)
> +			return -EINVAL;
> +
> +		fse->min_width = fmt->width;
> +		fse->max_width = fmt->width;
> +		fse->min_height = fmt->height;
> +		fse->max_height = fmt->height;
> +	} else {
> +		const struct unicam_format_info *fmtinfo;
> +
> +		fmtinfo = unicam_find_format_by_code(fse->code, pad);
> +		if (!fmtinfo)
> +			return -EINVAL;
> +
> +		if (pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> +			fse->min_width = UNICAM_IMAGE_MIN_WIDTH;
> +			fse->max_width = UNICAM_IMAGE_MAX_WIDTH;
> +			fse->min_height = UNICAM_IMAGE_MIN_HEIGHT;
> +			fse->max_height = UNICAM_IMAGE_MAX_HEIGHT;
> +		} else {
> +			fse->min_width = UNICAM_META_MIN_WIDTH;
> +			fse->max_width = UNICAM_META_MAX_WIDTH;
> +			fse->min_height = UNICAM_META_MIN_HEIGHT;
> +			fse->max_height = UNICAM_META_MAX_HEIGHT;
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +static int unicam_subdev_set_format(struct v4l2_subdev *sd,
> +				    struct v4l2_subdev_state *state,
> +				    struct v4l2_subdev_format *format)
> +{
> +	struct unicam_device *unicam = sd_to_unicam_device(sd);
> +	struct v4l2_mbus_framefmt *sink_format, *source_format;
> +	const struct unicam_format_info *fmtinfo;
> +	u32 source_pad, source_stream;
> +	int ret;
> +
> +	if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE &&
> +	    unicam->subdev.streaming)
> +		return -EBUSY;
> +
> +	/* No transcoding, source and sink formats must match. */
> +	if (unicam_sd_pad_is_source(format->pad))
> +		return v4l2_subdev_get_fmt(sd, state, format);
> +
> +	/*
> +	 * Allowed formats for the stream on the sink pad depend on what source
> +	 * pad the stream is routed to. Find the corresponding source pad and
> +	 * use it to validate the media bus code.
> +	 */
> +	ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
> +						    format->pad, format->stream,
> +						    &source_pad, &source_stream);
> +	if (ret)
> +		return ret;
> +
> +	fmtinfo = unicam_find_format_by_code(format->format.code, source_pad);
> +	if (!fmtinfo) {
> +		fmtinfo = source_pad == UNICAM_SD_PAD_SOURCE_IMAGE
> +			? &unicam_image_formats[0] : &unicam_meta_formats[0];
> +		format->format.code = fmtinfo->code;
> +	}
> +
> +	if (source_pad == UNICAM_SD_PAD_SOURCE_IMAGE) {
> +		format->format.width = clamp_t(unsigned int,
> +					       format->format.width,
> +					       UNICAM_IMAGE_MIN_WIDTH,
> +					       UNICAM_IMAGE_MAX_WIDTH);
> +		format->format.height = clamp_t(unsigned int,
> +						format->format.height,
> +						UNICAM_IMAGE_MIN_HEIGHT,
> +						UNICAM_IMAGE_MAX_HEIGHT);
> +		format->format.field = V4L2_FIELD_NONE;
> +	} else {
> +		format->format.width = clamp_t(unsigned int,
> +					       format->format.width,
> +					       UNICAM_META_MIN_WIDTH,
> +					       UNICAM_META_MAX_WIDTH);
> +		format->format.height = clamp_t(unsigned int,
> +						format->format.height,
> +						UNICAM_META_MIN_HEIGHT,
> +						UNICAM_META_MAX_HEIGHT);
> +		format->format.field = V4L2_FIELD_NONE;
> +
> +		/* Colorspace don't apply to metadata. */
> +		format->format.colorspace = 0;
> +		format->format.ycbcr_enc = 0;
> +		format->format.quantization = 0;
> +		format->format.xfer_func = 0;
> +	}
> +
> +	sink_format = v4l2_subdev_state_get_format(state, format->pad,
> +						   format->stream);
> +	source_format = v4l2_subdev_state_get_format(state, source_pad,
> +						     source_stream);
> +	*sink_format = format->format;
> +	*source_format = format->format;
> +
> +	return 0;
> +}
> +
> +static int unicam_subdev_set_routing(struct v4l2_subdev *sd,
> +				     struct v4l2_subdev_state *state,
> +				     enum v4l2_subdev_format_whence which,
> +				     struct v4l2_subdev_krouting *routing)
> +{
> +	struct unicam_device *unicam = sd_to_unicam_device(sd);
> +
> +	if (which == V4L2_SUBDEV_FORMAT_ACTIVE && unicam->subdev.streaming)
> +		return -EBUSY;
> +
> +	return __unicam_subdev_set_routing(sd, state, routing);
> +}
> +
> +static int unicam_sd_enable_streams(struct v4l2_subdev *sd,
> +				    struct v4l2_subdev_state *state, u32 pad,
> +				    u64 streams_mask)
> +{
> +	struct unicam_device *unicam = sd_to_unicam_device(sd);
> +	u32 other_pad, other_stream;
> +	int ret;
> +
> +	ret = v4l2_subdev_routing_find_opposite_end(&state->routing, pad, 0,
> +						    &other_pad, &other_stream);
> +	if (ret)
> +		return ret;
> +
> +	unicam->sequence = 0;
> +
> +	ret = v4l2_subdev_enable_streams(unicam->sensor.subdev,
> +					 unicam->sensor.pad->index,
> +					 BIT(other_stream));
> +	if (ret) {
> +		dev_err(unicam->dev, "stream on failed in subdev\n");
> +		return ret;
> +	}
> +
> +	unicam->subdev.streaming = true;
> +
> +	return 0;
> +}
> +
> +static int unicam_sd_disable_streams(struct v4l2_subdev *sd,
> +				     struct v4l2_subdev_state *state, u32 pad,
> +				     u64 streams_mask)
> +{
> +	struct unicam_device *unicam = sd_to_unicam_device(sd);
> +	u32 other_pad, other_stream;
> +	int ret;
> +
> +	ret = v4l2_subdev_routing_find_opposite_end(&state->routing, pad, 0,
> +						    &other_pad, &other_stream);
> +	if (ret)
> +		return ret;
> +
> +	v4l2_subdev_disable_streams(unicam->sensor.subdev,
> +				    unicam->sensor.pad->index,
> +				    BIT(other_stream));
> +
> +	unicam->subdev.streaming = false;
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_subdev_pad_ops unicam_subdev_pad_ops = {
> +	.enum_mbus_code		= unicam_subdev_enum_mbus_code,
> +	.enum_frame_size	= unicam_subdev_enum_frame_size,
> +	.get_fmt		= v4l2_subdev_get_fmt,
> +	.set_fmt		= unicam_subdev_set_format,
> +	.set_routing		= unicam_subdev_set_routing,
> +	.enable_streams		= unicam_sd_enable_streams,
> +	.disable_streams	= unicam_sd_disable_streams,
> +};
> +
> +static const struct v4l2_subdev_ops unicam_subdev_ops = {
> +	.pad			= &unicam_subdev_pad_ops,
> +};
> +
> +static const struct v4l2_subdev_internal_ops unicam_subdev_internal_ops = {
> +	.init_state		= unicam_subdev_init_state,
> +};
> +
> +static const struct media_entity_operations unicam_subdev_media_ops = {
> +	.link_validate		= v4l2_subdev_link_validate,
> +	.has_pad_interdep	= v4l2_subdev_has_pad_interdep,
> +};
> +
> +static int unicam_subdev_init(struct unicam_device *unicam)
> +{
> +	struct v4l2_subdev *sd = &unicam->subdev.sd;
> +	int ret;
> +
> +	v4l2_subdev_init(sd, &unicam_subdev_ops);
> +	sd->internal_ops = &unicam_subdev_internal_ops;
> +	v4l2_set_subdevdata(sd, unicam);
> +
> +	sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> +	sd->entity.ops = &unicam_subdev_media_ops;
> +	sd->dev = unicam->dev;
> +	sd->owner = THIS_MODULE;
> +	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_STREAMS;
> +
> +	strscpy(sd->name, "unicam", sizeof(sd->name));
> +
> +	unicam->subdev.pads[UNICAM_SD_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> +	unicam->subdev.pads[UNICAM_SD_PAD_SOURCE_IMAGE].flags = MEDIA_PAD_FL_SOURCE;
> +	unicam->subdev.pads[UNICAM_SD_PAD_SOURCE_METADATA].flags = MEDIA_PAD_FL_SOURCE;
> +
> +	ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(unicam->subdev.pads),
> +				     unicam->subdev.pads);
> +	if (ret) {
> +		dev_err(unicam->dev, "Failed to initialize media entity: %d\n",
> +			ret);
> +		return ret;
> +	}
> +
> +	ret = v4l2_subdev_init_finalize(sd);
> +	if (ret) {
> +		dev_err(unicam->dev, "Failed to initialize subdev: %d\n", ret);
> +		goto err_entity;
> +	}
> +
> +	ret = v4l2_device_register_subdev(&unicam->v4l2_dev, sd);
> +	if (ret) {
> +		dev_err(unicam->dev, "Failed to register subdev: %d\n", ret);
> +		goto err_subdev;
> +	}
> +
> +	return 0;
> +
> +err_subdev:
> +	v4l2_subdev_cleanup(sd);
> +err_entity:
> +	media_entity_cleanup(&sd->entity);
> +	return ret;
> +}
> +
> +static void unicam_subdev_cleanup(struct unicam_device *unicam)
> +{
> +	v4l2_subdev_cleanup(&unicam->subdev.sd);
> +	media_entity_cleanup(&unicam->subdev.sd.entity);
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * Videobuf2 queue operations
> + */
> +
> +static int unicam_queue_setup(struct vb2_queue *vq,
> +			      unsigned int *nbuffers,
> +			      unsigned int *nplanes,
> +			      unsigned int sizes[],
> +			      struct device *alloc_devs[])
> +{
> +	struct unicam_node *node = vb2_get_drv_priv(vq);
> +	u32 size = is_image_node(node) ? node->fmt.fmt.pix.sizeimage
> +		 : node->fmt.fmt.meta.buffersize;
> +
> +	if (vq->num_buffers + *nbuffers < 3)
> +		*nbuffers = 3 - vq->num_buffers;
> +
> +	if (*nplanes) {
> +		if (sizes[0] < size) {
> +			dev_dbg(node->dev->dev, "sizes[0] %i < size %u\n",
> +				sizes[0], size);
> +			return -EINVAL;
> +		}
> +		size = sizes[0];
> +	}
> +
> +	*nplanes = 1;
> +	sizes[0] = size;
> +
> +	return 0;
> +}
> +
> +static int unicam_buffer_prepare(struct vb2_buffer *vb)
> +{
> +	struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
> +	struct unicam_buffer *buf = to_unicam_buffer(vb);
> +	u32 size = is_image_node(node) ? node->fmt.fmt.pix.sizeimage
> +		 : node->fmt.fmt.meta.buffersize;
> +
> +	if (vb2_plane_size(vb, 0) < size) {
> +		dev_dbg(node->dev->dev,
> +			"data will not fit into plane (%lu < %u)\n",
> +			vb2_plane_size(vb, 0), size);
> +		return -EINVAL;
> +	}
> +
> +	buf->dma_addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
> +	buf->size = size;
> +
> +	vb2_set_plane_payload(&buf->vb.vb2_buf, 0, size);
> +
> +	return 0;
> +}
> +
> +static void unicam_return_buffers(struct unicam_node *node,
> +				  enum vb2_buffer_state state)
> +{
> +	struct unicam_buffer *buf, *tmp;
> +
> +	list_for_each_entry_safe(buf, tmp, &node->dma_queue, list) {
> +		list_del(&buf->list);
> +		vb2_buffer_done(&buf->vb.vb2_buf, state);
> +	}
> +
> +	if (node->cur_frm)
> +		vb2_buffer_done(&node->cur_frm->vb.vb2_buf,
> +				state);
> +	if (node->next_frm && node->cur_frm != node->next_frm)
> +		vb2_buffer_done(&node->next_frm->vb.vb2_buf,
> +				state);
> +
> +	node->cur_frm = NULL;
> +	node->next_frm = NULL;
> +}
> +
> +static int unicam_start_streaming(struct vb2_queue *vq, unsigned int count)
> +{
> +	struct unicam_node *node = vb2_get_drv_priv(vq);
> +	struct unicam_device *unicam = node->dev;
> +	struct v4l2_subdev_state *state;
> +	struct unicam_buffer *buf;
> +	unsigned long flags;
> +	int ret;
> +	u32 pad, stream;
> +	u32 remote_pad = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
> +					     : UNICAM_SD_PAD_SOURCE_METADATA;
> +
> +	/* Look for the route for the given pad and stream. */
> +	state = v4l2_subdev_lock_and_get_active_state(&unicam->subdev.sd);
> +	ret = v4l2_subdev_routing_find_opposite_end(&state->routing,
> +						    remote_pad, 0,
> +						    &pad, &stream);
> +	v4l2_subdev_unlock_state(state);
> +
> +	if (ret)
> +		goto err_return_buffers;
> +
> +	dev_dbg(unicam->dev, "Starting stream on %s: %u/%u -> %u/%u (%s)\n",
> +		unicam->subdev.sd.name, pad, stream, remote_pad, 0,
> +		is_metadata_node(node) ? "metadata" : "image");
> +
> +	/* The metadata node can't be started alone. */
> +	if (is_metadata_node(node)) {
> +		if (!unicam->node[UNICAM_IMAGE_NODE].streaming) {
> +			dev_err(unicam->dev,
> +				"Can't start metadata without image\n");
> +			ret = -EINVAL;
> +			goto err_return_buffers;
> +		}
> +
> +		spin_lock_irqsave(&node->dma_queue_lock, flags);
> +		buf = list_first_entry(&node->dma_queue,
> +				       struct unicam_buffer, list);
> +		dev_dbg(unicam->dev, "buffer %p\n", buf);
> +		node->cur_frm = buf;
> +		node->next_frm = buf;
> +		list_del(&buf->list);
> +		spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> +
> +		unicam_start_metadata(unicam, buf);
> +		node->streaming = true;
> +		return 0;
> +	}
> +
> +	ret = pm_runtime_resume_and_get(unicam->dev);
> +	if (ret < 0) {
> +		dev_err(unicam->dev, "PM runtime resume failed: %d\n", ret);
> +		goto err_return_buffers;
> +	}
> +
> +	ret = video_device_pipeline_start(&node->video_dev, &unicam->pipe);
> +	if (ret < 0) {
> +		dev_dbg(unicam->dev, "Failed to start media pipeline: %d\n", ret);
> +		goto err_pm_put;
> +	}
> +
> +	spin_lock_irqsave(&node->dma_queue_lock, flags);
> +	buf = list_first_entry(&node->dma_queue,
> +			       struct unicam_buffer, list);
> +	dev_dbg(unicam->dev, "buffer %p\n", buf);
> +	node->cur_frm = buf;
> +	node->next_frm = buf;
> +	list_del(&buf->list);
> +	spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> +
> +	unicam_start_rx(unicam, buf);
> +
> +	ret = v4l2_subdev_enable_streams(&unicam->subdev.sd, remote_pad, BIT(0));
> +	if (ret < 0) {
> +		dev_err(unicam->dev, "stream on failed in subdev\n");
> +		goto error_pipeline;
> +	}
> +
> +	node->streaming = true;
> +
> +	return 0;
> +
> +error_pipeline:
> +	video_device_pipeline_stop(&node->video_dev);
> +err_pm_put:
> +	pm_runtime_put_sync(unicam->dev);
> +err_return_buffers:
> +	unicam_return_buffers(node, VB2_BUF_STATE_QUEUED);
> +	return ret;
> +}
> +
> +static void unicam_stop_streaming(struct vb2_queue *vq)
> +{
> +	struct unicam_node *node = vb2_get_drv_priv(vq);
> +	struct unicam_device *unicam = node->dev;
> +	u32 remote_pad_index = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
> +						   : UNICAM_SD_PAD_SOURCE_METADATA;
> +
> +	node->streaming = false;
> +
> +	v4l2_subdev_disable_streams(&unicam->subdev.sd, remote_pad_index,
> +				    BIT(0));
> +
> +	/* We can stream only with the image node. */
> +	if (is_metadata_node(node)) {
> +		/*
> +		 * Allow the hardware to spin in the dummy buffer.
> +		 * This is only really needed if the embedded data pad is
> +		 * disabled before the image pad.
> +		 */
> +		unicam_wr_dma_addr(node, &node->dummy_buf);
> +		goto dequeue_buffers;
> +	}
> +
> +	unicam_disable(unicam);
> +
> +	video_device_pipeline_stop(&node->video_dev);
> +	pm_runtime_put(unicam->dev);
> +
> +dequeue_buffers:
> +	/* Clear all queued buffers for the node */
> +	unicam_return_buffers(node, VB2_BUF_STATE_ERROR);
> +}
> +
> +static void unicam_buffer_queue(struct vb2_buffer *vb)
> +{
> +	struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
> +	struct unicam_buffer *buf = to_unicam_buffer(vb);
> +
> +	spin_lock_irq(&node->dma_queue_lock);
> +	list_add_tail(&buf->list, &node->dma_queue);
> +	spin_unlock_irq(&node->dma_queue_lock);
> +}
> +
> +static const struct vb2_ops unicam_video_qops = {
> +	.queue_setup		= unicam_queue_setup,
> +	.wait_prepare		= vb2_ops_wait_prepare,
> +	.wait_finish		= vb2_ops_wait_finish,
> +	.buf_prepare		= unicam_buffer_prepare,
> +	.start_streaming	= unicam_start_streaming,
> +	.stop_streaming		= unicam_stop_streaming,
> +	.buf_queue		= unicam_buffer_queue,
> +};
> +
> +/* -----------------------------------------------------------------------------
> + *  V4L2 video device operations
> + */
> +
> +static int unicam_querycap(struct file *file, void *priv,
> +			   struct v4l2_capability *cap)
> +{
> +	strscpy(cap->driver, UNICAM_MODULE_NAME, sizeof(cap->driver));
> +	strscpy(cap->card, UNICAM_MODULE_NAME, sizeof(cap->card));
> +
> +	cap->capabilities |= V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_META_CAPTURE;
> +
> +	return 0;
> +}
> +
> +static int unicam_enum_fmt_vid(struct file *file, void  *priv,
> +			       struct v4l2_fmtdesc *f)
> +{
> +	unsigned int index;
> +	unsigned int i;
> +
> +	for (i = 0, index = 0; i < ARRAY_SIZE(unicam_image_formats); i++) {
> +		if (f->mbus_code && unicam_image_formats[i].code != f->mbus_code)
> +			continue;
> +
> +		if (index == f->index) {
> +			f->pixelformat = unicam_image_formats[i].fourcc;
> +			return 0;
> +		}
> +
> +		index++;
> +
> +		if (!unicam_image_formats[i].unpacked_fourcc)
> +			continue;
> +
> +		if (index == f->index) {
> +			f->pixelformat = unicam_image_formats[i].unpacked_fourcc;
> +			return 0;
> +		}
> +
> +		index++;
> +	}
> +
> +	return -EINVAL;
> +}
> +
> +static int unicam_g_fmt_vid(struct file *file, void *priv,
> +			    struct v4l2_format *f)
> +{
> +	struct unicam_node *node = video_drvdata(file);
> +
> +	*f = node->fmt;
> +
> +	return 0;
> +}
> +
> +static const struct unicam_format_info *
> +__unicam_try_fmt_vid(struct unicam_node *node, struct v4l2_pix_format *pix)
> +{
> +	const struct unicam_format_info *fmtinfo;
> +
> +	/*
> +	 * Default to the first format if the requested pixel format code isn't
> +	 * supported.
> +	 */
> +	fmtinfo = unicam_find_format_by_fourcc(pix->pixelformat,
> +					       UNICAM_SD_PAD_SOURCE_IMAGE);
> +	if (!fmtinfo) {
> +		fmtinfo = &unicam_image_formats[0];
> +		pix->pixelformat = fmtinfo->fourcc;
> +	}
> +
> +	unicam_calc_image_size_bpl(node->dev, fmtinfo, pix);
> +
> +	if (pix->field == V4L2_FIELD_ANY)
> +		pix->field = V4L2_FIELD_NONE;
> +
> +	return fmtinfo;
> +}
> +
> +static int unicam_try_fmt_vid(struct file *file, void *priv,
> +			      struct v4l2_format *f)
> +{
> +	struct unicam_node *node = video_drvdata(file);
> +
> +	__unicam_try_fmt_vid(node, &f->fmt.pix);
> +	return 0;
> +}
> +
> +static int unicam_s_fmt_vid(struct file *file, void *priv,
> +			    struct v4l2_format *f)
> +{
> +	struct unicam_node *node = video_drvdata(file);
> +
> +	if (vb2_is_busy(&node->buffer_queue))
> +		return -EBUSY;
> +
> +	node->fmtinfo = __unicam_try_fmt_vid(node, &f->fmt.pix);
> +	node->fmt = *f;
> +
> +	return 0;
> +}
> +
> +static int unicam_enum_fmt_meta(struct file *file, void *priv,
> +				struct v4l2_fmtdesc *f)
> +{
> +	unsigned int i, index;
> +
> +	for (i = 0, index = 0; i < ARRAY_SIZE(unicam_meta_formats); i++) {
> +		if (f->mbus_code && unicam_meta_formats[i].code != f->mbus_code)
> +			continue;
> +		if (!unicam_meta_formats[i].metadata_fmt)
> +			continue;
> +
> +		if (index == f->index) {
> +			f->pixelformat = unicam_meta_formats[i].fourcc;
> +			f->type = V4L2_BUF_TYPE_META_CAPTURE;
> +			return 0;
> +		}
> +		index++;
> +	}
> +
> +	return -EINVAL;
> +}
> +
> +static int unicam_g_fmt_meta(struct file *file, void *priv,
> +			     struct v4l2_format *f)
> +{
> +	struct unicam_node *node = video_drvdata(file);
> +
> +	f->fmt.meta = node->fmt.fmt.meta;
> +
> +	return 0;
> +}
> +
> +static const struct unicam_format_info *
> +__unicam_try_fmt_meta(struct unicam_node *node, struct v4l2_meta_format *meta)
> +{
> +	const struct unicam_format_info *fmtinfo;
> +
> +	/*
> +	 * Default to the first format if the requested pixel format code isn't
> +	 * supported.
> +	 */
> +	fmtinfo = unicam_find_format_by_fourcc(meta->dataformat,
> +					       UNICAM_SD_PAD_SOURCE_METADATA);
> +	if (!fmtinfo) {
> +		fmtinfo = &unicam_meta_formats[0];
> +		meta->dataformat = fmtinfo->fourcc;
> +	}
> +
> +	unicam_calc_meta_size_bpl(node->dev, fmtinfo, meta);
> +
> +	return fmtinfo;
> +}
> +
> +static int unicam_try_fmt_meta(struct file *file, void *priv,
> +			       struct v4l2_format *f)
> +{
> +	struct unicam_node *node = video_drvdata(file);
> +
> +	__unicam_try_fmt_vid(node, &f->fmt.pix);
> +	return 0;
> +}
> +
> +static int unicam_s_fmt_meta(struct file *file, void *priv,
> +			     struct v4l2_format *f)
> +{
> +	struct unicam_node *node = video_drvdata(file);
> +
> +	if (vb2_is_busy(&node->buffer_queue))
> +		return -EBUSY;
> +
> +	node->fmtinfo = __unicam_try_fmt_meta(node, &f->fmt.meta);
> +	node->fmt = *f;
> +
> +	return 0;
> +}
> +
> +static int unicam_enum_framesizes(struct file *file, void *fh,
> +				  struct v4l2_frmsizeenum *fsize)
> +{
> +	struct unicam_node *node = video_drvdata(file);
> +	int ret = -EINVAL;
> +
> +	if (fsize->index > 0)
> +		return ret;
> +
> +	if (is_image_node(node)) {
> +		if (!unicam_find_format_by_fourcc(fsize->pixel_format,
> +						  UNICAM_SD_PAD_SOURCE_IMAGE))
> +			return ret;
> +
> +		fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> +		fsize->stepwise.min_width = UNICAM_IMAGE_MIN_WIDTH;
> +		fsize->stepwise.max_width = UNICAM_IMAGE_MAX_WIDTH;
> +		fsize->stepwise.step_width = 1;
> +		fsize->stepwise.min_height = UNICAM_IMAGE_MIN_HEIGHT;
> +		fsize->stepwise.max_height = UNICAM_IMAGE_MAX_HEIGHT;
> +		fsize->stepwise.step_height = 1;
> +	} else {
> +		if (!unicam_find_format_by_fourcc(fsize->pixel_format,
> +						  UNICAM_SD_PAD_SOURCE_METADATA))
> +			return ret;
> +
> +		fsize->stepwise.min_width = UNICAM_META_MIN_WIDTH;
> +		fsize->stepwise.max_width = UNICAM_META_MAX_WIDTH;
> +		fsize->stepwise.step_width = 1;
> +		fsize->stepwise.min_height = UNICAM_META_MIN_HEIGHT;
> +		fsize->stepwise.max_height = UNICAM_META_MAX_HEIGHT;
> +		fsize->stepwise.step_height = 1;
> +	}
> +
> +	return 0;
> +}
> +
> +static int unicam_log_status(struct file *file, void *fh)
> +{
> +	struct unicam_node *node = video_drvdata(file);
> +	struct unicam_device *unicam = node->dev;
> +	u32 reg;
> +
> +	/* status for sub devices */
> +	v4l2_device_call_all(&unicam->v4l2_dev, 0, core, log_status);
> +
> +	dev_info(unicam->dev, "-----Receiver status-----\n");
> +	dev_info(unicam->dev, "V4L2 width/height:   %ux%u\n",
> +		 node->fmt.fmt.pix.width, node->fmt.fmt.pix.height);
> +	dev_info(unicam->dev, "Mediabus format:     %08x\n",
> +		 node->fmtinfo->code);
> +	dev_info(unicam->dev, "V4L2 format:         %08x\n",
> +		 node->fmt.fmt.pix.pixelformat);
> +	reg = unicam_reg_read(unicam, UNICAM_IPIPE);
> +	dev_info(unicam->dev, "Unpacking/packing:   %u / %u\n",
> +		 unicam_get_field(reg, UNICAM_PUM_MASK),
> +		 unicam_get_field(reg, UNICAM_PPM_MASK));
> +	dev_info(unicam->dev, "----Live data----\n");
> +	dev_info(unicam->dev, "Programmed stride:   %4u\n",
> +		 unicam_reg_read(unicam, UNICAM_IBLS));
> +	dev_info(unicam->dev, "Detected resolution: %ux%u\n",
> +		 unicam_reg_read(unicam, UNICAM_IHSTA),
> +		 unicam_reg_read(unicam, UNICAM_IVSTA));
> +	dev_info(unicam->dev, "Write pointer:       %08x\n",
> +		 unicam_reg_read(unicam, UNICAM_IBWP));
> +
> +	return 0;
> +}
> +
> +static int unicam_subscribe_event(struct v4l2_fh *fh,
> +				  const struct v4l2_event_subscription *sub)
> +{
> +	switch (sub->type) {
> +	case V4L2_EVENT_FRAME_SYNC:
> +		return v4l2_event_subscribe(fh, sub, 2, NULL);
> +	default:
> +		return -EINVAL;
> +	}
> +}
> +
> +static const struct v4l2_ioctl_ops unicam_ioctl_ops = {
> +	.vidioc_querycap		= unicam_querycap,
> +
> +	.vidioc_enum_fmt_vid_cap	= unicam_enum_fmt_vid,
> +	.vidioc_g_fmt_vid_cap		= unicam_g_fmt_vid,
> +	.vidioc_try_fmt_vid_cap		= unicam_try_fmt_vid,
> +	.vidioc_s_fmt_vid_cap		= unicam_s_fmt_vid,
> +
> +	.vidioc_enum_fmt_meta_cap	= unicam_enum_fmt_meta,
> +	.vidioc_g_fmt_meta_cap		= unicam_g_fmt_meta,
> +	.vidioc_try_fmt_meta_cap	= unicam_try_fmt_meta,
> +	.vidioc_s_fmt_meta_cap		= unicam_s_fmt_meta,
> +
> +	.vidioc_enum_framesizes		= unicam_enum_framesizes,
> +
> +	.vidioc_reqbufs			= vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs		= vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf		= vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf		= vb2_ioctl_querybuf,
> +	.vidioc_qbuf			= vb2_ioctl_qbuf,
> +	.vidioc_dqbuf			= vb2_ioctl_dqbuf,
> +	.vidioc_expbuf			= vb2_ioctl_expbuf,
> +	.vidioc_streamon		= vb2_ioctl_streamon,
> +	.vidioc_streamoff		= vb2_ioctl_streamoff,
> +
> +	.vidioc_log_status		= unicam_log_status,
> +	.vidioc_subscribe_event		= unicam_subscribe_event,
> +	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
> +};
> +
> +/* unicam capture driver file operations */
> +static const struct v4l2_file_operations unicam_fops = {
> +	.owner		= THIS_MODULE,
> +	.open           = v4l2_fh_open,
> +	.release        = vb2_fop_release,
> +	.poll		= vb2_fop_poll,
> +	.unlocked_ioctl = video_ioctl2,
> +	.mmap           = vb2_fop_mmap,
> +};
> +
> +static int unicam_video_link_validate(struct media_link *link)
> +{
> +	struct video_device *vdev =
> +		media_entity_to_video_device(link->sink->entity);
> +	struct v4l2_subdev *sd =
> +		media_entity_to_v4l2_subdev(link->source->entity);
> +	struct unicam_node *node = video_get_drvdata(vdev);
> +	const u32 pad = is_image_node(node) ? UNICAM_SD_PAD_SOURCE_IMAGE
> +		      : UNICAM_SD_PAD_SOURCE_METADATA;
> +	const struct v4l2_mbus_framefmt *format;
> +	struct v4l2_subdev_state *state;
> +	int ret = 0;
> +
> +	state = v4l2_subdev_lock_and_get_active_state(sd);
> +
> +	format = v4l2_subdev_state_get_format(state, pad, 0);
> +	if (!format) {
> +		ret = -EINVAL;
> +		goto out;
> +	}
> +
> +	if (is_image_node(node)) {
> +		const struct v4l2_pix_format *fmt = &node->fmt.fmt.pix;
> +
> +		if (node->fmtinfo->code != format->code ||
> +		    fmt->height != format->height ||
> +		    fmt->width != format->width ||
> +		    fmt->field != format->field) {
> +			dev_dbg(node->dev->dev,
> +				"image: (%u x %u) 0x%08x %s != (%u x %u) 0x%08x %s\n",
> +				fmt->width, fmt->height, node->fmtinfo->code,
> +				v4l2_field_names[fmt->field],
> +				format->width, format->height, format->code,
> +				v4l2_field_names[format->field]);
> +			ret = -EPIPE;
> +		};
> +	} else {
> +		const struct v4l2_meta_format *fmt = &node->fmt.fmt.meta;
> +
> +		if (node->fmtinfo->code != format->code ||
> +		    fmt->height != format->height ||
> +		    fmt->width != format->width) {
> +			dev_dbg(node->dev->dev,
> +				"meta: (%u x %u) 0x%04x != (%u x %u) 0x%04x\n",
> +				fmt->width, fmt->height, node->fmtinfo->code,
> +				format->width, format->height, format->code);
> +			ret = -EPIPE;
> +		};
> +	}
> +
> +out:
> +	v4l2_subdev_unlock_state(state);
> +	return ret;
> +}
> +
> +static const struct media_entity_operations unicam_video_media_ops = {
> +	.link_validate = unicam_video_link_validate,
> +};
> +
> +static void unicam_node_release(struct video_device *vdev)
> +{
> +	struct unicam_node *node = video_get_drvdata(vdev);
> +
> +	unicam_put(node->dev);
> +}
> +
> +static void unicam_set_default_format(struct unicam_node *node)
> +{
> +	if (is_image_node(node)) {
> +		struct v4l2_pix_format *fmt = &node->fmt.fmt.pix;
> +
> +		node->fmtinfo = &unicam_image_formats[0];
> +		node->fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> +
> +		v4l2_fill_pix_format(fmt, &unicam_default_image_format);
> +		fmt->pixelformat = node->fmtinfo->fourcc;
> +		unicam_calc_image_size_bpl(node->dev, node->fmtinfo, fmt);
> +	} else {
> +		struct v4l2_meta_format *fmt = &node->fmt.fmt.meta;
> +
> +		node->fmtinfo = &unicam_meta_formats[0];
> +		node->fmt.type = V4L2_BUF_TYPE_META_CAPTURE;
> +
> +		fmt->dataformat = node->fmtinfo->fourcc;
> +		fmt->width = unicam_default_meta_format.width;
> +		fmt->height = unicam_default_meta_format.height;
> +		unicam_calc_meta_size_bpl(node->dev, node->fmtinfo, fmt);
> +	}
> +}
> +
> +static int unicam_register_node(struct unicam_device *unicam,
> +				enum unicam_node_type type)
> +{
> +	const u32 pad_index = type == UNICAM_IMAGE_NODE
> +			    ? UNICAM_SD_PAD_SOURCE_IMAGE
> +			    : UNICAM_SD_PAD_SOURCE_METADATA;
> +	struct unicam_node *node = &unicam->node[type];
> +	struct video_device *vdev = &node->video_dev;
> +	struct vb2_queue *q = &node->buffer_queue;
> +	int ret;
> +
> +	node->dev = unicam_get(unicam);
> +	node->id = type;
> +
> +	spin_lock_init(&node->dma_queue_lock);
> +	mutex_init(&node->lock);
> +
> +	INIT_LIST_HEAD(&node->dma_queue);
> +
> +	/* Initialize the videobuf2 queue. */
> +	q->type = type == UNICAM_IMAGE_NODE ? V4L2_BUF_TYPE_VIDEO_CAPTURE
> +					    : V4L2_BUF_TYPE_META_CAPTURE;
> +	q->io_modes = VB2_MMAP | VB2_DMABUF;
> +	q->drv_priv = node;
> +	q->ops = &unicam_video_qops;
> +	q->mem_ops = &vb2_dma_contig_memops;
> +	q->buf_struct_size = sizeof(struct unicam_buffer);
> +	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> +	q->lock = &node->lock;
> +	q->min_queued_buffers = 1;
> +	q->dev = unicam->dev;
> +
> +	ret = vb2_queue_init(q);
> +	if (ret) {
> +		dev_err(unicam->dev, "vb2_queue_init() failed\n");
> +		goto err_unicam_put;
> +	}
> +
> +	/* Initialize the video device. */
> +	vdev->release = unicam_node_release;
> +	vdev->fops = &unicam_fops;
> +	vdev->ioctl_ops = &unicam_ioctl_ops;
> +	vdev->v4l2_dev = &unicam->v4l2_dev;
> +	vdev->vfl_dir = VFL_DIR_RX;
> +	vdev->queue = q;
> +	vdev->lock = &node->lock;
> +	vdev->device_caps = type == UNICAM_IMAGE_NODE
> +			  ? V4L2_CAP_VIDEO_CAPTURE : V4L2_CAP_META_CAPTURE;
> +	vdev->device_caps |= V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;
> +	vdev->entity.ops = &unicam_video_media_ops;
> +
> +	snprintf(vdev->name, sizeof(vdev->name), "%s-%s", UNICAM_MODULE_NAME,
> +		 type == UNICAM_IMAGE_NODE ? "image" : "embedded");
> +
> +	video_set_drvdata(vdev, node);
> +
> +	if (type == UNICAM_IMAGE_NODE)
> +		vdev->entity.flags |= MEDIA_ENT_FL_DEFAULT;
> +
> +	node->pad.flags = MEDIA_PAD_FL_SINK;
> +
> +	ret = media_entity_pads_init(&vdev->entity, 1, &node->pad);
> +	if (ret)
> +		goto err_unicam_put;
> +
> +	node->dummy_buf.size = UNICAM_DUMMY_BUF_SIZE;
> +	node->dummy_buf_cpu_addr = dma_alloc_coherent(unicam->dev,
> +						      node->dummy_buf.size,
> +						      &node->dummy_buf.dma_addr,
> +						      GFP_KERNEL);
> +	if (!node->dummy_buf_cpu_addr) {
> +		dev_err(unicam->dev, "Unable to allocate dummy buffer.\n");
> +		ret = -ENOMEM;
> +		goto err_entity_cleanup;
> +	}
> +
> +	unicam_set_default_format(node);
> +
> +	ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> +	if (ret) {
> +		dev_err(unicam->dev, "Unable to register video device %s\n",
> +			vdev->name);
> +		goto err_dma_free;
> +	}
> +
> +	node->registered = true;
> +
> +	ret = media_create_pad_link(&unicam->subdev.sd.entity,
> +				    pad_index,
> +				    &node->video_dev.entity,
> +				    0,
> +				    MEDIA_LNK_FL_ENABLED |
> +				    MEDIA_LNK_FL_IMMUTABLE);
> +	if (ret) {
> +		/*
> +		 * No need for cleanup, the caller will unregister the
> +		 * video device, which will drop the reference on the
> +		 * device and trigger the cleanup.
> +		 */
> +		dev_err(unicam->dev, "Unable to create pad link for %s\n",
> +			unicam->sensor.subdev->name);
> +		return ret;
> +	}
> +
> +	return 0;
> +
> +err_dma_free:
> +	dma_free_coherent(unicam->dev, node->dummy_buf.size,
> +			  node->dummy_buf_cpu_addr,
> +			  node->dummy_buf.dma_addr);
> +err_entity_cleanup:
> +	media_entity_cleanup(&vdev->entity);
> +err_unicam_put:
> +	unicam_put(unicam);
> +	return ret;
> +}
> +
> +static void unicam_unregister_nodes(struct unicam_device *unicam)
> +{
> +	unsigned int i;
> +
> +	for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> +		struct unicam_node *node = &unicam->node[i];
> +
> +		if (node->dummy_buf_cpu_addr)
> +			dma_free_coherent(unicam->dev, node->dummy_buf.size,
> +					  node->dummy_buf_cpu_addr,
> +					  node->dummy_buf.dma_addr);
> +
> +		if (node->registered) {
> +			video_unregister_device(&node->video_dev);
> +			node->registered = false;
> +		}
> +	}
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * Power management
> + */
> +
> +static int unicam_runtime_resume(struct device *dev)
> +{
> +	struct unicam_device *unicam = dev_get_drvdata(dev);
> +	int ret;
> +
> +	ret = clk_set_min_rate(unicam->vpu_clock, UNICAM_MIN_VPU_CLOCK_RATE);
> +	if (ret) {
> +		dev_err(unicam->dev, "failed to set up VPU clock\n");
> +		return ret;
> +	}
> +
> +	ret = clk_prepare_enable(unicam->vpu_clock);
> +	if (ret) {
> +		dev_err(unicam->dev, "Failed to enable VPU clock: %d\n", ret);
> +		goto err_vpu_clock;
> +	}
> +
> +	ret = clk_set_rate(unicam->clock, 100 * 1000 * 1000);
> +	if (ret) {
> +		dev_err(unicam->dev, "failed to set up CSI clock\n");
> +		goto err_vpu_prepare;
> +	}
> +
> +	ret = clk_prepare_enable(unicam->clock);
> +	if (ret) {
> +		dev_err(unicam->dev, "Failed to enable CSI clock: %d\n", ret);
> +		goto err_vpu_prepare;
> +	}
> +
> +	return 0;
> +
> +err_vpu_prepare:
> +	clk_disable_unprepare(unicam->vpu_clock);
> +err_vpu_clock:
> +	if (clk_set_min_rate(unicam->vpu_clock, 0))
> +		dev_err(unicam->dev, "failed to reset the VPU clock\n");
> +
> +	return ret;
> +}
> +
> +static int unicam_runtime_suspend(struct device *dev)
> +{
> +	struct unicam_device *unicam = dev_get_drvdata(dev);
> +
> +	clk_disable_unprepare(unicam->clock);
> +
> +	if (clk_set_min_rate(unicam->vpu_clock, 0))
> +		dev_err(unicam->dev, "failed to reset the VPU clock\n");
> +
> +	clk_disable_unprepare(unicam->vpu_clock);
> +
> +	return 0;
> +}
> +
> +static const struct dev_pm_ops unicam_pm_ops = {
> +	RUNTIME_PM_OPS(unicam_runtime_suspend, unicam_runtime_resume, NULL)
> +};
> +
> +/* -----------------------------------------------------------------------------
> + * V4L2 async notifier
> + */
> +
> +static int unicam_async_bound(struct v4l2_async_notifier *notifier,
> +			      struct v4l2_subdev *subdev,
> +			      struct v4l2_async_connection *asc)
> +{
> +	struct unicam_device *unicam = notifier_to_unicam_device(notifier);
> +	struct media_pad *sink = &unicam->subdev.pads[UNICAM_SD_PAD_SINK];
> +	struct media_pad *source;
> +	int ret;
> +
> +	dev_dbg(unicam->dev, "Using sensor %s for capture\n",
> +		subdev->name);
> +
> +	ret = v4l2_create_fwnode_links_to_pad(subdev, sink, MEDIA_LNK_FL_ENABLED |
> +					      MEDIA_LNK_FL_IMMUTABLE);
> +	if (ret)
> +		return ret;
> +
> +	source = media_pad_remote_pad_unique(sink);
> +	if (!source) {
> +		dev_err(unicam->dev, "No connected sensor pad\n");
> +		return -ENOTCONN;
> +	}
> +
> +	unicam->sensor.subdev = subdev;
> +	unicam->sensor.pad = source;
> +
> +	return 0;
> +}
> +
> +static int unicam_async_complete(struct v4l2_async_notifier *notifier)
> +{
> +	struct unicam_device *unicam = notifier_to_unicam_device(notifier);
> +	int ret;
> +
> +	ret = unicam_register_node(unicam, UNICAM_IMAGE_NODE);
> +	if (ret) {
> +		dev_err(unicam->dev, "Unable to register image video device.\n");
> +		goto unregister;
> +	}
> +
> +	ret = unicam_register_node(unicam, UNICAM_METADATA_NODE);
> +	if (ret) {
> +		dev_err(unicam->dev, "Unable to register metadata video device.\n");
> +		goto unregister;
> +	}
> +
> +	ret = v4l2_device_register_subdev_nodes(&unicam->v4l2_dev);
> +	if (ret) {
> +		dev_err(unicam->dev, "Unable to register subdev nodes.\n");
> +		goto unregister;
> +	}
> +
> +	return 0;
> +
> +unregister:
> +	unicam_unregister_nodes(unicam);
> +	unicam_put(unicam);
> +
> +	return ret;
> +}
> +
> +static const struct v4l2_async_notifier_operations unicam_async_ops = {
> +	.bound = unicam_async_bound,
> +	.complete = unicam_async_complete,
> +};
> +
> +static int unicam_async_nf_init(struct unicam_device *unicam)
> +{
> +	struct v4l2_fwnode_endpoint ep = { };

If the bus-type property is mandatory and you have no stated defaults
anywhere, this is fine. I.e. all the relevant properties would need to be
mandatory.

> +	struct fwnode_handle *ep_handle;
> +	struct v4l2_async_connection *asc;
> +	int ret;
> +
> +	ret = of_property_read_u32(unicam->dev->of_node, "brcm,num-data-lanes",
> +				   &unicam->max_data_lanes);
> +	if (ret < 0) {
> +		dev_err(unicam->dev, "Missing %s DT property\n",
> +			"brcm,num-data-lanes");
> +		return -EINVAL;
> +	}
> +
> +	/* Get and parse the local endpoint. */
> +	ep_handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(unicam->dev), 0, 0,
> +						    FWNODE_GRAPH_ENDPOINT_NEXT);
> +	if (!ep_handle) {
> +		dev_err(unicam->dev, "No endpoint found\n");
> +		return -ENODEV;
> +	}
> +
> +	ret = v4l2_fwnode_endpoint_parse(ep_handle, &ep);
> +	if (ret) {
> +		dev_err(unicam->dev, "Failed to parse endpoint: %d\n", ret);
> +		goto error;
> +	}
> +
> +	unicam->bus_type = ep.bus_type;
> +
> +	switch (ep.bus_type) {
> +	case V4L2_MBUS_CSI2_DPHY: {
> +		unsigned int num_data_lanes = ep.bus.mipi_csi2.num_data_lanes;
> +
> +		if (num_data_lanes != 1 && num_data_lanes != 2 &&
> +		    num_data_lanes != 4) {
> +			dev_err(unicam->dev, "%u data lanes not supported\n",
> +				num_data_lanes);
> +			goto error;
> +		}
> +
> +		if (num_data_lanes > unicam->max_data_lanes) {
> +			dev_err(unicam->dev,
> +				"Endpoint uses %u data lanes when %u are supported\n",
> +				num_data_lanes, unicam->max_data_lanes);
> +			goto error;
> +		}
> +
> +		unicam->active_data_lanes = num_data_lanes;
> +		unicam->bus_flags = ep.bus.mipi_csi2.flags;
> +		break;
> +	}
> +
> +	case V4L2_MBUS_CCP2:
> +		unicam->max_data_lanes = 1;
> +		unicam->active_data_lanes = 1;
> +		unicam->bus_flags = ep.bus.mipi_csi1.strobe;
> +		break;
> +
> +	default:
> +		/* Unsupported bus type */
> +		dev_err(unicam->dev, "Unsupported bus type %u\n", ep.bus_type);
> +		goto error;
> +	}
> +
> +	/* Initialize and register the async notifier. */
> +	v4l2_async_nf_init(&unicam->notifier, &unicam->v4l2_dev);
> +
> +	asc = v4l2_async_nf_add_fwnode_remote(&unicam->notifier, ep_handle,
> +					      struct v4l2_async_connection);
> +	fwnode_handle_put(ep_handle);
> +	ep_handle = NULL;
> +
> +	if (IS_ERR(asc)) {
> +		ret = PTR_ERR(asc);
> +		dev_err(unicam->dev, "Failed to add entry to notifier: %d\n",
> +			ret);
> +		goto error;
> +	}
> +
> +	unicam->notifier.ops = &unicam_async_ops;
> +
> +	ret = v4l2_async_nf_register(&unicam->notifier);
> +	if (ret) {
> +		dev_err(unicam->dev, "Error registering device notifier: %d\n",
> +			ret);
> +		goto error;
> +	}
> +
> +	return 0;
> +
> +error:
> +	fwnode_handle_put(ep_handle);
> +	return ret;
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * Probe & remove
> + */
> +
> +static int unicam_media_init(struct unicam_device *unicam)
> +{
> +	int ret;
> +
> +	unicam->mdev.dev = unicam->dev;
> +	strscpy(unicam->mdev.model, UNICAM_MODULE_NAME,
> +		sizeof(unicam->mdev.model));
> +	strscpy(unicam->mdev.serial, "", sizeof(unicam->mdev.serial));

Isn't the field already zeroed?

> +	unicam->mdev.hw_revision = 0;
> +
> +	media_device_init(&unicam->mdev);
> +
> +	unicam->v4l2_dev.mdev = &unicam->mdev;
> +
> +	ret = v4l2_device_register(unicam->dev, &unicam->v4l2_dev);
> +	if (ret < 0) {
> +		dev_err(unicam->dev, "Unable to register v4l2 device\n");
> +		goto err_media_cleanup;
> +	}
> +
> +	ret = media_device_register(&unicam->mdev);
> +	if (ret < 0) {
> +		dev_err(unicam->dev,
> +			"Unable to register media-controller device\n");
> +		goto err_v4l2_unregister;
> +	}
> +
> +	return 0;
> +
> +err_v4l2_unregister:
> +	v4l2_device_unregister(&unicam->v4l2_dev);
> +err_media_cleanup:
> +	media_device_cleanup(&unicam->mdev);
> +	return ret;
> +}
> +
> +static int unicam_probe(struct platform_device *pdev)
> +{
> +	struct unicam_device *unicam;
> +	int ret;
> +
> +	unicam = kzalloc(sizeof(*unicam), GFP_KERNEL);
> +	if (!unicam)
> +		return -ENOMEM;
> +
> +	kref_init(&unicam->kref);
> +	unicam->dev = &pdev->dev;
> +	platform_set_drvdata(pdev, unicam);
> +
> +	unicam->base = devm_platform_ioremap_resource_byname(pdev, "unicam");
> +	if (IS_ERR(unicam->base)) {
> +		ret = PTR_ERR(unicam->base);
> +		goto err_unicam_put;
> +	}
> +
> +	unicam->clk_gate_base = devm_platform_ioremap_resource_byname(pdev, "cmi");
> +	if (IS_ERR(unicam->clk_gate_base)) {
> +		ret = PTR_ERR(unicam->clk_gate_base);
> +		goto err_unicam_put;
> +	}
> +
> +	unicam->clock = devm_clk_get(&pdev->dev, "lp");
> +	if (IS_ERR(unicam->clock)) {
> +		dev_err(unicam->dev, "Failed to get lp clock\n");
> +		ret = PTR_ERR(unicam->clock);
> +		goto err_unicam_put;
> +	}
> +
> +	unicam->vpu_clock = devm_clk_get(&pdev->dev, "vpu");
> +	if (IS_ERR(unicam->vpu_clock)) {
> +		dev_err(unicam->dev, "Failed to get vpu clock\n");
> +		ret = PTR_ERR(unicam->vpu_clock);
> +		goto err_unicam_put;
> +	}
> +
> +	ret = platform_get_irq(pdev, 0);
> +	if (ret <= 0) {
> +		dev_err(&pdev->dev, "No IRQ resource\n");
> +		ret = -EINVAL;
> +		goto err_unicam_put;
> +	}
> +
> +	ret = devm_request_irq(&pdev->dev, ret, unicam_isr, 0,
> +			       "unicam_capture0", unicam);
> +	if (ret) {
> +		dev_err(&pdev->dev, "Unable to request interrupt\n");
> +		ret = -EINVAL;
> +		goto err_unicam_put;
> +	}
> +
> +	/* Enable the block power domain. */
> +	pm_runtime_enable(&pdev->dev);
> +
> +	ret = unicam_media_init(unicam);
> +	if (ret)
> +		goto err_pm_runtime;
> +
> +	ret = unicam_subdev_init(unicam);
> +	if (ret)
> +		goto err_media_unregister;
> +
> +	ret = unicam_async_nf_init(unicam);
> +	if (ret)
> +		goto err_subdev_unregister;
> +
> +	return 0;
> +
> +err_subdev_unregister:
> +	unicam_subdev_cleanup(unicam);
> +err_media_unregister:
> +	media_device_unregister(&unicam->mdev);
> +err_pm_runtime:
> +	pm_runtime_disable(&pdev->dev);
> +err_unicam_put:
> +	unicam_put(unicam);
> +
> +	return ret;
> +}
> +
> +static int unicam_remove(struct platform_device *pdev)
> +{
> +	struct unicam_device *unicam = platform_get_drvdata(pdev);
> +
> +	unicam_unregister_nodes(unicam);
> +	v4l2_device_unregister(&unicam->v4l2_dev);
> +	media_device_unregister(&unicam->mdev);
> +	v4l2_async_nf_unregister(&unicam->notifier);
> +
> +	unicam_subdev_cleanup(unicam);
> +
> +	unicam_put(unicam);
> +
> +	pm_runtime_disable(&pdev->dev);
> +
> +	return 0;
> +}
> +
> +static const struct of_device_id unicam_of_match[] = {
> +	{ .compatible = "brcm,bcm2835-unicam", },
> +	{ /* sentinel */ },
> +};
> +MODULE_DEVICE_TABLE(of, unicam_of_match);
> +
> +static struct platform_driver unicam_driver = {
> +	.probe		= unicam_probe,
> +	.remove		= unicam_remove,
> +	.driver = {
> +		.name	= UNICAM_MODULE_NAME,
> +		.pm	= pm_ptr(&unicam_pm_ops),
> +		.of_match_table = of_match_ptr(unicam_of_match),
> +	},
> +};
> +
> +module_platform_driver(unicam_driver);
> +
> +MODULE_AUTHOR("Dave Stevenson <dave.stevenson@raspberrypi.com>");
> +MODULE_DESCRIPTION("BCM2835 Unicam driver");
> +MODULE_LICENSE("GPL");

-- 
Kind regards,

Sakari Ailus

Re: [PATCH v6 08/15] dt-bindings: media: Add bindings for bcm2835-unicam

[Laurent Pinchart]

Hi Sakari,

On Mon, Mar 25, 2024 at 06:28:11PM +0000, Sakari Ailus wrote:
> On Fri, Mar 01, 2024 at 11:32:23PM +0200, Laurent Pinchart wrote:
> > From: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > 
> > Introduce the dt-bindings documentation for bcm2835 CCP2/CSI2 Unicam
> > camera interface.
> > 
> > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > Co-developed-by: Naushir Patuck <naush@raspberrypi.com>
> > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > Co-developed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > Reviewed-by: Rob Herring <robh@kernel.org>
> > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > ---
> > Changes since v5:
> > 
> > - Squash MAINTAINERS changes in
> > 
> > Changes since v3:
> > 
> > - Make MAINTAINERS its own patch
> > - Describe the reg and clocks correctly
> > - Use a vendor entry for the number of data lanes
> > ---
> >  .../bindings/media/brcm,bcm2835-unicam.yaml   | 117 ++++++++++++++++++
> >  MAINTAINERS                                   |   6 +
> >  2 files changed, 123 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> > 
> > diff --git a/Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml b/Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> > new file mode 100644
> > index 000000000000..1938ace23b3d
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> > @@ -0,0 +1,117 @@
> > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> > +%YAML 1.2
> > +---
> > +$id: http://devicetree.org/schemas/media/brcm,bcm2835-unicam.yaml#
> > +$schema: http://devicetree.org/meta-schemas/core.yaml#
> > +
> > +title: Broadcom BCM283x Camera Interface (Unicam)
> > +
> > +maintainers:
> > +  - Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
> > +
> > +description: |-
> > +  The Unicam block on BCM283x SoCs is the receiver for either
> > +  CSI-2 or CCP2 data from image sensors or similar devices.
> > +
> > +  The main platform using this SoC is the Raspberry Pi family of boards.  On
> > +  the Pi the VideoCore firmware can also control this hardware block, and
> > +  driving it from two different processors will cause issues.  To avoid this,
> > +  the firmware checks the device tree configuration during boot. If it finds
> > +  device tree nodes whose name starts with 'csi' then it will stop the firmware
> > +  accessing the block, and it can then safely be used via the device tree
> > +  binding.
> > +
> > +properties:
> > +  compatible:
> > +    const: brcm,bcm2835-unicam
> > +
> > +  reg:
> > +    items:
> > +      - description: Unicam block.
> > +      - description: Clock Manager Image (CMI) block.
> > +
> > +  reg-names:
> > +    items:
> > +      - const: unicam
> > +      - const: cmi
> > +
> > +  interrupts:
> > +    maxItems: 1
> > +
> > +  clocks:
> > +    items:
> > +      - description: Clock to drive the LP state machine of Unicam.
> > +      - description: Clock for the VPU (core clock).
> > +
> > +  clock-names:
> > +    items:
> > +      - const: lp
> > +      - const: vpu
> > +
> > +  power-domains:
> > +    items:
> > +      - description: Unicam power domain
> > +
> > +  brcm,num-data-lanes:
> > +    $ref: /schemas/types.yaml#/definitions/uint32
> > +    enum: [ 2, 4 ]
> > +    description: |
> > +      Number of CSI-2 data lanes supported by this Unicam instance. The number
> > +      of data lanes actively used is specified with the data-lanes endpoint
> > +      property.
> > +
> > +  port:
> > +    $ref: /schemas/graph.yaml#/$defs/port-base
> > +    unevaluatedProperties: false
> > +
> > +    properties:
> > +      endpoint:
> > +        $ref: /schemas/media/video-interfaces.yaml#
> > +        unevaluatedProperties: false
> > +
> > +        properties:
> > +          data-lanes: true
> > +
> > +        required:
> > +          - data-lanes
> 
> As the device supports multiple data interfaces (at least so it seems when
> looking at the driver code), you need to list the bus-type property here,
> too.

Good point, I'll add

        properties:
	  bus-type:
	    enum: [ 3, 4 ]
	required:
	  - bus-type

Should I also change unevaluatedProperties to additionalProperties for
the endpoint node, to reject any other property (and the explicitly list
remote-endpoint as an allowed property) ? The result would be 

      endpoint:
        $ref: /schemas/media/video-interfaces.yaml#
        additionalProperties: false

        properties:
          bus-type:
            enum: [ 3, 4 ]

          data-lanes: true
          remote-endpoint: true

        required:
          - bus-type
          - data-lanes
          - remote-endpoint

> > +
> > +    required:
> > +      - endpoint
> > +
> > +required:
> > +  - compatible
> > +  - reg
> > +  - reg-names
> > +  - interrupts
> > +  - clocks
> > +  - clock-names
> > +  - power-domains
> > +  - brcm,num-data-lanes
> > +  - port
> > +
> > +additionalProperties: False
> > +
> > +examples:
> > +  - |
> > +    #include <dt-bindings/clock/bcm2835.h>
> > +    #include <dt-bindings/interrupt-controller/arm-gic.h>
> > +    #include <dt-bindings/power/raspberrypi-power.h>
> > +    csi1: csi@7e801000 {
> > +        compatible = "brcm,bcm2835-unicam";
> > +        reg = <0x7e801000 0x800>,
> > +              <0x7e802004 0x4>;
> > +        reg-names = "unicam", "cmi";
> > +        interrupts = <GIC_SPI 103 IRQ_TYPE_LEVEL_HIGH>;
> > +        clocks = <&clocks BCM2835_CLOCK_CAM1>,
> > +                 <&firmware_clocks 4>;
> > +        clock-names = "lp", "vpu";
> > +        power-domains = <&power RPI_POWER_DOMAIN_UNICAM1>;
> > +        brcm,num-data-lanes = <2>;
> > +        port {
> > +                csi1_ep: endpoint {
> > +                        remote-endpoint = <&imx219_0>;
> > +                        data-lanes = <1 2>;
> > +                };
> > +        };
> > +    };
> > +...
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index fada59148cb5..e50a59654e6e 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -3997,6 +3997,12 @@ N:	bcm113*
> >  N:	bcm216*
> >  N:	kona
> >  
> > +BROADCOM BCM2835 CAMERA DRIVERS
> > +M:	Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
> > +L:	linux-media@vger.kernel.org
> > +S:	Maintained
> > +F:	Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> > +
> >  BROADCOM BCM47XX MIPS ARCHITECTURE
> >  M:	Hauke Mehrtens <hauke@hauke-m.de>
> >  M:	Rafał Miłecki <zajec5@gmail.com>

-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 09/15] media: bcm2835-unicam: Add support for CCP2/CSI2 camera interface

[Laurent Pinchart]

Hi Sakari,

On Mon, Mar 25, 2024 at 06:36:49PM +0000, Sakari Ailus wrote:
> On Fri, Mar 01, 2024 at 11:32:24PM +0200, Laurent Pinchart wrote:
> > From: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > 
> > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > It is represented as two video device nodes: unicam-image and
> > unicam-embedded which are connected to an internal subdev (named
> > unicam-subdev) in order to manage streams routing.
> > 
> > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > Co-developed-by: Naushir Patuck <naush@raspberrypi.com>
> > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > Co-developed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
> Thanks for submitting this, it's the cleanest and neatest Unicom driver

Unicam, or if you insist Unicorn, but not Unicom :-)

> I've ever seen!
> 
> Some mostly unimportant comments below, however the bus-type issue needs to
> be addressed.
> 
> > ---
> > Changes since v5:
> > 
> > - Move to drivers/media/platform/broadcom/
> > - Port to the upstream V4L2 streams API
> > - Rebase on latest metadata API proposal
> > - Add missing error message
> > - Drop unneeded documentation block for unicam_isr()
> > - Drop unneeded dev_dbg() and dev_err() messages
> > - Drop unneeded streams_mask and fmt checks
> > - Drop unused unicam_sd_pad_is_sink()
> > - Drop unneeded includes
> > - Drop v4l2_ctrl_subscribe_event() call
> > - Use pm_runtime_resume_and_get()
> > - Indentation and line wrap fixes
> > - Let the framework set bus_info
> > - Use v4l2_fwnode_endpoint_parse()
> > - Fix media device cleanup
> > - Drop lane reordering checks
> > - Fix subdev state locking
> > - Drop extra debug messages
> > - Move clock handling to runtime PM handlers
> > - Reorder functions
> > - Rename init functions for more clarity
> > - Initialize runtime PM earlier
> > - Clarify error messages
> > - Simplify subdev init with local variable
> > - Fix subdev cleanup
> > - Fix typos and indentation
> > - Don't initialize local variables needlessly
> > - Simplify num lanes check
> > - Fix metadata handling in subdev set_fmt
> > - Drop manual fallback to .s_stream()
> > - Pass v4l2_pix_format to unicam_calc_format_size_bpl()
> > - Simplify unicam_set_default_format()
> > - Fix default format settings
> > - Add busy check in unicam_s_fmt_meta()
> > - Add missing \n at end of format strings
> > - Fix metadata handling in subdev set_fmt
> > - Fix locking when starting streaming
> > - Return buffers from start streaming fails
> > - Fix format validation for metadata node
> > - Use video_device_pipeline_{start,stop}() helpers
> > - Simplify format enumeration
> > - Drop unset variable
> > - Update MAINTAINERS entry
> > - Update to the upstream v4l2_async_nf API
> > - Update to the latest subdev routing API
> > - Update to the latest subdev state API
> > - Move from subdev .init_cfg() to .init_state()
> > - Update to the latest videobuf2 API
> > - Fix v4l2_subdev_enable_streams() error check
> > - Use correct pad for the connected subdev
> > - Return buffers to vb2 when start streaming fails
> > - Improve debugging in start streaming handler
> > - Simplify DMA address management
> > - Drop comment about bcm2835-camera driver
> > - Clarify comments that explain min/max sizes
> > - Pass v4l2_pix_format to unicam_try_fmt()
> > - Drop unneeded local variables
> > - Rename image-related constants and functions
> > - Turn unicam_fmt.metadata_fmt into bool
> > - Rename unicam_fmt to unicam_format_info
> > - Rename unicam_format_info variables to fmtinfo
> > - Rename unicam_node.v_fmt to fmt
> > - Add metadata formats for RAW10, RAW12 and RAW14
> > - Make metadata formats line-based
> > - Validate format on metadata video device
> > - Add Co-devlopped-by tags
> > 
> > Changes since v3:
> > 
> > - Add the vendor prefix for DT name
> > - Use the reg-names in DT parsing
> > - Remove MAINTAINERS entry
> > 
> > Changes since v2:
> > 
> > - Change code organization
> > - Remove unused variables
> > - Correct the fmt_meta functions
> > - Rewrite the start/stop streaming
> >   - You can now start the image node alone, but not the metadata one
> >   - The buffers are allocated per-node
> >   - only the required stream is started, if the route exists and is
> >     enabled
> > - Prefix the macros with UNICAM_ to not have too generic names
> > - Drop colorspace support
> > 
> > Changes since v1:
> > 
> > - Replace the unicam_{info,debug,error} macros with dev_*()
> > ---
> >  MAINTAINERS                                   |    1 +
> >  drivers/media/platform/Kconfig                |    1 +
> >  drivers/media/platform/Makefile               |    1 +
> >  drivers/media/platform/broadcom/Kconfig       |   23 +
> >  drivers/media/platform/broadcom/Makefile      |    3 +
> >  .../platform/broadcom/bcm2835-unicam-regs.h   |  255 ++
> >  .../media/platform/broadcom/bcm2835-unicam.c  | 2607 +++++++++++++++++
> >  7 files changed, 2891 insertions(+)
> >  create mode 100644 drivers/media/platform/broadcom/Kconfig
> >  create mode 100644 drivers/media/platform/broadcom/Makefile
> >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam.c
> > 
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index e50a59654e6e..cc350729f467 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -4002,6 +4002,7 @@ M:	Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
> >  L:	linux-media@vger.kernel.org
> >  S:	Maintained
> >  F:	Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> > +F:	drivers/media/platform/bcm2835/
> >  
> >  BROADCOM BCM47XX MIPS ARCHITECTURE
> >  M:	Hauke Mehrtens <hauke@hauke-m.de>
> > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > index 91e54215de3a..2d79bfc68c15 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -67,6 +67,7 @@ source "drivers/media/platform/amlogic/Kconfig"
> >  source "drivers/media/platform/amphion/Kconfig"
> >  source "drivers/media/platform/aspeed/Kconfig"
> >  source "drivers/media/platform/atmel/Kconfig"
> > +source "drivers/media/platform/broadcom/Kconfig"
> >  source "drivers/media/platform/cadence/Kconfig"
> >  source "drivers/media/platform/chips-media/Kconfig"
> >  source "drivers/media/platform/intel/Kconfig"
> > diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> > index 3296ec1ebe16..da17301f7439 100644
> > --- a/drivers/media/platform/Makefile
> > +++ b/drivers/media/platform/Makefile
> > @@ -10,6 +10,7 @@ obj-y += amlogic/
> >  obj-y += amphion/
> >  obj-y += aspeed/
> >  obj-y += atmel/
> > +obj-y += broadcom/
> >  obj-y += cadence/
> >  obj-y += chips-media/
> >  obj-y += intel/
> > diff --git a/drivers/media/platform/broadcom/Kconfig b/drivers/media/platform/broadcom/Kconfig
> > new file mode 100644
> > index 000000000000..cc2c9afcc948
> > --- /dev/null
> > +++ b/drivers/media/platform/broadcom/Kconfig
> > @@ -0,0 +1,23 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +config VIDEO_BCM2835_UNICAM
> > +	tristate "Broadcom BCM283x/BCM271x Unicam video capture driver"
> > +	depends on ARCH_BCM2835 || COMPILE_TEST
> > +	depends on PM
> > +	depends on VIDEO_DEV
> > +	select MEDIA_CONTROLLER
> > +	select V4L2_FWNODE
> > +	select VIDEO_V4L2_SUBDEV_API
> > +	select VIDEOBUF2_DMA_CONTIG
> > +	help
> > +	  Say Y here to enable support for the BCM283x/BCM271x CSI-2 receiver.
> > +	  This is a V4L2 driver that controls the CSI-2 receiver directly,
> > +	  independently from the VC4 firmware.
> > +
> > +	  This driver is mutually exclusive with the use of bcm2835-camera. The
> > +	  firmware will disable all access to the peripheral from within the
> > +	  firmware if it finds a DT node using it, and bcm2835-camera will
> > +	  therefore fail to probe.
> > +
> > +	  To compile this driver as a module, choose M here. The module will be
> > +	  called bcm2835-unicam.
> > diff --git a/drivers/media/platform/broadcom/Makefile b/drivers/media/platform/broadcom/Makefile
> > new file mode 100644
> > index 000000000000..03d2045aba2e
> > --- /dev/null
> > +++ b/drivers/media/platform/broadcom/Makefile
> > @@ -0,0 +1,3 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +obj-$(CONFIG_VIDEO_BCM2835_UNICAM) += bcm2835-unicam.o
> > diff --git a/drivers/media/platform/broadcom/bcm2835-unicam-regs.h b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > new file mode 100644
> > index 000000000000..84775fd2fac5
> > --- /dev/null
> > +++ b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > @@ -0,0 +1,255 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +
> > +/*
> > + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> 
> Anything up to 2024?

Not really. The registers haven't really changed :-) I'll update the
copyright in the .c file though.

> > + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> > + */

[snip]

> > diff --git a/drivers/media/platform/broadcom/bcm2835-unicam.c b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > new file mode 100644
> > index 000000000000..716c89b8a217
> > --- /dev/null
> > +++ b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > @@ -0,0 +1,2607 @@

[snip]

> > +static irqreturn_t unicam_isr(int irq, void *dev)
> > +{
> > +	struct unicam_device *unicam = dev;
> > +	unsigned int lines_done = unicam_get_lines_done(dev);
> > +	unsigned int sequence = unicam->sequence;
> > +	unsigned int i;
> > +	u32 ista, sta;
> > +	bool fe;
> > +	u64 ts;
> > +
> > +	sta = unicam_reg_read(unicam, UNICAM_STA);
> > +	/* Write value back to clear the interrupts */
> > +	unicam_reg_write(unicam, UNICAM_STA, sta);
> > +
> > +	ista = unicam_reg_read(unicam, UNICAM_ISTA);
> > +	/* Write value back to clear the interrupts */
> > +	unicam_reg_write(unicam, UNICAM_ISTA, ista);
> > +
> > +	dev_dbg(unicam->dev, "ISR: ISTA: 0x%X, STA: 0x%X, sequence %d, lines done %d\n",
> > +		ista, sta, sequence, lines_done);
> > +
> > +	if (!(sta & (UNICAM_IS | UNICAM_PI0)))
> > +		return IRQ_HANDLED;
> > +
> > +	/*
> > +	 * Look for either the Frame End interrupt or the Packet Capture status
> > +	 * to signal a frame end.
> > +	 */
> > +	fe = ista & UNICAM_FEI || sta & UNICAM_PI0;
> > +
> > +	/*
> > +	 * We must run the frame end handler first. If we have a valid next_frm
> > +	 * and we get a simultaneout FE + FS interrupt, running the FS handler
> > +	 * first would null out the next_frm ptr and we would have lost the
> > +	 * buffer forever.
> > +	 */
> > +	if (fe) {
> > +		/*
> > +		 * Ensure we have swapped buffers already as we can't
> > +		 * stop the peripheral. If no buffer is available, use a
> > +		 * dummy buffer to dump out frames until we get a new buffer
> > +		 * to use.
> > +		 */
> > +		for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > +			if (!unicam->node[i].streaming)
> > +				continue;
> > +
> > +			/*
> > +			 * If cur_frm == next_frm, it means we have not had
> > +			 * a chance to swap buffers, likely due to having
> > +			 * multiple interrupts occurring simultaneously (like FE
> > +			 * + FS + LS). In this case, we cannot signal the buffer
> > +			 * as complete, as the HW will reuse that buffer.
> > +			 */
> > +			if (unicam->node[i].cur_frm &&
> > +			    unicam->node[i].cur_frm != unicam->node[i].next_frm)
> > +				unicam_process_buffer_complete(&unicam->node[i],
> > +							       sequence);
> > +			unicam->node[i].cur_frm = unicam->node[i].next_frm;
> > +		}
> > +		unicam->sequence++;
> 
> Does access to this data need to be serialised somehow.

Given that it's only accessed from the interrupt handler (beside
start_streaming time, before starting the hardware), I don't think so.

> > +	}
> > +
> > +	if (ista & UNICAM_FSI) {
> > +		/*
> > +		 * Timestamp is to be when the first data byte was captured,
> > +		 * aka frame start.
> > +		 */
> > +		ts = ktime_get_ns();
> > +		for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > +			if (!unicam->node[i].streaming)
> > +				continue;
> > +
> > +			if (unicam->node[i].cur_frm)
> > +				unicam->node[i].cur_frm->vb.vb2_buf.timestamp =
> > +								ts;
> > +			else
> > +				dev_dbg(unicam->v4l2_dev.dev,
> > +					"ISR: [%d] Dropping frame, buffer not available at FS\n",
> > +					i);
> > +			/*
> > +			 * Set the next frame output to go to a dummy frame
> > +			 * if we have not managed to obtain another frame
> > +			 * from the queue.
> > +			 */
> > +			unicam_schedule_dummy_buffer(&unicam->node[i]);
> > +		}
> > +
> > +		unicam_queue_event_sof(unicam);
> > +	}
> > +
> > +	/*
> > +	 * Cannot swap buffer at frame end, there may be a race condition
> > +	 * where the HW does not actually swap it if the new frame has
> > +	 * already started.
> > +	 */
> > +	if (ista & (UNICAM_FSI | UNICAM_LCI) && !fe) {
> > +		for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > +			if (!unicam->node[i].streaming)
> > +				continue;
> > +
> > +			spin_lock(&unicam->node[i].dma_queue_lock);
> > +			if (!list_empty(&unicam->node[i].dma_queue) &&
> > +			    !unicam->node[i].next_frm)
> > +				unicam_schedule_next_buffer(&unicam->node[i]);
> > +			spin_unlock(&unicam->node[i].dma_queue_lock);
> > +		}
> > +	}
> > +
> > +	if (unicam_reg_read(unicam, UNICAM_ICTL) & UNICAM_FCM) {
> > +		/* Switch out of trigger mode if selected */
> > +		unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> > +		unicam_reg_write_field(unicam, UNICAM_ICTL, 0, UNICAM_FCM);
> > +	}
> > +	return IRQ_HANDLED;
> > +}
> > +
> > +static void unicam_set_packing_config(struct unicam_device *unicam)
> > +{
> > +	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > +	u32 pack, unpack;
> > +	u32 val;
> > +
> > +	if (node->fmt.fmt.pix.pixelformat == node->fmtinfo->fourcc) {
> > +		unpack = UNICAM_PUM_NONE;
> > +		pack = UNICAM_PPM_NONE;
> > +	} else {
> > +		switch (node->fmtinfo->depth) {
> > +		case 8:
> > +			unpack = UNICAM_PUM_UNPACK8;
> > +			break;
> > +		case 10:
> > +			unpack = UNICAM_PUM_UNPACK10;
> > +			break;
> > +		case 12:
> > +			unpack = UNICAM_PUM_UNPACK12;
> > +			break;
> > +		case 14:
> > +			unpack = UNICAM_PUM_UNPACK14;
> > +			break;
> > +		case 16:
> > +			unpack = UNICAM_PUM_UNPACK16;
> > +			break;
> > +		default:
> > +			unpack = UNICAM_PUM_NONE;
> > +			break;
> > +		}
> > +
> > +		/* Repacking is always to 16bpp */
> > +		pack = UNICAM_PPM_PACK16;
> 
> Also 8-bit data?

Not that I know of. The 8-bit entries in unicam_image_formats have no
.unpacked_fourcc field, so the condition in the if above will always be
true for those as they can only be selected by setting the pixel format
to fmtinfo->fourcc.

> > +	}
> > +
> > +	val = 0;
> 
> You could do initialisation in declaration.

Yes, but I think it's more readable to keep all the code that affects
the 'val' variable together.

> > +	unicam_set_field(&val, unpack, UNICAM_PUM_MASK);
> > +	unicam_set_field(&val, pack, UNICAM_PPM_MASK);
> > +	unicam_reg_write(unicam, UNICAM_IPIPE, val);
> > +}
> > +
> > +static void unicam_cfg_image_id(struct unicam_device *unicam)
> > +{
> > +	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > +
> > +	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> > +		/* CSI2 mode, hardcode VC 0 for now. */
> > +		unicam_reg_write(unicam, UNICAM_IDI0,
> > +				 (0 << 6) | node->fmtinfo->csi_dt);
> > +	} else {
> > +		/* CCP2 mode */
> > +		unicam_reg_write(unicam, UNICAM_IDI0,
> > +				 0x80 | node->fmtinfo->csi_dt);
> > +	}
> > +}
> > +
> > +static void unicam_enable_ed(struct unicam_device *unicam)
> > +{
> > +	u32 val = unicam_reg_read(unicam, UNICAM_DCS);
> > +
> > +	unicam_set_field(&val, 2, UNICAM_EDL_MASK);
> > +	/* Do not wrap at the end of the embedded data buffer */
> > +	unicam_set_field(&val, 0, UNICAM_DBOB);
> > +
> > +	unicam_reg_write(unicam, UNICAM_DCS, val);
> > +}
> > +
> > +static void unicam_start_rx(struct unicam_device *unicam,
> > +			    struct unicam_buffer *buf)
> > +{
> > +	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > +	int line_int_freq = node->fmt.fmt.pix.height >> 2;
> > +	unsigned int i;
> > +	u32 val;
> > +
> > +	if (line_int_freq < 128)
> > +		line_int_freq = 128;
> 
> 	line_int_freq = max(line_int_freq, 128);
Ack.

> > +
> > +	/* Enable lane clocks */
> > +	val = 1;
> 
> Initialise in the loop initialisation below, I'd say.

How about

	val = 0x55 & GENMASK(unicam->pipe.num_data_lanes * 2 - 1, 0);

?

> > +	for (i = 0; i < unicam->active_data_lanes; i++)
> > +		val = val << 2 | 1;
> > +	unicam_clk_write(unicam, val);
> > +
> > +	/* Basic init */
> > +	unicam_reg_write(unicam, UNICAM_CTRL, UNICAM_MEM);
> > +
> > +	/* Enable analogue control, and leave in reset. */
> > +	val = UNICAM_AR;
> > +	unicam_set_field(&val, 7, UNICAM_CTATADJ_MASK);
> > +	unicam_set_field(&val, 7, UNICAM_PTATADJ_MASK);
> > +	unicam_reg_write(unicam, UNICAM_ANA, val);
> > +	usleep_range(1000, 2000);
> > +
> > +	/* Come out of reset */
> > +	unicam_reg_write_field(unicam, UNICAM_ANA, 0, UNICAM_AR);
> > +
> > +	/* Peripheral reset */
> > +	unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPR);
> > +	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPR);
> > +
> > +	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPE);
> > +
> > +	/* Enable Rx control. */
> > +	val = unicam_reg_read(unicam, UNICAM_CTRL);
> > +	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> > +		unicam_set_field(&val, UNICAM_CPM_CSI2, UNICAM_CPM_MASK);
> > +		unicam_set_field(&val, UNICAM_DCM_STROBE, UNICAM_DCM_MASK);
> > +	} else {
> > +		unicam_set_field(&val, UNICAM_CPM_CCP2, UNICAM_CPM_MASK);
> > +		unicam_set_field(&val, unicam->bus_flags, UNICAM_DCM_MASK);
> > +	}
> > +	/* Packet framer timeout */
> > +	unicam_set_field(&val, 0xf, UNICAM_PFT_MASK);
> > +	unicam_set_field(&val, 128, UNICAM_OET_MASK);
> > +	unicam_reg_write(unicam, UNICAM_CTRL, val);
> > +
> > +	unicam_reg_write(unicam, UNICAM_IHWIN, 0);
> > +	unicam_reg_write(unicam, UNICAM_IVWIN, 0);
> > +
> > +	/* AXI bus access QoS setup */
> > +	val = unicam_reg_read(unicam, UNICAM_PRI);
> > +	unicam_set_field(&val, 0, UNICAM_BL_MASK);
> > +	unicam_set_field(&val, 0, UNICAM_BS_MASK);
> > +	unicam_set_field(&val, 0xe, UNICAM_PP_MASK);
> > +	unicam_set_field(&val, 8, UNICAM_NP_MASK);
> > +	unicam_set_field(&val, 2, UNICAM_PT_MASK);
> > +	unicam_set_field(&val, 1, UNICAM_PE);
> > +	unicam_reg_write(unicam, UNICAM_PRI, val);
> > +
> > +	unicam_reg_write_field(unicam, UNICAM_ANA, 0, UNICAM_DDL);
> > +
> > +	/* Always start in trigger frame capture mode (UNICAM_FCM set) */
> > +	val = UNICAM_FSIE | UNICAM_FEIE | UNICAM_FCM | UNICAM_IBOB;
> > +	unicam_set_field(&val, line_int_freq, UNICAM_LCIE_MASK);
> > +	unicam_reg_write(unicam, UNICAM_ICTL, val);
> > +	unicam_reg_write(unicam, UNICAM_STA, UNICAM_STA_MASK_ALL);
> > +	unicam_reg_write(unicam, UNICAM_ISTA, UNICAM_ISTA_MASK_ALL);
> > +
> > +	/* tclk_term_en */
> > +	unicam_reg_write_field(unicam, UNICAM_CLT, 2, UNICAM_CLT1_MASK);
> > +	/* tclk_settle */
> > +	unicam_reg_write_field(unicam, UNICAM_CLT, 6, UNICAM_CLT2_MASK);
> > +	/* td_term_en */
> > +	unicam_reg_write_field(unicam, UNICAM_DLT, 2, UNICAM_DLT1_MASK);
> > +	/* ths_settle */
> > +	unicam_reg_write_field(unicam, UNICAM_DLT, 6, UNICAM_DLT2_MASK);
> > +	/* trx_enable */
> > +	unicam_reg_write_field(unicam, UNICAM_DLT, 0, UNICAM_DLT3_MASK);
> > +
> > +	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_SOE);
> > +
> > +	/* Packet compare setup - required to avoid missing frame ends */
> > +	val = 0;
> > +	unicam_set_field(&val, 1, UNICAM_PCE);
> > +	unicam_set_field(&val, 1, UNICAM_GI);
> > +	unicam_set_field(&val, 1, UNICAM_CPH);
> > +	unicam_set_field(&val, 0, UNICAM_PCVC_MASK);
> > +	unicam_set_field(&val, 1, UNICAM_PCDT_MASK);
> > +	unicam_reg_write(unicam, UNICAM_CMP0, val);
> > +
> > +	/* Enable clock lane and set up terminations */
> > +	val = 0;
> > +	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> > +		/* CSI2 */
> > +		unicam_set_field(&val, 1, UNICAM_CLE);
> > +		unicam_set_field(&val, 1, UNICAM_CLLPE);
> > +		if (!(unicam->bus_flags & V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK)) {
> > +			unicam_set_field(&val, 1, UNICAM_CLTRE);
> > +			unicam_set_field(&val, 1, UNICAM_CLHSE);
> > +		}
> > +	} else {
> > +		/* CCP2 */
> > +		unicam_set_field(&val, 1, UNICAM_CLE);
> > +		unicam_set_field(&val, 1, UNICAM_CLHSE);
> > +		unicam_set_field(&val, 1, UNICAM_CLTRE);
> > +	}
> > +	unicam_reg_write(unicam, UNICAM_CLK, val);
> > +
> > +	/*
> > +	 * Enable required data lanes with appropriate terminations.
> > +	 * The same value needs to be written to UNICAM_DATn registers for
> > +	 * the active lanes, and 0 for inactive ones.
> > +	 */
> > +	val = 0;
> > +	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> > +		/* CSI2 */
> > +		unicam_set_field(&val, 1, UNICAM_DLE);
> > +		unicam_set_field(&val, 1, UNICAM_DLLPE);
> > +		if (!(unicam->bus_flags & V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK)) {
> > +			unicam_set_field(&val, 1, UNICAM_DLTRE);
> > +			unicam_set_field(&val, 1, UNICAM_DLHSE);
> > +		}
> > +	} else {
> > +		/* CCP2 */
> > +		unicam_set_field(&val, 1, UNICAM_DLE);
> > +		unicam_set_field(&val, 1, UNICAM_DLHSE);
> > +		unicam_set_field(&val, 1, UNICAM_DLTRE);
> > +	}
> > +	unicam_reg_write(unicam, UNICAM_DAT0, val);
> > +
> > +	if (unicam->active_data_lanes == 1)
> > +		val = 0;
> > +	unicam_reg_write(unicam, UNICAM_DAT1, val);
> > +
> > +	if (unicam->max_data_lanes > 2) {
> > +		/*
> > +		 * Registers UNICAM_DAT2 and UNICAM_DAT3 only valid if the
> > +		 * instance supports more than 2 data lanes.
> > +		 */
> > +		if (unicam->active_data_lanes == 2)
> > +			val = 0;
> > +		unicam_reg_write(unicam, UNICAM_DAT2, val);
> > +
> > +		if (unicam->active_data_lanes == 3)
> > +			val = 0;
> > +		unicam_reg_write(unicam, UNICAM_DAT3, val);
> > +	}
> > +
> > +	unicam_reg_write(unicam, UNICAM_IBLS,
> > +			 node->fmt.fmt.pix.bytesperline);
> > +	unicam_wr_dma_addr(&unicam->node[UNICAM_IMAGE_NODE], buf);
> > +	unicam_set_packing_config(unicam);
> > +	unicam_cfg_image_id(unicam);
> > +
> > +	val = unicam_reg_read(unicam, UNICAM_MISC);
> > +	unicam_set_field(&val, 1, UNICAM_FL0);
> > +	unicam_set_field(&val, 1, UNICAM_FL1);
> > +	unicam_reg_write(unicam, UNICAM_MISC, val);
> > +
> > +	/* Enable peripheral */
> > +	unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPE);
> > +
> > +	/* Load image pointers */
> > +	unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_LIP_MASK);
> > +
> > +	/*
> > +	 * Enable trigger only for the first frame to
> > +	 * sync correctly to the FS from the source.
> > +	 */
> > +	unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> > +}

[snip]

> > +static int unicam_async_nf_init(struct unicam_device *unicam)
> > +{
> > +	struct v4l2_fwnode_endpoint ep = { };
> 
> If the bus-type property is mandatory and you have no stated defaults
> anywhere, this is fine. I.e. all the relevant properties would need to be
> mandatory.

They are, as far as I can tell (well, the clock-noncontinuous property
is not mandatory, but that's expected as it's a flag).

> > +	struct fwnode_handle *ep_handle;
> > +	struct v4l2_async_connection *asc;
> > +	int ret;
> > +
> > +	ret = of_property_read_u32(unicam->dev->of_node, "brcm,num-data-lanes",
> > +				   &unicam->max_data_lanes);
> > +	if (ret < 0) {
> > +		dev_err(unicam->dev, "Missing %s DT property\n",
> > +			"brcm,num-data-lanes");
> > +		return -EINVAL;
> > +	}
> > +
> > +	/* Get and parse the local endpoint. */
> > +	ep_handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(unicam->dev), 0, 0,
> > +						    FWNODE_GRAPH_ENDPOINT_NEXT);
> > +	if (!ep_handle) {
> > +		dev_err(unicam->dev, "No endpoint found\n");
> > +		return -ENODEV;
> > +	}
> > +
> > +	ret = v4l2_fwnode_endpoint_parse(ep_handle, &ep);
> > +	if (ret) {
> > +		dev_err(unicam->dev, "Failed to parse endpoint: %d\n", ret);
> > +		goto error;
> > +	}
> > +
> > +	unicam->bus_type = ep.bus_type;
> > +
> > +	switch (ep.bus_type) {
> > +	case V4L2_MBUS_CSI2_DPHY: {
> > +		unsigned int num_data_lanes = ep.bus.mipi_csi2.num_data_lanes;
> > +
> > +		if (num_data_lanes != 1 && num_data_lanes != 2 &&
> > +		    num_data_lanes != 4) {
> > +			dev_err(unicam->dev, "%u data lanes not supported\n",
> > +				num_data_lanes);
> > +			goto error;
> > +		}
> > +
> > +		if (num_data_lanes > unicam->max_data_lanes) {
> > +			dev_err(unicam->dev,
> > +				"Endpoint uses %u data lanes when %u are supported\n",
> > +				num_data_lanes, unicam->max_data_lanes);
> > +			goto error;
> > +		}
> > +
> > +		unicam->active_data_lanes = num_data_lanes;
> > +		unicam->bus_flags = ep.bus.mipi_csi2.flags;
> > +		break;
> > +	}
> > +
> > +	case V4L2_MBUS_CCP2:
> > +		unicam->max_data_lanes = 1;
> > +		unicam->active_data_lanes = 1;
> > +		unicam->bus_flags = ep.bus.mipi_csi1.strobe;
> > +		break;
> > +
> > +	default:
> > +		/* Unsupported bus type */
> > +		dev_err(unicam->dev, "Unsupported bus type %u\n", ep.bus_type);
> > +		goto error;
> > +	}
> > +
> > +	/* Initialize and register the async notifier. */
> > +	v4l2_async_nf_init(&unicam->notifier, &unicam->v4l2_dev);
> > +
> > +	asc = v4l2_async_nf_add_fwnode_remote(&unicam->notifier, ep_handle,
> > +					      struct v4l2_async_connection);
> > +	fwnode_handle_put(ep_handle);
> > +	ep_handle = NULL;
> > +
> > +	if (IS_ERR(asc)) {
> > +		ret = PTR_ERR(asc);
> > +		dev_err(unicam->dev, "Failed to add entry to notifier: %d\n",
> > +			ret);
> > +		goto error;
> > +	}
> > +
> > +	unicam->notifier.ops = &unicam_async_ops;
> > +
> > +	ret = v4l2_async_nf_register(&unicam->notifier);
> > +	if (ret) {
> > +		dev_err(unicam->dev, "Error registering device notifier: %d\n",
> > +			ret);
> > +		goto error;
> > +	}
> > +
> > +	return 0;
> > +
> > +error:
> > +	fwnode_handle_put(ep_handle);
> > +	return ret;
> > +}
> > +
> > +/* -----------------------------------------------------------------------------
> > + * Probe & remove
> > + */
> > +
> > +static int unicam_media_init(struct unicam_device *unicam)
> > +{
> > +	int ret;
> > +
> > +	unicam->mdev.dev = unicam->dev;
> > +	strscpy(unicam->mdev.model, UNICAM_MODULE_NAME,
> > +		sizeof(unicam->mdev.model));
> > +	strscpy(unicam->mdev.serial, "", sizeof(unicam->mdev.serial));
> 
> Isn't the field already zeroed?

Indeed. I'll drop this.

> 
> > +	unicam->mdev.hw_revision = 0;
> > +
> > +	media_device_init(&unicam->mdev);
> > +
> > +	unicam->v4l2_dev.mdev = &unicam->mdev;
> > +
> > +	ret = v4l2_device_register(unicam->dev, &unicam->v4l2_dev);
> > +	if (ret < 0) {
> > +		dev_err(unicam->dev, "Unable to register v4l2 device\n");
> > +		goto err_media_cleanup;
> > +	}
> > +
> > +	ret = media_device_register(&unicam->mdev);
> > +	if (ret < 0) {
> > +		dev_err(unicam->dev,
> > +			"Unable to register media-controller device\n");
> > +		goto err_v4l2_unregister;
> > +	}
> > +
> > +	return 0;
> > +
> > +err_v4l2_unregister:
> > +	v4l2_device_unregister(&unicam->v4l2_dev);
> > +err_media_cleanup:
> > +	media_device_cleanup(&unicam->mdev);
> > +	return ret;
> > +}

[snip]

-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 09/15] media: bcm2835-unicam: Add support for CCP2/CSI2 camera interface

[Laurent Pinchart]

On Tue, Mar 26, 2024 at 03:37:09AM +0200, Laurent Pinchart wrote:
> Hi Sakari,
> 
> On Mon, Mar 25, 2024 at 06:36:49PM +0000, Sakari Ailus wrote:
> > On Fri, Mar 01, 2024 at 11:32:24PM +0200, Laurent Pinchart wrote:
> > > From: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > 
> > > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > > It is represented as two video device nodes: unicam-image and
> > > unicam-embedded which are connected to an internal subdev (named
> > > unicam-subdev) in order to manage streams routing.
> > > 
> > > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > Co-developed-by: Naushir Patuck <naush@raspberrypi.com>
> > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > > Co-developed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > > Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > > Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > 
> > Thanks for submitting this, it's the cleanest and neatest Unicom driver
> 
> Unicam, or if you insist Unicorn, but not Unicom :-)
> 
> > I've ever seen!
> > 
> > Some mostly unimportant comments below, however the bus-type issue needs to
> > be addressed.
> > 
> > > ---
> > > Changes since v5:
> > > 
> > > - Move to drivers/media/platform/broadcom/
> > > - Port to the upstream V4L2 streams API
> > > - Rebase on latest metadata API proposal
> > > - Add missing error message
> > > - Drop unneeded documentation block for unicam_isr()
> > > - Drop unneeded dev_dbg() and dev_err() messages
> > > - Drop unneeded streams_mask and fmt checks
> > > - Drop unused unicam_sd_pad_is_sink()
> > > - Drop unneeded includes
> > > - Drop v4l2_ctrl_subscribe_event() call
> > > - Use pm_runtime_resume_and_get()
> > > - Indentation and line wrap fixes
> > > - Let the framework set bus_info
> > > - Use v4l2_fwnode_endpoint_parse()
> > > - Fix media device cleanup
> > > - Drop lane reordering checks
> > > - Fix subdev state locking
> > > - Drop extra debug messages
> > > - Move clock handling to runtime PM handlers
> > > - Reorder functions
> > > - Rename init functions for more clarity
> > > - Initialize runtime PM earlier
> > > - Clarify error messages
> > > - Simplify subdev init with local variable
> > > - Fix subdev cleanup
> > > - Fix typos and indentation
> > > - Don't initialize local variables needlessly
> > > - Simplify num lanes check
> > > - Fix metadata handling in subdev set_fmt
> > > - Drop manual fallback to .s_stream()
> > > - Pass v4l2_pix_format to unicam_calc_format_size_bpl()
> > > - Simplify unicam_set_default_format()
> > > - Fix default format settings
> > > - Add busy check in unicam_s_fmt_meta()
> > > - Add missing \n at end of format strings
> > > - Fix metadata handling in subdev set_fmt
> > > - Fix locking when starting streaming
> > > - Return buffers from start streaming fails
> > > - Fix format validation for metadata node
> > > - Use video_device_pipeline_{start,stop}() helpers
> > > - Simplify format enumeration
> > > - Drop unset variable
> > > - Update MAINTAINERS entry
> > > - Update to the upstream v4l2_async_nf API
> > > - Update to the latest subdev routing API
> > > - Update to the latest subdev state API
> > > - Move from subdev .init_cfg() to .init_state()
> > > - Update to the latest videobuf2 API
> > > - Fix v4l2_subdev_enable_streams() error check
> > > - Use correct pad for the connected subdev
> > > - Return buffers to vb2 when start streaming fails
> > > - Improve debugging in start streaming handler
> > > - Simplify DMA address management
> > > - Drop comment about bcm2835-camera driver
> > > - Clarify comments that explain min/max sizes
> > > - Pass v4l2_pix_format to unicam_try_fmt()
> > > - Drop unneeded local variables
> > > - Rename image-related constants and functions
> > > - Turn unicam_fmt.metadata_fmt into bool
> > > - Rename unicam_fmt to unicam_format_info
> > > - Rename unicam_format_info variables to fmtinfo
> > > - Rename unicam_node.v_fmt to fmt
> > > - Add metadata formats for RAW10, RAW12 and RAW14
> > > - Make metadata formats line-based
> > > - Validate format on metadata video device
> > > - Add Co-devlopped-by tags
> > > 
> > > Changes since v3:
> > > 
> > > - Add the vendor prefix for DT name
> > > - Use the reg-names in DT parsing
> > > - Remove MAINTAINERS entry
> > > 
> > > Changes since v2:
> > > 
> > > - Change code organization
> > > - Remove unused variables
> > > - Correct the fmt_meta functions
> > > - Rewrite the start/stop streaming
> > >   - You can now start the image node alone, but not the metadata one
> > >   - The buffers are allocated per-node
> > >   - only the required stream is started, if the route exists and is
> > >     enabled
> > > - Prefix the macros with UNICAM_ to not have too generic names
> > > - Drop colorspace support
> > > 
> > > Changes since v1:
> > > 
> > > - Replace the unicam_{info,debug,error} macros with dev_*()
> > > ---
> > >  MAINTAINERS                                   |    1 +
> > >  drivers/media/platform/Kconfig                |    1 +
> > >  drivers/media/platform/Makefile               |    1 +
> > >  drivers/media/platform/broadcom/Kconfig       |   23 +
> > >  drivers/media/platform/broadcom/Makefile      |    3 +
> > >  .../platform/broadcom/bcm2835-unicam-regs.h   |  255 ++
> > >  .../media/platform/broadcom/bcm2835-unicam.c  | 2607 +++++++++++++++++
> > >  7 files changed, 2891 insertions(+)
> > >  create mode 100644 drivers/media/platform/broadcom/Kconfig
> > >  create mode 100644 drivers/media/platform/broadcom/Makefile
> > >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam.c
> > > 
> > > diff --git a/MAINTAINERS b/MAINTAINERS
> > > index e50a59654e6e..cc350729f467 100644
> > > --- a/MAINTAINERS
> > > +++ b/MAINTAINERS
> > > @@ -4002,6 +4002,7 @@ M:	Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
> > >  L:	linux-media@vger.kernel.org
> > >  S:	Maintained
> > >  F:	Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> > > +F:	drivers/media/platform/bcm2835/
> > >  
> > >  BROADCOM BCM47XX MIPS ARCHITECTURE
> > >  M:	Hauke Mehrtens <hauke@hauke-m.de>
> > > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > > index 91e54215de3a..2d79bfc68c15 100644
> > > --- a/drivers/media/platform/Kconfig
> > > +++ b/drivers/media/platform/Kconfig
> > > @@ -67,6 +67,7 @@ source "drivers/media/platform/amlogic/Kconfig"
> > >  source "drivers/media/platform/amphion/Kconfig"
> > >  source "drivers/media/platform/aspeed/Kconfig"
> > >  source "drivers/media/platform/atmel/Kconfig"
> > > +source "drivers/media/platform/broadcom/Kconfig"
> > >  source "drivers/media/platform/cadence/Kconfig"
> > >  source "drivers/media/platform/chips-media/Kconfig"
> > >  source "drivers/media/platform/intel/Kconfig"
> > > diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> > > index 3296ec1ebe16..da17301f7439 100644
> > > --- a/drivers/media/platform/Makefile
> > > +++ b/drivers/media/platform/Makefile
> > > @@ -10,6 +10,7 @@ obj-y += amlogic/
> > >  obj-y += amphion/
> > >  obj-y += aspeed/
> > >  obj-y += atmel/
> > > +obj-y += broadcom/
> > >  obj-y += cadence/
> > >  obj-y += chips-media/
> > >  obj-y += intel/
> > > diff --git a/drivers/media/platform/broadcom/Kconfig b/drivers/media/platform/broadcom/Kconfig
> > > new file mode 100644
> > > index 000000000000..cc2c9afcc948
> > > --- /dev/null
> > > +++ b/drivers/media/platform/broadcom/Kconfig
> > > @@ -0,0 +1,23 @@
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +
> > > +config VIDEO_BCM2835_UNICAM
> > > +	tristate "Broadcom BCM283x/BCM271x Unicam video capture driver"
> > > +	depends on ARCH_BCM2835 || COMPILE_TEST
> > > +	depends on PM
> > > +	depends on VIDEO_DEV
> > > +	select MEDIA_CONTROLLER
> > > +	select V4L2_FWNODE
> > > +	select VIDEO_V4L2_SUBDEV_API
> > > +	select VIDEOBUF2_DMA_CONTIG
> > > +	help
> > > +	  Say Y here to enable support for the BCM283x/BCM271x CSI-2 receiver.
> > > +	  This is a V4L2 driver that controls the CSI-2 receiver directly,
> > > +	  independently from the VC4 firmware.
> > > +
> > > +	  This driver is mutually exclusive with the use of bcm2835-camera. The
> > > +	  firmware will disable all access to the peripheral from within the
> > > +	  firmware if it finds a DT node using it, and bcm2835-camera will
> > > +	  therefore fail to probe.
> > > +
> > > +	  To compile this driver as a module, choose M here. The module will be
> > > +	  called bcm2835-unicam.
> > > diff --git a/drivers/media/platform/broadcom/Makefile b/drivers/media/platform/broadcom/Makefile
> > > new file mode 100644
> > > index 000000000000..03d2045aba2e
> > > --- /dev/null
> > > +++ b/drivers/media/platform/broadcom/Makefile
> > > @@ -0,0 +1,3 @@
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +
> > > +obj-$(CONFIG_VIDEO_BCM2835_UNICAM) += bcm2835-unicam.o
> > > diff --git a/drivers/media/platform/broadcom/bcm2835-unicam-regs.h b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > > new file mode 100644
> > > index 000000000000..84775fd2fac5
> > > --- /dev/null
> > > +++ b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > > @@ -0,0 +1,255 @@
> > > +/* SPDX-License-Identifier: GPL-2.0-only */
> > > +
> > > +/*
> > > + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> > 
> > Anything up to 2024?
> 
> Not really. The registers haven't really changed :-) I'll update the
> copyright in the .c file though.
> 
> > > + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > + */
> 
> [snip]
> 
> > > diff --git a/drivers/media/platform/broadcom/bcm2835-unicam.c b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > > new file mode 100644
> > > index 000000000000..716c89b8a217
> > > --- /dev/null
> > > +++ b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > > @@ -0,0 +1,2607 @@
> 
> [snip]
> 
> > > +static irqreturn_t unicam_isr(int irq, void *dev)
> > > +{
> > > +	struct unicam_device *unicam = dev;
> > > +	unsigned int lines_done = unicam_get_lines_done(dev);
> > > +	unsigned int sequence = unicam->sequence;
> > > +	unsigned int i;
> > > +	u32 ista, sta;
> > > +	bool fe;
> > > +	u64 ts;
> > > +
> > > +	sta = unicam_reg_read(unicam, UNICAM_STA);
> > > +	/* Write value back to clear the interrupts */
> > > +	unicam_reg_write(unicam, UNICAM_STA, sta);
> > > +
> > > +	ista = unicam_reg_read(unicam, UNICAM_ISTA);
> > > +	/* Write value back to clear the interrupts */
> > > +	unicam_reg_write(unicam, UNICAM_ISTA, ista);
> > > +
> > > +	dev_dbg(unicam->dev, "ISR: ISTA: 0x%X, STA: 0x%X, sequence %d, lines done %d\n",
> > > +		ista, sta, sequence, lines_done);
> > > +
> > > +	if (!(sta & (UNICAM_IS | UNICAM_PI0)))
> > > +		return IRQ_HANDLED;
> > > +
> > > +	/*
> > > +	 * Look for either the Frame End interrupt or the Packet Capture status
> > > +	 * to signal a frame end.
> > > +	 */
> > > +	fe = ista & UNICAM_FEI || sta & UNICAM_PI0;
> > > +
> > > +	/*
> > > +	 * We must run the frame end handler first. If we have a valid next_frm
> > > +	 * and we get a simultaneout FE + FS interrupt, running the FS handler
> > > +	 * first would null out the next_frm ptr and we would have lost the
> > > +	 * buffer forever.
> > > +	 */
> > > +	if (fe) {
> > > +		/*
> > > +		 * Ensure we have swapped buffers already as we can't
> > > +		 * stop the peripheral. If no buffer is available, use a
> > > +		 * dummy buffer to dump out frames until we get a new buffer
> > > +		 * to use.
> > > +		 */
> > > +		for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > > +			if (!unicam->node[i].streaming)
> > > +				continue;
> > > +
> > > +			/*
> > > +			 * If cur_frm == next_frm, it means we have not had
> > > +			 * a chance to swap buffers, likely due to having
> > > +			 * multiple interrupts occurring simultaneously (like FE
> > > +			 * + FS + LS). In this case, we cannot signal the buffer
> > > +			 * as complete, as the HW will reuse that buffer.
> > > +			 */
> > > +			if (unicam->node[i].cur_frm &&
> > > +			    unicam->node[i].cur_frm != unicam->node[i].next_frm)
> > > +				unicam_process_buffer_complete(&unicam->node[i],
> > > +							       sequence);
> > > +			unicam->node[i].cur_frm = unicam->node[i].next_frm;
> > > +		}
> > > +		unicam->sequence++;
> > 
> > Does access to this data need to be serialised somehow.
> 
> Given that it's only accessed from the interrupt handler (beside
> start_streaming time, before starting the hardware), I don't think so.
> 
> > > +	}
> > > +
> > > +	if (ista & UNICAM_FSI) {
> > > +		/*
> > > +		 * Timestamp is to be when the first data byte was captured,
> > > +		 * aka frame start.
> > > +		 */
> > > +		ts = ktime_get_ns();
> > > +		for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > > +			if (!unicam->node[i].streaming)
> > > +				continue;
> > > +
> > > +			if (unicam->node[i].cur_frm)
> > > +				unicam->node[i].cur_frm->vb.vb2_buf.timestamp =
> > > +								ts;
> > > +			else
> > > +				dev_dbg(unicam->v4l2_dev.dev,
> > > +					"ISR: [%d] Dropping frame, buffer not available at FS\n",
> > > +					i);
> > > +			/*
> > > +			 * Set the next frame output to go to a dummy frame
> > > +			 * if we have not managed to obtain another frame
> > > +			 * from the queue.
> > > +			 */
> > > +			unicam_schedule_dummy_buffer(&unicam->node[i]);
> > > +		}
> > > +
> > > +		unicam_queue_event_sof(unicam);
> > > +	}
> > > +
> > > +	/*
> > > +	 * Cannot swap buffer at frame end, there may be a race condition
> > > +	 * where the HW does not actually swap it if the new frame has
> > > +	 * already started.
> > > +	 */
> > > +	if (ista & (UNICAM_FSI | UNICAM_LCI) && !fe) {
> > > +		for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > > +			if (!unicam->node[i].streaming)
> > > +				continue;
> > > +
> > > +			spin_lock(&unicam->node[i].dma_queue_lock);
> > > +			if (!list_empty(&unicam->node[i].dma_queue) &&
> > > +			    !unicam->node[i].next_frm)
> > > +				unicam_schedule_next_buffer(&unicam->node[i]);
> > > +			spin_unlock(&unicam->node[i].dma_queue_lock);
> > > +		}
> > > +	}
> > > +
> > > +	if (unicam_reg_read(unicam, UNICAM_ICTL) & UNICAM_FCM) {
> > > +		/* Switch out of trigger mode if selected */
> > > +		unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> > > +		unicam_reg_write_field(unicam, UNICAM_ICTL, 0, UNICAM_FCM);
> > > +	}
> > > +	return IRQ_HANDLED;
> > > +}
> > > +
> > > +static void unicam_set_packing_config(struct unicam_device *unicam)
> > > +{
> > > +	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > > +	u32 pack, unpack;
> > > +	u32 val;
> > > +
> > > +	if (node->fmt.fmt.pix.pixelformat == node->fmtinfo->fourcc) {
> > > +		unpack = UNICAM_PUM_NONE;
> > > +		pack = UNICAM_PPM_NONE;
> > > +	} else {
> > > +		switch (node->fmtinfo->depth) {
> > > +		case 8:
> > > +			unpack = UNICAM_PUM_UNPACK8;
> > > +			break;
> > > +		case 10:
> > > +			unpack = UNICAM_PUM_UNPACK10;
> > > +			break;
> > > +		case 12:
> > > +			unpack = UNICAM_PUM_UNPACK12;
> > > +			break;
> > > +		case 14:
> > > +			unpack = UNICAM_PUM_UNPACK14;
> > > +			break;
> > > +		case 16:
> > > +			unpack = UNICAM_PUM_UNPACK16;
> > > +			break;
> > > +		default:
> > > +			unpack = UNICAM_PUM_NONE;
> > > +			break;
> > > +		}
> > > +
> > > +		/* Repacking is always to 16bpp */
> > > +		pack = UNICAM_PPM_PACK16;
> > 
> > Also 8-bit data?
> 
> Not that I know of. The 8-bit entries in unicam_image_formats have no
> .unpacked_fourcc field, so the condition in the if above will always be
> true for those as they can only be selected by setting the pixel format
> to fmtinfo->fourcc.
> 
> > > +	}
> > > +
> > > +	val = 0;
> > 
> > You could do initialisation in declaration.
> 
> Yes, but I think it's more readable to keep all the code that affects
> the 'val' variable together.
> 
> > > +	unicam_set_field(&val, unpack, UNICAM_PUM_MASK);
> > > +	unicam_set_field(&val, pack, UNICAM_PPM_MASK);
> > > +	unicam_reg_write(unicam, UNICAM_IPIPE, val);
> > > +}
> > > +
> > > +static void unicam_cfg_image_id(struct unicam_device *unicam)
> > > +{
> > > +	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > > +
> > > +	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> > > +		/* CSI2 mode, hardcode VC 0 for now. */
> > > +		unicam_reg_write(unicam, UNICAM_IDI0,
> > > +				 (0 << 6) | node->fmtinfo->csi_dt);
> > > +	} else {
> > > +		/* CCP2 mode */
> > > +		unicam_reg_write(unicam, UNICAM_IDI0,
> > > +				 0x80 | node->fmtinfo->csi_dt);
> > > +	}
> > > +}
> > > +
> > > +static void unicam_enable_ed(struct unicam_device *unicam)
> > > +{
> > > +	u32 val = unicam_reg_read(unicam, UNICAM_DCS);
> > > +
> > > +	unicam_set_field(&val, 2, UNICAM_EDL_MASK);
> > > +	/* Do not wrap at the end of the embedded data buffer */
> > > +	unicam_set_field(&val, 0, UNICAM_DBOB);
> > > +
> > > +	unicam_reg_write(unicam, UNICAM_DCS, val);
> > > +}
> > > +
> > > +static void unicam_start_rx(struct unicam_device *unicam,
> > > +			    struct unicam_buffer *buf)
> > > +{
> > > +	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > > +	int line_int_freq = node->fmt.fmt.pix.height >> 2;
> > > +	unsigned int i;
> > > +	u32 val;
> > > +
> > > +	if (line_int_freq < 128)
> > > +		line_int_freq = 128;
> > 
> > 	line_int_freq = max(line_int_freq, 128);
> Ack.
> 
> > > +
> > > +	/* Enable lane clocks */
> > > +	val = 1;
> > 
> > Initialise in the loop initialisation below, I'd say.
> 
> How about
> 
> 	val = 0x55 & GENMASK(unicam->pipe.num_data_lanes * 2 - 1, 0);

I meant

 	val = 0x155 & GENMASK(unicam->pipe.num_data_lanes * 2 + 1, 0);

Maybe a comment would be useful ?

	/*
	 * Enable lane clocks. The register is structured as follows:
	 *
	 * [9:8] - DAT3
	 * [7:6] - DAT2
	 * [5:4] - DAT1
	 * [3:2] - DAT0
	 * [1:0] - CLK
	 *
	 * Enabled lane must be set to b01, and disabled lanes to b00. The clock
	 * lane is always enabled.
	 */
	val = 0x155 & GENMASK(unicam->pipe.num_data_lanes * 2 + 1, 0);

> > > +	for (i = 0; i < unicam->active_data_lanes; i++)
> > > +		val = val << 2 | 1;
> > > +	unicam_clk_write(unicam, val);
> > > +
> > > +	/* Basic init */
> > > +	unicam_reg_write(unicam, UNICAM_CTRL, UNICAM_MEM);
> > > +
> > > +	/* Enable analogue control, and leave in reset. */
> > > +	val = UNICAM_AR;
> > > +	unicam_set_field(&val, 7, UNICAM_CTATADJ_MASK);
> > > +	unicam_set_field(&val, 7, UNICAM_PTATADJ_MASK);
> > > +	unicam_reg_write(unicam, UNICAM_ANA, val);
> > > +	usleep_range(1000, 2000);
> > > +
> > > +	/* Come out of reset */
> > > +	unicam_reg_write_field(unicam, UNICAM_ANA, 0, UNICAM_AR);
> > > +
> > > +	/* Peripheral reset */
> > > +	unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPR);
> > > +	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPR);
> > > +
> > > +	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPE);
> > > +
> > > +	/* Enable Rx control. */
> > > +	val = unicam_reg_read(unicam, UNICAM_CTRL);
> > > +	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> > > +		unicam_set_field(&val, UNICAM_CPM_CSI2, UNICAM_CPM_MASK);
> > > +		unicam_set_field(&val, UNICAM_DCM_STROBE, UNICAM_DCM_MASK);
> > > +	} else {
> > > +		unicam_set_field(&val, UNICAM_CPM_CCP2, UNICAM_CPM_MASK);
> > > +		unicam_set_field(&val, unicam->bus_flags, UNICAM_DCM_MASK);
> > > +	}
> > > +	/* Packet framer timeout */
> > > +	unicam_set_field(&val, 0xf, UNICAM_PFT_MASK);
> > > +	unicam_set_field(&val, 128, UNICAM_OET_MASK);
> > > +	unicam_reg_write(unicam, UNICAM_CTRL, val);
> > > +
> > > +	unicam_reg_write(unicam, UNICAM_IHWIN, 0);
> > > +	unicam_reg_write(unicam, UNICAM_IVWIN, 0);
> > > +
> > > +	/* AXI bus access QoS setup */
> > > +	val = unicam_reg_read(unicam, UNICAM_PRI);
> > > +	unicam_set_field(&val, 0, UNICAM_BL_MASK);
> > > +	unicam_set_field(&val, 0, UNICAM_BS_MASK);
> > > +	unicam_set_field(&val, 0xe, UNICAM_PP_MASK);
> > > +	unicam_set_field(&val, 8, UNICAM_NP_MASK);
> > > +	unicam_set_field(&val, 2, UNICAM_PT_MASK);
> > > +	unicam_set_field(&val, 1, UNICAM_PE);
> > > +	unicam_reg_write(unicam, UNICAM_PRI, val);
> > > +
> > > +	unicam_reg_write_field(unicam, UNICAM_ANA, 0, UNICAM_DDL);
> > > +
> > > +	/* Always start in trigger frame capture mode (UNICAM_FCM set) */
> > > +	val = UNICAM_FSIE | UNICAM_FEIE | UNICAM_FCM | UNICAM_IBOB;
> > > +	unicam_set_field(&val, line_int_freq, UNICAM_LCIE_MASK);
> > > +	unicam_reg_write(unicam, UNICAM_ICTL, val);
> > > +	unicam_reg_write(unicam, UNICAM_STA, UNICAM_STA_MASK_ALL);
> > > +	unicam_reg_write(unicam, UNICAM_ISTA, UNICAM_ISTA_MASK_ALL);
> > > +
> > > +	/* tclk_term_en */
> > > +	unicam_reg_write_field(unicam, UNICAM_CLT, 2, UNICAM_CLT1_MASK);
> > > +	/* tclk_settle */
> > > +	unicam_reg_write_field(unicam, UNICAM_CLT, 6, UNICAM_CLT2_MASK);
> > > +	/* td_term_en */
> > > +	unicam_reg_write_field(unicam, UNICAM_DLT, 2, UNICAM_DLT1_MASK);
> > > +	/* ths_settle */
> > > +	unicam_reg_write_field(unicam, UNICAM_DLT, 6, UNICAM_DLT2_MASK);
> > > +	/* trx_enable */
> > > +	unicam_reg_write_field(unicam, UNICAM_DLT, 0, UNICAM_DLT3_MASK);
> > > +
> > > +	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_SOE);
> > > +
> > > +	/* Packet compare setup - required to avoid missing frame ends */
> > > +	val = 0;
> > > +	unicam_set_field(&val, 1, UNICAM_PCE);
> > > +	unicam_set_field(&val, 1, UNICAM_GI);
> > > +	unicam_set_field(&val, 1, UNICAM_CPH);
> > > +	unicam_set_field(&val, 0, UNICAM_PCVC_MASK);
> > > +	unicam_set_field(&val, 1, UNICAM_PCDT_MASK);
> > > +	unicam_reg_write(unicam, UNICAM_CMP0, val);
> > > +
> > > +	/* Enable clock lane and set up terminations */
> > > +	val = 0;
> > > +	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> > > +		/* CSI2 */
> > > +		unicam_set_field(&val, 1, UNICAM_CLE);
> > > +		unicam_set_field(&val, 1, UNICAM_CLLPE);
> > > +		if (!(unicam->bus_flags & V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK)) {
> > > +			unicam_set_field(&val, 1, UNICAM_CLTRE);
> > > +			unicam_set_field(&val, 1, UNICAM_CLHSE);
> > > +		}
> > > +	} else {
> > > +		/* CCP2 */
> > > +		unicam_set_field(&val, 1, UNICAM_CLE);
> > > +		unicam_set_field(&val, 1, UNICAM_CLHSE);
> > > +		unicam_set_field(&val, 1, UNICAM_CLTRE);
> > > +	}
> > > +	unicam_reg_write(unicam, UNICAM_CLK, val);
> > > +
> > > +	/*
> > > +	 * Enable required data lanes with appropriate terminations.
> > > +	 * The same value needs to be written to UNICAM_DATn registers for
> > > +	 * the active lanes, and 0 for inactive ones.
> > > +	 */
> > > +	val = 0;
> > > +	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> > > +		/* CSI2 */
> > > +		unicam_set_field(&val, 1, UNICAM_DLE);
> > > +		unicam_set_field(&val, 1, UNICAM_DLLPE);
> > > +		if (!(unicam->bus_flags & V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK)) {
> > > +			unicam_set_field(&val, 1, UNICAM_DLTRE);
> > > +			unicam_set_field(&val, 1, UNICAM_DLHSE);
> > > +		}
> > > +	} else {
> > > +		/* CCP2 */
> > > +		unicam_set_field(&val, 1, UNICAM_DLE);
> > > +		unicam_set_field(&val, 1, UNICAM_DLHSE);
> > > +		unicam_set_field(&val, 1, UNICAM_DLTRE);
> > > +	}
> > > +	unicam_reg_write(unicam, UNICAM_DAT0, val);
> > > +
> > > +	if (unicam->active_data_lanes == 1)
> > > +		val = 0;
> > > +	unicam_reg_write(unicam, UNICAM_DAT1, val);
> > > +
> > > +	if (unicam->max_data_lanes > 2) {
> > > +		/*
> > > +		 * Registers UNICAM_DAT2 and UNICAM_DAT3 only valid if the
> > > +		 * instance supports more than 2 data lanes.
> > > +		 */
> > > +		if (unicam->active_data_lanes == 2)
> > > +			val = 0;
> > > +		unicam_reg_write(unicam, UNICAM_DAT2, val);
> > > +
> > > +		if (unicam->active_data_lanes == 3)
> > > +			val = 0;
> > > +		unicam_reg_write(unicam, UNICAM_DAT3, val);
> > > +	}
> > > +
> > > +	unicam_reg_write(unicam, UNICAM_IBLS,
> > > +			 node->fmt.fmt.pix.bytesperline);
> > > +	unicam_wr_dma_addr(&unicam->node[UNICAM_IMAGE_NODE], buf);
> > > +	unicam_set_packing_config(unicam);
> > > +	unicam_cfg_image_id(unicam);
> > > +
> > > +	val = unicam_reg_read(unicam, UNICAM_MISC);
> > > +	unicam_set_field(&val, 1, UNICAM_FL0);
> > > +	unicam_set_field(&val, 1, UNICAM_FL1);
> > > +	unicam_reg_write(unicam, UNICAM_MISC, val);
> > > +
> > > +	/* Enable peripheral */
> > > +	unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPE);
> > > +
> > > +	/* Load image pointers */
> > > +	unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_LIP_MASK);
> > > +
> > > +	/*
> > > +	 * Enable trigger only for the first frame to
> > > +	 * sync correctly to the FS from the source.
> > > +	 */
> > > +	unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> > > +}
> 
> [snip]
> 
> > > +static int unicam_async_nf_init(struct unicam_device *unicam)
> > > +{
> > > +	struct v4l2_fwnode_endpoint ep = { };
> > 
> > If the bus-type property is mandatory and you have no stated defaults
> > anywhere, this is fine. I.e. all the relevant properties would need to be
> > mandatory.
> 
> They are, as far as I can tell (well, the clock-noncontinuous property
> is not mandatory, but that's expected as it's a flag).
> 
> > > +	struct fwnode_handle *ep_handle;
> > > +	struct v4l2_async_connection *asc;
> > > +	int ret;
> > > +
> > > +	ret = of_property_read_u32(unicam->dev->of_node, "brcm,num-data-lanes",
> > > +				   &unicam->max_data_lanes);
> > > +	if (ret < 0) {
> > > +		dev_err(unicam->dev, "Missing %s DT property\n",
> > > +			"brcm,num-data-lanes");
> > > +		return -EINVAL;
> > > +	}
> > > +
> > > +	/* Get and parse the local endpoint. */
> > > +	ep_handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(unicam->dev), 0, 0,
> > > +						    FWNODE_GRAPH_ENDPOINT_NEXT);
> > > +	if (!ep_handle) {
> > > +		dev_err(unicam->dev, "No endpoint found\n");
> > > +		return -ENODEV;
> > > +	}
> > > +
> > > +	ret = v4l2_fwnode_endpoint_parse(ep_handle, &ep);
> > > +	if (ret) {
> > > +		dev_err(unicam->dev, "Failed to parse endpoint: %d\n", ret);
> > > +		goto error;
> > > +	}
> > > +
> > > +	unicam->bus_type = ep.bus_type;
> > > +
> > > +	switch (ep.bus_type) {
> > > +	case V4L2_MBUS_CSI2_DPHY: {
> > > +		unsigned int num_data_lanes = ep.bus.mipi_csi2.num_data_lanes;
> > > +
> > > +		if (num_data_lanes != 1 && num_data_lanes != 2 &&
> > > +		    num_data_lanes != 4) {
> > > +			dev_err(unicam->dev, "%u data lanes not supported\n",
> > > +				num_data_lanes);
> > > +			goto error;
> > > +		}
> > > +
> > > +		if (num_data_lanes > unicam->max_data_lanes) {
> > > +			dev_err(unicam->dev,
> > > +				"Endpoint uses %u data lanes when %u are supported\n",
> > > +				num_data_lanes, unicam->max_data_lanes);
> > > +			goto error;
> > > +		}
> > > +
> > > +		unicam->active_data_lanes = num_data_lanes;
> > > +		unicam->bus_flags = ep.bus.mipi_csi2.flags;
> > > +		break;
> > > +	}
> > > +
> > > +	case V4L2_MBUS_CCP2:
> > > +		unicam->max_data_lanes = 1;
> > > +		unicam->active_data_lanes = 1;
> > > +		unicam->bus_flags = ep.bus.mipi_csi1.strobe;
> > > +		break;
> > > +
> > > +	default:
> > > +		/* Unsupported bus type */
> > > +		dev_err(unicam->dev, "Unsupported bus type %u\n", ep.bus_type);
> > > +		goto error;
> > > +	}
> > > +
> > > +	/* Initialize and register the async notifier. */
> > > +	v4l2_async_nf_init(&unicam->notifier, &unicam->v4l2_dev);
> > > +
> > > +	asc = v4l2_async_nf_add_fwnode_remote(&unicam->notifier, ep_handle,
> > > +					      struct v4l2_async_connection);
> > > +	fwnode_handle_put(ep_handle);
> > > +	ep_handle = NULL;
> > > +
> > > +	if (IS_ERR(asc)) {
> > > +		ret = PTR_ERR(asc);
> > > +		dev_err(unicam->dev, "Failed to add entry to notifier: %d\n",
> > > +			ret);
> > > +		goto error;
> > > +	}
> > > +
> > > +	unicam->notifier.ops = &unicam_async_ops;
> > > +
> > > +	ret = v4l2_async_nf_register(&unicam->notifier);
> > > +	if (ret) {
> > > +		dev_err(unicam->dev, "Error registering device notifier: %d\n",
> > > +			ret);
> > > +		goto error;
> > > +	}
> > > +
> > > +	return 0;
> > > +
> > > +error:
> > > +	fwnode_handle_put(ep_handle);
> > > +	return ret;
> > > +}
> > > +
> > > +/* -----------------------------------------------------------------------------
> > > + * Probe & remove
> > > + */
> > > +
> > > +static int unicam_media_init(struct unicam_device *unicam)
> > > +{
> > > +	int ret;
> > > +
> > > +	unicam->mdev.dev = unicam->dev;
> > > +	strscpy(unicam->mdev.model, UNICAM_MODULE_NAME,
> > > +		sizeof(unicam->mdev.model));
> > > +	strscpy(unicam->mdev.serial, "", sizeof(unicam->mdev.serial));
> > 
> > Isn't the field already zeroed?
> 
> Indeed. I'll drop this.
> 
> > 
> > > +	unicam->mdev.hw_revision = 0;
> > > +
> > > +	media_device_init(&unicam->mdev);
> > > +
> > > +	unicam->v4l2_dev.mdev = &unicam->mdev;
> > > +
> > > +	ret = v4l2_device_register(unicam->dev, &unicam->v4l2_dev);
> > > +	if (ret < 0) {
> > > +		dev_err(unicam->dev, "Unable to register v4l2 device\n");
> > > +		goto err_media_cleanup;
> > > +	}
> > > +
> > > +	ret = media_device_register(&unicam->mdev);
> > > +	if (ret < 0) {
> > > +		dev_err(unicam->dev,
> > > +			"Unable to register media-controller device\n");
> > > +		goto err_v4l2_unregister;
> > > +	}
> > > +
> > > +	return 0;
> > > +
> > > +err_v4l2_unregister:
> > > +	v4l2_device_unregister(&unicam->v4l2_dev);
> > > +err_media_cleanup:
> > > +	media_device_cleanup(&unicam->mdev);
> > > +	return ret;
> > > +}
> 
> [snip]

-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 09/15] media: bcm2835-unicam: Add support for CCP2/CSI2 camera interface

[Dave Stevenson]

Hi Sakari & Laurent

On Tue, 26 Mar 2024 at 01:37, Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> Hi Sakari,
>
> On Mon, Mar 25, 2024 at 06:36:49PM +0000, Sakari Ailus wrote:
> > On Fri, Mar 01, 2024 at 11:32:24PM +0200, Laurent Pinchart wrote:
> > > From: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > >
> > > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > > It is represented as two video device nodes: unicam-image and
> > > unicam-embedded which are connected to an internal subdev (named
> > > unicam-subdev) in order to manage streams routing.
> > >
> > > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > Co-developed-by: Naushir Patuck <naush@raspberrypi.com>
> > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > > Co-developed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > > Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > > Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> >
> > Thanks for submitting this, it's the cleanest and neatest Unicom driver
>
> Unicam, or if you insist Unicorn, but not Unicom :-)
>
> > I've ever seen!
> >
> > Some mostly unimportant comments below, however the bus-type issue needs to
> > be addressed.
> >
> > > ---
> > > Changes since v5:
> > >
> > > - Move to drivers/media/platform/broadcom/
> > > - Port to the upstream V4L2 streams API
> > > - Rebase on latest metadata API proposal
> > > - Add missing error message
> > > - Drop unneeded documentation block for unicam_isr()
> > > - Drop unneeded dev_dbg() and dev_err() messages
> > > - Drop unneeded streams_mask and fmt checks
> > > - Drop unused unicam_sd_pad_is_sink()
> > > - Drop unneeded includes
> > > - Drop v4l2_ctrl_subscribe_event() call
> > > - Use pm_runtime_resume_and_get()
> > > - Indentation and line wrap fixes
> > > - Let the framework set bus_info
> > > - Use v4l2_fwnode_endpoint_parse()
> > > - Fix media device cleanup
> > > - Drop lane reordering checks
> > > - Fix subdev state locking
> > > - Drop extra debug messages
> > > - Move clock handling to runtime PM handlers
> > > - Reorder functions
> > > - Rename init functions for more clarity
> > > - Initialize runtime PM earlier
> > > - Clarify error messages
> > > - Simplify subdev init with local variable
> > > - Fix subdev cleanup
> > > - Fix typos and indentation
> > > - Don't initialize local variables needlessly
> > > - Simplify num lanes check
> > > - Fix metadata handling in subdev set_fmt
> > > - Drop manual fallback to .s_stream()
> > > - Pass v4l2_pix_format to unicam_calc_format_size_bpl()
> > > - Simplify unicam_set_default_format()
> > > - Fix default format settings
> > > - Add busy check in unicam_s_fmt_meta()
> > > - Add missing \n at end of format strings
> > > - Fix metadata handling in subdev set_fmt
> > > - Fix locking when starting streaming
> > > - Return buffers from start streaming fails
> > > - Fix format validation for metadata node
> > > - Use video_device_pipeline_{start,stop}() helpers
> > > - Simplify format enumeration
> > > - Drop unset variable
> > > - Update MAINTAINERS entry
> > > - Update to the upstream v4l2_async_nf API
> > > - Update to the latest subdev routing API
> > > - Update to the latest subdev state API
> > > - Move from subdev .init_cfg() to .init_state()
> > > - Update to the latest videobuf2 API
> > > - Fix v4l2_subdev_enable_streams() error check
> > > - Use correct pad for the connected subdev
> > > - Return buffers to vb2 when start streaming fails
> > > - Improve debugging in start streaming handler
> > > - Simplify DMA address management
> > > - Drop comment about bcm2835-camera driver
> > > - Clarify comments that explain min/max sizes
> > > - Pass v4l2_pix_format to unicam_try_fmt()
> > > - Drop unneeded local variables
> > > - Rename image-related constants and functions
> > > - Turn unicam_fmt.metadata_fmt into bool
> > > - Rename unicam_fmt to unicam_format_info
> > > - Rename unicam_format_info variables to fmtinfo
> > > - Rename unicam_node.v_fmt to fmt
> > > - Add metadata formats for RAW10, RAW12 and RAW14
> > > - Make metadata formats line-based
> > > - Validate format on metadata video device
> > > - Add Co-devlopped-by tags
> > >
> > > Changes since v3:
> > >
> > > - Add the vendor prefix for DT name
> > > - Use the reg-names in DT parsing
> > > - Remove MAINTAINERS entry
> > >
> > > Changes since v2:
> > >
> > > - Change code organization
> > > - Remove unused variables
> > > - Correct the fmt_meta functions
> > > - Rewrite the start/stop streaming
> > >   - You can now start the image node alone, but not the metadata one
> > >   - The buffers are allocated per-node
> > >   - only the required stream is started, if the route exists and is
> > >     enabled
> > > - Prefix the macros with UNICAM_ to not have too generic names
> > > - Drop colorspace support
> > >
> > > Changes since v1:
> > >
> > > - Replace the unicam_{info,debug,error} macros with dev_*()
> > > ---
> > >  MAINTAINERS                                   |    1 +
> > >  drivers/media/platform/Kconfig                |    1 +
> > >  drivers/media/platform/Makefile               |    1 +
> > >  drivers/media/platform/broadcom/Kconfig       |   23 +
> > >  drivers/media/platform/broadcom/Makefile      |    3 +
> > >  .../platform/broadcom/bcm2835-unicam-regs.h   |  255 ++
> > >  .../media/platform/broadcom/bcm2835-unicam.c  | 2607 +++++++++++++++++
> > >  7 files changed, 2891 insertions(+)
> > >  create mode 100644 drivers/media/platform/broadcom/Kconfig
> > >  create mode 100644 drivers/media/platform/broadcom/Makefile
> > >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam.c
> > >
> > > diff --git a/MAINTAINERS b/MAINTAINERS
> > > index e50a59654e6e..cc350729f467 100644
> > > --- a/MAINTAINERS
> > > +++ b/MAINTAINERS
> > > @@ -4002,6 +4002,7 @@ M:    Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
> > >  L: linux-media@vger.kernel.org
> > >  S: Maintained
> > >  F: Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> > > +F: drivers/media/platform/bcm2835/
> > >
> > >  BROADCOM BCM47XX MIPS ARCHITECTURE
> > >  M: Hauke Mehrtens <hauke@hauke-m.de>
> > > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > > index 91e54215de3a..2d79bfc68c15 100644
> > > --- a/drivers/media/platform/Kconfig
> > > +++ b/drivers/media/platform/Kconfig
> > > @@ -67,6 +67,7 @@ source "drivers/media/platform/amlogic/Kconfig"
> > >  source "drivers/media/platform/amphion/Kconfig"
> > >  source "drivers/media/platform/aspeed/Kconfig"
> > >  source "drivers/media/platform/atmel/Kconfig"
> > > +source "drivers/media/platform/broadcom/Kconfig"
> > >  source "drivers/media/platform/cadence/Kconfig"
> > >  source "drivers/media/platform/chips-media/Kconfig"
> > >  source "drivers/media/platform/intel/Kconfig"
> > > diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> > > index 3296ec1ebe16..da17301f7439 100644
> > > --- a/drivers/media/platform/Makefile
> > > +++ b/drivers/media/platform/Makefile
> > > @@ -10,6 +10,7 @@ obj-y += amlogic/
> > >  obj-y += amphion/
> > >  obj-y += aspeed/
> > >  obj-y += atmel/
> > > +obj-y += broadcom/
> > >  obj-y += cadence/
> > >  obj-y += chips-media/
> > >  obj-y += intel/
> > > diff --git a/drivers/media/platform/broadcom/Kconfig b/drivers/media/platform/broadcom/Kconfig
> > > new file mode 100644
> > > index 000000000000..cc2c9afcc948
> > > --- /dev/null
> > > +++ b/drivers/media/platform/broadcom/Kconfig
> > > @@ -0,0 +1,23 @@
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +
> > > +config VIDEO_BCM2835_UNICAM
> > > +   tristate "Broadcom BCM283x/BCM271x Unicam video capture driver"
> > > +   depends on ARCH_BCM2835 || COMPILE_TEST
> > > +   depends on PM
> > > +   depends on VIDEO_DEV
> > > +   select MEDIA_CONTROLLER
> > > +   select V4L2_FWNODE
> > > +   select VIDEO_V4L2_SUBDEV_API
> > > +   select VIDEOBUF2_DMA_CONTIG
> > > +   help
> > > +     Say Y here to enable support for the BCM283x/BCM271x CSI-2 receiver.
> > > +     This is a V4L2 driver that controls the CSI-2 receiver directly,
> > > +     independently from the VC4 firmware.
> > > +
> > > +     This driver is mutually exclusive with the use of bcm2835-camera. The
> > > +     firmware will disable all access to the peripheral from within the
> > > +     firmware if it finds a DT node using it, and bcm2835-camera will
> > > +     therefore fail to probe.
> > > +
> > > +     To compile this driver as a module, choose M here. The module will be
> > > +     called bcm2835-unicam.
> > > diff --git a/drivers/media/platform/broadcom/Makefile b/drivers/media/platform/broadcom/Makefile
> > > new file mode 100644
> > > index 000000000000..03d2045aba2e
> > > --- /dev/null
> > > +++ b/drivers/media/platform/broadcom/Makefile
> > > @@ -0,0 +1,3 @@
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +
> > > +obj-$(CONFIG_VIDEO_BCM2835_UNICAM) += bcm2835-unicam.o
> > > diff --git a/drivers/media/platform/broadcom/bcm2835-unicam-regs.h b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > > new file mode 100644
> > > index 000000000000..84775fd2fac5
> > > --- /dev/null
> > > +++ b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > > @@ -0,0 +1,255 @@
> > > +/* SPDX-License-Identifier: GPL-2.0-only */
> > > +
> > > +/*
> > > + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> >
> > Anything up to 2024?
>
> Not really. The registers haven't really changed :-) I'll update the
> copyright in the .c file though.
>
> > > + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > + */
>
> [snip]
>
> > > diff --git a/drivers/media/platform/broadcom/bcm2835-unicam.c b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > > new file mode 100644
> > > index 000000000000..716c89b8a217
> > > --- /dev/null
> > > +++ b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > > @@ -0,0 +1,2607 @@
>
> [snip]
>
> > > +static irqreturn_t unicam_isr(int irq, void *dev)
> > > +{
> > > +   struct unicam_device *unicam = dev;
> > > +   unsigned int lines_done = unicam_get_lines_done(dev);
> > > +   unsigned int sequence = unicam->sequence;
> > > +   unsigned int i;
> > > +   u32 ista, sta;
> > > +   bool fe;
> > > +   u64 ts;
> > > +
> > > +   sta = unicam_reg_read(unicam, UNICAM_STA);
> > > +   /* Write value back to clear the interrupts */
> > > +   unicam_reg_write(unicam, UNICAM_STA, sta);
> > > +
> > > +   ista = unicam_reg_read(unicam, UNICAM_ISTA);
> > > +   /* Write value back to clear the interrupts */
> > > +   unicam_reg_write(unicam, UNICAM_ISTA, ista);
> > > +
> > > +   dev_dbg(unicam->dev, "ISR: ISTA: 0x%X, STA: 0x%X, sequence %d, lines done %d\n",
> > > +           ista, sta, sequence, lines_done);
> > > +
> > > +   if (!(sta & (UNICAM_IS | UNICAM_PI0)))
> > > +           return IRQ_HANDLED;
> > > +
> > > +   /*
> > > +    * Look for either the Frame End interrupt or the Packet Capture status
> > > +    * to signal a frame end.
> > > +    */
> > > +   fe = ista & UNICAM_FEI || sta & UNICAM_PI0;
> > > +
> > > +   /*
> > > +    * We must run the frame end handler first. If we have a valid next_frm
> > > +    * and we get a simultaneout FE + FS interrupt, running the FS handler
> > > +    * first would null out the next_frm ptr and we would have lost the
> > > +    * buffer forever.
> > > +    */
> > > +   if (fe) {
> > > +           /*
> > > +            * Ensure we have swapped buffers already as we can't
> > > +            * stop the peripheral. If no buffer is available, use a
> > > +            * dummy buffer to dump out frames until we get a new buffer
> > > +            * to use.
> > > +            */
> > > +           for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > > +                   if (!unicam->node[i].streaming)
> > > +                           continue;
> > > +
> > > +                   /*
> > > +                    * If cur_frm == next_frm, it means we have not had
> > > +                    * a chance to swap buffers, likely due to having
> > > +                    * multiple interrupts occurring simultaneously (like FE
> > > +                    * + FS + LS). In this case, we cannot signal the buffer
> > > +                    * as complete, as the HW will reuse that buffer.
> > > +                    */
> > > +                   if (unicam->node[i].cur_frm &&
> > > +                       unicam->node[i].cur_frm != unicam->node[i].next_frm)
> > > +                           unicam_process_buffer_complete(&unicam->node[i],
> > > +                                                          sequence);
> > > +                   unicam->node[i].cur_frm = unicam->node[i].next_frm;
> > > +           }
> > > +           unicam->sequence++;
> >
> > Does access to this data need to be serialised somehow.
>
> Given that it's only accessed from the interrupt handler (beside
> start_streaming time, before starting the hardware), I don't think so.
>
> > > +   }
> > > +
> > > +   if (ista & UNICAM_FSI) {
> > > +           /*
> > > +            * Timestamp is to be when the first data byte was captured,
> > > +            * aka frame start.
> > > +            */
> > > +           ts = ktime_get_ns();
> > > +           for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > > +                   if (!unicam->node[i].streaming)
> > > +                           continue;
> > > +
> > > +                   if (unicam->node[i].cur_frm)
> > > +                           unicam->node[i].cur_frm->vb.vb2_buf.timestamp =
> > > +                                                           ts;
> > > +                   else
> > > +                           dev_dbg(unicam->v4l2_dev.dev,
> > > +                                   "ISR: [%d] Dropping frame, buffer not available at FS\n",
> > > +                                   i);
> > > +                   /*
> > > +                    * Set the next frame output to go to a dummy frame
> > > +                    * if we have not managed to obtain another frame
> > > +                    * from the queue.
> > > +                    */
> > > +                   unicam_schedule_dummy_buffer(&unicam->node[i]);
> > > +           }
> > > +
> > > +           unicam_queue_event_sof(unicam);
> > > +   }
> > > +
> > > +   /*
> > > +    * Cannot swap buffer at frame end, there may be a race condition
> > > +    * where the HW does not actually swap it if the new frame has
> > > +    * already started.
> > > +    */
> > > +   if (ista & (UNICAM_FSI | UNICAM_LCI) && !fe) {
> > > +           for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > > +                   if (!unicam->node[i].streaming)
> > > +                           continue;
> > > +
> > > +                   spin_lock(&unicam->node[i].dma_queue_lock);
> > > +                   if (!list_empty(&unicam->node[i].dma_queue) &&
> > > +                       !unicam->node[i].next_frm)
> > > +                           unicam_schedule_next_buffer(&unicam->node[i]);
> > > +                   spin_unlock(&unicam->node[i].dma_queue_lock);
> > > +           }
> > > +   }
> > > +
> > > +   if (unicam_reg_read(unicam, UNICAM_ICTL) & UNICAM_FCM) {
> > > +           /* Switch out of trigger mode if selected */
> > > +           unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> > > +           unicam_reg_write_field(unicam, UNICAM_ICTL, 0, UNICAM_FCM);
> > > +   }
> > > +   return IRQ_HANDLED;
> > > +}
> > > +
> > > +static void unicam_set_packing_config(struct unicam_device *unicam)
> > > +{
> > > +   struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > > +   u32 pack, unpack;
> > > +   u32 val;
> > > +
> > > +   if (node->fmt.fmt.pix.pixelformat == node->fmtinfo->fourcc) {
> > > +           unpack = UNICAM_PUM_NONE;
> > > +           pack = UNICAM_PPM_NONE;
> > > +   } else {
> > > +           switch (node->fmtinfo->depth) {
> > > +           case 8:
> > > +                   unpack = UNICAM_PUM_UNPACK8;
> > > +                   break;
> > > +           case 10:
> > > +                   unpack = UNICAM_PUM_UNPACK10;
> > > +                   break;
> > > +           case 12:
> > > +                   unpack = UNICAM_PUM_UNPACK12;
> > > +                   break;
> > > +           case 14:
> > > +                   unpack = UNICAM_PUM_UNPACK14;
> > > +                   break;
> > > +           case 16:
> > > +                   unpack = UNICAM_PUM_UNPACK16;
> > > +                   break;
> > > +           default:
> > > +                   unpack = UNICAM_PUM_NONE;
> > > +                   break;
> > > +           }
> > > +
> > > +           /* Repacking is always to 16bpp */
> > > +           pack = UNICAM_PPM_PACK16;
> >
> > Also 8-bit data?
>
> Not that I know of. The 8-bit entries in unicam_image_formats have no
> .unpacked_fourcc field, so the condition in the if above will always be
> true for those as they can only be selected by setting the pixel format
> to fmtinfo->fourcc.

Correct.
The unpacking / packing doesn't bit shift, hence 10P can be unpacked
to 10, 12P to 12, and 14P to 14, but there is no 8bit unpacked into
the LSBs of 16 bit words defined as a V4L2 format.

You can configure packing to produce 10P, 12P, etc, but without the
bitshift it has no real use.

MSBs get dropped if you try packing to a smaller bit depth than the
source, so packing to 8 bit from any other depth results in unusable
output.

If being really critical, then there is no point in having the case 8
and case 16 in this switch as they will never be selected.

  Dave

> > > +   }
<snip>

Re: [PATCH v6 09/15] media: bcm2835-unicam: Add support for CCP2/CSI2 camera interface

[Laurent Pinchart]

Hi Dave,

On Tue, Mar 26, 2024 at 03:23:42PM +0000, Dave Stevenson wrote:
> On Tue, 26 Mar 2024 at 01:37, Laurent Pinchart wrote:
> > On Mon, Mar 25, 2024 at 06:36:49PM +0000, Sakari Ailus wrote:
> > > On Fri, Mar 01, 2024 at 11:32:24PM +0200, Laurent Pinchart wrote:
> > > > From: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > >
> > > > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > > > It is represented as two video device nodes: unicam-image and
> > > > unicam-embedded which are connected to an internal subdev (named
> > > > unicam-subdev) in order to manage streams routing.
> > > >
> > > > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > Co-developed-by: Naushir Patuck <naush@raspberrypi.com>
> > > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > > > Co-developed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > > > Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > > > Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > >
> > > Thanks for submitting this, it's the cleanest and neatest Unicom driver
> >
> > Unicam, or if you insist Unicorn, but not Unicom :-)
> >
> > > I've ever seen!
> > >
> > > Some mostly unimportant comments below, however the bus-type issue needs to
> > > be addressed.
> > >
> > > > ---
> > > > Changes since v5:
> > > >
> > > > - Move to drivers/media/platform/broadcom/
> > > > - Port to the upstream V4L2 streams API
> > > > - Rebase on latest metadata API proposal
> > > > - Add missing error message
> > > > - Drop unneeded documentation block for unicam_isr()
> > > > - Drop unneeded dev_dbg() and dev_err() messages
> > > > - Drop unneeded streams_mask and fmt checks
> > > > - Drop unused unicam_sd_pad_is_sink()
> > > > - Drop unneeded includes
> > > > - Drop v4l2_ctrl_subscribe_event() call
> > > > - Use pm_runtime_resume_and_get()
> > > > - Indentation and line wrap fixes
> > > > - Let the framework set bus_info
> > > > - Use v4l2_fwnode_endpoint_parse()
> > > > - Fix media device cleanup
> > > > - Drop lane reordering checks
> > > > - Fix subdev state locking
> > > > - Drop extra debug messages
> > > > - Move clock handling to runtime PM handlers
> > > > - Reorder functions
> > > > - Rename init functions for more clarity
> > > > - Initialize runtime PM earlier
> > > > - Clarify error messages
> > > > - Simplify subdev init with local variable
> > > > - Fix subdev cleanup
> > > > - Fix typos and indentation
> > > > - Don't initialize local variables needlessly
> > > > - Simplify num lanes check
> > > > - Fix metadata handling in subdev set_fmt
> > > > - Drop manual fallback to .s_stream()
> > > > - Pass v4l2_pix_format to unicam_calc_format_size_bpl()
> > > > - Simplify unicam_set_default_format()
> > > > - Fix default format settings
> > > > - Add busy check in unicam_s_fmt_meta()
> > > > - Add missing \n at end of format strings
> > > > - Fix metadata handling in subdev set_fmt
> > > > - Fix locking when starting streaming
> > > > - Return buffers from start streaming fails
> > > > - Fix format validation for metadata node
> > > > - Use video_device_pipeline_{start,stop}() helpers
> > > > - Simplify format enumeration
> > > > - Drop unset variable
> > > > - Update MAINTAINERS entry
> > > > - Update to the upstream v4l2_async_nf API
> > > > - Update to the latest subdev routing API
> > > > - Update to the latest subdev state API
> > > > - Move from subdev .init_cfg() to .init_state()
> > > > - Update to the latest videobuf2 API
> > > > - Fix v4l2_subdev_enable_streams() error check
> > > > - Use correct pad for the connected subdev
> > > > - Return buffers to vb2 when start streaming fails
> > > > - Improve debugging in start streaming handler
> > > > - Simplify DMA address management
> > > > - Drop comment about bcm2835-camera driver
> > > > - Clarify comments that explain min/max sizes
> > > > - Pass v4l2_pix_format to unicam_try_fmt()
> > > > - Drop unneeded local variables
> > > > - Rename image-related constants and functions
> > > > - Turn unicam_fmt.metadata_fmt into bool
> > > > - Rename unicam_fmt to unicam_format_info
> > > > - Rename unicam_format_info variables to fmtinfo
> > > > - Rename unicam_node.v_fmt to fmt
> > > > - Add metadata formats for RAW10, RAW12 and RAW14
> > > > - Make metadata formats line-based
> > > > - Validate format on metadata video device
> > > > - Add Co-devlopped-by tags
> > > >
> > > > Changes since v3:
> > > >
> > > > - Add the vendor prefix for DT name
> > > > - Use the reg-names in DT parsing
> > > > - Remove MAINTAINERS entry
> > > >
> > > > Changes since v2:
> > > >
> > > > - Change code organization
> > > > - Remove unused variables
> > > > - Correct the fmt_meta functions
> > > > - Rewrite the start/stop streaming
> > > >   - You can now start the image node alone, but not the metadata one
> > > >   - The buffers are allocated per-node
> > > >   - only the required stream is started, if the route exists and is
> > > >     enabled
> > > > - Prefix the macros with UNICAM_ to not have too generic names
> > > > - Drop colorspace support
> > > >
> > > > Changes since v1:
> > > >
> > > > - Replace the unicam_{info,debug,error} macros with dev_*()
> > > > ---
> > > >  MAINTAINERS                                   |    1 +
> > > >  drivers/media/platform/Kconfig                |    1 +
> > > >  drivers/media/platform/Makefile               |    1 +
> > > >  drivers/media/platform/broadcom/Kconfig       |   23 +
> > > >  drivers/media/platform/broadcom/Makefile      |    3 +
> > > >  .../platform/broadcom/bcm2835-unicam-regs.h   |  255 ++
> > > >  .../media/platform/broadcom/bcm2835-unicam.c  | 2607 +++++++++++++++++
> > > >  7 files changed, 2891 insertions(+)
> > > >  create mode 100644 drivers/media/platform/broadcom/Kconfig
> > > >  create mode 100644 drivers/media/platform/broadcom/Makefile
> > > >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > > >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam.c
> > > >
> > > > diff --git a/MAINTAINERS b/MAINTAINERS
> > > > index e50a59654e6e..cc350729f467 100644
> > > > --- a/MAINTAINERS
> > > > +++ b/MAINTAINERS
> > > > @@ -4002,6 +4002,7 @@ M:    Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
> > > >  L: linux-media@vger.kernel.org
> > > >  S: Maintained
> > > >  F: Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> > > > +F: drivers/media/platform/bcm2835/
> > > >
> > > >  BROADCOM BCM47XX MIPS ARCHITECTURE
> > > >  M: Hauke Mehrtens <hauke@hauke-m.de>
> > > > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > > > index 91e54215de3a..2d79bfc68c15 100644
> > > > --- a/drivers/media/platform/Kconfig
> > > > +++ b/drivers/media/platform/Kconfig
> > > > @@ -67,6 +67,7 @@ source "drivers/media/platform/amlogic/Kconfig"
> > > >  source "drivers/media/platform/amphion/Kconfig"
> > > >  source "drivers/media/platform/aspeed/Kconfig"
> > > >  source "drivers/media/platform/atmel/Kconfig"
> > > > +source "drivers/media/platform/broadcom/Kconfig"
> > > >  source "drivers/media/platform/cadence/Kconfig"
> > > >  source "drivers/media/platform/chips-media/Kconfig"
> > > >  source "drivers/media/platform/intel/Kconfig"
> > > > diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> > > > index 3296ec1ebe16..da17301f7439 100644
> > > > --- a/drivers/media/platform/Makefile
> > > > +++ b/drivers/media/platform/Makefile
> > > > @@ -10,6 +10,7 @@ obj-y += amlogic/
> > > >  obj-y += amphion/
> > > >  obj-y += aspeed/
> > > >  obj-y += atmel/
> > > > +obj-y += broadcom/
> > > >  obj-y += cadence/
> > > >  obj-y += chips-media/
> > > >  obj-y += intel/
> > > > diff --git a/drivers/media/platform/broadcom/Kconfig b/drivers/media/platform/broadcom/Kconfig
> > > > new file mode 100644
> > > > index 000000000000..cc2c9afcc948
> > > > --- /dev/null
> > > > +++ b/drivers/media/platform/broadcom/Kconfig
> > > > @@ -0,0 +1,23 @@
> > > > +# SPDX-License-Identifier: GPL-2.0
> > > > +
> > > > +config VIDEO_BCM2835_UNICAM
> > > > +   tristate "Broadcom BCM283x/BCM271x Unicam video capture driver"
> > > > +   depends on ARCH_BCM2835 || COMPILE_TEST
> > > > +   depends on PM
> > > > +   depends on VIDEO_DEV
> > > > +   select MEDIA_CONTROLLER
> > > > +   select V4L2_FWNODE
> > > > +   select VIDEO_V4L2_SUBDEV_API
> > > > +   select VIDEOBUF2_DMA_CONTIG
> > > > +   help
> > > > +     Say Y here to enable support for the BCM283x/BCM271x CSI-2 receiver.
> > > > +     This is a V4L2 driver that controls the CSI-2 receiver directly,
> > > > +     independently from the VC4 firmware.
> > > > +
> > > > +     This driver is mutually exclusive with the use of bcm2835-camera. The
> > > > +     firmware will disable all access to the peripheral from within the
> > > > +     firmware if it finds a DT node using it, and bcm2835-camera will
> > > > +     therefore fail to probe.
> > > > +
> > > > +     To compile this driver as a module, choose M here. The module will be
> > > > +     called bcm2835-unicam.
> > > > diff --git a/drivers/media/platform/broadcom/Makefile b/drivers/media/platform/broadcom/Makefile
> > > > new file mode 100644
> > > > index 000000000000..03d2045aba2e
> > > > --- /dev/null
> > > > +++ b/drivers/media/platform/broadcom/Makefile
> > > > @@ -0,0 +1,3 @@
> > > > +# SPDX-License-Identifier: GPL-2.0
> > > > +
> > > > +obj-$(CONFIG_VIDEO_BCM2835_UNICAM) += bcm2835-unicam.o
> > > > diff --git a/drivers/media/platform/broadcom/bcm2835-unicam-regs.h b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > > > new file mode 100644
> > > > index 000000000000..84775fd2fac5
> > > > --- /dev/null
> > > > +++ b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > > > @@ -0,0 +1,255 @@
> > > > +/* SPDX-License-Identifier: GPL-2.0-only */
> > > > +
> > > > +/*
> > > > + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> > >
> > > Anything up to 2024?
> >
> > Not really. The registers haven't really changed :-) I'll update the
> > copyright in the .c file though.
> >
> > > > + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > + */
> >
> > [snip]
> >
> > > > diff --git a/drivers/media/platform/broadcom/bcm2835-unicam.c b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > > > new file mode 100644
> > > > index 000000000000..716c89b8a217
> > > > --- /dev/null
> > > > +++ b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > > > @@ -0,0 +1,2607 @@
> >
> > [snip]
> >
> > > > +static irqreturn_t unicam_isr(int irq, void *dev)
> > > > +{
> > > > +   struct unicam_device *unicam = dev;
> > > > +   unsigned int lines_done = unicam_get_lines_done(dev);
> > > > +   unsigned int sequence = unicam->sequence;
> > > > +   unsigned int i;
> > > > +   u32 ista, sta;
> > > > +   bool fe;
> > > > +   u64 ts;
> > > > +
> > > > +   sta = unicam_reg_read(unicam, UNICAM_STA);
> > > > +   /* Write value back to clear the interrupts */
> > > > +   unicam_reg_write(unicam, UNICAM_STA, sta);
> > > > +
> > > > +   ista = unicam_reg_read(unicam, UNICAM_ISTA);
> > > > +   /* Write value back to clear the interrupts */
> > > > +   unicam_reg_write(unicam, UNICAM_ISTA, ista);
> > > > +
> > > > +   dev_dbg(unicam->dev, "ISR: ISTA: 0x%X, STA: 0x%X, sequence %d, lines done %d\n",
> > > > +           ista, sta, sequence, lines_done);
> > > > +
> > > > +   if (!(sta & (UNICAM_IS | UNICAM_PI0)))
> > > > +           return IRQ_HANDLED;
> > > > +
> > > > +   /*
> > > > +    * Look for either the Frame End interrupt or the Packet Capture status
> > > > +    * to signal a frame end.
> > > > +    */
> > > > +   fe = ista & UNICAM_FEI || sta & UNICAM_PI0;
> > > > +
> > > > +   /*
> > > > +    * We must run the frame end handler first. If we have a valid next_frm
> > > > +    * and we get a simultaneout FE + FS interrupt, running the FS handler
> > > > +    * first would null out the next_frm ptr and we would have lost the
> > > > +    * buffer forever.
> > > > +    */
> > > > +   if (fe) {
> > > > +           /*
> > > > +            * Ensure we have swapped buffers already as we can't
> > > > +            * stop the peripheral. If no buffer is available, use a
> > > > +            * dummy buffer to dump out frames until we get a new buffer
> > > > +            * to use.
> > > > +            */
> > > > +           for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > > > +                   if (!unicam->node[i].streaming)
> > > > +                           continue;
> > > > +
> > > > +                   /*
> > > > +                    * If cur_frm == next_frm, it means we have not had
> > > > +                    * a chance to swap buffers, likely due to having
> > > > +                    * multiple interrupts occurring simultaneously (like FE
> > > > +                    * + FS + LS). In this case, we cannot signal the buffer
> > > > +                    * as complete, as the HW will reuse that buffer.
> > > > +                    */
> > > > +                   if (unicam->node[i].cur_frm &&
> > > > +                       unicam->node[i].cur_frm != unicam->node[i].next_frm)
> > > > +                           unicam_process_buffer_complete(&unicam->node[i],
> > > > +                                                          sequence);
> > > > +                   unicam->node[i].cur_frm = unicam->node[i].next_frm;
> > > > +           }
> > > > +           unicam->sequence++;
> > >
> > > Does access to this data need to be serialised somehow.
> >
> > Given that it's only accessed from the interrupt handler (beside
> > start_streaming time, before starting the hardware), I don't think so.
> >
> > > > +   }
> > > > +
> > > > +   if (ista & UNICAM_FSI) {
> > > > +           /*
> > > > +            * Timestamp is to be when the first data byte was captured,
> > > > +            * aka frame start.
> > > > +            */
> > > > +           ts = ktime_get_ns();
> > > > +           for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > > > +                   if (!unicam->node[i].streaming)
> > > > +                           continue;
> > > > +
> > > > +                   if (unicam->node[i].cur_frm)
> > > > +                           unicam->node[i].cur_frm->vb.vb2_buf.timestamp =
> > > > +                                                           ts;
> > > > +                   else
> > > > +                           dev_dbg(unicam->v4l2_dev.dev,
> > > > +                                   "ISR: [%d] Dropping frame, buffer not available at FS\n",
> > > > +                                   i);
> > > > +                   /*
> > > > +                    * Set the next frame output to go to a dummy frame
> > > > +                    * if we have not managed to obtain another frame
> > > > +                    * from the queue.
> > > > +                    */
> > > > +                   unicam_schedule_dummy_buffer(&unicam->node[i]);
> > > > +           }
> > > > +
> > > > +           unicam_queue_event_sof(unicam);
> > > > +   }
> > > > +
> > > > +   /*
> > > > +    * Cannot swap buffer at frame end, there may be a race condition
> > > > +    * where the HW does not actually swap it if the new frame has
> > > > +    * already started.
> > > > +    */
> > > > +   if (ista & (UNICAM_FSI | UNICAM_LCI) && !fe) {
> > > > +           for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > > > +                   if (!unicam->node[i].streaming)
> > > > +                           continue;
> > > > +
> > > > +                   spin_lock(&unicam->node[i].dma_queue_lock);
> > > > +                   if (!list_empty(&unicam->node[i].dma_queue) &&
> > > > +                       !unicam->node[i].next_frm)
> > > > +                           unicam_schedule_next_buffer(&unicam->node[i]);
> > > > +                   spin_unlock(&unicam->node[i].dma_queue_lock);
> > > > +           }
> > > > +   }
> > > > +
> > > > +   if (unicam_reg_read(unicam, UNICAM_ICTL) & UNICAM_FCM) {
> > > > +           /* Switch out of trigger mode if selected */
> > > > +           unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> > > > +           unicam_reg_write_field(unicam, UNICAM_ICTL, 0, UNICAM_FCM);
> > > > +   }
> > > > +   return IRQ_HANDLED;
> > > > +}
> > > > +
> > > > +static void unicam_set_packing_config(struct unicam_device *unicam)
> > > > +{
> > > > +   struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > > > +   u32 pack, unpack;
> > > > +   u32 val;
> > > > +
> > > > +   if (node->fmt.fmt.pix.pixelformat == node->fmtinfo->fourcc) {
> > > > +           unpack = UNICAM_PUM_NONE;
> > > > +           pack = UNICAM_PPM_NONE;
> > > > +   } else {
> > > > +           switch (node->fmtinfo->depth) {
> > > > +           case 8:
> > > > +                   unpack = UNICAM_PUM_UNPACK8;
> > > > +                   break;
> > > > +           case 10:
> > > > +                   unpack = UNICAM_PUM_UNPACK10;
> > > > +                   break;
> > > > +           case 12:
> > > > +                   unpack = UNICAM_PUM_UNPACK12;
> > > > +                   break;
> > > > +           case 14:
> > > > +                   unpack = UNICAM_PUM_UNPACK14;
> > > > +                   break;
> > > > +           case 16:
> > > > +                   unpack = UNICAM_PUM_UNPACK16;
> > > > +                   break;
> > > > +           default:
> > > > +                   unpack = UNICAM_PUM_NONE;
> > > > +                   break;
> > > > +           }
> > > > +
> > > > +           /* Repacking is always to 16bpp */
> > > > +           pack = UNICAM_PPM_PACK16;
> > >
> > > Also 8-bit data?
> >
> > Not that I know of. The 8-bit entries in unicam_image_formats have no
> > .unpacked_fourcc field, so the condition in the if above will always be
> > true for those as they can only be selected by setting the pixel format
> > to fmtinfo->fourcc.
> 
> Correct.
> The unpacking / packing doesn't bit shift, hence 10P can be unpacked
> to 10, 12P to 12, and 14P to 14, but there is no 8bit unpacked into
> the LSBs of 16 bit words defined as a V4L2 format.
> 
> You can configure packing to produce 10P, 12P, etc, but without the
> bitshift it has no real use.
> 
> MSBs get dropped if you try packing to a smaller bit depth than the
> source, so packing to 8 bit from any other depth results in unusable
> output.
> 
> If being really critical, then there is no point in having the case 8
> and case 16 in this switch as they will never be selected.

As we have space to spare in the unicam_format_info structure due to
padding, I'll move the unpack value there and drop the switch.

> > > > +   }
>
> <snip>

-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 09/15] media: bcm2835-unicam: Add support for CCP2/CSI2 camera interface

[Dave Stevenson]

On Tue, 26 Mar 2024 at 16:42, Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> Hi Dave,
>
> On Tue, Mar 26, 2024 at 03:23:42PM +0000, Dave Stevenson wrote:
> > On Tue, 26 Mar 2024 at 01:37, Laurent Pinchart wrote:
> > > On Mon, Mar 25, 2024 at 06:36:49PM +0000, Sakari Ailus wrote:
> > > > On Fri, Mar 01, 2024 at 11:32:24PM +0200, Laurent Pinchart wrote:
> > > > > From: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > >
> > > > > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > > > > It is represented as two video device nodes: unicam-image and
> > > > > unicam-embedded which are connected to an internal subdev (named
> > > > > unicam-subdev) in order to manage streams routing.
> > > > >
> > > > > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > > Co-developed-by: Naushir Patuck <naush@raspberrypi.com>
> > > > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > > > > Co-developed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > > > > Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > > > > Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > >
> > > > Thanks for submitting this, it's the cleanest and neatest Unicom driver
> > >
> > > Unicam, or if you insist Unicorn, but not Unicom :-)
> > >
> > > > I've ever seen!
> > > >
> > > > Some mostly unimportant comments below, however the bus-type issue needs to
> > > > be addressed.
> > > >
> > > > > ---
> > > > > Changes since v5:
> > > > >
> > > > > - Move to drivers/media/platform/broadcom/
> > > > > - Port to the upstream V4L2 streams API
> > > > > - Rebase on latest metadata API proposal
> > > > > - Add missing error message
> > > > > - Drop unneeded documentation block for unicam_isr()
> > > > > - Drop unneeded dev_dbg() and dev_err() messages
> > > > > - Drop unneeded streams_mask and fmt checks
> > > > > - Drop unused unicam_sd_pad_is_sink()
> > > > > - Drop unneeded includes
> > > > > - Drop v4l2_ctrl_subscribe_event() call
> > > > > - Use pm_runtime_resume_and_get()
> > > > > - Indentation and line wrap fixes
> > > > > - Let the framework set bus_info
> > > > > - Use v4l2_fwnode_endpoint_parse()
> > > > > - Fix media device cleanup
> > > > > - Drop lane reordering checks
> > > > > - Fix subdev state locking
> > > > > - Drop extra debug messages
> > > > > - Move clock handling to runtime PM handlers
> > > > > - Reorder functions
> > > > > - Rename init functions for more clarity
> > > > > - Initialize runtime PM earlier
> > > > > - Clarify error messages
> > > > > - Simplify subdev init with local variable
> > > > > - Fix subdev cleanup
> > > > > - Fix typos and indentation
> > > > > - Don't initialize local variables needlessly
> > > > > - Simplify num lanes check
> > > > > - Fix metadata handling in subdev set_fmt
> > > > > - Drop manual fallback to .s_stream()
> > > > > - Pass v4l2_pix_format to unicam_calc_format_size_bpl()
> > > > > - Simplify unicam_set_default_format()
> > > > > - Fix default format settings
> > > > > - Add busy check in unicam_s_fmt_meta()
> > > > > - Add missing \n at end of format strings
> > > > > - Fix metadata handling in subdev set_fmt
> > > > > - Fix locking when starting streaming
> > > > > - Return buffers from start streaming fails
> > > > > - Fix format validation for metadata node
> > > > > - Use video_device_pipeline_{start,stop}() helpers
> > > > > - Simplify format enumeration
> > > > > - Drop unset variable
> > > > > - Update MAINTAINERS entry
> > > > > - Update to the upstream v4l2_async_nf API
> > > > > - Update to the latest subdev routing API
> > > > > - Update to the latest subdev state API
> > > > > - Move from subdev .init_cfg() to .init_state()
> > > > > - Update to the latest videobuf2 API
> > > > > - Fix v4l2_subdev_enable_streams() error check
> > > > > - Use correct pad for the connected subdev
> > > > > - Return buffers to vb2 when start streaming fails
> > > > > - Improve debugging in start streaming handler
> > > > > - Simplify DMA address management
> > > > > - Drop comment about bcm2835-camera driver
> > > > > - Clarify comments that explain min/max sizes
> > > > > - Pass v4l2_pix_format to unicam_try_fmt()
> > > > > - Drop unneeded local variables
> > > > > - Rename image-related constants and functions
> > > > > - Turn unicam_fmt.metadata_fmt into bool
> > > > > - Rename unicam_fmt to unicam_format_info
> > > > > - Rename unicam_format_info variables to fmtinfo
> > > > > - Rename unicam_node.v_fmt to fmt
> > > > > - Add metadata formats for RAW10, RAW12 and RAW14
> > > > > - Make metadata formats line-based
> > > > > - Validate format on metadata video device
> > > > > - Add Co-devlopped-by tags
> > > > >
> > > > > Changes since v3:
> > > > >
> > > > > - Add the vendor prefix for DT name
> > > > > - Use the reg-names in DT parsing
> > > > > - Remove MAINTAINERS entry
> > > > >
> > > > > Changes since v2:
> > > > >
> > > > > - Change code organization
> > > > > - Remove unused variables
> > > > > - Correct the fmt_meta functions
> > > > > - Rewrite the start/stop streaming
> > > > >   - You can now start the image node alone, but not the metadata one
> > > > >   - The buffers are allocated per-node
> > > > >   - only the required stream is started, if the route exists and is
> > > > >     enabled
> > > > > - Prefix the macros with UNICAM_ to not have too generic names
> > > > > - Drop colorspace support
> > > > >
> > > > > Changes since v1:
> > > > >
> > > > > - Replace the unicam_{info,debug,error} macros with dev_*()
> > > > > ---
> > > > >  MAINTAINERS                                   |    1 +
> > > > >  drivers/media/platform/Kconfig                |    1 +
> > > > >  drivers/media/platform/Makefile               |    1 +
> > > > >  drivers/media/platform/broadcom/Kconfig       |   23 +
> > > > >  drivers/media/platform/broadcom/Makefile      |    3 +
> > > > >  .../platform/broadcom/bcm2835-unicam-regs.h   |  255 ++
> > > > >  .../media/platform/broadcom/bcm2835-unicam.c  | 2607 +++++++++++++++++
> > > > >  7 files changed, 2891 insertions(+)
> > > > >  create mode 100644 drivers/media/platform/broadcom/Kconfig
> > > > >  create mode 100644 drivers/media/platform/broadcom/Makefile
> > > > >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > > > >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam.c
> > > > >
> > > > > diff --git a/MAINTAINERS b/MAINTAINERS
> > > > > index e50a59654e6e..cc350729f467 100644
> > > > > --- a/MAINTAINERS
> > > > > +++ b/MAINTAINERS
> > > > > @@ -4002,6 +4002,7 @@ M:    Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
> > > > >  L: linux-media@vger.kernel.org
> > > > >  S: Maintained
> > > > >  F: Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> > > > > +F: drivers/media/platform/bcm2835/
> > > > >
> > > > >  BROADCOM BCM47XX MIPS ARCHITECTURE
> > > > >  M: Hauke Mehrtens <hauke@hauke-m.de>
> > > > > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > > > > index 91e54215de3a..2d79bfc68c15 100644
> > > > > --- a/drivers/media/platform/Kconfig
> > > > > +++ b/drivers/media/platform/Kconfig
> > > > > @@ -67,6 +67,7 @@ source "drivers/media/platform/amlogic/Kconfig"
> > > > >  source "drivers/media/platform/amphion/Kconfig"
> > > > >  source "drivers/media/platform/aspeed/Kconfig"
> > > > >  source "drivers/media/platform/atmel/Kconfig"
> > > > > +source "drivers/media/platform/broadcom/Kconfig"
> > > > >  source "drivers/media/platform/cadence/Kconfig"
> > > > >  source "drivers/media/platform/chips-media/Kconfig"
> > > > >  source "drivers/media/platform/intel/Kconfig"
> > > > > diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> > > > > index 3296ec1ebe16..da17301f7439 100644
> > > > > --- a/drivers/media/platform/Makefile
> > > > > +++ b/drivers/media/platform/Makefile
> > > > > @@ -10,6 +10,7 @@ obj-y += amlogic/
> > > > >  obj-y += amphion/
> > > > >  obj-y += aspeed/
> > > > >  obj-y += atmel/
> > > > > +obj-y += broadcom/
> > > > >  obj-y += cadence/
> > > > >  obj-y += chips-media/
> > > > >  obj-y += intel/
> > > > > diff --git a/drivers/media/platform/broadcom/Kconfig b/drivers/media/platform/broadcom/Kconfig
> > > > > new file mode 100644
> > > > > index 000000000000..cc2c9afcc948
> > > > > --- /dev/null
> > > > > +++ b/drivers/media/platform/broadcom/Kconfig
> > > > > @@ -0,0 +1,23 @@
> > > > > +# SPDX-License-Identifier: GPL-2.0
> > > > > +
> > > > > +config VIDEO_BCM2835_UNICAM
> > > > > +   tristate "Broadcom BCM283x/BCM271x Unicam video capture driver"
> > > > > +   depends on ARCH_BCM2835 || COMPILE_TEST
> > > > > +   depends on PM
> > > > > +   depends on VIDEO_DEV
> > > > > +   select MEDIA_CONTROLLER
> > > > > +   select V4L2_FWNODE
> > > > > +   select VIDEO_V4L2_SUBDEV_API
> > > > > +   select VIDEOBUF2_DMA_CONTIG
> > > > > +   help
> > > > > +     Say Y here to enable support for the BCM283x/BCM271x CSI-2 receiver.
> > > > > +     This is a V4L2 driver that controls the CSI-2 receiver directly,
> > > > > +     independently from the VC4 firmware.
> > > > > +
> > > > > +     This driver is mutually exclusive with the use of bcm2835-camera. The
> > > > > +     firmware will disable all access to the peripheral from within the
> > > > > +     firmware if it finds a DT node using it, and bcm2835-camera will
> > > > > +     therefore fail to probe.
> > > > > +
> > > > > +     To compile this driver as a module, choose M here. The module will be
> > > > > +     called bcm2835-unicam.
> > > > > diff --git a/drivers/media/platform/broadcom/Makefile b/drivers/media/platform/broadcom/Makefile
> > > > > new file mode 100644
> > > > > index 000000000000..03d2045aba2e
> > > > > --- /dev/null
> > > > > +++ b/drivers/media/platform/broadcom/Makefile
> > > > > @@ -0,0 +1,3 @@
> > > > > +# SPDX-License-Identifier: GPL-2.0
> > > > > +
> > > > > +obj-$(CONFIG_VIDEO_BCM2835_UNICAM) += bcm2835-unicam.o
> > > > > diff --git a/drivers/media/platform/broadcom/bcm2835-unicam-regs.h b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > > > > new file mode 100644
> > > > > index 000000000000..84775fd2fac5
> > > > > --- /dev/null
> > > > > +++ b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > > > > @@ -0,0 +1,255 @@
> > > > > +/* SPDX-License-Identifier: GPL-2.0-only */
> > > > > +
> > > > > +/*
> > > > > + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> > > >
> > > > Anything up to 2024?
> > >
> > > Not really. The registers haven't really changed :-) I'll update the
> > > copyright in the .c file though.
> > >
> > > > > + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > > + */
> > >
> > > [snip]
> > >
> > > > > diff --git a/drivers/media/platform/broadcom/bcm2835-unicam.c b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > > > > new file mode 100644
> > > > > index 000000000000..716c89b8a217
> > > > > --- /dev/null
> > > > > +++ b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > > > > @@ -0,0 +1,2607 @@
> > >
> > > [snip]
> > >
> > > > > +static irqreturn_t unicam_isr(int irq, void *dev)
> > > > > +{
> > > > > +   struct unicam_device *unicam = dev;
> > > > > +   unsigned int lines_done = unicam_get_lines_done(dev);
> > > > > +   unsigned int sequence = unicam->sequence;
> > > > > +   unsigned int i;
> > > > > +   u32 ista, sta;
> > > > > +   bool fe;
> > > > > +   u64 ts;
> > > > > +
> > > > > +   sta = unicam_reg_read(unicam, UNICAM_STA);
> > > > > +   /* Write value back to clear the interrupts */
> > > > > +   unicam_reg_write(unicam, UNICAM_STA, sta);
> > > > > +
> > > > > +   ista = unicam_reg_read(unicam, UNICAM_ISTA);
> > > > > +   /* Write value back to clear the interrupts */
> > > > > +   unicam_reg_write(unicam, UNICAM_ISTA, ista);
> > > > > +
> > > > > +   dev_dbg(unicam->dev, "ISR: ISTA: 0x%X, STA: 0x%X, sequence %d, lines done %d\n",
> > > > > +           ista, sta, sequence, lines_done);
> > > > > +
> > > > > +   if (!(sta & (UNICAM_IS | UNICAM_PI0)))
> > > > > +           return IRQ_HANDLED;
> > > > > +
> > > > > +   /*
> > > > > +    * Look for either the Frame End interrupt or the Packet Capture status
> > > > > +    * to signal a frame end.
> > > > > +    */
> > > > > +   fe = ista & UNICAM_FEI || sta & UNICAM_PI0;
> > > > > +
> > > > > +   /*
> > > > > +    * We must run the frame end handler first. If we have a valid next_frm
> > > > > +    * and we get a simultaneout FE + FS interrupt, running the FS handler
> > > > > +    * first would null out the next_frm ptr and we would have lost the
> > > > > +    * buffer forever.
> > > > > +    */
> > > > > +   if (fe) {
> > > > > +           /*
> > > > > +            * Ensure we have swapped buffers already as we can't
> > > > > +            * stop the peripheral. If no buffer is available, use a
> > > > > +            * dummy buffer to dump out frames until we get a new buffer
> > > > > +            * to use.
> > > > > +            */
> > > > > +           for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > > > > +                   if (!unicam->node[i].streaming)
> > > > > +                           continue;
> > > > > +
> > > > > +                   /*
> > > > > +                    * If cur_frm == next_frm, it means we have not had
> > > > > +                    * a chance to swap buffers, likely due to having
> > > > > +                    * multiple interrupts occurring simultaneously (like FE
> > > > > +                    * + FS + LS). In this case, we cannot signal the buffer
> > > > > +                    * as complete, as the HW will reuse that buffer.
> > > > > +                    */
> > > > > +                   if (unicam->node[i].cur_frm &&
> > > > > +                       unicam->node[i].cur_frm != unicam->node[i].next_frm)
> > > > > +                           unicam_process_buffer_complete(&unicam->node[i],
> > > > > +                                                          sequence);
> > > > > +                   unicam->node[i].cur_frm = unicam->node[i].next_frm;
> > > > > +           }
> > > > > +           unicam->sequence++;
> > > >
> > > > Does access to this data need to be serialised somehow.
> > >
> > > Given that it's only accessed from the interrupt handler (beside
> > > start_streaming time, before starting the hardware), I don't think so.
> > >
> > > > > +   }
> > > > > +
> > > > > +   if (ista & UNICAM_FSI) {
> > > > > +           /*
> > > > > +            * Timestamp is to be when the first data byte was captured,
> > > > > +            * aka frame start.
> > > > > +            */
> > > > > +           ts = ktime_get_ns();
> > > > > +           for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > > > > +                   if (!unicam->node[i].streaming)
> > > > > +                           continue;
> > > > > +
> > > > > +                   if (unicam->node[i].cur_frm)
> > > > > +                           unicam->node[i].cur_frm->vb.vb2_buf.timestamp =
> > > > > +                                                           ts;
> > > > > +                   else
> > > > > +                           dev_dbg(unicam->v4l2_dev.dev,
> > > > > +                                   "ISR: [%d] Dropping frame, buffer not available at FS\n",
> > > > > +                                   i);
> > > > > +                   /*
> > > > > +                    * Set the next frame output to go to a dummy frame
> > > > > +                    * if we have not managed to obtain another frame
> > > > > +                    * from the queue.
> > > > > +                    */
> > > > > +                   unicam_schedule_dummy_buffer(&unicam->node[i]);
> > > > > +           }
> > > > > +
> > > > > +           unicam_queue_event_sof(unicam);
> > > > > +   }
> > > > > +
> > > > > +   /*
> > > > > +    * Cannot swap buffer at frame end, there may be a race condition
> > > > > +    * where the HW does not actually swap it if the new frame has
> > > > > +    * already started.
> > > > > +    */
> > > > > +   if (ista & (UNICAM_FSI | UNICAM_LCI) && !fe) {
> > > > > +           for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > > > > +                   if (!unicam->node[i].streaming)
> > > > > +                           continue;
> > > > > +
> > > > > +                   spin_lock(&unicam->node[i].dma_queue_lock);
> > > > > +                   if (!list_empty(&unicam->node[i].dma_queue) &&
> > > > > +                       !unicam->node[i].next_frm)
> > > > > +                           unicam_schedule_next_buffer(&unicam->node[i]);
> > > > > +                   spin_unlock(&unicam->node[i].dma_queue_lock);
> > > > > +           }
> > > > > +   }
> > > > > +
> > > > > +   if (unicam_reg_read(unicam, UNICAM_ICTL) & UNICAM_FCM) {
> > > > > +           /* Switch out of trigger mode if selected */
> > > > > +           unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> > > > > +           unicam_reg_write_field(unicam, UNICAM_ICTL, 0, UNICAM_FCM);
> > > > > +   }
> > > > > +   return IRQ_HANDLED;
> > > > > +}
> > > > > +
> > > > > +static void unicam_set_packing_config(struct unicam_device *unicam)
> > > > > +{
> > > > > +   struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > > > > +   u32 pack, unpack;
> > > > > +   u32 val;
> > > > > +
> > > > > +   if (node->fmt.fmt.pix.pixelformat == node->fmtinfo->fourcc) {
> > > > > +           unpack = UNICAM_PUM_NONE;
> > > > > +           pack = UNICAM_PPM_NONE;
> > > > > +   } else {
> > > > > +           switch (node->fmtinfo->depth) {
> > > > > +           case 8:
> > > > > +                   unpack = UNICAM_PUM_UNPACK8;
> > > > > +                   break;
> > > > > +           case 10:
> > > > > +                   unpack = UNICAM_PUM_UNPACK10;
> > > > > +                   break;
> > > > > +           case 12:
> > > > > +                   unpack = UNICAM_PUM_UNPACK12;
> > > > > +                   break;
> > > > > +           case 14:
> > > > > +                   unpack = UNICAM_PUM_UNPACK14;
> > > > > +                   break;
> > > > > +           case 16:
> > > > > +                   unpack = UNICAM_PUM_UNPACK16;
> > > > > +                   break;
> > > > > +           default:
> > > > > +                   unpack = UNICAM_PUM_NONE;
> > > > > +                   break;
> > > > > +           }
> > > > > +
> > > > > +           /* Repacking is always to 16bpp */
> > > > > +           pack = UNICAM_PPM_PACK16;
> > > >
> > > > Also 8-bit data?
> > >
> > > Not that I know of. The 8-bit entries in unicam_image_formats have no
> > > .unpacked_fourcc field, so the condition in the if above will always be
> > > true for those as they can only be selected by setting the pixel format
> > > to fmtinfo->fourcc.
> >
> > Correct.
> > The unpacking / packing doesn't bit shift, hence 10P can be unpacked
> > to 10, 12P to 12, and 14P to 14, but there is no 8bit unpacked into
> > the LSBs of 16 bit words defined as a V4L2 format.
> >
> > You can configure packing to produce 10P, 12P, etc, but without the
> > bitshift it has no real use.
> >
> > MSBs get dropped if you try packing to a smaller bit depth than the
> > source, so packing to 8 bit from any other depth results in unusable
> > output.
> >
> > If being really critical, then there is no point in having the case 8
> > and case 16 in this switch as they will never be selected.
>
> As we have space to spare in the unicam_format_info structure due to
> padding, I'll move the unpack value there and drop the switch.

As it ends up being a 1:1 mapping, that would work.

Originally I believed unicam did the bit-shifting which made offering
the flexibility useful, but that proved to be wrong.

  Dave

> > > > > +   }
> >
> > <snip>
>
> --
> Regards,
>
> Laurent Pinchart

Re: [PATCH v6 08/15] dt-bindings: media: Add bindings for bcm2835-unicam

[Sakari Ailus]

Hi Laurent,

On Tue, Mar 26, 2024 at 01:02:03AM +0200, Laurent Pinchart wrote:
> Hi Sakari,
> 
> On Mon, Mar 25, 2024 at 06:28:11PM +0000, Sakari Ailus wrote:
> > On Fri, Mar 01, 2024 at 11:32:23PM +0200, Laurent Pinchart wrote:
> > > From: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > 
> > > Introduce the dt-bindings documentation for bcm2835 CCP2/CSI2 Unicam
> > > camera interface.
> > > 
> > > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > Co-developed-by: Naushir Patuck <naush@raspberrypi.com>
> > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > > Co-developed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > > Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > Reviewed-by: Rob Herring <robh@kernel.org>
> > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > ---
> > > Changes since v5:
> > > 
> > > - Squash MAINTAINERS changes in
> > > 
> > > Changes since v3:
> > > 
> > > - Make MAINTAINERS its own patch
> > > - Describe the reg and clocks correctly
> > > - Use a vendor entry for the number of data lanes
> > > ---
> > >  .../bindings/media/brcm,bcm2835-unicam.yaml   | 117 ++++++++++++++++++
> > >  MAINTAINERS                                   |   6 +
> > >  2 files changed, 123 insertions(+)
> > >  create mode 100644 Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> > > 
> > > diff --git a/Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml b/Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> > > new file mode 100644
> > > index 000000000000..1938ace23b3d
> > > --- /dev/null
> > > +++ b/Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> > > @@ -0,0 +1,117 @@
> > > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> > > +%YAML 1.2
> > > +---
> > > +$id: http://devicetree.org/schemas/media/brcm,bcm2835-unicam.yaml#
> > > +$schema: http://devicetree.org/meta-schemas/core.yaml#
> > > +
> > > +title: Broadcom BCM283x Camera Interface (Unicam)
> > > +
> > > +maintainers:
> > > +  - Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
> > > +
> > > +description: |-
> > > +  The Unicam block on BCM283x SoCs is the receiver for either
> > > +  CSI-2 or CCP2 data from image sensors or similar devices.
> > > +
> > > +  The main platform using this SoC is the Raspberry Pi family of boards.  On
> > > +  the Pi the VideoCore firmware can also control this hardware block, and
> > > +  driving it from two different processors will cause issues.  To avoid this,
> > > +  the firmware checks the device tree configuration during boot. If it finds
> > > +  device tree nodes whose name starts with 'csi' then it will stop the firmware
> > > +  accessing the block, and it can then safely be used via the device tree
> > > +  binding.
> > > +
> > > +properties:
> > > +  compatible:
> > > +    const: brcm,bcm2835-unicam
> > > +
> > > +  reg:
> > > +    items:
> > > +      - description: Unicam block.
> > > +      - description: Clock Manager Image (CMI) block.
> > > +
> > > +  reg-names:
> > > +    items:
> > > +      - const: unicam
> > > +      - const: cmi
> > > +
> > > +  interrupts:
> > > +    maxItems: 1
> > > +
> > > +  clocks:
> > > +    items:
> > > +      - description: Clock to drive the LP state machine of Unicam.
> > > +      - description: Clock for the VPU (core clock).
> > > +
> > > +  clock-names:
> > > +    items:
> > > +      - const: lp
> > > +      - const: vpu
> > > +
> > > +  power-domains:
> > > +    items:
> > > +      - description: Unicam power domain
> > > +
> > > +  brcm,num-data-lanes:
> > > +    $ref: /schemas/types.yaml#/definitions/uint32
> > > +    enum: [ 2, 4 ]
> > > +    description: |
> > > +      Number of CSI-2 data lanes supported by this Unicam instance. The number
> > > +      of data lanes actively used is specified with the data-lanes endpoint
> > > +      property.
> > > +
> > > +  port:
> > > +    $ref: /schemas/graph.yaml#/$defs/port-base
> > > +    unevaluatedProperties: false
> > > +
> > > +    properties:
> > > +      endpoint:
> > > +        $ref: /schemas/media/video-interfaces.yaml#
> > > +        unevaluatedProperties: false
> > > +
> > > +        properties:
> > > +          data-lanes: true
> > > +
> > > +        required:
> > > +          - data-lanes
> > 
> > As the device supports multiple data interfaces (at least so it seems when
> > looking at the driver code), you need to list the bus-type property here,
> > too.
> 
> Good point, I'll add
> 
>         properties:
> 	  bus-type:
> 	    enum: [ 3, 4 ]
> 	required:
> 	  - bus-type
> 
> Should I also change unevaluatedProperties to additionalProperties for
> the endpoint node, to reject any other property (and the explicitly list
> remote-endpoint as an allowed property) ? The result would be 
> 
>       endpoint:
>         $ref: /schemas/media/video-interfaces.yaml#
>         additionalProperties: false
> 
>         properties:
>           bus-type:
>             enum: [ 3, 4 ]
> 
>           data-lanes: true
>           remote-endpoint: true
> 
>         required:
>           - bus-type
>           - data-lanes
>           - remote-endpoint

Seems good to me.

> 
> > > +
> > > +    required:
> > > +      - endpoint
> > > +
> > > +required:
> > > +  - compatible
> > > +  - reg
> > > +  - reg-names
> > > +  - interrupts
> > > +  - clocks
> > > +  - clock-names
> > > +  - power-domains
> > > +  - brcm,num-data-lanes
> > > +  - port
> > > +
> > > +additionalProperties: False
> > > +
> > > +examples:
> > > +  - |
> > > +    #include <dt-bindings/clock/bcm2835.h>
> > > +    #include <dt-bindings/interrupt-controller/arm-gic.h>
> > > +    #include <dt-bindings/power/raspberrypi-power.h>
> > > +    csi1: csi@7e801000 {
> > > +        compatible = "brcm,bcm2835-unicam";
> > > +        reg = <0x7e801000 0x800>,
> > > +              <0x7e802004 0x4>;
> > > +        reg-names = "unicam", "cmi";
> > > +        interrupts = <GIC_SPI 103 IRQ_TYPE_LEVEL_HIGH>;
> > > +        clocks = <&clocks BCM2835_CLOCK_CAM1>,
> > > +                 <&firmware_clocks 4>;
> > > +        clock-names = "lp", "vpu";
> > > +        power-domains = <&power RPI_POWER_DOMAIN_UNICAM1>;
> > > +        brcm,num-data-lanes = <2>;
> > > +        port {
> > > +                csi1_ep: endpoint {
> > > +                        remote-endpoint = <&imx219_0>;
> > > +                        data-lanes = <1 2>;
> > > +                };
> > > +        };
> > > +    };
> > > +...
> > > diff --git a/MAINTAINERS b/MAINTAINERS
> > > index fada59148cb5..e50a59654e6e 100644
> > > --- a/MAINTAINERS
> > > +++ b/MAINTAINERS
> > > @@ -3997,6 +3997,12 @@ N:	bcm113*
> > >  N:	bcm216*
> > >  N:	kona
> > >  
> > > +BROADCOM BCM2835 CAMERA DRIVERS
> > > +M:	Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
> > > +L:	linux-media@vger.kernel.org
> > > +S:	Maintained
> > > +F:	Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> > > +
> > >  BROADCOM BCM47XX MIPS ARCHITECTURE
> > >  M:	Hauke Mehrtens <hauke@hauke-m.de>
> > >  M:	Rafał Miłecki <zajec5@gmail.com>
> 
> -- 
> Regards,
> 
> Laurent Pinchart

-- 
Regards,

Sakari Ailus

Re: [PATCH v6 09/15] media: bcm2835-unicam: Add support for CCP2/CSI2 camera interface

[Sakari Ailus]

Hi Laurent,

On Tue, Mar 26, 2024 at 03:50:28AM +0200, Laurent Pinchart wrote:
> On Tue, Mar 26, 2024 at 03:37:09AM +0200, Laurent Pinchart wrote:
> > Hi Sakari,
> > 
> > On Mon, Mar 25, 2024 at 06:36:49PM +0000, Sakari Ailus wrote:
> > > On Fri, Mar 01, 2024 at 11:32:24PM +0200, Laurent Pinchart wrote:
> > > > From: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > 
> > > > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > > > It is represented as two video device nodes: unicam-image and
> > > > unicam-embedded which are connected to an internal subdev (named
> > > > unicam-subdev) in order to manage streams routing.
> > > > 
> > > > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > Co-developed-by: Naushir Patuck <naush@raspberrypi.com>
> > > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > > > Co-developed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > > > Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
> > > > Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > 
> > > Thanks for submitting this, it's the cleanest and neatest Unicom driver
> > 
> > Unicam, or if you insist Unicorn, but not Unicom :-)

A Unicorn driver? Hmm. Maybe Unicam is indeed the best choice here?

> > 
> > > I've ever seen!
> > > 
> > > Some mostly unimportant comments below, however the bus-type issue needs to
> > > be addressed.
> > > 
> > > > ---
> > > > Changes since v5:
> > > > 
> > > > - Move to drivers/media/platform/broadcom/
> > > > - Port to the upstream V4L2 streams API
> > > > - Rebase on latest metadata API proposal
> > > > - Add missing error message
> > > > - Drop unneeded documentation block for unicam_isr()
> > > > - Drop unneeded dev_dbg() and dev_err() messages
> > > > - Drop unneeded streams_mask and fmt checks
> > > > - Drop unused unicam_sd_pad_is_sink()
> > > > - Drop unneeded includes
> > > > - Drop v4l2_ctrl_subscribe_event() call
> > > > - Use pm_runtime_resume_and_get()
> > > > - Indentation and line wrap fixes
> > > > - Let the framework set bus_info
> > > > - Use v4l2_fwnode_endpoint_parse()
> > > > - Fix media device cleanup
> > > > - Drop lane reordering checks
> > > > - Fix subdev state locking
> > > > - Drop extra debug messages
> > > > - Move clock handling to runtime PM handlers
> > > > - Reorder functions
> > > > - Rename init functions for more clarity
> > > > - Initialize runtime PM earlier
> > > > - Clarify error messages
> > > > - Simplify subdev init with local variable
> > > > - Fix subdev cleanup
> > > > - Fix typos and indentation
> > > > - Don't initialize local variables needlessly
> > > > - Simplify num lanes check
> > > > - Fix metadata handling in subdev set_fmt
> > > > - Drop manual fallback to .s_stream()
> > > > - Pass v4l2_pix_format to unicam_calc_format_size_bpl()
> > > > - Simplify unicam_set_default_format()
> > > > - Fix default format settings
> > > > - Add busy check in unicam_s_fmt_meta()
> > > > - Add missing \n at end of format strings
> > > > - Fix metadata handling in subdev set_fmt
> > > > - Fix locking when starting streaming
> > > > - Return buffers from start streaming fails
> > > > - Fix format validation for metadata node
> > > > - Use video_device_pipeline_{start,stop}() helpers
> > > > - Simplify format enumeration
> > > > - Drop unset variable
> > > > - Update MAINTAINERS entry
> > > > - Update to the upstream v4l2_async_nf API
> > > > - Update to the latest subdev routing API
> > > > - Update to the latest subdev state API
> > > > - Move from subdev .init_cfg() to .init_state()
> > > > - Update to the latest videobuf2 API
> > > > - Fix v4l2_subdev_enable_streams() error check
> > > > - Use correct pad for the connected subdev
> > > > - Return buffers to vb2 when start streaming fails
> > > > - Improve debugging in start streaming handler
> > > > - Simplify DMA address management
> > > > - Drop comment about bcm2835-camera driver
> > > > - Clarify comments that explain min/max sizes
> > > > - Pass v4l2_pix_format to unicam_try_fmt()
> > > > - Drop unneeded local variables
> > > > - Rename image-related constants and functions
> > > > - Turn unicam_fmt.metadata_fmt into bool
> > > > - Rename unicam_fmt to unicam_format_info
> > > > - Rename unicam_format_info variables to fmtinfo
> > > > - Rename unicam_node.v_fmt to fmt
> > > > - Add metadata formats for RAW10, RAW12 and RAW14
> > > > - Make metadata formats line-based
> > > > - Validate format on metadata video device
> > > > - Add Co-devlopped-by tags
> > > > 
> > > > Changes since v3:
> > > > 
> > > > - Add the vendor prefix for DT name
> > > > - Use the reg-names in DT parsing
> > > > - Remove MAINTAINERS entry
> > > > 
> > > > Changes since v2:
> > > > 
> > > > - Change code organization
> > > > - Remove unused variables
> > > > - Correct the fmt_meta functions
> > > > - Rewrite the start/stop streaming
> > > >   - You can now start the image node alone, but not the metadata one
> > > >   - The buffers are allocated per-node
> > > >   - only the required stream is started, if the route exists and is
> > > >     enabled
> > > > - Prefix the macros with UNICAM_ to not have too generic names
> > > > - Drop colorspace support
> > > > 
> > > > Changes since v1:
> > > > 
> > > > - Replace the unicam_{info,debug,error} macros with dev_*()
> > > > ---
> > > >  MAINTAINERS                                   |    1 +
> > > >  drivers/media/platform/Kconfig                |    1 +
> > > >  drivers/media/platform/Makefile               |    1 +
> > > >  drivers/media/platform/broadcom/Kconfig       |   23 +
> > > >  drivers/media/platform/broadcom/Makefile      |    3 +
> > > >  .../platform/broadcom/bcm2835-unicam-regs.h   |  255 ++
> > > >  .../media/platform/broadcom/bcm2835-unicam.c  | 2607 +++++++++++++++++
> > > >  7 files changed, 2891 insertions(+)
> > > >  create mode 100644 drivers/media/platform/broadcom/Kconfig
> > > >  create mode 100644 drivers/media/platform/broadcom/Makefile
> > > >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > > >  create mode 100644 drivers/media/platform/broadcom/bcm2835-unicam.c
> > > > 
> > > > diff --git a/MAINTAINERS b/MAINTAINERS
> > > > index e50a59654e6e..cc350729f467 100644
> > > > --- a/MAINTAINERS
> > > > +++ b/MAINTAINERS
> > > > @@ -4002,6 +4002,7 @@ M:	Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
> > > >  L:	linux-media@vger.kernel.org
> > > >  S:	Maintained
> > > >  F:	Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> > > > +F:	drivers/media/platform/bcm2835/
> > > >  
> > > >  BROADCOM BCM47XX MIPS ARCHITECTURE
> > > >  M:	Hauke Mehrtens <hauke@hauke-m.de>
> > > > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > > > index 91e54215de3a..2d79bfc68c15 100644
> > > > --- a/drivers/media/platform/Kconfig
> > > > +++ b/drivers/media/platform/Kconfig
> > > > @@ -67,6 +67,7 @@ source "drivers/media/platform/amlogic/Kconfig"
> > > >  source "drivers/media/platform/amphion/Kconfig"
> > > >  source "drivers/media/platform/aspeed/Kconfig"
> > > >  source "drivers/media/platform/atmel/Kconfig"
> > > > +source "drivers/media/platform/broadcom/Kconfig"
> > > >  source "drivers/media/platform/cadence/Kconfig"
> > > >  source "drivers/media/platform/chips-media/Kconfig"
> > > >  source "drivers/media/platform/intel/Kconfig"
> > > > diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> > > > index 3296ec1ebe16..da17301f7439 100644
> > > > --- a/drivers/media/platform/Makefile
> > > > +++ b/drivers/media/platform/Makefile
> > > > @@ -10,6 +10,7 @@ obj-y += amlogic/
> > > >  obj-y += amphion/
> > > >  obj-y += aspeed/
> > > >  obj-y += atmel/
> > > > +obj-y += broadcom/
> > > >  obj-y += cadence/
> > > >  obj-y += chips-media/
> > > >  obj-y += intel/
> > > > diff --git a/drivers/media/platform/broadcom/Kconfig b/drivers/media/platform/broadcom/Kconfig
> > > > new file mode 100644
> > > > index 000000000000..cc2c9afcc948
> > > > --- /dev/null
> > > > +++ b/drivers/media/platform/broadcom/Kconfig
> > > > @@ -0,0 +1,23 @@
> > > > +# SPDX-License-Identifier: GPL-2.0
> > > > +
> > > > +config VIDEO_BCM2835_UNICAM
> > > > +	tristate "Broadcom BCM283x/BCM271x Unicam video capture driver"
> > > > +	depends on ARCH_BCM2835 || COMPILE_TEST
> > > > +	depends on PM
> > > > +	depends on VIDEO_DEV
> > > > +	select MEDIA_CONTROLLER
> > > > +	select V4L2_FWNODE
> > > > +	select VIDEO_V4L2_SUBDEV_API
> > > > +	select VIDEOBUF2_DMA_CONTIG
> > > > +	help
> > > > +	  Say Y here to enable support for the BCM283x/BCM271x CSI-2 receiver.
> > > > +	  This is a V4L2 driver that controls the CSI-2 receiver directly,
> > > > +	  independently from the VC4 firmware.
> > > > +
> > > > +	  This driver is mutually exclusive with the use of bcm2835-camera. The
> > > > +	  firmware will disable all access to the peripheral from within the
> > > > +	  firmware if it finds a DT node using it, and bcm2835-camera will
> > > > +	  therefore fail to probe.
> > > > +
> > > > +	  To compile this driver as a module, choose M here. The module will be
> > > > +	  called bcm2835-unicam.
> > > > diff --git a/drivers/media/platform/broadcom/Makefile b/drivers/media/platform/broadcom/Makefile
> > > > new file mode 100644
> > > > index 000000000000..03d2045aba2e
> > > > --- /dev/null
> > > > +++ b/drivers/media/platform/broadcom/Makefile
> > > > @@ -0,0 +1,3 @@
> > > > +# SPDX-License-Identifier: GPL-2.0
> > > > +
> > > > +obj-$(CONFIG_VIDEO_BCM2835_UNICAM) += bcm2835-unicam.o
> > > > diff --git a/drivers/media/platform/broadcom/bcm2835-unicam-regs.h b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > > > new file mode 100644
> > > > index 000000000000..84775fd2fac5
> > > > --- /dev/null
> > > > +++ b/drivers/media/platform/broadcom/bcm2835-unicam-regs.h
> > > > @@ -0,0 +1,255 @@
> > > > +/* SPDX-License-Identifier: GPL-2.0-only */
> > > > +
> > > > +/*
> > > > + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> > > 
> > > Anything up to 2024?
> > 
> > Not really. The registers haven't really changed :-) I'll update the
> > copyright in the .c file though.
> > 
> > > > + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > + */
> > 
> > [snip]
> > 
> > > > diff --git a/drivers/media/platform/broadcom/bcm2835-unicam.c b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > > > new file mode 100644
> > > > index 000000000000..716c89b8a217
> > > > --- /dev/null
> > > > +++ b/drivers/media/platform/broadcom/bcm2835-unicam.c
> > > > @@ -0,0 +1,2607 @@
> > 
> > [snip]
> > 
> > > > +static irqreturn_t unicam_isr(int irq, void *dev)
> > > > +{
> > > > +	struct unicam_device *unicam = dev;
> > > > +	unsigned int lines_done = unicam_get_lines_done(dev);
> > > > +	unsigned int sequence = unicam->sequence;
> > > > +	unsigned int i;
> > > > +	u32 ista, sta;
> > > > +	bool fe;
> > > > +	u64 ts;
> > > > +
> > > > +	sta = unicam_reg_read(unicam, UNICAM_STA);
> > > > +	/* Write value back to clear the interrupts */
> > > > +	unicam_reg_write(unicam, UNICAM_STA, sta);
> > > > +
> > > > +	ista = unicam_reg_read(unicam, UNICAM_ISTA);
> > > > +	/* Write value back to clear the interrupts */
> > > > +	unicam_reg_write(unicam, UNICAM_ISTA, ista);
> > > > +
> > > > +	dev_dbg(unicam->dev, "ISR: ISTA: 0x%X, STA: 0x%X, sequence %d, lines done %d\n",
> > > > +		ista, sta, sequence, lines_done);
> > > > +
> > > > +	if (!(sta & (UNICAM_IS | UNICAM_PI0)))
> > > > +		return IRQ_HANDLED;
> > > > +
> > > > +	/*
> > > > +	 * Look for either the Frame End interrupt or the Packet Capture status
> > > > +	 * to signal a frame end.
> > > > +	 */
> > > > +	fe = ista & UNICAM_FEI || sta & UNICAM_PI0;
> > > > +
> > > > +	/*
> > > > +	 * We must run the frame end handler first. If we have a valid next_frm
> > > > +	 * and we get a simultaneout FE + FS interrupt, running the FS handler
> > > > +	 * first would null out the next_frm ptr and we would have lost the
> > > > +	 * buffer forever.
> > > > +	 */
> > > > +	if (fe) {
> > > > +		/*
> > > > +		 * Ensure we have swapped buffers already as we can't
> > > > +		 * stop the peripheral. If no buffer is available, use a
> > > > +		 * dummy buffer to dump out frames until we get a new buffer
> > > > +		 * to use.
> > > > +		 */
> > > > +		for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > > > +			if (!unicam->node[i].streaming)
> > > > +				continue;
> > > > +
> > > > +			/*
> > > > +			 * If cur_frm == next_frm, it means we have not had
> > > > +			 * a chance to swap buffers, likely due to having
> > > > +			 * multiple interrupts occurring simultaneously (like FE
> > > > +			 * + FS + LS). In this case, we cannot signal the buffer
> > > > +			 * as complete, as the HW will reuse that buffer.
> > > > +			 */
> > > > +			if (unicam->node[i].cur_frm &&
> > > > +			    unicam->node[i].cur_frm != unicam->node[i].next_frm)
> > > > +				unicam_process_buffer_complete(&unicam->node[i],
> > > > +							       sequence);
> > > > +			unicam->node[i].cur_frm = unicam->node[i].next_frm;
> > > > +		}
> > > > +		unicam->sequence++;
> > > 
> > > Does access to this data need to be serialised somehow.
> > 
> > Given that it's only accessed from the interrupt handler (beside
> > start_streaming time, before starting the hardware), I don't think so.

Ack. I guess a memory barrier would be theoretically needed although in
practice other locks will be taken so you might not have issues.

> > 
> > > > +	}
> > > > +
> > > > +	if (ista & UNICAM_FSI) {
> > > > +		/*
> > > > +		 * Timestamp is to be when the first data byte was captured,
> > > > +		 * aka frame start.
> > > > +		 */
> > > > +		ts = ktime_get_ns();
> > > > +		for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > > > +			if (!unicam->node[i].streaming)
> > > > +				continue;
> > > > +
> > > > +			if (unicam->node[i].cur_frm)
> > > > +				unicam->node[i].cur_frm->vb.vb2_buf.timestamp =
> > > > +								ts;
> > > > +			else
> > > > +				dev_dbg(unicam->v4l2_dev.dev,
> > > > +					"ISR: [%d] Dropping frame, buffer not available at FS\n",
> > > > +					i);
> > > > +			/*
> > > > +			 * Set the next frame output to go to a dummy frame
> > > > +			 * if we have not managed to obtain another frame
> > > > +			 * from the queue.
> > > > +			 */
> > > > +			unicam_schedule_dummy_buffer(&unicam->node[i]);
> > > > +		}
> > > > +
> > > > +		unicam_queue_event_sof(unicam);
> > > > +	}
> > > > +
> > > > +	/*
> > > > +	 * Cannot swap buffer at frame end, there may be a race condition
> > > > +	 * where the HW does not actually swap it if the new frame has
> > > > +	 * already started.
> > > > +	 */
> > > > +	if (ista & (UNICAM_FSI | UNICAM_LCI) && !fe) {
> > > > +		for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > > > +			if (!unicam->node[i].streaming)
> > > > +				continue;
> > > > +
> > > > +			spin_lock(&unicam->node[i].dma_queue_lock);
> > > > +			if (!list_empty(&unicam->node[i].dma_queue) &&
> > > > +			    !unicam->node[i].next_frm)
> > > > +				unicam_schedule_next_buffer(&unicam->node[i]);
> > > > +			spin_unlock(&unicam->node[i].dma_queue_lock);
> > > > +		}
> > > > +	}
> > > > +
> > > > +	if (unicam_reg_read(unicam, UNICAM_ICTL) & UNICAM_FCM) {
> > > > +		/* Switch out of trigger mode if selected */
> > > > +		unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> > > > +		unicam_reg_write_field(unicam, UNICAM_ICTL, 0, UNICAM_FCM);
> > > > +	}
> > > > +	return IRQ_HANDLED;
> > > > +}
> > > > +
> > > > +static void unicam_set_packing_config(struct unicam_device *unicam)
> > > > +{
> > > > +	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > > > +	u32 pack, unpack;
> > > > +	u32 val;
> > > > +
> > > > +	if (node->fmt.fmt.pix.pixelformat == node->fmtinfo->fourcc) {
> > > > +		unpack = UNICAM_PUM_NONE;
> > > > +		pack = UNICAM_PPM_NONE;
> > > > +	} else {
> > > > +		switch (node->fmtinfo->depth) {
> > > > +		case 8:
> > > > +			unpack = UNICAM_PUM_UNPACK8;
> > > > +			break;
> > > > +		case 10:
> > > > +			unpack = UNICAM_PUM_UNPACK10;
> > > > +			break;
> > > > +		case 12:
> > > > +			unpack = UNICAM_PUM_UNPACK12;
> > > > +			break;
> > > > +		case 14:
> > > > +			unpack = UNICAM_PUM_UNPACK14;
> > > > +			break;
> > > > +		case 16:
> > > > +			unpack = UNICAM_PUM_UNPACK16;
> > > > +			break;
> > > > +		default:
> > > > +			unpack = UNICAM_PUM_NONE;
> > > > +			break;
> > > > +		}
> > > > +
> > > > +		/* Repacking is always to 16bpp */
> > > > +		pack = UNICAM_PPM_PACK16;
> > > 
> > > Also 8-bit data?
> > 
> > Not that I know of. The 8-bit entries in unicam_image_formats have no
> > .unpacked_fourcc field, so the condition in the if above will always be
> > true for those as they can only be selected by setting the pixel format
> > to fmtinfo->fourcc.

Ok.

> > 
> > > > +	}
> > > > +
> > > > +	val = 0;
> > > 
> > > You could do initialisation in declaration.
> > 
> > Yes, but I think it's more readable to keep all the code that affects
> > the 'val' variable together.

If wal was a bit more descriptive name this would be a non-issue. Up to
you.

> > 
> > > > +	unicam_set_field(&val, unpack, UNICAM_PUM_MASK);
> > > > +	unicam_set_field(&val, pack, UNICAM_PPM_MASK);
> > > > +	unicam_reg_write(unicam, UNICAM_IPIPE, val);
> > > > +}
> > > > +
> > > > +static void unicam_cfg_image_id(struct unicam_device *unicam)
> > > > +{
> > > > +	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > > > +
> > > > +	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> > > > +		/* CSI2 mode, hardcode VC 0 for now. */
> > > > +		unicam_reg_write(unicam, UNICAM_IDI0,
> > > > +				 (0 << 6) | node->fmtinfo->csi_dt);
> > > > +	} else {
> > > > +		/* CCP2 mode */
> > > > +		unicam_reg_write(unicam, UNICAM_IDI0,
> > > > +				 0x80 | node->fmtinfo->csi_dt);
> > > > +	}
> > > > +}
> > > > +
> > > > +static void unicam_enable_ed(struct unicam_device *unicam)
> > > > +{
> > > > +	u32 val = unicam_reg_read(unicam, UNICAM_DCS);
> > > > +
> > > > +	unicam_set_field(&val, 2, UNICAM_EDL_MASK);
> > > > +	/* Do not wrap at the end of the embedded data buffer */
> > > > +	unicam_set_field(&val, 0, UNICAM_DBOB);
> > > > +
> > > > +	unicam_reg_write(unicam, UNICAM_DCS, val);
> > > > +}
> > > > +
> > > > +static void unicam_start_rx(struct unicam_device *unicam,
> > > > +			    struct unicam_buffer *buf)
> > > > +{
> > > > +	struct unicam_node *node = &unicam->node[UNICAM_IMAGE_NODE];
> > > > +	int line_int_freq = node->fmt.fmt.pix.height >> 2;
> > > > +	unsigned int i;
> > > > +	u32 val;
> > > > +
> > > > +	if (line_int_freq < 128)
> > > > +		line_int_freq = 128;
> > > 
> > > 	line_int_freq = max(line_int_freq, 128);
> > Ack.
> > 
> > > > +
> > > > +	/* Enable lane clocks */
> > > > +	val = 1;
> > > 
> > > Initialise in the loop initialisation below, I'd say.
> > 
> > How about
> > 
> > 	val = 0x55 & GENMASK(unicam->pipe.num_data_lanes * 2 - 1, 0);
> 
> I meant
> 
>  	val = 0x155 & GENMASK(unicam->pipe.num_data_lanes * 2 + 1, 0);
> 
> Maybe a comment would be useful ?
> 
> 	/*
> 	 * Enable lane clocks. The register is structured as follows:
> 	 *
> 	 * [9:8] - DAT3
> 	 * [7:6] - DAT2
> 	 * [5:4] - DAT1
> 	 * [3:2] - DAT0
> 	 * [1:0] - CLK
> 	 *
> 	 * Enabled lane must be set to b01, and disabled lanes to b00. The clock
> 	 * lane is always enabled.
> 	 */
> 	val = 0x155 & GENMASK(unicam->pipe.num_data_lanes * 2 + 1, 0);

Seems good to me.

> 
> > > > +	for (i = 0; i < unicam->active_data_lanes; i++)
> > > > +		val = val << 2 | 1;
> > > > +	unicam_clk_write(unicam, val);
> > > > +
> > > > +	/* Basic init */
> > > > +	unicam_reg_write(unicam, UNICAM_CTRL, UNICAM_MEM);
> > > > +
> > > > +	/* Enable analogue control, and leave in reset. */
> > > > +	val = UNICAM_AR;
> > > > +	unicam_set_field(&val, 7, UNICAM_CTATADJ_MASK);
> > > > +	unicam_set_field(&val, 7, UNICAM_PTATADJ_MASK);
> > > > +	unicam_reg_write(unicam, UNICAM_ANA, val);
> > > > +	usleep_range(1000, 2000);
> > > > +
> > > > +	/* Come out of reset */
> > > > +	unicam_reg_write_field(unicam, UNICAM_ANA, 0, UNICAM_AR);
> > > > +
> > > > +	/* Peripheral reset */
> > > > +	unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPR);
> > > > +	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPR);
> > > > +
> > > > +	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_CPE);
> > > > +
> > > > +	/* Enable Rx control. */
> > > > +	val = unicam_reg_read(unicam, UNICAM_CTRL);
> > > > +	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> > > > +		unicam_set_field(&val, UNICAM_CPM_CSI2, UNICAM_CPM_MASK);
> > > > +		unicam_set_field(&val, UNICAM_DCM_STROBE, UNICAM_DCM_MASK);
> > > > +	} else {
> > > > +		unicam_set_field(&val, UNICAM_CPM_CCP2, UNICAM_CPM_MASK);
> > > > +		unicam_set_field(&val, unicam->bus_flags, UNICAM_DCM_MASK);
> > > > +	}
> > > > +	/* Packet framer timeout */
> > > > +	unicam_set_field(&val, 0xf, UNICAM_PFT_MASK);
> > > > +	unicam_set_field(&val, 128, UNICAM_OET_MASK);
> > > > +	unicam_reg_write(unicam, UNICAM_CTRL, val);
> > > > +
> > > > +	unicam_reg_write(unicam, UNICAM_IHWIN, 0);
> > > > +	unicam_reg_write(unicam, UNICAM_IVWIN, 0);
> > > > +
> > > > +	/* AXI bus access QoS setup */
> > > > +	val = unicam_reg_read(unicam, UNICAM_PRI);
> > > > +	unicam_set_field(&val, 0, UNICAM_BL_MASK);
> > > > +	unicam_set_field(&val, 0, UNICAM_BS_MASK);
> > > > +	unicam_set_field(&val, 0xe, UNICAM_PP_MASK);
> > > > +	unicam_set_field(&val, 8, UNICAM_NP_MASK);
> > > > +	unicam_set_field(&val, 2, UNICAM_PT_MASK);
> > > > +	unicam_set_field(&val, 1, UNICAM_PE);
> > > > +	unicam_reg_write(unicam, UNICAM_PRI, val);
> > > > +
> > > > +	unicam_reg_write_field(unicam, UNICAM_ANA, 0, UNICAM_DDL);
> > > > +
> > > > +	/* Always start in trigger frame capture mode (UNICAM_FCM set) */
> > > > +	val = UNICAM_FSIE | UNICAM_FEIE | UNICAM_FCM | UNICAM_IBOB;
> > > > +	unicam_set_field(&val, line_int_freq, UNICAM_LCIE_MASK);
> > > > +	unicam_reg_write(unicam, UNICAM_ICTL, val);
> > > > +	unicam_reg_write(unicam, UNICAM_STA, UNICAM_STA_MASK_ALL);
> > > > +	unicam_reg_write(unicam, UNICAM_ISTA, UNICAM_ISTA_MASK_ALL);
> > > > +
> > > > +	/* tclk_term_en */
> > > > +	unicam_reg_write_field(unicam, UNICAM_CLT, 2, UNICAM_CLT1_MASK);
> > > > +	/* tclk_settle */
> > > > +	unicam_reg_write_field(unicam, UNICAM_CLT, 6, UNICAM_CLT2_MASK);
> > > > +	/* td_term_en */
> > > > +	unicam_reg_write_field(unicam, UNICAM_DLT, 2, UNICAM_DLT1_MASK);
> > > > +	/* ths_settle */
> > > > +	unicam_reg_write_field(unicam, UNICAM_DLT, 6, UNICAM_DLT2_MASK);
> > > > +	/* trx_enable */
> > > > +	unicam_reg_write_field(unicam, UNICAM_DLT, 0, UNICAM_DLT3_MASK);
> > > > +
> > > > +	unicam_reg_write_field(unicam, UNICAM_CTRL, 0, UNICAM_SOE);
> > > > +
> > > > +	/* Packet compare setup - required to avoid missing frame ends */
> > > > +	val = 0;
> > > > +	unicam_set_field(&val, 1, UNICAM_PCE);
> > > > +	unicam_set_field(&val, 1, UNICAM_GI);
> > > > +	unicam_set_field(&val, 1, UNICAM_CPH);
> > > > +	unicam_set_field(&val, 0, UNICAM_PCVC_MASK);
> > > > +	unicam_set_field(&val, 1, UNICAM_PCDT_MASK);
> > > > +	unicam_reg_write(unicam, UNICAM_CMP0, val);
> > > > +
> > > > +	/* Enable clock lane and set up terminations */
> > > > +	val = 0;
> > > > +	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> > > > +		/* CSI2 */
> > > > +		unicam_set_field(&val, 1, UNICAM_CLE);
> > > > +		unicam_set_field(&val, 1, UNICAM_CLLPE);
> > > > +		if (!(unicam->bus_flags & V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK)) {
> > > > +			unicam_set_field(&val, 1, UNICAM_CLTRE);
> > > > +			unicam_set_field(&val, 1, UNICAM_CLHSE);
> > > > +		}
> > > > +	} else {
> > > > +		/* CCP2 */
> > > > +		unicam_set_field(&val, 1, UNICAM_CLE);
> > > > +		unicam_set_field(&val, 1, UNICAM_CLHSE);
> > > > +		unicam_set_field(&val, 1, UNICAM_CLTRE);
> > > > +	}
> > > > +	unicam_reg_write(unicam, UNICAM_CLK, val);
> > > > +
> > > > +	/*
> > > > +	 * Enable required data lanes with appropriate terminations.
> > > > +	 * The same value needs to be written to UNICAM_DATn registers for
> > > > +	 * the active lanes, and 0 for inactive ones.
> > > > +	 */
> > > > +	val = 0;
> > > > +	if (unicam->bus_type == V4L2_MBUS_CSI2_DPHY) {
> > > > +		/* CSI2 */
> > > > +		unicam_set_field(&val, 1, UNICAM_DLE);
> > > > +		unicam_set_field(&val, 1, UNICAM_DLLPE);
> > > > +		if (!(unicam->bus_flags & V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK)) {
> > > > +			unicam_set_field(&val, 1, UNICAM_DLTRE);
> > > > +			unicam_set_field(&val, 1, UNICAM_DLHSE);
> > > > +		}
> > > > +	} else {
> > > > +		/* CCP2 */
> > > > +		unicam_set_field(&val, 1, UNICAM_DLE);
> > > > +		unicam_set_field(&val, 1, UNICAM_DLHSE);
> > > > +		unicam_set_field(&val, 1, UNICAM_DLTRE);
> > > > +	}
> > > > +	unicam_reg_write(unicam, UNICAM_DAT0, val);
> > > > +
> > > > +	if (unicam->active_data_lanes == 1)
> > > > +		val = 0;
> > > > +	unicam_reg_write(unicam, UNICAM_DAT1, val);
> > > > +
> > > > +	if (unicam->max_data_lanes > 2) {
> > > > +		/*
> > > > +		 * Registers UNICAM_DAT2 and UNICAM_DAT3 only valid if the
> > > > +		 * instance supports more than 2 data lanes.
> > > > +		 */
> > > > +		if (unicam->active_data_lanes == 2)
> > > > +			val = 0;
> > > > +		unicam_reg_write(unicam, UNICAM_DAT2, val);
> > > > +
> > > > +		if (unicam->active_data_lanes == 3)
> > > > +			val = 0;
> > > > +		unicam_reg_write(unicam, UNICAM_DAT3, val);
> > > > +	}
> > > > +
> > > > +	unicam_reg_write(unicam, UNICAM_IBLS,
> > > > +			 node->fmt.fmt.pix.bytesperline);
> > > > +	unicam_wr_dma_addr(&unicam->node[UNICAM_IMAGE_NODE], buf);
> > > > +	unicam_set_packing_config(unicam);
> > > > +	unicam_cfg_image_id(unicam);
> > > > +
> > > > +	val = unicam_reg_read(unicam, UNICAM_MISC);
> > > > +	unicam_set_field(&val, 1, UNICAM_FL0);
> > > > +	unicam_set_field(&val, 1, UNICAM_FL1);
> > > > +	unicam_reg_write(unicam, UNICAM_MISC, val);
> > > > +
> > > > +	/* Enable peripheral */
> > > > +	unicam_reg_write_field(unicam, UNICAM_CTRL, 1, UNICAM_CPE);
> > > > +
> > > > +	/* Load image pointers */
> > > > +	unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_LIP_MASK);
> > > > +
> > > > +	/*
> > > > +	 * Enable trigger only for the first frame to
> > > > +	 * sync correctly to the FS from the source.
> > > > +	 */
> > > > +	unicam_reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> > > > +}
> > 
> > [snip]
> > 
> > > > +static int unicam_async_nf_init(struct unicam_device *unicam)
> > > > +{
> > > > +	struct v4l2_fwnode_endpoint ep = { };
> > > 
> > > If the bus-type property is mandatory and you have no stated defaults
> > > anywhere, this is fine. I.e. all the relevant properties would need to be

Actually this is a non-issue nowadays: bus in struct v4l2_fwnode_endpoint
is no longer a union.

> > > mandatory.
> > 
> > They are, as far as I can tell (well, the clock-noncontinuous property
> > is not mandatory, but that's expected as it's a flag).
> > 
> > > > +	struct fwnode_handle *ep_handle;
> > > > +	struct v4l2_async_connection *asc;
> > > > +	int ret;
> > > > +
> > > > +	ret = of_property_read_u32(unicam->dev->of_node, "brcm,num-data-lanes",
> > > > +				   &unicam->max_data_lanes);
> > > > +	if (ret < 0) {
> > > > +		dev_err(unicam->dev, "Missing %s DT property\n",
> > > > +			"brcm,num-data-lanes");
> > > > +		return -EINVAL;
> > > > +	}
> > > > +
> > > > +	/* Get and parse the local endpoint. */
> > > > +	ep_handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(unicam->dev), 0, 0,
> > > > +						    FWNODE_GRAPH_ENDPOINT_NEXT);
> > > > +	if (!ep_handle) {
> > > > +		dev_err(unicam->dev, "No endpoint found\n");
> > > > +		return -ENODEV;
> > > > +	}
> > > > +
> > > > +	ret = v4l2_fwnode_endpoint_parse(ep_handle, &ep);
> > > > +	if (ret) {
> > > > +		dev_err(unicam->dev, "Failed to parse endpoint: %d\n", ret);
> > > > +		goto error;
> > > > +	}
> > > > +
> > > > +	unicam->bus_type = ep.bus_type;
> > > > +
> > > > +	switch (ep.bus_type) {
> > > > +	case V4L2_MBUS_CSI2_DPHY: {
> > > > +		unsigned int num_data_lanes = ep.bus.mipi_csi2.num_data_lanes;
> > > > +
> > > > +		if (num_data_lanes != 1 && num_data_lanes != 2 &&
> > > > +		    num_data_lanes != 4) {
> > > > +			dev_err(unicam->dev, "%u data lanes not supported\n",
> > > > +				num_data_lanes);
> > > > +			goto error;
> > > > +		}
> > > > +
> > > > +		if (num_data_lanes > unicam->max_data_lanes) {
> > > > +			dev_err(unicam->dev,
> > > > +				"Endpoint uses %u data lanes when %u are supported\n",
> > > > +				num_data_lanes, unicam->max_data_lanes);
> > > > +			goto error;
> > > > +		}
> > > > +
> > > > +		unicam->active_data_lanes = num_data_lanes;
> > > > +		unicam->bus_flags = ep.bus.mipi_csi2.flags;
> > > > +		break;
> > > > +	}
> > > > +
> > > > +	case V4L2_MBUS_CCP2:
> > > > +		unicam->max_data_lanes = 1;
> > > > +		unicam->active_data_lanes = 1;
> > > > +		unicam->bus_flags = ep.bus.mipi_csi1.strobe;
> > > > +		break;
> > > > +
> > > > +	default:
> > > > +		/* Unsupported bus type */
> > > > +		dev_err(unicam->dev, "Unsupported bus type %u\n", ep.bus_type);
> > > > +		goto error;
> > > > +	}
> > > > +
> > > > +	/* Initialize and register the async notifier. */
> > > > +	v4l2_async_nf_init(&unicam->notifier, &unicam->v4l2_dev);
> > > > +
> > > > +	asc = v4l2_async_nf_add_fwnode_remote(&unicam->notifier, ep_handle,
> > > > +					      struct v4l2_async_connection);
> > > > +	fwnode_handle_put(ep_handle);
> > > > +	ep_handle = NULL;
> > > > +
> > > > +	if (IS_ERR(asc)) {
> > > > +		ret = PTR_ERR(asc);
> > > > +		dev_err(unicam->dev, "Failed to add entry to notifier: %d\n",
> > > > +			ret);
> > > > +		goto error;
> > > > +	}
> > > > +
> > > > +	unicam->notifier.ops = &unicam_async_ops;
> > > > +
> > > > +	ret = v4l2_async_nf_register(&unicam->notifier);
> > > > +	if (ret) {
> > > > +		dev_err(unicam->dev, "Error registering device notifier: %d\n",
> > > > +			ret);
> > > > +		goto error;
> > > > +	}
> > > > +
> > > > +	return 0;
> > > > +
> > > > +error:
> > > > +	fwnode_handle_put(ep_handle);
> > > > +	return ret;
> > > > +}
> > > > +
> > > > +/* -----------------------------------------------------------------------------
> > > > + * Probe & remove
> > > > + */
> > > > +
> > > > +static int unicam_media_init(struct unicam_device *unicam)
> > > > +{
> > > > +	int ret;
> > > > +
> > > > +	unicam->mdev.dev = unicam->dev;
> > > > +	strscpy(unicam->mdev.model, UNICAM_MODULE_NAME,
> > > > +		sizeof(unicam->mdev.model));
> > > > +	strscpy(unicam->mdev.serial, "", sizeof(unicam->mdev.serial));
> > > 
> > > Isn't the field already zeroed?
> > 
> > Indeed. I'll drop this.
> > 
> > > 
> > > > +	unicam->mdev.hw_revision = 0;
> > > > +
> > > > +	media_device_init(&unicam->mdev);
> > > > +
> > > > +	unicam->v4l2_dev.mdev = &unicam->mdev;
> > > > +
> > > > +	ret = v4l2_device_register(unicam->dev, &unicam->v4l2_dev);
> > > > +	if (ret < 0) {
> > > > +		dev_err(unicam->dev, "Unable to register v4l2 device\n");
> > > > +		goto err_media_cleanup;
> > > > +	}
> > > > +
> > > > +	ret = media_device_register(&unicam->mdev);
> > > > +	if (ret < 0) {
> > > > +		dev_err(unicam->dev,
> > > > +			"Unable to register media-controller device\n");
> > > > +		goto err_v4l2_unregister;
> > > > +	}
> > > > +
> > > > +	return 0;
> > > > +
> > > > +err_v4l2_unregister:
> > > > +	v4l2_device_unregister(&unicam->v4l2_dev);
> > > > +err_media_cleanup:
> > > > +	media_device_cleanup(&unicam->mdev);
> > > > +	return ret;
> > > > +}
> > 
> > [snip]
> 

-- 
Kind regards,

Sakari Ailus