[PATCH 00/21] media: atmel: atmel-isc: implement media controller

[PATCH 00/21] media: atmel: atmel-isc: implement media controller

[Eugen Hristev]

This series is a first attempt to support media controller in the atmel
ISC and XISC drivers.
This series also includes the csi2dc driver which was previously sent in a
separate series:
https://www.spinics.net/lists/linux-media/msg181042.html
https://www.spinics.net/lists/linux-media/msg181044.html
The driver now addresses comments received in latest v5 series from last year.

The series includes some minor changes and fixes that improve the isc common
code base, like removing the enum frameintervals VIDIOC, fixing bytesperline
for planar formats, etc.

Many thanks to folks from libcamera who helped a lot with understanding
how a media controller driver should behave.

Feedback is welcome !


Eugen Hristev (21):
  MAINTAINERS: add microchip csi2dc
  dt-bindings: media: atmel: csi2dc: add bindings for microchip csi2dc
  media: atmel: introduce microchip csi2dc driver
  MAINTAINERS: atmel-isc: add new file atmel-isc-clk.c
  media: atmel: atmel-isc: split the clock code into separate source
    file
  media: atmel: atmel-isc: replace video device name with module name
  media: atmel: atmel-sama7g5-isc: fix ispck leftover
  media: atmel: atmel-isc-base: use streaming status when queueing
    buffers
  media: atmel: atmel-isc-base: remove frameintervals VIDIOC
  media: atmel: atmel-isc-base: report frame sizes as full supported
    range
  media: atmel: atmel-isc-base: implement mbus_code support in enumfmt
  media: atmel: atmel-isc-base: fix bytesperline value for planar
    formats
  MAINTAINERS: atmel-isc: add new file atmel-isc-mc.c
  media: atmel: atmel-isc: implement media controller
  ARM: dts: at91: sama7g5: add nodes for video capture
  ARM: configs: at91: sama7: add xisc and csi2dc
  ARM: multi_v7_defconfig: add atmel video pipeline modules
  media: atmel: atmel-sama5d2-isc: fix wrong mask in YUYV format check
  media: atmel: atmel-isc-base: use mutex to lock awb workqueue from
    streaming
  media: atmel: atmel-isc-base: add wb debug messages
  media: atmel: atmel-isc-base: clamp wb gain coefficients

 .../bindings/media/microchip,csi2dc.yaml      | 149 ++++
 MAINTAINERS                                   |   9 +
 arch/arm/boot/dts/sama7g5.dtsi                |  49 ++
 arch/arm/configs/multi_v7_defconfig           |   3 +
 arch/arm/configs/sama7_defconfig              |   2 +
 drivers/media/platform/atmel/Kconfig          |  15 +
 drivers/media/platform/atmel/Makefile         |   3 +-
 drivers/media/platform/atmel/atmel-isc-base.c | 515 ++++---------
 drivers/media/platform/atmel/atmel-isc-clk.c  | 316 ++++++++
 drivers/media/platform/atmel/atmel-isc-mc.c   | 235 ++++++
 drivers/media/platform/atmel/atmel-isc.h      |  33 +
 .../media/platform/atmel/atmel-sama5d2-isc.c  |  16 +-
 .../media/platform/atmel/atmel-sama7g5-isc.c  |  18 +-
 .../media/platform/atmel/microchip-csi2dc.c   | 700 ++++++++++++++++++
 14 files changed, 1686 insertions(+), 377 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
 create mode 100644 drivers/media/platform/atmel/atmel-isc-clk.c
 create mode 100644 drivers/media/platform/atmel/atmel-isc-mc.c
 create mode 100644 drivers/media/platform/atmel/microchip-csi2dc.c

-- 
2.25.1

[PATCH 01/21] MAINTAINERS: add microchip csi2dc

[Eugen Hristev]

Add Microchip CSI2DC driver in the list.

Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
 MAINTAINERS | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 186cfab07504..dab338f424db 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -12252,6 +12252,13 @@ L:	alsa-devel@alsa-project.org (moderated for non-subscribers)
 S:	Supported
 F:	sound/soc/atmel
 
+MICROCHIP CSI2DC DRIVER
+M:	Eugen Hristev <eugen.hristev@microchip.com>
+L:	linux-media@vger.kernel.org
+S:	Supported
+F:	Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
+F:	drivers/media/platform/atmel/microchip-csi2dc.c
+
 MICROCHIP ECC DRIVER
 M:	Tudor Ambarus <tudor.ambarus@microchip.com>
 L:	linux-crypto@vger.kernel.org
-- 
2.25.1

[PATCH 02/21] dt-bindings: media: atmel: csi2dc: add bindings for microchip csi2dc

[Eugen Hristev]

Add bindings documentation for Microchip CSI2 Demultiplexer controller.

CSI2DC is a demultiplexer from Synopsys IDI interface specification to
parallel interface connection or direct memory access.

Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
Changes in this version :
- fixed 'sink' name to be actually source.
- added dma properties and example with dma

Previous change log:
Changes in v5:
- modified bindings as per Rob Herring review

Changes in v4:
- Removed property for inter-line-delay and for clock continuous/non-continuous
- Removed virtual channel by reg for second endpoint

Changes in v3:
- Removed some text from description, as it was explained in the schema
- fixed other things as per Rob's review
- moved some text inside the schema, like the clock description

Changes in v2:
- fixed warnings reported by dt_binding_check


 .../bindings/media/microchip,csi2dc.yaml      | 149 ++++++++++++++++++
 1 file changed, 149 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/microchip,csi2dc.yaml

diff --git a/Documentation/devicetree/bindings/media/microchip,csi2dc.yaml b/Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
new file mode 100644
index 000000000000..d317478908d0
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
@@ -0,0 +1,149 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/microchip,csi2dc.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Microchip CSI2 Demux Controller (CSI2DC)
+
+maintainers:
+  - Eugen Hristev <eugen.hristev@microchip.com>
+
+description:
+  CSI2DC - Camera Serial Interface 2 Demux Controller
+
+  CSI2DC is a hardware block that receives incoming data from an IDI interface
+  and filters packets based on their data type and virtual channel identifier,
+  then converts the byte stream into a cross clock domain to a pixel stream
+  to a parallel interface that can be read by a sensor controller.
+  IDI interface is Synopsys proprietary.
+
+  CSI2DC provides two pipes, one video pipe and one data pipe. Video pipe
+  is connected to a sensor controller and the data pipe is accessible
+  as a DMA slave port to a DMA controller.
+
+  CSI2DC supports a single 'port' node as a sink port with Synopsys 32-bit
+  IDI interface. The connected endpoint must be a IDI interface compatible
+  device , that can provide 32-bit IDI interface connection as source port.
+  For graph video endpoints please refer to the bindings defined in
+  Documentation/devicetree/bindings/media/video-interfaces.txt.
+  This port is mandatory.
+
+  CSI2DC supports one 'port' node as source port with parallel interface.
+  This is called video pipe.
+  This port has an 'endpoint' that can be connected to a sink port of another
+  controller (next in pipeline).
+  Please refer to the bindings defined in
+  Documentation/devicetree/bindings/media/video-interfaces.txt.
+
+  CSI2DC also supports direct access to the data through AHB, via DMA channel,
+  called data pipe.
+  Because of this, the source 'port' child node (second) is not mandatory.
+  If the source 'port' child node is missing, only data pipe is available.
+  For data pipe to be available, a dma controller must be referenced.
+
+properties:
+  compatible:
+    const: microchip,sama7g5-csi2dc
+
+  reg:
+    maxItems: 1
+
+  clocks:
+    maxItems: 2
+
+  clock-names:
+    description:
+      CSI2DC must have two clocks to function correctly. One clock is the
+      peripheral clock for the inside functionality of the hardware block.
+      This is named 'pclk'. The second clock must be the cross domain clock,
+      in which CSI2DC will perform clock crossing. This clock must be fed
+      by the next controller in pipeline, which usually is a sensor controller.
+      Normally this clock should be given by this sensor controller who
+      is also a clock source. This clock is named 'scck', sensor controller clock.
+    items:
+      - const: pclk
+      - const: scck
+
+  dmas:
+    maxItems: 1
+
+  dma-names:
+    const: rx
+
+  ports:
+    type: object
+    description:
+      List of ports
+
+    properties:
+      port@0:
+        type: object
+        description:
+          Input port node, single endpoint describing the input port.
+      port@1:
+        type: object
+        description:
+          Output port node, single endpoint, describing the output port.
+
+additionalProperties: false
+
+required:
+  - compatible
+  - reg
+  - clocks
+  - clock-names
+  - ports
+
+examples:
+  # Example for connecting to a parallel sensor controller block
+  - |
+    csi2dc@e1404000 {
+        compatible = "microchip,sama7g5-csi2dc";
+        reg = <0xe1404000 0x500>;
+        clocks = <&pclk>, <&scck>;
+        clock-names = "pclk", "scck";
+
+        ports {
+               #address-cells = <1>;
+               #size-cells = <0>;
+               port@0 {
+                       reg = <0>; /* must be 0, first child port */
+                       csi2dc_in: endpoint { /* input from IDI interface */
+                               remote-endpoint = <&csi2host_out>;
+                       };
+               };
+
+               port@1 {
+                       reg = <1>; /* must be 1, second child port */
+                       csi2dc_out: endpoint {
+                               remote-endpoint = <&xisc_in>; /* output to sensor controller */
+                       };
+               };
+        };
+    };
+
+  # Example for connecting to a DMA master as an AHB slave
+  - |
+    #include <dt-bindings/dma/at91.h>
+    csi2dc@e1404000 {
+        compatible = "microchip,sama7g5-csi2dc";
+        reg = <0xe1404000 0x500>;
+        clocks = <&pclk>, <&scck>;
+        clock-names = "pclk", "scck";
+        dmas = <&dma0 AT91_XDMAC_DT_PERID(34)>;
+        dma-names = "rx";
+
+        ports {
+               #address-cells = <1>;
+               #size-cells = <0>;
+               port@0 {
+                       reg = <0>; /* must be 0, first child port */
+                       csi2dc_input: endpoint { /* input from IDI interface */
+                               remote-endpoint = <&csi2host_out>;
+                       };
+               };
+        };
+    };
+
+...
-- 
2.25.1

[PATCH 03/21] media: atmel: introduce microchip csi2dc driver

[Eugen Hristev]

Microchip CSI2DC (CSI2 Demultiplexer Controller) is a misc bridge device
that converts a byte stream in IDI Synopsys format (coming from a CSI2HOST)
to a pixel stream that can be captured by a sensor controller.

Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
Changes in this revision:
- addressed comments by Jacopo and Laurent as in this thread:
https://www.spinics.net/lists/linux-media/msg181044.html

Previous change log :
Changes in v5:
- only in bindings

Changes in v4:
- now using get_mbus_config ops to get data from the subdevice, like the
virtual channel id, and the clock type.
- now having possibility to select any of the RAW10 data modes
- at completion time, select which formats are also available in the subdevice,
and move to the dynamic list accordingly
- changed the pipeline integration, do not advertise subdev ready at probe time.
wait until completion is done, and then start a workqueue that will register
this device as a subdevice for the next element in pipeline.
- moved the s_power code into a different function called now csi2dc_power
that is called with CONFIG_PM functions. This is also called at completion,
to have the device ready in case CONFIG_PM is not selected on the platform.
- merged try_fmt into set_fmt
- driver cleanup, wrapped lines over 80 characters

Changes in v2:
- moved driver to platform/atmel
- fixed minor things as per Sakari's review
- still some things from v2 review are not yet addressed, to be followed up


 drivers/media/platform/atmel/Kconfig          |  15 +
 drivers/media/platform/atmel/Makefile         |   1 +
 .../media/platform/atmel/microchip-csi2dc.c   | 700 ++++++++++++++++++
 3 files changed, 716 insertions(+)
 create mode 100644 drivers/media/platform/atmel/microchip-csi2dc.c

diff --git a/drivers/media/platform/atmel/Kconfig b/drivers/media/platform/atmel/Kconfig
index dda2f27da317..f83bee373d82 100644
--- a/drivers/media/platform/atmel/Kconfig
+++ b/drivers/media/platform/atmel/Kconfig
@@ -40,3 +40,18 @@ config VIDEO_ATMEL_ISI
 	help
 	  This module makes the ATMEL Image Sensor Interface available
 	  as a v4l2 device.
+
+config VIDEO_MICROCHIP_CSI2DC
+	tristate "Microchip CSI2 Demux Controller"
+	depends on VIDEO_V4L2 && COMMON_CLK && OF
+	depends on ARCH_AT91 || COMPILE_TEST
+	select MEDIA_CONTROLLER
+	select VIDEO_V4L2_SUBDEV_API
+	select V4L2_FWNODE
+	help
+	  CSI2 Demux Controller driver. CSI2DC is a helper chip
+	  that converts IDI interface byte stream to a parallel pixel stream.
+	  It supports various RAW formats as input.
+
+	  To compile this driver as a module, choose M here: the
+	  module will be called microchip-csi2dc.
diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
index 46d264ab7948..39f0a7eba702 100644
--- a/drivers/media/platform/atmel/Makefile
+++ b/drivers/media/platform/atmel/Makefile
@@ -6,3 +6,4 @@ obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
 obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o
 obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel-isc.o
 obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel-xisc.o
+obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += microchip-csi2dc.o
diff --git a/drivers/media/platform/atmel/microchip-csi2dc.c b/drivers/media/platform/atmel/microchip-csi2dc.c
new file mode 100644
index 000000000000..277b86988eee
--- /dev/null
+++ b/drivers/media/platform/atmel/microchip-csi2dc.c
@@ -0,0 +1,700 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Microchip CSI2 Demux Controller (CSI2DC) driver
+ *
+ * Copyright (C) 2018 Microchip Technology, Inc.
+ *
+ * Author: Eugen Hristev <eugen.hristev@microchip.com>
+ *
+ */
+
+#include <linux/clk.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/videodev2.h>
+
+#include <media/v4l2-device.h>
+#include <media/v4l2-fwnode.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-dma-contig.h>
+
+/* Global configuration register */
+#define CSI2DC_GCFG			0x0
+
+/* MIPI sensor pixel clock is free running */
+#define CSI2DC_GCFG_MIPIFRN		BIT(0)
+/* Output waveform inter-line minimum delay */
+#define CSI2DC_GCFG_HLC(v)		((v) << 4)
+#define CSI2DC_GCFG_HLC_MASK		GENMASK(7, 4)
+/* SAMA7G5 requires a HLC delay of 15 */
+#define SAMA7G5_HLC			(15)
+
+/* Global control register */
+#define CSI2DC_GCTLR			0x04
+#define CSI2DC_GCTLR_SWRST		BIT(0)
+
+/* Global status register */
+#define CSI2DC_GS			0x08
+
+/* SSP interrupt status register */
+#define CSI2DC_SSPIS			0x28
+/* Pipe update register */
+#define CSI2DC_PU			0xC0
+/* Video pipe attributes update */
+#define CSI2DC_PU_VP			BIT(0)
+
+/* Pipe update status register */
+#define CSI2DC_PUS			0xC4
+
+/* Video pipeline enable register */
+#define CSI2DC_VPE			0xF8
+#define CSI2DC_VPE_ENABLE		BIT(0)
+
+/* Video pipeline configuration register */
+#define CSI2DC_VPCFG			0xFC
+/* Data type */
+#define CSI2DC_VPCFG_DT(v)		((v) << 0)
+#define CSI2DC_VPCFG_DT_MASK		GENMASK(5, 0)
+/* Virtual channel identifier */
+#define CSI2DC_VPCFG_VC(v)		((v) << 6)
+#define CSI2DC_VPCFG_VC_MASK		GENMASK(7, 6)
+/* Decompression enable */
+#define CSI2DC_VPCFG_DE			BIT(8)
+/* Decoder mode */
+#define CSI2DC_VPCFG_DM(v)		((v) << 9)
+#define CSI2DC_VPCFG_DM_DECODER8TO12	0
+/* Decoder predictor 2 selection */
+#define CSI2DC_VPCFG_DP2		BIT(12)
+/* Recommended memory storage */
+#define CSI2DC_VPCFG_RMS		BIT(13)
+/* Post adjustment */
+#define CSI2DC_VPCFG_PA			BIT(14)
+
+/* Video pipeline column register */
+#define CSI2DC_VPCOL			0x100
+/* Column number */
+#define CSI2DC_VPCOL_COL(v)		((v) << 0)
+#define CSI2DC_VPCOL_COL_MASK		GENMASK(15, 0)
+
+/* Video pipeline row register */
+#define CSI2DC_VPROW			0x104
+/* Row number */
+#define CSI2DC_VPROW_ROW(v)		((v) << 0)
+#define CSI2DC_VPROW_ROW_MASK		GENMASK(15, 0)
+
+/* Version register */
+#define CSI2DC_VERSION			0x1FC
+
+/* register read/write helpers */
+#define csi2dc_readl(st, reg)		readl_relaxed((st)->base + (reg))
+#define csi2dc_writel(st, reg, val)	writel_relaxed((val), \
+					(st)->base + (reg))
+
+/* supported RAW data types */
+#define CSI2DC_DT_RAW6			0x28
+#define CSI2DC_DT_RAW7			0x29
+#define CSI2DC_DT_RAW8			0x2A
+#define CSI2DC_DT_RAW10			0x2B
+#define CSI2DC_DT_RAW12			0x2C
+#define CSI2DC_DT_RAW14			0x2D
+
+/*
+ * struct csi2dc_format - CSI2DC format type struct
+ * @mbus_code:		Media bus code for the format
+ * @dt:			Data type constant for this format
+ */
+struct csi2dc_format {
+	u32				mbus_code;
+	u32				dt;
+};
+
+static const struct csi2dc_format csi2dc_formats[] = {
+	{
+		.mbus_code =		MEDIA_BUS_FMT_SRGGB10_1X10,
+		.dt =			CSI2DC_DT_RAW10,
+	}, {
+		.mbus_code =		MEDIA_BUS_FMT_SBGGR10_1X10,
+		.dt =			CSI2DC_DT_RAW10,
+	}, {
+		.mbus_code =		MEDIA_BUS_FMT_SGRBG10_1X10,
+		.dt =			CSI2DC_DT_RAW10,
+	}, {
+		.mbus_code =		MEDIA_BUS_FMT_SGBRG10_1X10,
+		.dt =			CSI2DC_DT_RAW10,
+	},
+};
+
+enum mipi_csi_pads {
+	CSI2DC_PAD_SINK			= 0,
+	CSI2DC_PAD_SOURCE		= 1,
+	CSI2DC_PADS_NUM			= 2,
+};
+
+/*
+ * struct csi2dc_device - CSI2DC device driver data/config struct
+ * @base:		Register map base address
+ * @csi2dc_sd:		v4l2 subdevice for the csi2dc device
+ *			This is the subdevice that the csi2dc device itself
+ *			registers in v4l2 subsystem
+ * @dev:		struct device for this csi2dc device
+ * @pclk:		Peripheral clock reference
+ *			Input clock that clocks the hardware block internal
+ *			logic
+ * @scck:		Sensor Controller clock reference
+ *			Input clock that is used to generate the pixel clock
+ * @format:		Current saved format used in g/s fmt
+ * @cur_fmt:		Current state format
+ * @try_fmt:		Try format that is being tried
+ * @pads:		Media entity pads for the csi2dc subdevice
+ * @clk_gated:		Whether the clock is gated or free running
+ * @video_pipe:		Whether video pipeline is configured
+ * @vc:			Current set virtual channel
+ * @asd:		Async subdevice for async bound of the underlying subdev
+ * @notifier:		Async notifier that is used to bound the underlying
+ *			subdevice to the csi2dc subdevice
+ * @input_sd:		Reference to the underlying subdevice bound to the
+ *			csi2dc subdevice
+ * @remote_pad:		Pad number of the underlying subdevice that is linked
+ *			to the csi2dc subdevice sink pad.
+ */
+struct csi2dc_device {
+	void __iomem			*base;
+	struct v4l2_subdev		csi2dc_sd;
+	struct device			*dev;
+	struct clk			*pclk;
+	struct clk			*scck;
+
+	struct v4l2_mbus_framefmt	 format;
+
+	const struct csi2dc_format	*cur_fmt;
+	const struct csi2dc_format	*try_fmt;
+
+	struct media_pad		pads[CSI2DC_PADS_NUM];
+
+	bool				clk_gated;
+	bool				video_pipe;
+	u32				vc;
+
+	struct v4l2_async_subdev	*asd;
+	struct v4l2_async_notifier	notifier;
+
+	struct v4l2_subdev		*input_sd;
+
+	u32				remote_pad;
+};
+
+static void csi2dc_vp_update(struct csi2dc_device *csi2dc)
+{
+	u32 vp;
+
+	if (!csi2dc->cur_fmt) {
+		dev_err(csi2dc->dev, "format must be configured first\n");
+		return;
+	}
+
+	if (!csi2dc->video_pipe) {
+		dev_err(csi2dc->dev, "video pipeline unavailable\n");
+		return;
+	}
+
+	vp = CSI2DC_VPCFG_DT(csi2dc->cur_fmt->dt) & CSI2DC_VPCFG_DT_MASK;
+	vp |= CSI2DC_VPCFG_VC(csi2dc->vc) & CSI2DC_VPCFG_VC_MASK;
+	vp &= ~CSI2DC_VPCFG_DE;
+	vp |= CSI2DC_VPCFG_DM(CSI2DC_VPCFG_DM_DECODER8TO12);
+	vp &= ~CSI2DC_VPCFG_DP2;
+	vp &= ~CSI2DC_VPCFG_RMS;
+	vp |= CSI2DC_VPCFG_PA;
+
+	csi2dc_writel(csi2dc, CSI2DC_VPCFG, vp);
+	csi2dc_writel(csi2dc, CSI2DC_VPE, CSI2DC_VPE_ENABLE);
+	csi2dc_writel(csi2dc, CSI2DC_PU, CSI2DC_PU_VP);
+}
+
+static inline struct csi2dc_device *
+csi2dc_sd_to_csi2dc_device(struct v4l2_subdev *csi2dc_sd)
+{
+	return container_of(csi2dc_sd, struct csi2dc_device, csi2dc_sd);
+}
+
+static int csi2dc_enum_mbus_code(struct v4l2_subdev *csi2dc_sd,
+				 struct v4l2_subdev_state *sd_state,
+				 struct v4l2_subdev_mbus_code_enum *code)
+{
+	if (code->index >= ARRAY_SIZE(csi2dc_formats))
+		return -EINVAL;
+
+	code->code = csi2dc_formats[code->index].mbus_code;
+
+	return 0;
+}
+
+static int csi2dc_get_fmt(struct v4l2_subdev *csi2dc_sd,
+			  struct v4l2_subdev_state *sd_state,
+			  struct v4l2_subdev_format *format)
+{
+	struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
+
+	format->format = csi2dc->format;
+
+	return 0;
+}
+
+static int csi2dc_set_fmt(struct v4l2_subdev *csi2dc_sd,
+			  struct v4l2_subdev_state *sd_state,
+			  struct v4l2_subdev_format *req_fmt)
+{
+	struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
+	const struct csi2dc_format *fmt;
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(csi2dc_formats);  i++) {
+		fmt = &csi2dc_formats[i];
+		if (req_fmt->format.code == fmt->mbus_code)
+			csi2dc->try_fmt = fmt;
+		fmt++;
+	}
+
+	/* in case we could not find the desired format, default to something */
+	if (!csi2dc->try_fmt  ||
+	    req_fmt->format.code != csi2dc->try_fmt->mbus_code) {
+		csi2dc->try_fmt = &csi2dc_formats[0];
+
+		dev_dbg(csi2dc->dev,
+			"CSI2DC unsupported format 0x%x, defaulting to 0x%x\n",
+			req_fmt->format.code, csi2dc_formats[0].mbus_code);
+
+		req_fmt->format.code = csi2dc_formats[0].mbus_code;
+	}
+
+	req_fmt->format.colorspace = V4L2_COLORSPACE_SRGB;
+	req_fmt->format.field = V4L2_FIELD_NONE;
+
+	/* save the format for later requests */
+	csi2dc->format = req_fmt->format;
+
+	/* if we are just trying, we are done */
+	if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY)
+		return 0;
+
+	csi2dc->cur_fmt = csi2dc->try_fmt;
+
+	dev_dbg(csi2dc->dev, "new format set: 0x%x\n", req_fmt->format.code);
+
+	return 0;
+}
+
+static int csi2dc_power(struct csi2dc_device *csi2dc, int on)
+{
+	int ret = 0;
+
+	if (on) {
+		ret = clk_prepare_enable(csi2dc->pclk);
+		if (ret) {
+			dev_err(csi2dc->dev, "failed to enable pclk: %d\n", ret);
+			return ret;
+		}
+
+		ret = clk_prepare_enable(csi2dc->scck);
+		if (ret)
+			dev_err(csi2dc->dev,
+				"failed to enable scck: %d\n", ret);
+
+		/* if powering up, deassert reset line */
+		csi2dc_writel(csi2dc, CSI2DC_GCTLR, CSI2DC_GCTLR_SWRST);
+	} else {
+		clk_disable_unprepare(csi2dc->scck);
+
+		/* if powering down, assert reset line */
+		csi2dc_writel(csi2dc, CSI2DC_GCTLR, !CSI2DC_GCTLR_SWRST);
+
+		clk_disable_unprepare(csi2dc->pclk);
+	}
+
+	return ret;
+}
+
+static int csi2dc_s_stream(struct v4l2_subdev *csi2dc_sd, int enable)
+{
+	struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
+	int ret;
+
+	if (enable) {
+		ret = pm_runtime_resume_and_get(csi2dc->dev);
+		if (ret < 0)
+			return ret;
+		csi2dc_vp_update(csi2dc);
+	} else {
+		pm_runtime_put_sync(csi2dc->dev);
+	}
+
+	return v4l2_subdev_call(csi2dc->input_sd, video, s_stream, enable);
+}
+
+static const struct v4l2_subdev_pad_ops csi2dc_pad_ops = {
+	.enum_mbus_code = csi2dc_enum_mbus_code,
+	.set_fmt = csi2dc_set_fmt,
+	.get_fmt = csi2dc_get_fmt,
+};
+
+static const struct v4l2_subdev_video_ops csi2dc_video_ops = {
+	.s_stream = csi2dc_s_stream,
+};
+
+static const struct v4l2_subdev_ops csi2dc_subdev_ops = {
+	.pad = &csi2dc_pad_ops,
+	.video = &csi2dc_video_ops,
+};
+
+static int csi2dc_get_mbus_config(struct csi2dc_device *csi2dc)
+{
+	struct v4l2_mbus_config mbus_config = { 0 };
+	int ret;
+
+	ret = v4l2_subdev_call(csi2dc->input_sd, pad, get_mbus_config,
+			       csi2dc->remote_pad, &mbus_config);
+	if (ret == -ENOIOCTLCMD) {
+		dev_dbg(csi2dc->dev,
+			"no remote mbus configuration available\n");
+		goto csi2dc_get_mbus_config_defaults;
+	}
+
+	if (ret) {
+		dev_err(csi2dc->dev,
+			"failed to get remote mbus configuration\n");
+		goto csi2dc_get_mbus_config_defaults;
+	}
+
+	if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_0)
+		csi2dc->vc = 0;
+	else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_1)
+		csi2dc->vc = 1;
+	else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_2)
+		csi2dc->vc = 2;
+	else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_3)
+		csi2dc->vc = 3;
+
+	dev_dbg(csi2dc->dev, "subdev sending on channel %d\n", csi2dc->vc);
+
+	csi2dc->clk_gated = mbus_config.flags &
+				V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK;
+
+	dev_dbg(csi2dc->dev, "%s clock\n",
+		csi2dc->clk_gated ? "gated" : "free running");
+
+	return 0;
+
+csi2dc_get_mbus_config_defaults:
+	csi2dc->vc = 0; /* Virtual ID 0 by default */
+	csi2dc->clk_gated = false; /* Free running clock by default */
+
+	return 0;
+}
+
+static int csi2dc_async_bound(struct v4l2_async_notifier *notifier,
+			      struct v4l2_subdev *subdev,
+			      struct v4l2_async_subdev *asd)
+{
+	struct csi2dc_device *csi2dc = container_of(notifier,
+						struct csi2dc_device, notifier);
+	int pad;
+	int ret;
+
+	csi2dc->input_sd = subdev;
+
+	pad = media_entity_get_fwnode_pad(&subdev->entity,
+					  asd->match.fwnode,
+					  MEDIA_PAD_FL_SOURCE);
+	if (pad < 0) {
+		dev_err(csi2dc->dev, "Failed to find pad for %s\n",
+			subdev->name);
+		return pad;
+	}
+
+	csi2dc->remote_pad = pad;
+
+	csi2dc_get_mbus_config(csi2dc);
+
+	ret = media_create_pad_link(&csi2dc->input_sd->entity,
+				    csi2dc->remote_pad,
+				    &csi2dc->csi2dc_sd.entity, 0,
+				    MEDIA_LNK_FL_ENABLED);
+	if (ret < 0) {
+		dev_err(csi2dc->dev,
+			"Failed to create pad link: %s to %s\n",
+			csi2dc->input_sd->entity.name,
+			csi2dc->csi2dc_sd.entity.name);
+		return ret;
+	}
+
+	dev_dbg(csi2dc->dev, "link with %s pad: %d\n",
+		csi2dc->input_sd->name, csi2dc->remote_pad);
+
+	ret = pm_runtime_resume_and_get(csi2dc->dev);
+	if (ret < 0)
+		return ret;
+
+	csi2dc_writel(csi2dc, CSI2DC_GCFG,
+		      (SAMA7G5_HLC & CSI2DC_GCFG_HLC_MASK) |
+		      (csi2dc->clk_gated ? 0 : CSI2DC_GCFG_MIPIFRN));
+
+	csi2dc_writel(csi2dc, CSI2DC_VPCOL,
+		      CSI2DC_VPCOL_COL(0xFFF) & CSI2DC_VPCOL_COL_MASK);
+	csi2dc_writel(csi2dc, CSI2DC_VPROW,
+		      CSI2DC_VPROW_ROW(0xFFF) & CSI2DC_VPROW_ROW_MASK);
+
+	pm_runtime_put_sync(csi2dc->dev);
+
+	return ret;
+}
+
+static const struct v4l2_async_notifier_operations csi2dc_async_ops = {
+	.bound = csi2dc_async_bound,
+};
+
+static void csi2dc_cleanup_notifier(struct csi2dc_device *csi2dc)
+{
+	v4l2_async_notifier_unregister(&csi2dc->notifier);
+	v4l2_async_notifier_cleanup(&csi2dc->notifier);
+}
+
+static int csi2dc_prepare_notifier(struct csi2dc_device *csi2dc,
+				   struct device_node *input_node)
+{
+	int ret = 0;
+
+	v4l2_async_notifier_init(&csi2dc->notifier);
+
+	csi2dc->asd = v4l2_async_notifier_add_fwnode_remote_subdev
+			(&csi2dc->notifier, of_fwnode_handle(input_node),
+			struct v4l2_async_subdev);
+
+	of_node_put(input_node);
+
+	if (IS_ERR(csi2dc->asd)) {
+		ret = PTR_ERR(csi2dc->asd);
+		dev_err(csi2dc->dev,
+			"failed to add async notifier for node %pOF: %d\n",
+			input_node, ret);
+		v4l2_async_notifier_cleanup(&csi2dc->notifier);
+		return ret;
+	}
+
+	csi2dc->notifier.ops = &csi2dc_async_ops;
+
+	ret = v4l2_async_subdev_notifier_register(&csi2dc->csi2dc_sd,
+						  &csi2dc->notifier);
+
+	if (ret) {
+		dev_err(csi2dc->dev, "fail to register async notifier: %d\n",
+			ret);
+		v4l2_async_notifier_cleanup(&csi2dc->notifier);
+	}
+
+	return ret;
+}
+
+static int csi2dc_of_parse(struct csi2dc_device *csi2dc,
+			   struct device_node *of_node)
+{
+	struct device_node *input_node, *output_node;
+	struct v4l2_fwnode_endpoint input_endpoint = { 0 },
+				    output_endpoint = { 0 };
+	int ret;
+
+	output_endpoint.bus_type = V4L2_MBUS_PARALLEL;
+
+	input_node = of_graph_get_next_endpoint(of_node, NULL);
+
+	if (!input_node) {
+		dev_err(csi2dc->dev,
+			"missing port node at %pOF, input node is mandatory.\n",
+			of_node);
+		return -EINVAL;
+	}
+
+	ret = v4l2_fwnode_endpoint_parse(of_fwnode_handle(input_node),
+					 &input_endpoint);
+
+	if (ret) {
+		dev_err(csi2dc->dev, "endpoint not defined at %pOF\n", of_node);
+		return ret;
+	}
+
+	output_node = of_graph_get_next_endpoint(of_node, input_node);
+
+	if (output_node)
+		ret = v4l2_fwnode_endpoint_parse(of_fwnode_handle(output_node),
+						 &output_endpoint);
+
+	if (!output_node || ret) {
+		dev_info(csi2dc->dev,
+			 "missing output node at %pOF, data pipe available only.\n",
+			 of_node);
+	} else {
+		csi2dc->video_pipe = true;
+
+		dev_dbg(csi2dc->dev, "block %pOF %d.%d->%d.%d video pipeline\n",
+			of_node, input_endpoint.base.port,
+			input_endpoint.base.id, output_endpoint.base.port,
+			output_endpoint.base.id);
+	}
+
+	of_node_put(output_node);
+	of_node_put(input_node);
+
+	/* prepare async notifier for subdevice completion */
+	return csi2dc_prepare_notifier(csi2dc, input_node);
+}
+
+static int csi2dc_probe(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct csi2dc_device *csi2dc;
+	struct resource *res = NULL;
+	int ret = 0;
+	u32 ver;
+
+	csi2dc = devm_kzalloc(dev, sizeof(*csi2dc), GFP_KERNEL);
+	if (!csi2dc)
+		return -ENOMEM;
+
+	csi2dc->dev = dev;
+
+	res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+
+	csi2dc->base = devm_ioremap_resource(dev, res);
+	if (IS_ERR(csi2dc->base)) {
+		dev_err(dev, "base address not set\n");
+		return PTR_ERR(csi2dc->base);
+	}
+
+	csi2dc->pclk = devm_clk_get(dev, "pclk");
+	if (IS_ERR(csi2dc->pclk)) {
+		ret = PTR_ERR(csi2dc->pclk);
+		dev_err(dev, "failed to get pclk: %d\n", ret);
+		return ret;
+	}
+
+	csi2dc->scck = devm_clk_get(dev, "scck");
+	if (IS_ERR(csi2dc->scck)) {
+		ret = PTR_ERR(csi2dc->scck);
+		dev_err(dev, "failed to get scck: %d\n", ret);
+		return ret;
+	}
+
+	v4l2_subdev_init(&csi2dc->csi2dc_sd, &csi2dc_subdev_ops);
+
+	csi2dc->csi2dc_sd.owner = THIS_MODULE;
+	csi2dc->csi2dc_sd.dev = dev;
+	snprintf(csi2dc->csi2dc_sd.name, sizeof(csi2dc->csi2dc_sd.name),
+		 "csi2dc");
+
+	csi2dc->csi2dc_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+	csi2dc->csi2dc_sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
+	v4l2_set_subdevdata(&csi2dc->csi2dc_sd, pdev);
+
+	platform_set_drvdata(pdev, csi2dc);
+
+	ret = csi2dc_of_parse(csi2dc, dev->of_node);
+	if (ret)
+		goto csi2dc_probe_cleanup_entity;
+
+	csi2dc->pads[CSI2DC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+	if (csi2dc->video_pipe)
+		csi2dc->pads[CSI2DC_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
+
+	ret = media_entity_pads_init(&csi2dc->csi2dc_sd.entity,
+				     csi2dc->video_pipe ? CSI2DC_PADS_NUM : 1,
+				     csi2dc->pads);
+	if (ret < 0) {
+		dev_err(dev, "media entity init failed\n");
+		goto csi2dc_probe_cleanup_entity;
+	}
+
+	/* turn power on to validate capabilities */
+	ret = csi2dc_power(csi2dc, true);
+	if (ret < 0)
+		goto csi2dc_probe_cleanup_entity;
+
+	pm_runtime_set_active(dev);
+	pm_runtime_enable(dev);
+	ver = csi2dc_readl(csi2dc, CSI2DC_VERSION);
+	pm_request_idle(dev);
+
+	/*
+	 * we must register the subdev after PM runtime has been requested,
+	 * otherwise we might bound immediately and request pm_runtime_resume
+	 * before runtime_enable.
+	 */
+	ret = v4l2_async_register_subdev(&csi2dc->csi2dc_sd);
+	if (ret) {
+		dev_err(csi2dc->dev, "failed to register the subdevice\n");
+		goto csi2dc_probe_cleanup_entity;
+	}
+
+	dev_info(dev, "Microchip CSI2DC version %x\n", ver);
+
+	return 0;
+
+csi2dc_probe_cleanup_entity:
+	media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
+
+	return ret;
+}
+
+static int csi2dc_remove(struct platform_device *pdev)
+{
+	struct csi2dc_device *csi2dc = platform_get_drvdata(pdev);
+
+	pm_runtime_disable(&pdev->dev);
+
+	v4l2_async_unregister_subdev(&csi2dc->csi2dc_sd);
+	csi2dc_cleanup_notifier(csi2dc);
+	media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
+
+	return 0;
+}
+
+static int __maybe_unused csi2dc_runtime_suspend(struct device *dev)
+{
+	struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
+
+	return csi2dc_power(csi2dc, false);
+}
+
+static int __maybe_unused csi2dc_runtime_resume(struct device *dev)
+{
+	struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
+
+	return csi2dc_power(csi2dc, true);
+}
+
+static const struct dev_pm_ops csi2dc_dev_pm_ops = {
+	SET_RUNTIME_PM_OPS(csi2dc_runtime_suspend, csi2dc_runtime_resume, NULL)
+};
+
+static const struct of_device_id csi2dc_of_match[] = {
+	{ .compatible = "microchip,sama7g5-csi2dc" },
+	{ }
+};
+
+MODULE_DEVICE_TABLE(of, csi2dc_of_match);
+
+static struct platform_driver csi2dc_driver = {
+	.probe	= csi2dc_probe,
+	.remove = csi2dc_remove,
+	.driver = {
+		.name =			"microchip-csi2dc",
+		.pm =			&csi2dc_dev_pm_ops,
+		.of_match_table =	of_match_ptr(csi2dc_of_match),
+	},
+};
+
+module_platform_driver(csi2dc_driver);
+
+MODULE_AUTHOR("Eugen Hristev <eugen.hristev@microchip.com>");
+MODULE_DESCRIPTION("Microchip CSI2 Demux Controller driver");
+MODULE_LICENSE("GPL v2");
-- 
2.25.1

[PATCH 04/21] MAINTAINERS: atmel-isc: add new file atmel-isc-clk.c

[Eugen Hristev]

Add new file atmel-isc-clk.c to MAINTAINERS entry.

Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index dab338f424db..e151716c8d1e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -12279,6 +12279,7 @@ S:	Supported
 F:	Documentation/devicetree/bindings/media/atmel,isc.yaml
 F:	Documentation/devicetree/bindings/media/microchip,xisc.yaml
 F:	drivers/media/platform/atmel/atmel-isc-base.c
+F:	drivers/media/platform/atmel/atmel-isc-clk.c
 F:	drivers/media/platform/atmel/atmel-isc-regs.h
 F:	drivers/media/platform/atmel/atmel-isc.h
 F:	drivers/media/platform/atmel/atmel-sama5d2-isc.c
-- 
2.25.1

[PATCH 05/21] media: atmel: atmel-isc: split the clock code into separate source file

[Eugen Hristev]

The atmel-isc-base is getting crowded. Split the clock functions into
atmel-isc-clk.c.

Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
 drivers/media/platform/atmel/Makefile         |   2 +-
 drivers/media/platform/atmel/atmel-isc-base.c | 294 ----------------
 drivers/media/platform/atmel/atmel-isc-clk.c  | 316 ++++++++++++++++++
 3 files changed, 317 insertions(+), 295 deletions(-)
 create mode 100644 drivers/media/platform/atmel/atmel-isc-clk.c

diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
index 39f0a7eba702..1f6fe7427769 100644
--- a/drivers/media/platform/atmel/Makefile
+++ b/drivers/media/platform/atmel/Makefile
@@ -3,7 +3,7 @@ atmel-isc-objs = atmel-sama5d2-isc.o
 atmel-xisc-objs = atmel-sama7g5-isc.o
 
 obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
-obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o
+obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o atmel-isc-clk.o
 obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel-isc.o
 obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel-xisc.o
 obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += microchip-csi2dc.o
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index ebf264b980f9..f532fd03e807 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -8,9 +8,6 @@
  * Author: Eugen Hristev <eugen.hristev@microchip.com>
  *
  */
-
-#include <linux/clk.h>
-#include <linux/clkdev.h>
 #include <linux/clk-provider.h>
 #include <linux/delay.h>
 #include <linux/interrupt.h>
@@ -100,297 +97,6 @@ static inline void isc_reset_awb_ctrls(struct isc_device *isc)
 	}
 }
 
-static int isc_wait_clk_stable(struct clk_hw *hw)
-{
-	struct isc_clk *isc_clk = to_isc_clk(hw);
-	struct regmap *regmap = isc_clk->regmap;
-	unsigned long timeout = jiffies + usecs_to_jiffies(1000);
-	unsigned int status;
-
-	while (time_before(jiffies, timeout)) {
-		regmap_read(regmap, ISC_CLKSR, &status);
-		if (!(status & ISC_CLKSR_SIP))
-			return 0;
-
-		usleep_range(10, 250);
-	}
-
-	return -ETIMEDOUT;
-}
-
-static int isc_clk_prepare(struct clk_hw *hw)
-{
-	struct isc_clk *isc_clk = to_isc_clk(hw);
-	int ret;
-
-	ret = pm_runtime_resume_and_get(isc_clk->dev);
-	if (ret < 0)
-		return ret;
-
-	return isc_wait_clk_stable(hw);
-}
-
-static void isc_clk_unprepare(struct clk_hw *hw)
-{
-	struct isc_clk *isc_clk = to_isc_clk(hw);
-
-	isc_wait_clk_stable(hw);
-
-	pm_runtime_put_sync(isc_clk->dev);
-}
-
-static int isc_clk_enable(struct clk_hw *hw)
-{
-	struct isc_clk *isc_clk = to_isc_clk(hw);
-	u32 id = isc_clk->id;
-	struct regmap *regmap = isc_clk->regmap;
-	unsigned long flags;
-	unsigned int status;
-
-	dev_dbg(isc_clk->dev, "ISC CLK: %s, id = %d, div = %d, parent id = %d\n",
-		__func__, id, isc_clk->div, isc_clk->parent_id);
-
-	spin_lock_irqsave(&isc_clk->lock, flags);
-	regmap_update_bits(regmap, ISC_CLKCFG,
-			   ISC_CLKCFG_DIV_MASK(id) | ISC_CLKCFG_SEL_MASK(id),
-			   (isc_clk->div << ISC_CLKCFG_DIV_SHIFT(id)) |
-			   (isc_clk->parent_id << ISC_CLKCFG_SEL_SHIFT(id)));
-
-	regmap_write(regmap, ISC_CLKEN, ISC_CLK(id));
-	spin_unlock_irqrestore(&isc_clk->lock, flags);
-
-	regmap_read(regmap, ISC_CLKSR, &status);
-	if (status & ISC_CLK(id))
-		return 0;
-	else
-		return -EINVAL;
-}
-
-static void isc_clk_disable(struct clk_hw *hw)
-{
-	struct isc_clk *isc_clk = to_isc_clk(hw);
-	u32 id = isc_clk->id;
-	unsigned long flags;
-
-	spin_lock_irqsave(&isc_clk->lock, flags);
-	regmap_write(isc_clk->regmap, ISC_CLKDIS, ISC_CLK(id));
-	spin_unlock_irqrestore(&isc_clk->lock, flags);
-}
-
-static int isc_clk_is_enabled(struct clk_hw *hw)
-{
-	struct isc_clk *isc_clk = to_isc_clk(hw);
-	u32 status;
-	int ret;
-
-	ret = pm_runtime_resume_and_get(isc_clk->dev);
-	if (ret < 0)
-		return 0;
-
-	regmap_read(isc_clk->regmap, ISC_CLKSR, &status);
-
-	pm_runtime_put_sync(isc_clk->dev);
-
-	return status & ISC_CLK(isc_clk->id) ? 1 : 0;
-}
-
-static unsigned long
-isc_clk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
-{
-	struct isc_clk *isc_clk = to_isc_clk(hw);
-
-	return DIV_ROUND_CLOSEST(parent_rate, isc_clk->div + 1);
-}
-
-static int isc_clk_determine_rate(struct clk_hw *hw,
-				   struct clk_rate_request *req)
-{
-	struct isc_clk *isc_clk = to_isc_clk(hw);
-	long best_rate = -EINVAL;
-	int best_diff = -1;
-	unsigned int i, div;
-
-	for (i = 0; i < clk_hw_get_num_parents(hw); i++) {
-		struct clk_hw *parent;
-		unsigned long parent_rate;
-
-		parent = clk_hw_get_parent_by_index(hw, i);
-		if (!parent)
-			continue;
-
-		parent_rate = clk_hw_get_rate(parent);
-		if (!parent_rate)
-			continue;
-
-		for (div = 1; div < ISC_CLK_MAX_DIV + 2; div++) {
-			unsigned long rate;
-			int diff;
-
-			rate = DIV_ROUND_CLOSEST(parent_rate, div);
-			diff = abs(req->rate - rate);
-
-			if (best_diff < 0 || best_diff > diff) {
-				best_rate = rate;
-				best_diff = diff;
-				req->best_parent_rate = parent_rate;
-				req->best_parent_hw = parent;
-			}
-
-			if (!best_diff || rate < req->rate)
-				break;
-		}
-
-		if (!best_diff)
-			break;
-	}
-
-	dev_dbg(isc_clk->dev,
-		"ISC CLK: %s, best_rate = %ld, parent clk: %s @ %ld\n",
-		__func__, best_rate,
-		__clk_get_name((req->best_parent_hw)->clk),
-		req->best_parent_rate);
-
-	if (best_rate < 0)
-		return best_rate;
-
-	req->rate = best_rate;
-
-	return 0;
-}
-
-static int isc_clk_set_parent(struct clk_hw *hw, u8 index)
-{
-	struct isc_clk *isc_clk = to_isc_clk(hw);
-
-	if (index >= clk_hw_get_num_parents(hw))
-		return -EINVAL;
-
-	isc_clk->parent_id = index;
-
-	return 0;
-}
-
-static u8 isc_clk_get_parent(struct clk_hw *hw)
-{
-	struct isc_clk *isc_clk = to_isc_clk(hw);
-
-	return isc_clk->parent_id;
-}
-
-static int isc_clk_set_rate(struct clk_hw *hw,
-			     unsigned long rate,
-			     unsigned long parent_rate)
-{
-	struct isc_clk *isc_clk = to_isc_clk(hw);
-	u32 div;
-
-	if (!rate)
-		return -EINVAL;
-
-	div = DIV_ROUND_CLOSEST(parent_rate, rate);
-	if (div > (ISC_CLK_MAX_DIV + 1) || !div)
-		return -EINVAL;
-
-	isc_clk->div = div - 1;
-
-	return 0;
-}
-
-static const struct clk_ops isc_clk_ops = {
-	.prepare	= isc_clk_prepare,
-	.unprepare	= isc_clk_unprepare,
-	.enable		= isc_clk_enable,
-	.disable	= isc_clk_disable,
-	.is_enabled	= isc_clk_is_enabled,
-	.recalc_rate	= isc_clk_recalc_rate,
-	.determine_rate	= isc_clk_determine_rate,
-	.set_parent	= isc_clk_set_parent,
-	.get_parent	= isc_clk_get_parent,
-	.set_rate	= isc_clk_set_rate,
-};
-
-static int isc_clk_register(struct isc_device *isc, unsigned int id)
-{
-	struct regmap *regmap = isc->regmap;
-	struct device_node *np = isc->dev->of_node;
-	struct isc_clk *isc_clk;
-	struct clk_init_data init;
-	const char *clk_name = np->name;
-	const char *parent_names[3];
-	int num_parents;
-
-	if (id == ISC_ISPCK && !isc->ispck_required)
-		return 0;
-
-	num_parents = of_clk_get_parent_count(np);
-	if (num_parents < 1 || num_parents > 3)
-		return -EINVAL;
-
-	if (num_parents > 2 && id == ISC_ISPCK)
-		num_parents = 2;
-
-	of_clk_parent_fill(np, parent_names, num_parents);
-
-	if (id == ISC_MCK)
-		of_property_read_string(np, "clock-output-names", &clk_name);
-	else
-		clk_name = "isc-ispck";
-
-	init.parent_names	= parent_names;
-	init.num_parents	= num_parents;
-	init.name		= clk_name;
-	init.ops		= &isc_clk_ops;
-	init.flags		= CLK_SET_RATE_GATE | CLK_SET_PARENT_GATE;
-
-	isc_clk = &isc->isc_clks[id];
-	isc_clk->hw.init	= &init;
-	isc_clk->regmap		= regmap;
-	isc_clk->id		= id;
-	isc_clk->dev		= isc->dev;
-	spin_lock_init(&isc_clk->lock);
-
-	isc_clk->clk = clk_register(isc->dev, &isc_clk->hw);
-	if (IS_ERR(isc_clk->clk)) {
-		dev_err(isc->dev, "%s: clock register fail\n", clk_name);
-		return PTR_ERR(isc_clk->clk);
-	} else if (id == ISC_MCK)
-		of_clk_add_provider(np, of_clk_src_simple_get, isc_clk->clk);
-
-	return 0;
-}
-
-int isc_clk_init(struct isc_device *isc)
-{
-	unsigned int i;
-	int ret;
-
-	for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++)
-		isc->isc_clks[i].clk = ERR_PTR(-EINVAL);
-
-	for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
-		ret = isc_clk_register(isc, i);
-		if (ret)
-			return ret;
-	}
-
-	return 0;
-}
-EXPORT_SYMBOL_GPL(isc_clk_init);
-
-void isc_clk_cleanup(struct isc_device *isc)
-{
-	unsigned int i;
-
-	of_clk_del_provider(isc->dev->of_node);
-
-	for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
-		struct isc_clk *isc_clk = &isc->isc_clks[i];
-
-		if (!IS_ERR(isc_clk->clk))
-			clk_unregister(isc_clk->clk);
-	}
-}
-EXPORT_SYMBOL_GPL(isc_clk_cleanup);
 
 static int isc_queue_setup(struct vb2_queue *vq,
 			    unsigned int *nbuffers, unsigned int *nplanes,
diff --git a/drivers/media/platform/atmel/atmel-isc-clk.c b/drivers/media/platform/atmel/atmel-isc-clk.c
new file mode 100644
index 000000000000..d650caade396
--- /dev/null
+++ b/drivers/media/platform/atmel/atmel-isc-clk.c
@@ -0,0 +1,316 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Microchip Image Sensor Controller (ISC) common clock driver setup
+ *
+ * Copyright (C) 2016-2019 Microchip Technology, Inc.
+ *
+ * Author: Songjun Wu
+ * Author: Eugen Hristev <eugen.hristev@microchip.com>
+ *
+ */
+#include <linux/clk.h>
+#include <linux/clkdev.h>
+#include <linux/clk-provider.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/regmap.h>
+
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "atmel-isc-regs.h"
+#include "atmel-isc.h"
+
+static int isc_wait_clk_stable(struct clk_hw *hw)
+{
+	struct isc_clk *isc_clk = to_isc_clk(hw);
+	struct regmap *regmap = isc_clk->regmap;
+	unsigned long timeout = jiffies + usecs_to_jiffies(1000);
+	unsigned int status;
+
+	while (time_before(jiffies, timeout)) {
+		regmap_read(regmap, ISC_CLKSR, &status);
+		if (!(status & ISC_CLKSR_SIP))
+			return 0;
+
+		usleep_range(10, 250);
+	}
+
+	return -ETIMEDOUT;
+}
+
+static int isc_clk_prepare(struct clk_hw *hw)
+{
+	struct isc_clk *isc_clk = to_isc_clk(hw);
+	int ret;
+
+	ret = pm_runtime_resume_and_get(isc_clk->dev);
+	if (ret < 0)
+		return ret;
+
+	return isc_wait_clk_stable(hw);
+}
+
+static void isc_clk_unprepare(struct clk_hw *hw)
+{
+	struct isc_clk *isc_clk = to_isc_clk(hw);
+
+	isc_wait_clk_stable(hw);
+
+	pm_runtime_put_sync(isc_clk->dev);
+}
+
+static int isc_clk_enable(struct clk_hw *hw)
+{
+	struct isc_clk *isc_clk = to_isc_clk(hw);
+	u32 id = isc_clk->id;
+	struct regmap *regmap = isc_clk->regmap;
+	unsigned long flags;
+	unsigned int status;
+
+	dev_dbg(isc_clk->dev, "ISC CLK: %s, id = %d, div = %d, parent id = %d\n",
+		__func__, id, isc_clk->div, isc_clk->parent_id);
+
+	spin_lock_irqsave(&isc_clk->lock, flags);
+	regmap_update_bits(regmap, ISC_CLKCFG,
+			   ISC_CLKCFG_DIV_MASK(id) | ISC_CLKCFG_SEL_MASK(id),
+			   (isc_clk->div << ISC_CLKCFG_DIV_SHIFT(id)) |
+			   (isc_clk->parent_id << ISC_CLKCFG_SEL_SHIFT(id)));
+
+	regmap_write(regmap, ISC_CLKEN, ISC_CLK(id));
+	spin_unlock_irqrestore(&isc_clk->lock, flags);
+
+	regmap_read(regmap, ISC_CLKSR, &status);
+	if (status & ISC_CLK(id))
+		return 0;
+	else
+		return -EINVAL;
+}
+
+static void isc_clk_disable(struct clk_hw *hw)
+{
+	struct isc_clk *isc_clk = to_isc_clk(hw);
+	u32 id = isc_clk->id;
+	unsigned long flags;
+
+	spin_lock_irqsave(&isc_clk->lock, flags);
+	regmap_write(isc_clk->regmap, ISC_CLKDIS, ISC_CLK(id));
+	spin_unlock_irqrestore(&isc_clk->lock, flags);
+}
+
+static int isc_clk_is_enabled(struct clk_hw *hw)
+{
+	struct isc_clk *isc_clk = to_isc_clk(hw);
+	u32 status;
+	int ret;
+
+	ret = pm_runtime_resume_and_get(isc_clk->dev);
+	if (ret < 0)
+		return 0;
+
+	regmap_read(isc_clk->regmap, ISC_CLKSR, &status);
+
+	pm_runtime_put_sync(isc_clk->dev);
+
+	return status & ISC_CLK(isc_clk->id) ? 1 : 0;
+}
+
+static unsigned long
+isc_clk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
+{
+	struct isc_clk *isc_clk = to_isc_clk(hw);
+
+	return DIV_ROUND_CLOSEST(parent_rate, isc_clk->div + 1);
+}
+
+static int isc_clk_determine_rate(struct clk_hw *hw,
+				  struct clk_rate_request *req)
+{
+	struct isc_clk *isc_clk = to_isc_clk(hw);
+	long best_rate = -EINVAL;
+	int best_diff = -1;
+	unsigned int i, div;
+
+	for (i = 0; i < clk_hw_get_num_parents(hw); i++) {
+		struct clk_hw *parent;
+		unsigned long parent_rate;
+
+		parent = clk_hw_get_parent_by_index(hw, i);
+		if (!parent)
+			continue;
+
+		parent_rate = clk_hw_get_rate(parent);
+		if (!parent_rate)
+			continue;
+
+		for (div = 1; div < ISC_CLK_MAX_DIV + 2; div++) {
+			unsigned long rate;
+			int diff;
+
+			rate = DIV_ROUND_CLOSEST(parent_rate, div);
+			diff = abs(req->rate - rate);
+
+			if (best_diff < 0 || best_diff > diff) {
+				best_rate = rate;
+				best_diff = diff;
+				req->best_parent_rate = parent_rate;
+				req->best_parent_hw = parent;
+			}
+
+			if (!best_diff || rate < req->rate)
+				break;
+		}
+
+		if (!best_diff)
+			break;
+	}
+
+	dev_dbg(isc_clk->dev,
+		"ISC CLK: %s, best_rate = %ld, parent clk: %s @ %ld\n",
+		__func__, best_rate,
+		__clk_get_name((req->best_parent_hw)->clk),
+		req->best_parent_rate);
+
+	if (best_rate < 0)
+		return best_rate;
+
+	req->rate = best_rate;
+
+	return 0;
+}
+
+static int isc_clk_set_parent(struct clk_hw *hw, u8 index)
+{
+	struct isc_clk *isc_clk = to_isc_clk(hw);
+
+	if (index >= clk_hw_get_num_parents(hw))
+		return -EINVAL;
+
+	isc_clk->parent_id = index;
+
+	return 0;
+}
+
+static u8 isc_clk_get_parent(struct clk_hw *hw)
+{
+	struct isc_clk *isc_clk = to_isc_clk(hw);
+
+	return isc_clk->parent_id;
+}
+
+static int isc_clk_set_rate(struct clk_hw *hw,
+			    unsigned long rate,
+			    unsigned long parent_rate)
+{
+	struct isc_clk *isc_clk = to_isc_clk(hw);
+	u32 div;
+
+	if (!rate)
+		return -EINVAL;
+
+	div = DIV_ROUND_CLOSEST(parent_rate, rate);
+	if (div > (ISC_CLK_MAX_DIV + 1) || !div)
+		return -EINVAL;
+
+	isc_clk->div = div - 1;
+
+	return 0;
+}
+
+static const struct clk_ops isc_clk_ops = {
+	.prepare	= isc_clk_prepare,
+	.unprepare	= isc_clk_unprepare,
+	.enable		= isc_clk_enable,
+	.disable	= isc_clk_disable,
+	.is_enabled	= isc_clk_is_enabled,
+	.recalc_rate	= isc_clk_recalc_rate,
+	.determine_rate	= isc_clk_determine_rate,
+	.set_parent	= isc_clk_set_parent,
+	.get_parent	= isc_clk_get_parent,
+	.set_rate	= isc_clk_set_rate,
+};
+
+static int isc_clk_register(struct isc_device *isc, unsigned int id)
+{
+	struct regmap *regmap = isc->regmap;
+	struct device_node *np = isc->dev->of_node;
+	struct isc_clk *isc_clk;
+	struct clk_init_data init;
+	const char *clk_name = np->name;
+	const char *parent_names[3];
+	int num_parents;
+
+	if (id == ISC_ISPCK && !isc->ispck_required)
+		return 0;
+
+	num_parents = of_clk_get_parent_count(np);
+	if (num_parents < 1 || num_parents > 3)
+		return -EINVAL;
+
+	if (num_parents > 2 && id == ISC_ISPCK)
+		num_parents = 2;
+
+	of_clk_parent_fill(np, parent_names, num_parents);
+
+	if (id == ISC_MCK)
+		of_property_read_string(np, "clock-output-names", &clk_name);
+	else
+		clk_name = "isc-ispck";
+
+	init.parent_names	= parent_names;
+	init.num_parents	= num_parents;
+	init.name		= clk_name;
+	init.ops		= &isc_clk_ops;
+	init.flags		= CLK_SET_RATE_GATE | CLK_SET_PARENT_GATE;
+
+	isc_clk = &isc->isc_clks[id];
+	isc_clk->hw.init	= &init;
+	isc_clk->regmap		= regmap;
+	isc_clk->id		= id;
+	isc_clk->dev		= isc->dev;
+	spin_lock_init(&isc_clk->lock);
+
+	isc_clk->clk = clk_register(isc->dev, &isc_clk->hw);
+	if (IS_ERR(isc_clk->clk)) {
+		dev_err(isc->dev, "%s: clock register fail\n", clk_name);
+		return PTR_ERR(isc_clk->clk);
+	} else if (id == ISC_MCK) {
+		of_clk_add_provider(np, of_clk_src_simple_get, isc_clk->clk);
+	}
+
+	return 0;
+}
+
+int isc_clk_init(struct isc_device *isc)
+{
+	unsigned int i;
+	int ret;
+
+	for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++)
+		isc->isc_clks[i].clk = ERR_PTR(-EINVAL);
+
+	for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
+		ret = isc_clk_register(isc, i);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(isc_clk_init);
+
+void isc_clk_cleanup(struct isc_device *isc)
+{
+	unsigned int i;
+
+	of_clk_del_provider(isc->dev->of_node);
+
+	for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
+		struct isc_clk *isc_clk = &isc->isc_clks[i];
+
+		if (!IS_ERR(isc_clk->clk))
+			clk_unregister(isc_clk->clk);
+	}
+}
+EXPORT_SYMBOL_GPL(isc_clk_cleanup);
-- 
2.25.1

[PATCH 06/21] media: atmel: atmel-isc: replace video device name with module name

[Eugen Hristev]

To have consistency with future media controller development,
replace the video device name with KBUILD_MODNAME.

Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
 drivers/media/platform/atmel/atmel-isc-base.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index f532fd03e807..f3e89a892373 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -1887,7 +1887,7 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
 	}
 
 	/* Register video device */
-	strscpy(vdev->name, "microchip-isc", sizeof(vdev->name));
+	strscpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name));
 	vdev->release		= video_device_release_empty;
 	vdev->fops		= &isc_fops;
 	vdev->ioctl_ops		= &isc_ioctl_ops;
-- 
2.25.1

[PATCH 07/21] media: atmel: atmel-sama7g5-isc: fix ispck leftover

[Eugen Hristev]

The ispck is not used for sama7g5 variant of the ISC.
Calls to ispck have to be removed also from module insert/removal.

Fixes: d7f26849ed7c ("media: atmel: fix the ispck initialization")
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
 drivers/media/platform/atmel/atmel-sama7g5-isc.c | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
index 9c05acafd072..6a5d3f7ce75e 100644
--- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
@@ -555,7 +555,6 @@ static int microchip_xisc_remove(struct platform_device *pdev)
 
 	v4l2_device_unregister(&isc->v4l2_dev);
 
-	clk_disable_unprepare(isc->ispck);
 	clk_disable_unprepare(isc->hclock);
 
 	isc_clk_cleanup(isc);
@@ -567,7 +566,6 @@ static int __maybe_unused xisc_runtime_suspend(struct device *dev)
 {
 	struct isc_device *isc = dev_get_drvdata(dev);
 
-	clk_disable_unprepare(isc->ispck);
 	clk_disable_unprepare(isc->hclock);
 
 	return 0;
@@ -582,10 +580,6 @@ static int __maybe_unused xisc_runtime_resume(struct device *dev)
 	if (ret)
 		return ret;
 
-	ret = clk_prepare_enable(isc->ispck);
-	if (ret)
-		clk_disable_unprepare(isc->hclock);
-
 	return ret;
 }
 
-- 
2.25.1

[PATCH 08/21] media: atmel: atmel-isc-base: use streaming status when queueing buffers

[Eugen Hristev]

During experiments with libcamera, it looks like vb2_is_streaming returns
true before our start streaming is called.
Order of operations is streamon -> queue -> start_streaming
ISC would have started the DMA immediately when a buffer is being added
to the vbqueue if the queue is streaming.
It is more safe to start the DMA after the start streaming of the driver is
called.
Thus, even if vb2queue is streaming, add the buffer to the dma queue of the
driver instead of actually starting the DMA process, if the start streaming
has not been called yet.

Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
 drivers/media/platform/atmel/atmel-isc-base.c | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index f3e89a892373..6f14cc549543 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -442,12 +442,14 @@ static void isc_buffer_queue(struct vb2_buffer *vb)
 	unsigned long flags;
 
 	spin_lock_irqsave(&isc->dma_queue_lock, flags);
-	if (!isc->cur_frm && list_empty(&isc->dma_queue) &&
-		vb2_is_streaming(vb->vb2_queue)) {
+
+	if (!isc->cur_frm && list_empty(&isc->dma_queue) && !isc->stop) {
 		isc->cur_frm = buf;
 		isc_start_dma(isc);
-	} else
+	} else {
 		list_add_tail(&buf->list, &isc->dma_queue);
+	}
+
 	spin_unlock_irqrestore(&isc->dma_queue_lock, flags);
 }
 
@@ -1015,7 +1017,7 @@ static int isc_s_fmt_vid_cap(struct file *file, void *priv,
 {
 	struct isc_device *isc = video_drvdata(file);
 
-	if (vb2_is_streaming(&isc->vb2_vidq))
+	if (!isc->stop)
 		return -EBUSY;
 
 	return isc_set_fmt(isc, f);
@@ -1537,7 +1539,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
 
 		isc_update_awb_ctrls(isc);
 
-		if (vb2_is_streaming(&isc->vb2_vidq)) {
+		if (!isc->stop) {
 			/*
 			 * If we are streaming, we can update profile to
 			 * have the new settings in place.
@@ -1553,8 +1555,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
 		}
 
 		/* if we have autowhitebalance on, start histogram procedure */
-		if (ctrls->awb == ISC_WB_AUTO &&
-		    vb2_is_streaming(&isc->vb2_vidq) &&
+		if (ctrls->awb == ISC_WB_AUTO && !isc->stop &&
 		    ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
 			isc_set_histogram(isc, true);
 
@@ -1830,6 +1831,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
 	struct vb2_queue *q = &isc->vb2_vidq;
 	int ret = 0;
 
+	isc->stop = true;
+
 	INIT_WORK(&isc->awb_work, isc_awb_work);
 
 	ret = v4l2_device_register_subdev_nodes(&isc->v4l2_dev);
-- 
2.25.1

[PATCH 09/21] media: atmel: atmel-isc-base: remove frameintervals VIDIOC

[Eugen Hristev]

VIDIOC_ENUM_FRAMEINTERVALS is not recommended for a top video driver.
The frame rate is defined by the sensor subdevice, thus it can be queried
directly by anyone interested in the frame intervals.

Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
 drivers/media/platform/atmel/atmel-isc-base.c | 37 -------------------
 1 file changed, 37 deletions(-)

diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index 6f14cc549543..8537ad73d160 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -1108,42 +1108,6 @@ static int isc_enum_framesizes(struct file *file, void *fh,
 	return 0;
 }
 
-static int isc_enum_frameintervals(struct file *file, void *fh,
-				    struct v4l2_frmivalenum *fival)
-{
-	struct isc_device *isc = video_drvdata(file);
-	struct v4l2_subdev_frame_interval_enum fie = {
-		.code = isc->config.sd_format->mbus_code,
-		.index = fival->index,
-		.width = fival->width,
-		.height = fival->height,
-		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
-	};
-	int ret = -EINVAL;
-	unsigned int i;
-
-	for (i = 0; i < isc->num_user_formats; i++)
-		if (isc->user_formats[i]->fourcc == fival->pixel_format)
-			ret = 0;
-
-	for (i = 0; i < isc->controller_formats_size; i++)
-		if (isc->controller_formats[i].fourcc == fival->pixel_format)
-			ret = 0;
-
-	if (ret)
-		return ret;
-
-	ret = v4l2_subdev_call(isc->current_subdev->sd, pad,
-			       enum_frame_interval, NULL, &fie);
-	if (ret)
-		return ret;
-
-	fival->type = V4L2_FRMIVAL_TYPE_DISCRETE;
-	fival->discrete = fie.interval;
-
-	return 0;
-}
-
 static const struct v4l2_ioctl_ops isc_ioctl_ops = {
 	.vidioc_querycap		= isc_querycap,
 	.vidioc_enum_fmt_vid_cap	= isc_enum_fmt_vid_cap,
@@ -1168,7 +1132,6 @@ static const struct v4l2_ioctl_ops isc_ioctl_ops = {
 	.vidioc_g_parm			= isc_g_parm,
 	.vidioc_s_parm			= isc_s_parm,
 	.vidioc_enum_framesizes		= isc_enum_framesizes,
-	.vidioc_enum_frameintervals	= isc_enum_frameintervals,
 
 	.vidioc_log_status		= v4l2_ctrl_log_status,
 	.vidioc_subscribe_event		= v4l2_ctrl_subscribe_event,
-- 
2.25.1

[PATCH 10/21] media: atmel: atmel-isc-base: report frame sizes as full supported range

[Eugen Hristev]

The ISC supports a full broad range of frame sizes.
Until now, the subdevice was queried for possible frame sizes and these
were reported to the user space.
However, the ISC should not care about which frame sizes the subdev supports,
as long as this frame size is supported.
Thus, report a continuous range from smallest frame size up to the max
resolution.

Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
 drivers/media/platform/atmel/atmel-isc-base.c | 22 +++++++++----------
 1 file changed, 10 insertions(+), 12 deletions(-)

diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index 8537ad73d160..2dd2511c7be1 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -1077,14 +1077,12 @@ static int isc_enum_framesizes(struct file *file, void *fh,
 			       struct v4l2_frmsizeenum *fsize)
 {
 	struct isc_device *isc = video_drvdata(file);
-	struct v4l2_subdev_frame_size_enum fse = {
-		.code = isc->config.sd_format->mbus_code,
-		.index = fsize->index,
-		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
-	};
 	int ret = -EINVAL;
 	int i;
 
+	if (fsize->index)
+		return -EINVAL;
+
 	for (i = 0; i < isc->num_user_formats; i++)
 		if (isc->user_formats[i]->fourcc == fsize->pixel_format)
 			ret = 0;
@@ -1096,14 +1094,14 @@ static int isc_enum_framesizes(struct file *file, void *fh,
 	if (ret)
 		return ret;
 
-	ret = v4l2_subdev_call(isc->current_subdev->sd, pad, enum_frame_size,
-			       NULL, &fse);
-	if (ret)
-		return ret;
+	fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
 
-	fsize->type = V4L2_FRMSIZE_TYPE_DISCRETE;
-	fsize->discrete.width = fse.max_width;
-	fsize->discrete.height = fse.max_height;
+	fsize->stepwise.min_width = 16;
+	fsize->stepwise.max_width = isc->max_width;
+	fsize->stepwise.min_height = 16;
+	fsize->stepwise.max_height = isc->max_height;
+	fsize->stepwise.step_width = 1;
+	fsize->stepwise.step_height = 1;
 
 	return 0;
 }
-- 
2.25.1

[PATCH 11/21] media: atmel: atmel-isc-base: implement mbus_code support in enumfmt

[Eugen Hristev]

If enumfmt is called with an mbus_code, the enumfmt handler should only
return the formats that are supported for this mbus_code.
To make it more easy to understand the formats, changed the report order
to report first the native formats, and after that the formats that the ISC
can convert to.

Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
 drivers/media/platform/atmel/atmel-isc-base.c | 51 ++++++++++++++++---
 1 file changed, 43 insertions(+), 8 deletions(-)

diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index 2dd2511c7be1..1f7fbe5e4d79 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -499,21 +499,56 @@ static int isc_enum_fmt_vid_cap(struct file *file, void *priv,
 	u32 index = f->index;
 	u32 i, supported_index;
 
-	if (index < isc->controller_formats_size) {
-		f->pixelformat = isc->controller_formats[index].fourcc;
-		return 0;
+	supported_index = 0;
+
+	for (i = 0; i < isc->formats_list_size; i++) {
+		if (!isc->formats_list[i].sd_support)
+			continue;
+		/*
+		 * If specific mbus_code is requested, provide only
+		 * supported formats with this mbus code
+		 */
+		if (f->mbus_code && f->mbus_code !=
+		    isc->formats_list[i].mbus_code)
+			continue;
+		if (supported_index == index) {
+			f->pixelformat = isc->formats_list[i].fourcc;
+			return 0;
+		}
+		supported_index++;
 	}
 
-	index -= isc->controller_formats_size;
+	/*
+	 * If the sensor does not support this mbus_code whatsoever,
+	 * there is no reason to advertise any of our output formats
+	 */
+	if (supported_index == 0)
+		return -EINVAL;
+
+	/*
+	 * If the sensor uses a format that is not raw, then we cannot
+	 * convert it to any of the formats that we usually can with a
+	 * RAW sensor. Thus, do not advertise them.
+	 */
+	if (!isc->config.sd_format ||
+	    !ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
+		return -EINVAL;
 
+	/*
+	 * Iterate again through the formats that we can convert to.
+	 * However, to avoid duplicates, skip the formats that
+	 * the sensor already supports directly
+	 */
+	index -= supported_index;
 	supported_index = 0;
 
-	for (i = 0; i < isc->formats_list_size; i++) {
-		if (!ISC_IS_FORMAT_RAW(isc->formats_list[i].mbus_code) ||
-		    !isc->formats_list[i].sd_support)
+	for (i = 0; i < isc->controller_formats_size; i++) {
+		/* if this format is already supported by sensor, skip it */
+		if (find_format_by_fourcc(isc, isc->controller_formats[i].fourcc))
 			continue;
 		if (supported_index == index) {
-			f->pixelformat = isc->formats_list[i].fourcc;
+			f->pixelformat =
+				isc->controller_formats[i].fourcc;
 			return 0;
 		}
 		supported_index++;
-- 
2.25.1

[PATCH 12/21] media: atmel: atmel-isc-base: fix bytesperline value for planar formats

[Eugen Hristev]

The bytesperline field of the pixfmt should be only for the first plane
in case of planar formats like YUV420 or YUV422.
The bytesperline is used by the driver to compute the framesize.
We have to report a different bpp (bytes per pixel) to v4l2 in bytesperline
than the actual bpp.
For example for YUV420, the real bpp is 12, but the first plane has only
8 bpp. Thus we report a bytesperline 8*width instead of 12*width.
However, for real framezise we have to compute 12*width*height.
Hence added a new variable to hold this information and to correctly
compute the frame size.

Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
 drivers/media/platform/atmel/atmel-isc-base.c | 19 +++++++++++++++++--
 drivers/media/platform/atmel/atmel-isc.h      |  4 ++++
 2 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index 1f7fbe5e4d79..f782a9c0c07d 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -655,6 +655,7 @@ static int isc_try_configure_rlp_dma(struct isc_device *isc, bool direct_dump)
 		isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED8;
 		isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
 		isc->try_config.bpp = 8;
+		isc->try_config.bpp_v4l2 = 8;
 		break;
 	case V4L2_PIX_FMT_SBGGR10:
 	case V4L2_PIX_FMT_SGBRG10:
@@ -664,6 +665,7 @@ static int isc_try_configure_rlp_dma(struct isc_device *isc, bool direct_dump)
 		isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16;
 		isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
 		isc->try_config.bpp = 16;
+		isc->try_config.bpp_v4l2 = 16;
 		break;
 	case V4L2_PIX_FMT_SBGGR12:
 	case V4L2_PIX_FMT_SGBRG12:
@@ -673,24 +675,28 @@ static int isc_try_configure_rlp_dma(struct isc_device *isc, bool direct_dump)
 		isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16;
 		isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
 		isc->try_config.bpp = 16;
+		isc->try_config.bpp_v4l2 = 16;
 		break;
 	case V4L2_PIX_FMT_RGB565:
 		isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_RGB565;
 		isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16;
 		isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
 		isc->try_config.bpp = 16;
+		isc->try_config.bpp_v4l2 = 16;
 		break;
 	case V4L2_PIX_FMT_ARGB444:
 		isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_ARGB444;
 		isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16;
 		isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
 		isc->try_config.bpp = 16;
+		isc->try_config.bpp_v4l2 = 16;
 		break;
 	case V4L2_PIX_FMT_ARGB555:
 		isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_ARGB555;
 		isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16;
 		isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
 		isc->try_config.bpp = 16;
+		isc->try_config.bpp_v4l2 = 16;
 		break;
 	case V4L2_PIX_FMT_ABGR32:
 	case V4L2_PIX_FMT_XBGR32:
@@ -698,42 +704,49 @@ static int isc_try_configure_rlp_dma(struct isc_device *isc, bool direct_dump)
 		isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED32;
 		isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
 		isc->try_config.bpp = 32;
+		isc->try_config.bpp_v4l2 = 32;
 		break;
 	case V4L2_PIX_FMT_YUV420:
 		isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_YYCC;
 		isc->try_config.dcfg_imode = ISC_DCFG_IMODE_YC420P;
 		isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PLANAR;
 		isc->try_config.bpp = 12;
+		isc->try_config.bpp_v4l2 = 8; /* only first plane */
 		break;
 	case V4L2_PIX_FMT_YUV422P:
 		isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_YYCC;
 		isc->try_config.dcfg_imode = ISC_DCFG_IMODE_YC422P;
 		isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PLANAR;
 		isc->try_config.bpp = 16;
+		isc->try_config.bpp_v4l2 = 8; /* only first plane */
 		break;
 	case V4L2_PIX_FMT_YUYV:
 		isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_YCYC | ISC_RLP_CFG_YMODE_YUYV;
 		isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED32;
 		isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
 		isc->try_config.bpp = 16;
+		isc->try_config.bpp_v4l2 = 16;
 		break;
 	case V4L2_PIX_FMT_UYVY:
 		isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_YCYC | ISC_RLP_CFG_YMODE_UYVY;
 		isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED32;
 		isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
 		isc->try_config.bpp = 16;
+		isc->try_config.bpp_v4l2 = 16;
 		break;
 	case V4L2_PIX_FMT_VYUY:
 		isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_YCYC | ISC_RLP_CFG_YMODE_VYUY;
 		isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED32;
 		isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
 		isc->try_config.bpp = 16;
+		isc->try_config.bpp_v4l2 = 16;
 		break;
 	case V4L2_PIX_FMT_GREY:
 		isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_DATY8;
 		isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED8;
 		isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
 		isc->try_config.bpp = 8;
+		isc->try_config.bpp_v4l2 = 8;
 		break;
 	case V4L2_PIX_FMT_Y16:
 		isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_DATY10 | ISC_RLP_CFG_LSH;
@@ -743,6 +756,7 @@ static int isc_try_configure_rlp_dma(struct isc_device *isc, bool direct_dump)
 		isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16;
 		isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
 		isc->try_config.bpp = 16;
+		isc->try_config.bpp_v4l2 = 16;
 		break;
 	default:
 		return -EINVAL;
@@ -991,8 +1005,9 @@ static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f,
 		pixfmt->height = isc->max_height;
 
 	pixfmt->field = V4L2_FIELD_NONE;
-	pixfmt->bytesperline = (pixfmt->width * isc->try_config.bpp) >> 3;
-	pixfmt->sizeimage = pixfmt->bytesperline * pixfmt->height;
+	pixfmt->bytesperline = (pixfmt->width * isc->try_config.bpp_v4l2) >> 3;
+	pixfmt->sizeimage = ((pixfmt->width * isc->try_config.bpp) >> 3) *
+			     pixfmt->height;
 
 	if (code)
 		*code = mbus_code;
diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
index 2bfcb135ef13..dd45e9ca22a9 100644
--- a/drivers/media/platform/atmel/atmel-isc.h
+++ b/drivers/media/platform/atmel/atmel-isc.h
@@ -95,6 +95,9 @@ struct isc_format {
 			configuration.
  * @fourcc:		Fourcc code for this format.
  * @bpp:		Bytes per pixel in the current format.
+ * @bpp_v4l2:		Bytes per pixel in the current format, for v4l2.
+			This differs from 'bpp' in the sense that in planar
+			formats, it refers only to the first plane.
  * @rlp_cfg_mode:	Configuration of the RLP (rounding, limiting packaging)
  * @dcfg_imode:		Configuration of the input of the DMA module
  * @dctrl_dview:	Configuration of the output of the DMA module
@@ -105,6 +108,7 @@ struct fmt_config {
 
 	u32			fourcc;
 	u8			bpp;
+	u8			bpp_v4l2;
 
 	u32			rlp_cfg_mode;
 	u32			dcfg_imode;
-- 
2.25.1

[PATCH 13/21] MAINTAINERS: atmel-isc: add new file atmel-isc-mc.c

[Eugen Hristev]

Add new file atmel-isc-mc.c to MAINTAINERS entry.

Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index e151716c8d1e..228363d2afe7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -12280,6 +12280,7 @@ F:	Documentation/devicetree/bindings/media/atmel,isc.yaml
 F:	Documentation/devicetree/bindings/media/microchip,xisc.yaml
 F:	drivers/media/platform/atmel/atmel-isc-base.c
 F:	drivers/media/platform/atmel/atmel-isc-clk.c
+F:	drivers/media/platform/atmel/atmel-isc-mc.c
 F:	drivers/media/platform/atmel/atmel-isc-regs.h
 F:	drivers/media/platform/atmel/atmel-isc.h
 F:	drivers/media/platform/atmel/atmel-sama5d2-isc.c
-- 
2.25.1

[PATCH 14/21] media: atmel: atmel-isc: implement media controller

[Eugen Hristev]

Implement the support for media-controller.
This means that the capabilities of the driver have changed and now
it also advertises the IO_MC .
The driver will register it's media device, and add the video entity to this
media device. The subdevices are registered to the same media device.
The ISC will have a base entity which is auto-detected as atmel_isc_base.
It will also register a subdevice that allows cropping of the incoming frame
to the maximum frame size supported by the ISC.
The ISC will create a link between the subdevice that is asynchronously
registered and the atmel_isc_scaler entity.
Then, the atmel_isc_scaler and atmel_isc_base are connected through another
link.

Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
 drivers/media/platform/atmel/Makefile         |   2 +-
 drivers/media/platform/atmel/atmel-isc-base.c |  20 +-
 drivers/media/platform/atmel/atmel-isc-mc.c   | 235 ++++++++++++++++++
 drivers/media/platform/atmel/atmel-isc.h      |  28 +++
 .../media/platform/atmel/atmel-sama5d2-isc.c  |  14 +-
 .../media/platform/atmel/atmel-sama7g5-isc.c  |  12 +-
 6 files changed, 306 insertions(+), 5 deletions(-)
 create mode 100644 drivers/media/platform/atmel/atmel-isc-mc.c

diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
index 1f6fe7427769..90220eaefa20 100644
--- a/drivers/media/platform/atmel/Makefile
+++ b/drivers/media/platform/atmel/Makefile
@@ -3,7 +3,7 @@ atmel-isc-objs = atmel-sama5d2-isc.o
 atmel-xisc-objs = atmel-sama7g5-isc.o
 
 obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
-obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o atmel-isc-clk.o
+obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o atmel-isc-clk.o atmel-isc-mc.o
 obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel-isc.o
 obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel-xisc.o
 obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += microchip-csi2dc.o
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index f782a9c0c07d..a0e4bc75d1a2 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -1727,6 +1727,7 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
 					      struct isc_device, v4l2_dev);
 	struct isc_subdev_entity *subdev_entity =
 		container_of(notifier, struct isc_subdev_entity, notifier);
+	int pad;
 
 	if (video_is_registered(&isc->video_dev)) {
 		v4l2_err(&isc->v4l2_dev, "only supports one sub-device.\n");
@@ -1735,6 +1736,16 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
 
 	subdev_entity->sd = subdev;
 
+	pad = media_entity_get_fwnode_pad(&subdev->entity, asd->match.fwnode,
+					  MEDIA_PAD_FL_SOURCE);
+	if (pad < 0) {
+		v4l2_err(&isc->v4l2_dev, "failed to find pad for %s\n",
+			 subdev->name);
+		return pad;
+	}
+
+	isc->remote_pad = pad;
+
 	return 0;
 }
 
@@ -1910,7 +1921,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
 	vdev->queue		= q;
 	vdev->lock		= &isc->lock;
 	vdev->ctrl_handler	= &isc->ctrls.handler;
-	vdev->device_caps	= V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE;
+	vdev->device_caps	= V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE |
+				  V4L2_CAP_IO_MC;
 	video_set_drvdata(vdev, isc);
 
 	ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
@@ -1920,8 +1932,14 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
 		goto isc_async_complete_err;
 	}
 
+	ret = isc_mc_register(isc);
+	if (ret < 0)
+		goto isc_async_complete_unregister_device;
 	return 0;
 
+isc_async_complete_unregister_device:
+	video_unregister_device(vdev);
+
 isc_async_complete_err:
 	mutex_destroy(&isc->lock);
 	return ret;
diff --git a/drivers/media/platform/atmel/atmel-isc-mc.c b/drivers/media/platform/atmel/atmel-isc-mc.c
new file mode 100644
index 000000000000..fe9ebb46e270
--- /dev/null
+++ b/drivers/media/platform/atmel/atmel-isc-mc.c
@@ -0,0 +1,235 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Microchip Image Sensor Controller (ISC) Media Controller support
+ *
+ * Copyright (C) 2021 Microchip Technology, Inc.
+ *
+ * Author: Eugen Hristev <eugen.hristev@microchip.com>
+ *
+ */
+
+#include <linux/clk.h>
+#include <linux/clkdev.h>
+#include <linux/clk-provider.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <linux/platform_device.h>
+
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "atmel-isc-regs.h"
+#include "atmel-isc.h"
+
+static const struct media_device_ops isc_media_ops = {
+};
+
+static int isc_scaler_get_fmt(struct v4l2_subdev *sd,
+			      struct v4l2_subdev_state *sd_state,
+			      struct v4l2_subdev_format *format)
+{
+	struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
+
+	format->format = isc->scaler_format;
+
+	if (format->pad == 1) {
+		if (format->format.height > isc->max_height)
+			format->format.height = isc->max_height;
+		if (format->format.width > isc->max_width)
+			format->format.width = isc->max_width;
+	}
+
+	return 0;
+}
+
+static int isc_scaler_set_fmt(struct v4l2_subdev *sd,
+			      struct v4l2_subdev_state *sd_state,
+			      struct v4l2_subdev_format *req_fmt)
+{
+	struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
+
+	if (req_fmt->pad == 1) {
+		if (req_fmt->format.height > isc->max_height)
+			req_fmt->format.height = isc->max_height;
+		if (req_fmt->format.width > isc->max_width)
+			req_fmt->format.width = isc->max_width;
+	}
+
+	isc->scaler_format = req_fmt->format;
+
+	return 0;
+}
+
+static int isc_scaler_enum_mbus_code(struct v4l2_subdev *sd,
+				     struct v4l2_subdev_state *sd_state,
+				     struct v4l2_subdev_mbus_code_enum *code)
+{
+	struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
+	int supported_index = 0;
+	int i;
+
+	for (i = 0; i < isc->formats_list_size; i++) {
+		if (!isc->formats_list[i].sd_support)
+			continue;
+		if (supported_index == code->index) {
+			code->code = isc->formats_list[i].mbus_code;
+			return 0;
+		}
+		supported_index++;
+	}
+
+	return -EINVAL;
+}
+
+static int isc_scaler_g_sel(struct v4l2_subdev *sd,
+			    struct v4l2_subdev_state *sd_state,
+			    struct v4l2_subdev_selection *sel)
+{
+	struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
+
+	if (sel->pad == ISC_SCALER_PAD_SOURCE)
+		return -EINVAL;
+
+	if (sel->target != V4L2_SEL_TGT_CROP_BOUNDS &&
+	    sel->target != V4L2_SEL_TGT_CROP)
+		return -EINVAL;
+
+	sel->r.height = isc->max_height;
+	sel->r.width = isc->max_width;
+
+	sel->r.left = 0;
+	sel->r.top = 0;
+
+	return 0;
+}
+
+static const struct v4l2_subdev_pad_ops isc_scaler_pad_ops = {
+	.enum_mbus_code = isc_scaler_enum_mbus_code,
+	.set_fmt = isc_scaler_set_fmt,
+	.get_fmt = isc_scaler_get_fmt,
+	.get_selection = isc_scaler_g_sel,
+};
+
+static const struct v4l2_subdev_ops xisc_scaler_subdev_ops = {
+	.pad = &isc_scaler_pad_ops,
+};
+
+static int isc_init_own_sd(struct isc_device *isc)
+{
+	int ret;
+
+	v4l2_subdev_init(&isc->scaler_sd, &xisc_scaler_subdev_ops);
+
+	isc->scaler_sd.owner = THIS_MODULE;
+	isc->scaler_sd.dev = isc->dev;
+	snprintf(isc->scaler_sd.name, sizeof(isc->scaler_sd.name),
+		 "atmel_isc_scaler");
+
+	isc->scaler_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+	isc->scaler_sd.entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
+	isc->scaler_pads[ISC_SCALER_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+	isc->scaler_pads[ISC_SCALER_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
+
+	isc->scaler_format.height = isc->max_height;
+	isc->scaler_format.width = isc->max_width;
+
+	ret = media_entity_pads_init(&isc->scaler_sd.entity,
+				     ISC_SCALER_PADS_NUM,
+				     isc->scaler_pads);
+	if (ret < 0) {
+		dev_err(isc->dev, "scaler sd media entity init failed\n");
+		return ret;
+	}
+	ret = v4l2_device_register_subdev(&isc->v4l2_dev, &isc->scaler_sd);
+	if (ret < 0) {
+		dev_err(isc->dev, "scaler sd failed to register subdev\n");
+		return ret;
+	}
+
+	return ret;
+}
+
+int isc_mc_init(struct isc_device *isc, u32 ver)
+{
+	const struct of_device_id *match;
+	int ret;
+
+	isc->video_dev.entity.function = MEDIA_ENT_F_IO_V4L;
+	isc->video_dev.entity.flags = MEDIA_ENT_FL_DEFAULT;
+	isc->pads[ISC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+
+	ret = media_entity_pads_init(&isc->video_dev.entity, ISC_PADS_NUM,
+				     isc->pads);
+	if (ret < 0) {
+		dev_err(isc->dev, "media entity init failed\n");
+		return ret;
+	}
+
+	isc->mdev.dev = isc->dev;
+	isc->mdev.ops = &isc_media_ops;
+
+	match = of_match_node(isc->dev->driver->of_match_table,
+			      isc->dev->of_node);
+
+	strscpy(isc->mdev.driver_name, KBUILD_MODNAME,
+		sizeof(isc->mdev.driver_name));
+	strscpy(isc->mdev.model, match->compatible, sizeof(isc->mdev.model));
+	snprintf(isc->mdev.bus_info, sizeof(isc->mdev.bus_info), "platform:%s",
+		 isc->v4l2_dev.name);
+	isc->mdev.hw_revision = ver;
+
+	media_device_init(&isc->mdev);
+
+	isc->v4l2_dev.mdev = &isc->mdev;
+
+	return isc_init_own_sd(isc);
+}
+
+int isc_mc_register(struct isc_device *isc)
+{
+	int ret;
+
+	ret = media_create_pad_link(&isc->current_subdev->sd->entity,
+				    isc->remote_pad, &isc->scaler_sd.entity,
+				    ISC_SCALER_PAD_SINK,
+				    MEDIA_LNK_FL_ENABLED |
+				    MEDIA_LNK_FL_IMMUTABLE);
+
+	if (ret < 0) {
+		v4l2_err(&isc->v4l2_dev,
+			 "Failed to create pad link: %s to %s\n",
+			 isc->current_subdev->sd->entity.name,
+			 isc->scaler_sd.entity.name);
+		return ret;
+	}
+
+	dev_dbg(isc->dev, "link with %s pad: %d\n",
+		isc->current_subdev->sd->name, isc->remote_pad);
+
+	ret = media_create_pad_link(&isc->scaler_sd.entity,
+				    ISC_SCALER_PAD_SOURCE,
+				    &isc->video_dev.entity, ISC_PAD_SINK,
+				    MEDIA_LNK_FL_ENABLED |
+				    MEDIA_LNK_FL_IMMUTABLE);
+
+	if (ret < 0) {
+		v4l2_err(&isc->v4l2_dev,
+			 "Failed to create pad link: %s to %s\n",
+			 isc->scaler_sd.entity.name,
+			 isc->video_dev.entity.name);
+		return ret;
+	}
+
+	dev_dbg(isc->dev, "link with %s pad: %d\n", isc->scaler_sd.name,
+		ISC_SCALER_PAD_SOURCE);
+
+	return media_device_register(&isc->mdev);
+}
+
+void isc_mc_cleanup(struct isc_device *isc)
+{
+	media_entity_cleanup(&isc->video_dev.entity);
+}
diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
index dd45e9ca22a9..27292c50915e 100644
--- a/drivers/media/platform/atmel/atmel-isc.h
+++ b/drivers/media/platform/atmel/atmel-isc.h
@@ -176,6 +176,17 @@ struct isc_reg_offsets {
 	u32 his_entry;
 };
 
+enum isc_mc_pads {
+	ISC_PAD_SINK	= 0,
+	ISC_PADS_NUM	= 1,
+};
+
+enum isc_scaler_pads {
+	ISC_SCALER_PAD_SINK	= 0,
+	ISC_SCALER_PAD_SOURCE	= 1,
+	ISC_SCALER_PADS_NUM	= 2,
+};
+
 /*
  * struct isc_device - ISC device driver data/config struct
  * @regmap:		Register map
@@ -339,6 +350,19 @@ struct isc_device {
 	struct isc_format		*formats_list;
 	u32				controller_formats_size;
 	u32				formats_list_size;
+
+	struct {
+		struct media_pad		pads[ISC_PADS_NUM];
+		struct media_device		mdev;
+
+		u32				remote_pad;
+	};
+
+	struct {
+		struct v4l2_subdev		scaler_sd;
+		struct media_pad		scaler_pads[ISC_SCALER_PADS_NUM];
+		struct v4l2_mbus_framefmt	scaler_format;
+	};
 };
 
 extern const struct regmap_config isc_regmap_config;
@@ -350,4 +374,8 @@ int isc_clk_init(struct isc_device *isc);
 void isc_subdev_cleanup(struct isc_device *isc);
 void isc_clk_cleanup(struct isc_device *isc);
 
+int isc_mc_init(struct isc_device *isc, u32 ver);
+int isc_mc_register(struct isc_device *isc);
+void isc_mc_cleanup(struct isc_device *isc);
+
 #endif
diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
index e29a9193bac8..8c088ad899ea 100644
--- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
@@ -528,6 +528,12 @@ static int atmel_isc_probe(struct platform_device *pdev)
 			break;
 	}
 
+	regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
+
+	ret = isc_mc_init(isc, ver);
+	if (ret < 0)
+		goto isc_probe_mc_init_err;
+
 	pm_runtime_set_active(dev);
 	pm_runtime_enable(dev);
 	pm_request_idle(dev);
@@ -537,7 +543,7 @@ static int atmel_isc_probe(struct platform_device *pdev)
 	ret = clk_prepare_enable(isc->ispck);
 	if (ret) {
 		dev_err(dev, "failed to enable ispck: %d\n", ret);
-		goto cleanup_subdev;
+		goto isc_probe_mc_init_err;
 	}
 
 	/* ispck should be greater or equal to hclock */
@@ -547,7 +553,6 @@ static int atmel_isc_probe(struct platform_device *pdev)
 		goto unprepare_clk;
 	}
 
-	regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
 	dev_info(dev, "Microchip ISC version %x\n", ver);
 
 	return 0;
@@ -555,6 +560,9 @@ static int atmel_isc_probe(struct platform_device *pdev)
 unprepare_clk:
 	clk_disable_unprepare(isc->ispck);
 
+isc_probe_mc_init_err:
+	isc_mc_cleanup(isc);
+
 cleanup_subdev:
 	isc_subdev_cleanup(isc);
 
@@ -575,6 +583,8 @@ static int atmel_isc_remove(struct platform_device *pdev)
 
 	pm_runtime_disable(&pdev->dev);
 
+	isc_mc_cleanup(isc);
+
 	isc_subdev_cleanup(isc);
 
 	v4l2_device_unregister(&isc->v4l2_dev);
diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
index 6a5d3f7ce75e..3484971ff9be 100644
--- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
@@ -522,15 +522,23 @@ static int microchip_xisc_probe(struct platform_device *pdev)
 			break;
 	}
 
+	regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
+
+	ret = isc_mc_init(isc, ver);
+	if (ret < 0)
+		goto isc_probe_mc_init_err;
+
 	pm_runtime_set_active(dev);
 	pm_runtime_enable(dev);
 	pm_request_idle(dev);
 
-	regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
 	dev_info(dev, "Microchip XISC version %x\n", ver);
 
 	return 0;
 
+isc_probe_mc_init_err:
+	isc_mc_cleanup(isc);
+
 cleanup_subdev:
 	isc_subdev_cleanup(isc);
 
@@ -551,6 +559,8 @@ static int microchip_xisc_remove(struct platform_device *pdev)
 
 	pm_runtime_disable(&pdev->dev);
 
+	isc_mc_cleanup(isc);
+
 	isc_subdev_cleanup(isc);
 
 	v4l2_device_unregister(&isc->v4l2_dev);
-- 
2.25.1

[PATCH 15/21] ARM: dts: at91: sama7g5: add nodes for video capture

[Eugen Hristev]

Add node for the XISC (eXtended Image Sensor Controller) and CSI2DC
(csi2 demux controller).
These nodes represent the top level of the video capture hardware pipeline
and are directly connected in hardware.

Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
 arch/arm/boot/dts/sama7g5.dtsi | 49 ++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)

diff --git a/arch/arm/boot/dts/sama7g5.dtsi b/arch/arm/boot/dts/sama7g5.dtsi
index 6c58c151c6d9..a8dda38ff18b 100644
--- a/arch/arm/boot/dts/sama7g5.dtsi
+++ b/arch/arm/boot/dts/sama7g5.dtsi
@@ -203,6 +203,55 @@ sdmmc2: mmc@e120c000 {
 			status = "disabled";
 		};
 
+		csi2dc: csi2dc@e1404000 {
+			compatible = "microchip,sama7g5-csi2dc";
+			reg = <0xe1404000 0x500>;
+			clocks = <&pmc PMC_TYPE_PERIPHERAL 34>, <&xisc>;
+			clock-names = "pclk", "scck";
+			assigned-clocks = <&xisc>;
+			assigned-clock-rates = <266000000>;
+
+			ports {
+				#address-cells = <1>;
+				#size-cells = <0>;
+				port@0 {
+					reg = <0>;
+					csi2dc_in: endpoint {
+					};
+				};
+
+				port@1 {
+					reg = <1>;
+					csi2dc_out: endpoint {
+						bus-width = <12>;
+						hsync-active = <1>;
+						vsync-active = <1>;
+						remote-endpoint = <&xisc_in>;
+					};
+				};
+			};
+		};
+
+		xisc: xisc@e1408000 {
+			compatible = "microchip,sama7g5-isc";
+			reg = <0xe1408000 0x2000>;
+			interrupts = <GIC_SPI 56 IRQ_TYPE_LEVEL_HIGH>;
+			clocks = <&pmc PMC_TYPE_PERIPHERAL 56>;
+			clock-names = "hclock";
+			#clock-cells = <0>;
+			clock-output-names = "isc-mck";
+
+			port {
+				xisc_in: endpoint {
+					bus-type = <5>; /* Parallel */
+					bus-width = <12>;
+					hsync-active = <1>;
+					vsync-active = <1>;
+					remote-endpoint = <&csi2dc_out>;
+				};
+			};
+		};
+
 		pwm: pwm@e1604000 {
 			compatible = "microchip,sama7g5-pwm", "atmel,sama5d2-pwm";
 			reg = <0xe1604000 0x4000>;
-- 
2.25.1

[PATCH 16/21] ARM: configs: at91: sama7: add xisc and csi2dc

[Eugen Hristev]

Enable XISC and CSI2DC drivers.

Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
 arch/arm/configs/sama7_defconfig | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/arm/configs/sama7_defconfig b/arch/arm/configs/sama7_defconfig
index 938aae4bd80b..15978f2ab4ea 100644
--- a/arch/arm/configs/sama7_defconfig
+++ b/arch/arm/configs/sama7_defconfig
@@ -126,6 +126,8 @@ CONFIG_MEDIA_SUPPORT_FILTER=y
 CONFIG_MEDIA_CAMERA_SUPPORT=y
 CONFIG_MEDIA_PLATFORM_SUPPORT=y
 CONFIG_V4L_PLATFORM_DRIVERS=y
+CONFIG_VIDEO_ATMEL_XISC=y
+CONFIG_VIDEO_MICROCHIP_CSI2DC=y
 CONFIG_VIDEO_IMX219=m
 CONFIG_VIDEO_IMX274=m
 CONFIG_VIDEO_OV5647=m
-- 
2.25.1

[PATCH 17/21] ARM: multi_v7_defconfig: add atmel video pipeline modules

[Eugen Hristev]

Add drivers for the atmel video capture pipeline: atmel isc, xisc and
microchip csi2dc.

Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
 arch/arm/configs/multi_v7_defconfig | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/arm/configs/multi_v7_defconfig b/arch/arm/configs/multi_v7_defconfig
index b4f74454f20f..03ea4d0ca242 100644
--- a/arch/arm/configs/multi_v7_defconfig
+++ b/arch/arm/configs/multi_v7_defconfig
@@ -654,7 +654,10 @@ CONFIG_VIDEO_S5P_MIPI_CSIS=m
 CONFIG_VIDEO_EXYNOS_FIMC_LITE=m
 CONFIG_VIDEO_EXYNOS4_FIMC_IS=m
 CONFIG_VIDEO_RCAR_VIN=m
+CONFIG_VIDEO_ATMEL_ISC=m
+CONFIG_VIDEO_ATMEL_XISC=m
 CONFIG_VIDEO_ATMEL_ISI=m
+CONFIG_VIDEO_MICROCHIP_CSI2DC=m
 CONFIG_V4L_MEM2MEM_DRIVERS=y
 CONFIG_VIDEO_SAMSUNG_S5P_JPEG=m
 CONFIG_VIDEO_SAMSUNG_S5P_MFC=m
-- 
2.25.1

[PATCH 18/21] media: atmel: atmel-sama5d2-isc: fix wrong mask in YUYV format check

[Eugen Hristev]

While this does not happen in production, this check should be done
versus the mask, as checking with the YCYC value may not include
some bits that may be set.
Is it correct and safe to check the whole mask.

Fixes: 123aaf816b95 ("media: atmel: atmel-sama5d2-isc: fix YUYV format")
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
 drivers/media/platform/atmel/atmel-sama5d2-isc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
index 8c088ad899ea..6c9c5fb041d7 100644
--- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
@@ -267,7 +267,7 @@ static void isc_sama5d2_config_rlp(struct isc_device *isc)
 	 * Thus, if the YCYC mode is selected, replace it with the
 	 * sama5d2-compliant mode which is YYCC .
 	 */
-	if ((rlp_mode & ISC_RLP_CFG_MODE_YCYC) == ISC_RLP_CFG_MODE_YCYC) {
+	if ((rlp_mode & ISC_RLP_CFG_MODE_MASK) == ISC_RLP_CFG_MODE_YCYC) {
 		rlp_mode &= ~ISC_RLP_CFG_MODE_MASK;
 		rlp_mode |= ISC_RLP_CFG_MODE_YYCC;
 	}
-- 
2.25.1

[PATCH 19/21] media: atmel: atmel-isc-base: use mutex to lock awb workqueue from streaming

[Eugen Hristev]

The AWB workqueue runs in a kernel thread and needs to be synchronized
w.r.t. the streaming status.
It is possible that streaming is stopped while the AWB workq is running.
In this case it is likely that the check for isc->stop is done at one point
in time, but the AWB computations are done later, including a call to
isc_update_profile, which requires streaming to be started.
Thus , isc_update_profile will fail if during this operation sequence the
streaming was stopped.
To solve this issue, a mutex is added, that will serialize the awb work and
streaming stopping, with the mention that either streaming is stopped
completely including termination of the last frame is done, and after that
the AWB work can check stream status and stop; either first AWB work is
completed and after that the streaming can stop correctly.
The awb spin lock cannot be used since this spinlock is taken in the same
context and using it in the stop streaming will result in a recursion BUG.

Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
 drivers/media/platform/atmel/atmel-isc-base.c | 31 ++++++++++++++++---
 drivers/media/platform/atmel/atmel-isc.h      |  1 +
 2 files changed, 28 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index a0e4bc75d1a2..7ebe0a2d130f 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -402,6 +402,7 @@ static void isc_stop_streaming(struct vb2_queue *vq)
 	struct isc_buffer *buf;
 	int ret;
 
+	mutex_lock(&isc->awb_mutex);
 	v4l2_ctrl_activate(isc->do_wb_ctrl, false);
 
 	isc->stop = true;
@@ -411,6 +412,8 @@ static void isc_stop_streaming(struct vb2_queue *vq)
 		v4l2_err(&isc->v4l2_dev,
 			 "Timeout waiting for end of the capture\n");
 
+	mutex_unlock(&isc->awb_mutex);
+
 	/* Disable DMA interrupt */
 	regmap_write(isc->regmap, ISC_INTDIS, ISC_INT_DDONE);
 
@@ -1417,10 +1420,6 @@ static void isc_awb_work(struct work_struct *w)
 	u32 min, max;
 	int ret;
 
-	/* streaming is not active anymore */
-	if (isc->stop)
-		return;
-
 	if (ctrls->hist_stat != HIST_ENABLED)
 		return;
 
@@ -1471,7 +1470,24 @@ static void isc_awb_work(struct work_struct *w)
 	}
 	regmap_write(regmap, ISC_HIS_CFG + isc->offsets.his,
 		     hist_id | baysel | ISC_HIS_CFG_RAR);
+
+	/*
+	 * We have to make sure the streaming has not stopped meanwhile.
+	 * ISC requires a frame to clock the internal profile update.
+	 * To avoid issues, lock the sequence with a mutex
+	 */
+	mutex_lock(&isc->awb_mutex);
+
+	/* streaming is not active anymore */
+	if (isc->stop) {
+		mutex_unlock(&isc->awb_mutex);
+		return;
+	};
+
 	isc_update_profile(isc);
+
+	mutex_unlock(&isc->awb_mutex);
+
 	/* if awb has been disabled, we don't need to start another histogram */
 	if (ctrls->awb)
 		regmap_write(regmap, ISC_CTRLEN, ISC_CTRL_HISREQ);
@@ -1550,6 +1566,8 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
 
 		isc_update_awb_ctrls(isc);
 
+		mutex_lock(&isc->awb_mutex);
+
 		if (!isc->stop) {
 			/*
 			 * If we are streaming, we can update profile to
@@ -1564,6 +1582,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
 			 */
 			v4l2_ctrl_activate(isc->do_wb_ctrl, false);
 		}
+		mutex_unlock(&isc->awb_mutex);
 
 		/* if we have autowhitebalance on, start histogram procedure */
 		if (ctrls->awb == ISC_WB_AUTO && !isc->stop &&
@@ -1755,6 +1774,7 @@ static void isc_async_unbind(struct v4l2_async_notifier *notifier,
 {
 	struct isc_device *isc = container_of(notifier->v4l2_dev,
 					      struct isc_device, v4l2_dev);
+	mutex_destroy(&isc->awb_mutex);
 	cancel_work_sync(&isc->awb_work);
 	video_unregister_device(&isc->video_dev);
 	v4l2_ctrl_handler_free(&isc->ctrls.handler);
@@ -1866,6 +1886,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
 	isc->current_subdev = container_of(notifier,
 					   struct isc_subdev_entity, notifier);
 	mutex_init(&isc->lock);
+	mutex_init(&isc->awb_mutex);
+
 	init_completion(&isc->comp);
 
 	/* Initialize videobuf2 queue */
@@ -1941,6 +1963,7 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
 	video_unregister_device(vdev);
 
 isc_async_complete_err:
+	mutex_destroy(&isc->awb_mutex);
 	mutex_destroy(&isc->lock);
 	return ret;
 }
diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
index 27292c50915e..93ee000279f2 100644
--- a/drivers/media/platform/atmel/atmel-isc.h
+++ b/drivers/media/platform/atmel/atmel-isc.h
@@ -294,6 +294,7 @@ struct isc_device {
 	struct work_struct	awb_work;
 
 	struct mutex		lock; /* serialize access to file operations */
+	struct mutex		awb_mutex; /* serialize access to streaming status from awb work queue */
 	spinlock_t		awb_lock; /* serialize access to DMA buffers from awb work queue */
 
 	struct regmap_field	*pipeline[ISC_PIPE_LINE_NODE_NUM];
-- 
2.25.1

[PATCH 20/21] media: atmel: atmel-isc-base: add wb debug messages

[Eugen Hristev]

Add debug messages that make it easier to debug white balance algorithm.

Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
 drivers/media/platform/atmel/atmel-isc-base.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index 7ebe0a2d130f..071ac5eec7ec 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -1330,10 +1330,15 @@ static void isc_hist_count(struct isc_device *isc, u32 *min, u32 *max)
 
 	if (!*min)
 		*min = 1;
+
+	v4l2_dbg(1, debug, &isc->v4l2_dev,
+		 "isc wb: hist_id %u, hist_count %u",
+		 ctrls->hist_id, *hist_count);
 }
 
 static void isc_wb_update(struct isc_ctrls *ctrls)
 {
+	struct isc_device *isc = container_of(ctrls, struct isc_device, ctrls);
 	u32 *hist_count = &ctrls->hist_count[0];
 	u32 c, offset[4];
 	u64 avg = 0;
@@ -1350,6 +1355,9 @@ static void isc_wb_update(struct isc_ctrls *ctrls)
 		(u64)hist_count[ISC_HIS_CFG_MODE_GB];
 	avg >>= 1;
 
+	v4l2_dbg(1, debug, &isc->v4l2_dev,
+		 "isc wb: green components average %llu\n", avg);
+
 	/* Green histogram is null, nothing to do */
 	if (!avg)
 		return;
@@ -1402,9 +1410,15 @@ static void isc_wb_update(struct isc_ctrls *ctrls)
 		else
 			gw_gain[c] = 1 << 9;
 
+		v4l2_dbg(1, debug, &isc->v4l2_dev,
+			 "isc wb: component %d, s_gain %u, gw_gain %u\n",
+			 c, s_gain[c], gw_gain[c]);
 		/* multiply both gains and adjust for decimals */
 		ctrls->gain[c] = s_gain[c] * gw_gain[c];
 		ctrls->gain[c] >>= 9;
+		v4l2_dbg(1, debug, &isc->v4l2_dev,
+			 "isc wb: component %d, final gain %u\n",
+			 c, ctrls->gain[c]);
 	}
 }
 
@@ -1424,6 +1438,10 @@ static void isc_awb_work(struct work_struct *w)
 		return;
 
 	isc_hist_count(isc, &min, &max);
+
+	v4l2_dbg(1, debug, &isc->v4l2_dev,
+		 "isc wb mode %d: hist min %u , max %u\n", hist_id, min, max);
+
 	ctrls->hist_minmax[hist_id][HIST_MIN_INDEX] = min;
 	ctrls->hist_minmax[hist_id][HIST_MAX_INDEX] = max;
 
-- 
2.25.1

[PATCH 21/21] media: atmel: atmel-isc-base: clamp wb gain coefficients

[Eugen Hristev]

White balance computed gains can overflow above the 13 bits hardware
coefficient that can be used, in some specific scenarios like a subexposure
from the sensor when the image is mostly black.
In this case the computed gain has to be clamped to the maximum value
allowed by the hardware.

Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
 drivers/media/platform/atmel/atmel-isc-base.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index 071ac5eec7ec..53141de78f67 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -1416,6 +1416,10 @@ static void isc_wb_update(struct isc_ctrls *ctrls)
 		/* multiply both gains and adjust for decimals */
 		ctrls->gain[c] = s_gain[c] * gw_gain[c];
 		ctrls->gain[c] >>= 9;
+
+		/* make sure we are not out of range */
+		ctrls->gain[c] = clamp_val(ctrls->gain[c], 0, GENMASK(12, 0));
+
 		v4l2_dbg(1, debug, &isc->v4l2_dev,
 			 "isc wb: component %d, final gain %u\n",
 			 c, ctrls->gain[c]);
-- 
2.25.1

Re: [PATCH 02/21] dt-bindings: media: atmel: csi2dc: add bindings for microchip csi2dc

[Rob Herring]

On Fri, Oct 22, 2021 at 10:52:28AM +0300, Eugen Hristev wrote:
> Add bindings documentation for Microchip CSI2 Demultiplexer controller.
> 
> CSI2DC is a demultiplexer from Synopsys IDI interface specification to
> parallel interface connection or direct memory access.
> 
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> ---
> Changes in this version :
> - fixed 'sink' name to be actually source.
> - added dma properties and example with dma
> 
> Previous change log:
> Changes in v5:
> - modified bindings as per Rob Herring review
> 
> Changes in v4:
> - Removed property for inter-line-delay and for clock continuous/non-continuous
> - Removed virtual channel by reg for second endpoint
> 
> Changes in v3:
> - Removed some text from description, as it was explained in the schema
> - fixed other things as per Rob's review
> - moved some text inside the schema, like the clock description
> 
> Changes in v2:
> - fixed warnings reported by dt_binding_check
> 
> 
>  .../bindings/media/microchip,csi2dc.yaml      | 149 ++++++++++++++++++
>  1 file changed, 149 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
> 
> diff --git a/Documentation/devicetree/bindings/media/microchip,csi2dc.yaml b/Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
> new file mode 100644
> index 000000000000..d317478908d0
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
> @@ -0,0 +1,149 @@
> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/media/microchip,csi2dc.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Microchip CSI2 Demux Controller (CSI2DC)
> +
> +maintainers:
> +  - Eugen Hristev <eugen.hristev@microchip.com>
> +
> +description:
> +  CSI2DC - Camera Serial Interface 2 Demux Controller
> +
> +  CSI2DC is a hardware block that receives incoming data from an IDI interface
> +  and filters packets based on their data type and virtual channel identifier,
> +  then converts the byte stream into a cross clock domain to a pixel stream
> +  to a parallel interface that can be read by a sensor controller.
> +  IDI interface is Synopsys proprietary.
> +
> +  CSI2DC provides two pipes, one video pipe and one data pipe. Video pipe
> +  is connected to a sensor controller and the data pipe is accessible
> +  as a DMA slave port to a DMA controller.
> +
> +  CSI2DC supports a single 'port' node as a sink port with Synopsys 32-bit
> +  IDI interface. The connected endpoint must be a IDI interface compatible
> +  device , that can provide 32-bit IDI interface connection as source port.
> +  For graph video endpoints please refer to the bindings defined in
> +  Documentation/devicetree/bindings/media/video-interfaces.txt.
> +  This port is mandatory.
> +
> +  CSI2DC supports one 'port' node as source port with parallel interface.
> +  This is called video pipe.
> +  This port has an 'endpoint' that can be connected to a sink port of another
> +  controller (next in pipeline).
> +  Please refer to the bindings defined in
> +  Documentation/devicetree/bindings/media/video-interfaces.txt.
> +
> +  CSI2DC also supports direct access to the data through AHB, via DMA channel,
> +  called data pipe.
> +  Because of this, the source 'port' child node (second) is not mandatory.
> +  If the source 'port' child node is missing, only data pipe is available.
> +  For data pipe to be available, a dma controller must be referenced.
> +
> +properties:
> +  compatible:
> +    const: microchip,sama7g5-csi2dc
> +
> +  reg:
> +    maxItems: 1
> +
> +  clocks:
> +    maxItems: 2
> +
> +  clock-names:
> +    description:
> +      CSI2DC must have two clocks to function correctly. One clock is the
> +      peripheral clock for the inside functionality of the hardware block.
> +      This is named 'pclk'. The second clock must be the cross domain clock,
> +      in which CSI2DC will perform clock crossing. This clock must be fed
> +      by the next controller in pipeline, which usually is a sensor controller.
> +      Normally this clock should be given by this sensor controller who
> +      is also a clock source. This clock is named 'scck', sensor controller clock.
> +    items:
> +      - const: pclk
> +      - const: scck
> +
> +  dmas:
> +    maxItems: 1
> +
> +  dma-names:
> +    const: rx
> +
> +  ports:
> +    type: object
> +    description:
> +      List of ports
> +
> +    properties:
> +      port@0:
> +        type: object
> +        description:
> +          Input port node, single endpoint describing the input port.
> +      port@1:
> +        type: object
> +        description:
> +          Output port node, single endpoint, describing the output port.

These need references to graph.yaml schema. See examples in the tree 
now.

> +
> +additionalProperties: false
> +
> +required:
> +  - compatible
> +  - reg
> +  - clocks
> +  - clock-names
> +  - ports
> +
> +examples:
> +  # Example for connecting to a parallel sensor controller block
> +  - |
> +    csi2dc@e1404000 {
> +        compatible = "microchip,sama7g5-csi2dc";
> +        reg = <0xe1404000 0x500>;
> +        clocks = <&pclk>, <&scck>;
> +        clock-names = "pclk", "scck";
> +
> +        ports {
> +               #address-cells = <1>;
> +               #size-cells = <0>;
> +               port@0 {
> +                       reg = <0>; /* must be 0, first child port */
> +                       csi2dc_in: endpoint { /* input from IDI interface */
> +                               remote-endpoint = <&csi2host_out>;
> +                       };
> +               };
> +
> +               port@1 {
> +                       reg = <1>; /* must be 1, second child port */
> +                       csi2dc_out: endpoint {
> +                               remote-endpoint = <&xisc_in>; /* output to sensor controller */
> +                       };
> +               };
> +        };
> +    };
> +
> +  # Example for connecting to a DMA master as an AHB slave
> +  - |
> +    #include <dt-bindings/dma/at91.h>
> +    csi2dc@e1404000 {
> +        compatible = "microchip,sama7g5-csi2dc";
> +        reg = <0xe1404000 0x500>;
> +        clocks = <&pclk>, <&scck>;
> +        clock-names = "pclk", "scck";
> +        dmas = <&dma0 AT91_XDMAC_DT_PERID(34)>;
> +        dma-names = "rx";
> +
> +        ports {
> +               #address-cells = <1>;
> +               #size-cells = <0>;
> +               port@0 {
> +                       reg = <0>; /* must be 0, first child port */
> +                       csi2dc_input: endpoint { /* input from IDI interface */
> +                               remote-endpoint = <&csi2host_out>;
> +                       };
> +               };
> +        };
> +    };
> +
> +...
> -- 
> 2.25.1
> 
> 

Re: [PATCH 03/21] media: atmel: introduce microchip csi2dc driver

[Jacopo Mondi]

Hi Eugen,

On Fri, Oct 22, 2021 at 10:52:29AM +0300, Eugen Hristev wrote:
> Microchip CSI2DC (CSI2 Demultiplexer Controller) is a misc bridge device
> that converts a byte stream in IDI Synopsys format (coming from a CSI2HOST)
> to a pixel stream that can be captured by a sensor controller.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> ---
> Changes in this revision:
> - addressed comments by Jacopo and Laurent as in this thread:
> https://www.spinics.net/lists/linux-media/msg181044.html
>
> Previous change log :
> Changes in v5:
> - only in bindings
>
> Changes in v4:
> - now using get_mbus_config ops to get data from the subdevice, like the
> virtual channel id, and the clock type.
> - now having possibility to select any of the RAW10 data modes
> - at completion time, select which formats are also available in the subdevice,
> and move to the dynamic list accordingly
> - changed the pipeline integration, do not advertise subdev ready at probe time.
> wait until completion is done, and then start a workqueue that will register
> this device as a subdevice for the next element in pipeline.
> - moved the s_power code into a different function called now csi2dc_power
> that is called with CONFIG_PM functions. This is also called at completion,
> to have the device ready in case CONFIG_PM is not selected on the platform.
> - merged try_fmt into set_fmt
> - driver cleanup, wrapped lines over 80 characters
>
> Changes in v2:
> - moved driver to platform/atmel
> - fixed minor things as per Sakari's review
> - still some things from v2 review are not yet addressed, to be followed up
>
>
>  drivers/media/platform/atmel/Kconfig          |  15 +
>  drivers/media/platform/atmel/Makefile         |   1 +
>  .../media/platform/atmel/microchip-csi2dc.c   | 700 ++++++++++++++++++
>  3 files changed, 716 insertions(+)
>  create mode 100644 drivers/media/platform/atmel/microchip-csi2dc.c
>
> diff --git a/drivers/media/platform/atmel/Kconfig b/drivers/media/platform/atmel/Kconfig
> index dda2f27da317..f83bee373d82 100644
> --- a/drivers/media/platform/atmel/Kconfig
> +++ b/drivers/media/platform/atmel/Kconfig
> @@ -40,3 +40,18 @@ config VIDEO_ATMEL_ISI
>  	help
>  	  This module makes the ATMEL Image Sensor Interface available
>  	  as a v4l2 device.
> +
> +config VIDEO_MICROCHIP_CSI2DC
> +	tristate "Microchip CSI2 Demux Controller"
> +	depends on VIDEO_V4L2 && COMMON_CLK && OF
> +	depends on ARCH_AT91 || COMPILE_TEST
> +	select MEDIA_CONTROLLER
> +	select VIDEO_V4L2_SUBDEV_API
> +	select V4L2_FWNODE
> +	help
> +	  CSI2 Demux Controller driver. CSI2DC is a helper chip
> +	  that converts IDI interface byte stream to a parallel pixel stream.
> +	  It supports various RAW formats as input.
> +
> +	  To compile this driver as a module, choose M here: the
> +	  module will be called microchip-csi2dc.
> diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
> index 46d264ab7948..39f0a7eba702 100644
> --- a/drivers/media/platform/atmel/Makefile
> +++ b/drivers/media/platform/atmel/Makefile
> @@ -6,3 +6,4 @@ obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
>  obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o
>  obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel-isc.o
>  obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel-xisc.o
> +obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += microchip-csi2dc.o
> diff --git a/drivers/media/platform/atmel/microchip-csi2dc.c b/drivers/media/platform/atmel/microchip-csi2dc.c
> new file mode 100644
> index 000000000000..277b86988eee
> --- /dev/null
> +++ b/drivers/media/platform/atmel/microchip-csi2dc.c
> @@ -0,0 +1,700 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Microchip CSI2 Demux Controller (CSI2DC) driver
> + *
> + * Copyright (C) 2018 Microchip Technology, Inc.
> + *
> + * Author: Eugen Hristev <eugen.hristev@microchip.com>
> + *
> + */
> +
> +#include <linux/clk.h>
> +#include <linux/module.h>
> +#include <linux/of.h>

Isn't linux/mod_devicetable.h enough ?

> +#include <linux/of_graph.h>

You should probably move to use the fwnode_graph framwork instead of
of_graph. This driver depends on OF so it shouldn't be an issue but I
defer this to maintainers

> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/videodev2.h>
> +
> +#include <media/v4l2-device.h>

Do you need this include ?

> +#include <media/v4l2-fwnode.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/videobuf2-dma-contig.h>

Is this one needed as well ?

> +
> +/* Global configuration register */
> +#define CSI2DC_GCFG			0x0
> +
> +/* MIPI sensor pixel clock is free running */
> +#define CSI2DC_GCFG_MIPIFRN		BIT(0)
> +/* Output waveform inter-line minimum delay */
> +#define CSI2DC_GCFG_HLC(v)		((v) << 4)
> +#define CSI2DC_GCFG_HLC_MASK		GENMASK(7, 4)
> +/* SAMA7G5 requires a HLC delay of 15 */
> +#define SAMA7G5_HLC			(15)
> +
> +/* Global control register */
> +#define CSI2DC_GCTLR			0x04
> +#define CSI2DC_GCTLR_SWRST		BIT(0)
> +
> +/* Global status register */
> +#define CSI2DC_GS			0x08
> +
> +/* SSP interrupt status register */
> +#define CSI2DC_SSPIS			0x28
> +/* Pipe update register */
> +#define CSI2DC_PU			0xC0

What about using lowercase for hex values (I know there's not strict
rule, so keep what you like the most, but most drivers use lowercase

> +/* Video pipe attributes update */
> +#define CSI2DC_PU_VP			BIT(0)
> +
> +/* Pipe update status register */
> +#define CSI2DC_PUS			0xC4
> +
> +/* Video pipeline enable register */
> +#define CSI2DC_VPE			0xF8
> +#define CSI2DC_VPE_ENABLE		BIT(0)
> +
> +/* Video pipeline configuration register */
> +#define CSI2DC_VPCFG			0xFC
> +/* Data type */
> +#define CSI2DC_VPCFG_DT(v)		((v) << 0)
> +#define CSI2DC_VPCFG_DT_MASK		GENMASK(5, 0)
> +/* Virtual channel identifier */
> +#define CSI2DC_VPCFG_VC(v)		((v) << 6)
> +#define CSI2DC_VPCFG_VC_MASK		GENMASK(7, 6)
> +/* Decompression enable */
> +#define CSI2DC_VPCFG_DE			BIT(8)
> +/* Decoder mode */
> +#define CSI2DC_VPCFG_DM(v)		((v) << 9)
> +#define CSI2DC_VPCFG_DM_DECODER8TO12	0
> +/* Decoder predictor 2 selection */
> +#define CSI2DC_VPCFG_DP2		BIT(12)
> +/* Recommended memory storage */
> +#define CSI2DC_VPCFG_RMS		BIT(13)
> +/* Post adjustment */
> +#define CSI2DC_VPCFG_PA			BIT(14)
> +
> +/* Video pipeline column register */
> +#define CSI2DC_VPCOL			0x100
> +/* Column number */
> +#define CSI2DC_VPCOL_COL(v)		((v) << 0)
> +#define CSI2DC_VPCOL_COL_MASK		GENMASK(15, 0)
> +
> +/* Video pipeline row register */
> +#define CSI2DC_VPROW			0x104
> +/* Row number */
> +#define CSI2DC_VPROW_ROW(v)		((v) << 0)
> +#define CSI2DC_VPROW_ROW_MASK		GENMASK(15, 0)
> +
> +/* Version register */
> +#define CSI2DC_VERSION			0x1FC
> +
> +/* register read/write helpers */
> +#define csi2dc_readl(st, reg)		readl_relaxed((st)->base + (reg))
> +#define csi2dc_writel(st, reg, val)	writel_relaxed((val), \
> +					(st)->base + (reg))
> +
> +/* supported RAW data types */
> +#define CSI2DC_DT_RAW6			0x28
> +#define CSI2DC_DT_RAW7			0x29
> +#define CSI2DC_DT_RAW8			0x2A
> +#define CSI2DC_DT_RAW10			0x2B
> +#define CSI2DC_DT_RAW12			0x2C
> +#define CSI2DC_DT_RAW14			0x2D
> +
> +/*
> + * struct csi2dc_format - CSI2DC format type struct
> + * @mbus_code:		Media bus code for the format
> + * @dt:			Data type constant for this format
> + */
> +struct csi2dc_format {
> +	u32				mbus_code;
> +	u32				dt;
> +};
> +
> +static const struct csi2dc_format csi2dc_formats[] = {
> +	{
> +		.mbus_code =		MEDIA_BUS_FMT_SRGGB10_1X10,
> +		.dt =			CSI2DC_DT_RAW10,
> +	}, {
> +		.mbus_code =		MEDIA_BUS_FMT_SBGGR10_1X10,
> +		.dt =			CSI2DC_DT_RAW10,
> +	}, {
> +		.mbus_code =		MEDIA_BUS_FMT_SGRBG10_1X10,
> +		.dt =			CSI2DC_DT_RAW10,
> +	}, {
> +		.mbus_code =		MEDIA_BUS_FMT_SGBRG10_1X10,
> +		.dt =			CSI2DC_DT_RAW10,
> +	},
> +};

How unfortunate we don't have this in the core...

> +
> +enum mipi_csi_pads {
> +	CSI2DC_PAD_SINK			= 0,
> +	CSI2DC_PAD_SOURCE		= 1,
> +	CSI2DC_PADS_NUM			= 2,
> +};
> +
> +/*
> + * struct csi2dc_device - CSI2DC device driver data/config struct
> + * @base:		Register map base address
> + * @csi2dc_sd:		v4l2 subdevice for the csi2dc device
> + *			This is the subdevice that the csi2dc device itself
> + *			registers in v4l2 subsystem
> + * @dev:		struct device for this csi2dc device
> + * @pclk:		Peripheral clock reference
> + *			Input clock that clocks the hardware block internal
> + *			logic
> + * @scck:		Sensor Controller clock reference
> + *			Input clock that is used to generate the pixel clock
> + * @format:		Current saved format used in g/s fmt
> + * @cur_fmt:		Current state format
> + * @try_fmt:		Try format that is being tried
> + * @pads:		Media entity pads for the csi2dc subdevice
> + * @clk_gated:		Whether the clock is gated or free running
> + * @video_pipe:		Whether video pipeline is configured
> + * @vc:			Current set virtual channel
> + * @asd:		Async subdevice for async bound of the underlying subdev
> + * @notifier:		Async notifier that is used to bound the underlying
> + *			subdevice to the csi2dc subdevice
> + * @input_sd:		Reference to the underlying subdevice bound to the
> + *			csi2dc subdevice
> + * @remote_pad:		Pad number of the underlying subdevice that is linked
> + *			to the csi2dc subdevice sink pad.
> + */
> +struct csi2dc_device {
> +	void __iomem			*base;
> +	struct v4l2_subdev		csi2dc_sd;
> +	struct device			*dev;
> +	struct clk			*pclk;
> +	struct clk			*scck;
> +
> +	struct v4l2_mbus_framefmt	 format;
> +
> +	const struct csi2dc_format	*cur_fmt;
> +	const struct csi2dc_format	*try_fmt;
> +
> +	struct media_pad		pads[CSI2DC_PADS_NUM];
> +
> +	bool				clk_gated;
> +	bool				video_pipe;
> +	u32				vc;
> +
> +	struct v4l2_async_subdev	*asd;
> +	struct v4l2_async_notifier	notifier;
> +
> +	struct v4l2_subdev		*input_sd;
> +
> +	u32				remote_pad;
> +};
> +
> +static void csi2dc_vp_update(struct csi2dc_device *csi2dc)

Could you move this below closer to the only caller ?

> +{
> +	u32 vp;
> +
> +	if (!csi2dc->cur_fmt) {

You should probably initialize this to a default format

> +		dev_err(csi2dc->dev, "format must be configured first\n");
> +		return;
> +	}
> +
> +	if (!csi2dc->video_pipe) {

This is only called internally by the driver at s_stream() time, can
this happen ? Or rather won't you have a streamon call also when the
data pipe only is available ? In that case you would error out here

> +		dev_err(csi2dc->dev, "video pipeline unavailable\n");
> +		return;
> +	}
> +
> +	vp = CSI2DC_VPCFG_DT(csi2dc->cur_fmt->dt) & CSI2DC_VPCFG_DT_MASK;
> +	vp |= CSI2DC_VPCFG_VC(csi2dc->vc) & CSI2DC_VPCFG_VC_MASK;
> +	vp &= ~CSI2DC_VPCFG_DE;
> +	vp |= CSI2DC_VPCFG_DM(CSI2DC_VPCFG_DM_DECODER8TO12);
> +	vp &= ~CSI2DC_VPCFG_DP2;
> +	vp &= ~CSI2DC_VPCFG_RMS;
> +	vp |= CSI2DC_VPCFG_PA;
> +
> +	csi2dc_writel(csi2dc, CSI2DC_VPCFG, vp);
> +	csi2dc_writel(csi2dc, CSI2DC_VPE, CSI2DC_VPE_ENABLE);
> +	csi2dc_writel(csi2dc, CSI2DC_PU, CSI2DC_PU_VP);
> +}
> +
> +static inline struct csi2dc_device *
> +csi2dc_sd_to_csi2dc_device(struct v4l2_subdev *csi2dc_sd)
> +{
> +	return container_of(csi2dc_sd, struct csi2dc_device, csi2dc_sd);
> +}
> +
> +static int csi2dc_enum_mbus_code(struct v4l2_subdev *csi2dc_sd,
> +				 struct v4l2_subdev_state *sd_state,
> +				 struct v4l2_subdev_mbus_code_enum *code)
> +{
> +	if (code->index >= ARRAY_SIZE(csi2dc_formats))
> +		return -EINVAL;
> +
> +	code->code = csi2dc_formats[code->index].mbus_code;
> +
> +	return 0;
> +}
> +
> +static int csi2dc_get_fmt(struct v4l2_subdev *csi2dc_sd,
> +			  struct v4l2_subdev_state *sd_state,
> +			  struct v4l2_subdev_format *format)
> +{
> +	struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
> +
> +	format->format = csi2dc->format;
> +
> +	return 0;

You should support try formats by storing the format in the file
handle state in s_fmt and return it in case which == TRY

Grep for v4l2_subdev_get_try_format() for usage examples

> +}
> +
> +static int csi2dc_set_fmt(struct v4l2_subdev *csi2dc_sd,
> +			  struct v4l2_subdev_state *sd_state,
> +			  struct v4l2_subdev_format *req_fmt)
> +{
> +	struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
> +	const struct csi2dc_format *fmt;
> +	int i;

unsigned

> +
> +	for (i = 0; i < ARRAY_SIZE(csi2dc_formats);  i++) {
> +		fmt = &csi2dc_formats[i];
> +		if (req_fmt->format.code == fmt->mbus_code)
> +			csi2dc->try_fmt = fmt;

Shouldn't you break ?

> +		fmt++;
> +	}

And make this a simpler

        for (i = 0; i < ARRAY_SIZE(csi2dc_formats); i++) {
                if (req_fmt->format.code == csi2dc_formats[i].mbus_code)
                        break;
        }

        if (i == ARRAY_SIZE(csi2dc_formats)
                i = 0;

> +
> +	/* in case we could not find the desired format, default to something */
> +	if (!csi2dc->try_fmt  ||
> +	    req_fmt->format.code != csi2dc->try_fmt->mbus_code) {
> +		csi2dc->try_fmt = &csi2dc_formats[0];
> +
> +		dev_dbg(csi2dc->dev,
> +			"CSI2DC unsupported format 0x%x, defaulting to 0x%x\n",
> +			req_fmt->format.code, csi2dc_formats[0].mbus_code);
> +
> +		req_fmt->format.code = csi2dc_formats[0].mbus_code;
> +	}
> +
> +	req_fmt->format.colorspace = V4L2_COLORSPACE_SRGB;
> +	req_fmt->format.field = V4L2_FIELD_NONE;
> +
> +	/* save the format for later requests */

You should support TRY formats

> +	csi2dc->format = req_fmt->format;
> +
> +	/* if we are just trying, we are done */
> +	if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY)
> +		return 0;
> +
> +	csi2dc->cur_fmt = csi2dc->try_fmt;

        csi2dc->cur_fmt = &csi2dc_format[i];

So you can drop the try_fmt from the driver structure as it seems to
be used as a temporary variable here only.

> +
> +	dev_dbg(csi2dc->dev, "new format set: 0x%x\n", req_fmt->format.code);
> +
> +	return 0;
> +}
> +
> +static int csi2dc_power(struct csi2dc_device *csi2dc, int on)
> +{
> +	int ret = 0;
> +
> +	if (on) {
> +		ret = clk_prepare_enable(csi2dc->pclk);
> +		if (ret) {
> +			dev_err(csi2dc->dev, "failed to enable pclk: %d\n", ret);
> +			return ret;
> +		}
> +
> +		ret = clk_prepare_enable(csi2dc->scck);
> +		if (ret)
> +			dev_err(csi2dc->dev,
> +				"failed to enable scck: %d\n", ret);

Shouldn't you bail out here ?

> +
> +		/* if powering up, deassert reset line */
> +		csi2dc_writel(csi2dc, CSI2DC_GCTLR, CSI2DC_GCTLR_SWRST);
> +	} else {
> +		clk_disable_unprepare(csi2dc->scck);
> +
> +		/* if powering down, assert reset line */
> +		csi2dc_writel(csi2dc, CSI2DC_GCTLR, !CSI2DC_GCTLR_SWRST);

Isn't reverse order of activation better ?

                csi2dc_writel(..)
                clk_disable_unprepare(..)
                clk_disable_unprepare(..)
> +
> +		clk_disable_unprepare(csi2dc->pclk);
> +	}
> +
> +	return ret;
> +}
> +
> +static int csi2dc_s_stream(struct v4l2_subdev *csi2dc_sd, int enable)
> +{
> +	struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
> +	int ret;
> +
> +	if (enable) {
> +		ret = pm_runtime_resume_and_get(csi2dc->dev);
> +		if (ret < 0)
> +			return ret;
> +		csi2dc_vp_update(csi2dc);
> +	} else {
> +		pm_runtime_put_sync(csi2dc->dev);
> +	}
> +
> +	return v4l2_subdev_call(csi2dc->input_sd, video, s_stream, enable);

Should the remote subdev be started before and stopped after ?

> +}
> +
> +static const struct v4l2_subdev_pad_ops csi2dc_pad_ops = {
> +	.enum_mbus_code = csi2dc_enum_mbus_code,
> +	.set_fmt = csi2dc_set_fmt,
> +	.get_fmt = csi2dc_get_fmt,
> +};
> +
> +static const struct v4l2_subdev_video_ops csi2dc_video_ops = {
> +	.s_stream = csi2dc_s_stream,
> +};
> +
> +static const struct v4l2_subdev_ops csi2dc_subdev_ops = {
> +	.pad = &csi2dc_pad_ops,
> +	.video = &csi2dc_video_ops,
> +};
> +
> +static int csi2dc_get_mbus_config(struct csi2dc_device *csi2dc)
> +{
> +	struct v4l2_mbus_config mbus_config = { 0 };
> +	int ret;
> +
> +	ret = v4l2_subdev_call(csi2dc->input_sd, pad, get_mbus_config,
> +			       csi2dc->remote_pad, &mbus_config);
> +	if (ret == -ENOIOCTLCMD) {
> +		dev_dbg(csi2dc->dev,
> +			"no remote mbus configuration available\n");
> +		goto csi2dc_get_mbus_config_defaults;
> +	}
> +
> +	if (ret) {
> +		dev_err(csi2dc->dev,
> +			"failed to get remote mbus configuration\n");
> +		goto csi2dc_get_mbus_config_defaults;
> +	}
> +
> +	if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_0)
> +		csi2dc->vc = 0;
> +	else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_1)
> +		csi2dc->vc = 1;
> +	else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_2)
> +		csi2dc->vc = 2;
> +	else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_3)
> +		csi2dc->vc = 3;
> +
> +	dev_dbg(csi2dc->dev, "subdev sending on channel %d\n", csi2dc->vc);
> +
> +	csi2dc->clk_gated = mbus_config.flags &
> +				V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK;

This should come from the default clock-noncontinuous property in the
endpoint. It is available in the mbus_configuration only to support
subdevs that can change it at runtime, and if that's the case it's ok,
but I think it should be in the endpoint.

Speaking of remote subdevs, is there any driver available for IDI
transmitters ?

> +
> +	dev_dbg(csi2dc->dev, "%s clock\n",
> +		csi2dc->clk_gated ? "gated" : "free running");
> +
> +	return 0;
> +
> +csi2dc_get_mbus_config_defaults:
> +	csi2dc->vc = 0; /* Virtual ID 0 by default */
> +	csi2dc->clk_gated = false; /* Free running clock by default */
> +
> +	return 0;
> +}
> +
> +static int csi2dc_async_bound(struct v4l2_async_notifier *notifier,
> +			      struct v4l2_subdev *subdev,
> +			      struct v4l2_async_subdev *asd)
> +{
> +	struct csi2dc_device *csi2dc = container_of(notifier,
> +						struct csi2dc_device, notifier);
> +	int pad;
> +	int ret;
> +
> +	csi2dc->input_sd = subdev;
> +
> +	pad = media_entity_get_fwnode_pad(&subdev->entity,

You can use 'ret'

> +					  asd->match.fwnode,
> +					  MEDIA_PAD_FL_SOURCE);
> +	if (pad < 0) {
> +		dev_err(csi2dc->dev, "Failed to find pad for %s\n",
> +			subdev->name);
> +		return pad;
> +	}
> +
> +	csi2dc->remote_pad = pad;
> +
> +	csi2dc_get_mbus_config(csi2dc);

Ideally, as this is not fatal, you could move this at s_stream time to
fetch the most up-to-date configuration

> +
> +	ret = media_create_pad_link(&csi2dc->input_sd->entity,
> +				    csi2dc->remote_pad,
> +				    &csi2dc->csi2dc_sd.entity, 0,
> +				    MEDIA_LNK_FL_ENABLED);
> +	if (ret < 0) {

        if (ret)

> +		dev_err(csi2dc->dev,
> +			"Failed to create pad link: %s to %s\n",
> +			csi2dc->input_sd->entity.name,
> +			csi2dc->csi2dc_sd.entity.name);
> +		return ret;
> +	}
> +
> +	dev_dbg(csi2dc->dev, "link with %s pad: %d\n",
> +		csi2dc->input_sd->name, csi2dc->remote_pad);
> +
> +	ret = pm_runtime_resume_and_get(csi2dc->dev);
> +	if (ret < 0)
> +		return ret;
> +
> +	csi2dc_writel(csi2dc, CSI2DC_GCFG,
> +		      (SAMA7G5_HLC & CSI2DC_GCFG_HLC_MASK) |
> +		      (csi2dc->clk_gated ? 0 : CSI2DC_GCFG_MIPIFRN));
> +
> +	csi2dc_writel(csi2dc, CSI2DC_VPCOL,
> +		      CSI2DC_VPCOL_COL(0xFFF) & CSI2DC_VPCOL_COL_MASK);
> +	csi2dc_writel(csi2dc, CSI2DC_VPROW,
> +		      CSI2DC_VPROW_ROW(0xFFF) & CSI2DC_VPROW_ROW_MASK);
> +
> +	pm_runtime_put_sync(csi2dc->dev);

I would really move access to the HW to s_stream time if possible

> +
> +	return ret;
> +}
> +
> +static const struct v4l2_async_notifier_operations csi2dc_async_ops = {
> +	.bound = csi2dc_async_bound,
> +};
> +
> +static void csi2dc_cleanup_notifier(struct csi2dc_device *csi2dc)
> +{
> +	v4l2_async_notifier_unregister(&csi2dc->notifier);
> +	v4l2_async_notifier_cleanup(&csi2dc->notifier);
> +}
> +
> +static int csi2dc_prepare_notifier(struct csi2dc_device *csi2dc,
> +				   struct device_node *input_node)
> +{
> +	int ret = 0;
> +
> +	v4l2_async_notifier_init(&csi2dc->notifier);
> +
> +	csi2dc->asd = v4l2_async_notifier_add_fwnode_remote_subdev

do you need asd in the driver structure ?

> +			(&csi2dc->notifier, of_fwnode_handle(input_node),
> +			struct v4l2_async_subdev);
> +
> +	of_node_put(input_node);
> +
> +	if (IS_ERR(csi2dc->asd)) {
> +		ret = PTR_ERR(csi2dc->asd);
> +		dev_err(csi2dc->dev,
> +			"failed to add async notifier for node %pOF: %d\n",
> +			input_node, ret);
> +		v4l2_async_notifier_cleanup(&csi2dc->notifier);
> +		return ret;
> +	}
> +
> +	csi2dc->notifier.ops = &csi2dc_async_ops;
> +
> +	ret = v4l2_async_subdev_notifier_register(&csi2dc->csi2dc_sd,
> +						  &csi2dc->notifier);
> +
> +	if (ret) {
> +		dev_err(csi2dc->dev, "fail to register async notifier: %d\n",
> +			ret);
> +		v4l2_async_notifier_cleanup(&csi2dc->notifier);
> +	}
> +
> +	return ret;
> +}
> +
> +static int csi2dc_of_parse(struct csi2dc_device *csi2dc,
> +			   struct device_node *of_node)
> +{
> +	struct device_node *input_node, *output_node;
> +	struct v4l2_fwnode_endpoint input_endpoint = { 0 },
> +				    output_endpoint = { 0 };
> +	int ret;
> +
> +	output_endpoint.bus_type = V4L2_MBUS_PARALLEL;
> +
> +	input_node = of_graph_get_next_endpoint(of_node, NULL);
> +
> +	if (!input_node) {
> +		dev_err(csi2dc->dev,
> +			"missing port node at %pOF, input node is mandatory.\n",
> +			of_node);
> +		return -EINVAL;
> +	}
> +
> +	ret = v4l2_fwnode_endpoint_parse(of_fwnode_handle(input_node),
> +					 &input_endpoint);
> +
> +	if (ret) {
                of_node_put(input_node);

> +		dev_err(csi2dc->dev, "endpoint not defined at %pOF\n", of_node);
> +		return ret;
> +	}
> +
> +	output_node = of_graph_get_next_endpoint(of_node, input_node);
> +
> +	if (output_node)
> +		ret = v4l2_fwnode_endpoint_parse(of_fwnode_handle(output_node),
> +						 &output_endpoint);

        of_node_put(output_node);
> +
> +	if (!output_node || ret) {
> +		dev_info(csi2dc->dev,
> +			 "missing output node at %pOF, data pipe available only.\n",
> +			 of_node);
> +	} else {
> +		csi2dc->video_pipe = true;
> +
> +		dev_dbg(csi2dc->dev, "block %pOF %d.%d->%d.%d video pipeline\n",
> +			of_node, input_endpoint.base.port,
> +			input_endpoint.base.id, output_endpoint.base.port,
> +			output_endpoint.base.id);
> +	}
> +
> +	of_node_put(output_node);

Drop this if you put it above

> +	of_node_put(input_node);

Should you put input_node before passing it to the function ?
> +
> +	/* prepare async notifier for subdevice completion */
> +	return csi2dc_prepare_notifier(csi2dc, input_node);
> +}
> +
> +static int csi2dc_probe(struct platform_device *pdev)
> +{
> +	struct device *dev = &pdev->dev;
> +	struct csi2dc_device *csi2dc;
> +	struct resource *res = NULL;
> +	int ret = 0;
> +	u32 ver;
> +
> +	csi2dc = devm_kzalloc(dev, sizeof(*csi2dc), GFP_KERNEL);
> +	if (!csi2dc)
> +		return -ENOMEM;
> +
> +	csi2dc->dev = dev;
> +
> +	res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
> +
> +	csi2dc->base = devm_ioremap_resource(dev, res);

Should devm_platform_ioremap_resource(pdev, 0) be used here ?

> +	if (IS_ERR(csi2dc->base)) {
> +		dev_err(dev, "base address not set\n");
> +		return PTR_ERR(csi2dc->base);
> +	}
> +
> +	csi2dc->pclk = devm_clk_get(dev, "pclk");
> +	if (IS_ERR(csi2dc->pclk)) {
> +		ret = PTR_ERR(csi2dc->pclk);
> +		dev_err(dev, "failed to get pclk: %d\n", ret);
> +		return ret;
> +	}
> +
> +	csi2dc->scck = devm_clk_get(dev, "scck");
> +	if (IS_ERR(csi2dc->scck)) {
> +		ret = PTR_ERR(csi2dc->scck);
> +		dev_err(dev, "failed to get scck: %d\n", ret);
> +		return ret;
> +	}
> +
> +	v4l2_subdev_init(&csi2dc->csi2dc_sd, &csi2dc_subdev_ops);
> +
> +	csi2dc->csi2dc_sd.owner = THIS_MODULE;
> +	csi2dc->csi2dc_sd.dev = dev;
> +	snprintf(csi2dc->csi2dc_sd.name, sizeof(csi2dc->csi2dc_sd.name),
> +		 "csi2dc");
> +
> +	csi2dc->csi2dc_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> +	csi2dc->csi2dc_sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> +	v4l2_set_subdevdata(&csi2dc->csi2dc_sd, pdev);

Not used it seems

> +
> +	platform_set_drvdata(pdev, csi2dc);
> +
> +	ret = csi2dc_of_parse(csi2dc, dev->of_node);
> +	if (ret)
> +		goto csi2dc_probe_cleanup_entity;
> +
> +	csi2dc->pads[CSI2DC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> +	if (csi2dc->video_pipe)
> +		csi2dc->pads[CSI2DC_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
> +
> +	ret = media_entity_pads_init(&csi2dc->csi2dc_sd.entity,
> +				     csi2dc->video_pipe ? CSI2DC_PADS_NUM : 1,
> +				     csi2dc->pads);
> +	if (ret < 0) {
> +		dev_err(dev, "media entity init failed\n");
> +		goto csi2dc_probe_cleanup_entity;

Should you also clean up the notifier in the error path ?

> +	}
> +
> +	/* turn power on to validate capabilities */
> +	ret = csi2dc_power(csi2dc, true);
> +	if (ret < 0)
> +		goto csi2dc_probe_cleanup_entity;
> +
> +	pm_runtime_set_active(dev);
> +	pm_runtime_enable(dev);
> +	ver = csi2dc_readl(csi2dc, CSI2DC_VERSION);
> +	pm_request_idle(dev);
> +
> +	/*
> +	 * we must register the subdev after PM runtime has been requested,
> +	 * otherwise we might bound immediately and request pm_runtime_resume
> +	 * before runtime_enable.
> +	 */
> +	ret = v4l2_async_register_subdev(&csi2dc->csi2dc_sd);
> +	if (ret) {
> +		dev_err(csi2dc->dev, "failed to register the subdevice\n");
> +		goto csi2dc_probe_cleanup_entity;
> +	}
> +
> +	dev_info(dev, "Microchip CSI2DC version %x\n", ver);
> +
> +	return 0;
> +
> +csi2dc_probe_cleanup_entity:
> +	media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
> +
> +	return ret;
> +}
> +
> +static int csi2dc_remove(struct platform_device *pdev)
> +{
> +	struct csi2dc_device *csi2dc = platform_get_drvdata(pdev);
> +
> +	pm_runtime_disable(&pdev->dev);
> +
> +	v4l2_async_unregister_subdev(&csi2dc->csi2dc_sd);
> +	csi2dc_cleanup_notifier(csi2dc);
> +	media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
> +
> +	return 0;
> +}
> +
> +static int __maybe_unused csi2dc_runtime_suspend(struct device *dev)
> +{
> +	struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
> +
> +	return csi2dc_power(csi2dc, false);
> +}
> +
> +static int __maybe_unused csi2dc_runtime_resume(struct device *dev)
> +{
> +	struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
> +
> +	return csi2dc_power(csi2dc, true);
> +}
> +
> +static const struct dev_pm_ops csi2dc_dev_pm_ops = {
> +	SET_RUNTIME_PM_OPS(csi2dc_runtime_suspend, csi2dc_runtime_resume, NULL)
> +};
> +
> +static const struct of_device_id csi2dc_of_match[] = {
> +	{ .compatible = "microchip,sama7g5-csi2dc" },
> +	{ }
> +};
> +
> +MODULE_DEVICE_TABLE(of, csi2dc_of_match);
> +
> +static struct platform_driver csi2dc_driver = {
> +	.probe	= csi2dc_probe,
> +	.remove = csi2dc_remove,
> +	.driver = {
> +		.name =			"microchip-csi2dc",
> +		.pm =			&csi2dc_dev_pm_ops,
> +		.of_match_table =	of_match_ptr(csi2dc_of_match),
> +	},
> +};
> +
> +module_platform_driver(csi2dc_driver);
> +
> +MODULE_AUTHOR("Eugen Hristev <eugen.hristev@microchip.com>");
> +MODULE_DESCRIPTION("Microchip CSI2 Demux Controller driver");
> +MODULE_LICENSE("GPL v2");
> --
> 2.25.1
>

Re: [PATCH 03/21] media: atmel: introduce microchip csi2dc driver

[Eugen.Hristev]

On 11/2/21 7:22 PM, Jacopo Mondi wrote:
> Hi Eugen,
> 

Hi,

Thank you for your review. I will try to understand everything you said 
and come up with a new version. I still have some inline questions 
though, about some things which are still unclear.

> On Fri, Oct 22, 2021 at 10:52:29AM +0300, Eugen Hristev wrote:
>> Microchip CSI2DC (CSI2 Demultiplexer Controller) is a misc bridge device
>> that converts a byte stream in IDI Synopsys format (coming from a CSI2HOST)
>> to a pixel stream that can be captured by a sensor controller.
>>
>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>> ---
>> Changes in this revision:
>> - addressed comments by Jacopo and Laurent as in this thread:
>> https://www.spinics.net/lists/linux-media/msg181044.html
>>
>> Previous change log :
>> Changes in v5:
>> - only in bindings
>>
>> Changes in v4:
>> - now using get_mbus_config ops to get data from the subdevice, like the
>> virtual channel id, and the clock type.
>> - now having possibility to select any of the RAW10 data modes
>> - at completion time, select which formats are also available in the subdevice,
>> and move to the dynamic list accordingly
>> - changed the pipeline integration, do not advertise subdev ready at probe time.
>> wait until completion is done, and then start a workqueue that will register
>> this device as a subdevice for the next element in pipeline.
>> - moved the s_power code into a different function called now csi2dc_power
>> that is called with CONFIG_PM functions. This is also called at completion,
>> to have the device ready in case CONFIG_PM is not selected on the platform.
>> - merged try_fmt into set_fmt
>> - driver cleanup, wrapped lines over 80 characters
>>
>> Changes in v2:
>> - moved driver to platform/atmel
>> - fixed minor things as per Sakari's review
>> - still some things from v2 review are not yet addressed, to be followed up
>>
>>
>>   drivers/media/platform/atmel/Kconfig          |  15 +
>>   drivers/media/platform/atmel/Makefile         |   1 +
>>   .../media/platform/atmel/microchip-csi2dc.c   | 700 ++++++++++++++++++
>>   3 files changed, 716 insertions(+)
>>   create mode 100644 drivers/media/platform/atmel/microchip-csi2dc.c
>>
>> diff --git a/drivers/media/platform/atmel/Kconfig b/drivers/media/platform/atmel/Kconfig
>> index dda2f27da317..f83bee373d82 100644
>> --- a/drivers/media/platform/atmel/Kconfig
>> +++ b/drivers/media/platform/atmel/Kconfig
>> @@ -40,3 +40,18 @@ config VIDEO_ATMEL_ISI
>>        help
>>          This module makes the ATMEL Image Sensor Interface available
>>          as a v4l2 device.
>> +
>> +config VIDEO_MICROCHIP_CSI2DC
>> +     tristate "Microchip CSI2 Demux Controller"
>> +     depends on VIDEO_V4L2 && COMMON_CLK && OF
>> +     depends on ARCH_AT91 || COMPILE_TEST
>> +     select MEDIA_CONTROLLER
>> +     select VIDEO_V4L2_SUBDEV_API
>> +     select V4L2_FWNODE
>> +     help
>> +       CSI2 Demux Controller driver. CSI2DC is a helper chip
>> +       that converts IDI interface byte stream to a parallel pixel stream.
>> +       It supports various RAW formats as input.
>> +
>> +       To compile this driver as a module, choose M here: the
>> +       module will be called microchip-csi2dc.
>> diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
>> index 46d264ab7948..39f0a7eba702 100644
>> --- a/drivers/media/platform/atmel/Makefile
>> +++ b/drivers/media/platform/atmel/Makefile
>> @@ -6,3 +6,4 @@ obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
>>   obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o
>>   obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel-isc.o
>>   obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel-xisc.o
>> +obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += microchip-csi2dc.o
>> diff --git a/drivers/media/platform/atmel/microchip-csi2dc.c b/drivers/media/platform/atmel/microchip-csi2dc.c
>> new file mode 100644
>> index 000000000000..277b86988eee
>> --- /dev/null
>> +++ b/drivers/media/platform/atmel/microchip-csi2dc.c
>> @@ -0,0 +1,700 @@
>> +// SPDX-License-Identifier: GPL-2.0-only
>> +/*
>> + * Microchip CSI2 Demux Controller (CSI2DC) driver
>> + *
>> + * Copyright (C) 2018 Microchip Technology, Inc.
>> + *
>> + * Author: Eugen Hristev <eugen.hristev@microchip.com>
>> + *
>> + */
>> +
>> +#include <linux/clk.h>
>> +#include <linux/module.h>
>> +#include <linux/of.h>
> 
> Isn't linux/mod_devicetable.h enough ?
> 
>> +#include <linux/of_graph.h>
> 
> You should probably move to use the fwnode_graph framwork instead of
> of_graph. This driver depends on OF so it shouldn't be an issue but I
> defer this to maintainers
> 
>> +#include <linux/platform_device.h>
>> +#include <linux/pm_runtime.h>
>> +#include <linux/videodev2.h>
>> +
>> +#include <media/v4l2-device.h>
> 
> Do you need this include ?
> 
>> +#include <media/v4l2-fwnode.h>
>> +#include <media/v4l2-subdev.h>
>> +#include <media/videobuf2-dma-contig.h>
> 
> Is this one needed as well ?
> 
>> +
>> +/* Global configuration register */
>> +#define CSI2DC_GCFG                  0x0
>> +
>> +/* MIPI sensor pixel clock is free running */
>> +#define CSI2DC_GCFG_MIPIFRN          BIT(0)
>> +/* Output waveform inter-line minimum delay */
>> +#define CSI2DC_GCFG_HLC(v)           ((v) << 4)
>> +#define CSI2DC_GCFG_HLC_MASK         GENMASK(7, 4)
>> +/* SAMA7G5 requires a HLC delay of 15 */
>> +#define SAMA7G5_HLC                  (15)
>> +
>> +/* Global control register */
>> +#define CSI2DC_GCTLR                 0x04
>> +#define CSI2DC_GCTLR_SWRST           BIT(0)
>> +
>> +/* Global status register */
>> +#define CSI2DC_GS                    0x08
>> +
>> +/* SSP interrupt status register */
>> +#define CSI2DC_SSPIS                 0x28
>> +/* Pipe update register */
>> +#define CSI2DC_PU                    0xC0
> 
> What about using lowercase for hex values (I know there's not strict
> rule, so keep what you like the most, but most drivers use lowercase
> 
>> +/* Video pipe attributes update */
>> +#define CSI2DC_PU_VP                 BIT(0)
>> +
>> +/* Pipe update status register */
>> +#define CSI2DC_PUS                   0xC4
>> +
>> +/* Video pipeline enable register */
>> +#define CSI2DC_VPE                   0xF8
>> +#define CSI2DC_VPE_ENABLE            BIT(0)
>> +
>> +/* Video pipeline configuration register */
>> +#define CSI2DC_VPCFG                 0xFC
>> +/* Data type */
>> +#define CSI2DC_VPCFG_DT(v)           ((v) << 0)
>> +#define CSI2DC_VPCFG_DT_MASK         GENMASK(5, 0)
>> +/* Virtual channel identifier */
>> +#define CSI2DC_VPCFG_VC(v)           ((v) << 6)
>> +#define CSI2DC_VPCFG_VC_MASK         GENMASK(7, 6)
>> +/* Decompression enable */
>> +#define CSI2DC_VPCFG_DE                      BIT(8)
>> +/* Decoder mode */
>> +#define CSI2DC_VPCFG_DM(v)           ((v) << 9)
>> +#define CSI2DC_VPCFG_DM_DECODER8TO12 0
>> +/* Decoder predictor 2 selection */
>> +#define CSI2DC_VPCFG_DP2             BIT(12)
>> +/* Recommended memory storage */
>> +#define CSI2DC_VPCFG_RMS             BIT(13)
>> +/* Post adjustment */
>> +#define CSI2DC_VPCFG_PA                      BIT(14)
>> +
>> +/* Video pipeline column register */
>> +#define CSI2DC_VPCOL                 0x100
>> +/* Column number */
>> +#define CSI2DC_VPCOL_COL(v)          ((v) << 0)
>> +#define CSI2DC_VPCOL_COL_MASK                GENMASK(15, 0)
>> +
>> +/* Video pipeline row register */
>> +#define CSI2DC_VPROW                 0x104
>> +/* Row number */
>> +#define CSI2DC_VPROW_ROW(v)          ((v) << 0)
>> +#define CSI2DC_VPROW_ROW_MASK                GENMASK(15, 0)
>> +
>> +/* Version register */
>> +#define CSI2DC_VERSION                       0x1FC
>> +
>> +/* register read/write helpers */
>> +#define csi2dc_readl(st, reg)                readl_relaxed((st)->base + (reg))
>> +#define csi2dc_writel(st, reg, val)  writel_relaxed((val), \
>> +                                     (st)->base + (reg))
>> +
>> +/* supported RAW data types */
>> +#define CSI2DC_DT_RAW6                       0x28
>> +#define CSI2DC_DT_RAW7                       0x29
>> +#define CSI2DC_DT_RAW8                       0x2A
>> +#define CSI2DC_DT_RAW10                      0x2B
>> +#define CSI2DC_DT_RAW12                      0x2C
>> +#define CSI2DC_DT_RAW14                      0x2D
>> +
>> +/*
>> + * struct csi2dc_format - CSI2DC format type struct
>> + * @mbus_code:               Media bus code for the format
>> + * @dt:                      Data type constant for this format
>> + */
>> +struct csi2dc_format {
>> +     u32                             mbus_code;
>> +     u32                             dt;
>> +};
>> +
>> +static const struct csi2dc_format csi2dc_formats[] = {
>> +     {
>> +             .mbus_code =            MEDIA_BUS_FMT_SRGGB10_1X10,
>> +             .dt =                   CSI2DC_DT_RAW10,
>> +     }, {
>> +             .mbus_code =            MEDIA_BUS_FMT_SBGGR10_1X10,
>> +             .dt =                   CSI2DC_DT_RAW10,
>> +     }, {
>> +             .mbus_code =            MEDIA_BUS_FMT_SGRBG10_1X10,
>> +             .dt =                   CSI2DC_DT_RAW10,
>> +     }, {
>> +             .mbus_code =            MEDIA_BUS_FMT_SGBRG10_1X10,
>> +             .dt =                   CSI2DC_DT_RAW10,
>> +     },
>> +};
> 
> How unfortunate we don't have this in the core...
> 
>> +
>> +enum mipi_csi_pads {
>> +     CSI2DC_PAD_SINK                 = 0,
>> +     CSI2DC_PAD_SOURCE               = 1,
>> +     CSI2DC_PADS_NUM                 = 2,
>> +};
>> +
>> +/*
>> + * struct csi2dc_device - CSI2DC device driver data/config struct
>> + * @base:            Register map base address
>> + * @csi2dc_sd:               v4l2 subdevice for the csi2dc device
>> + *                   This is the subdevice that the csi2dc device itself
>> + *                   registers in v4l2 subsystem
>> + * @dev:             struct device for this csi2dc device
>> + * @pclk:            Peripheral clock reference
>> + *                   Input clock that clocks the hardware block internal
>> + *                   logic
>> + * @scck:            Sensor Controller clock reference
>> + *                   Input clock that is used to generate the pixel clock
>> + * @format:          Current saved format used in g/s fmt
>> + * @cur_fmt:         Current state format
>> + * @try_fmt:         Try format that is being tried
>> + * @pads:            Media entity pads for the csi2dc subdevice
>> + * @clk_gated:               Whether the clock is gated or free running
>> + * @video_pipe:              Whether video pipeline is configured
>> + * @vc:                      Current set virtual channel
>> + * @asd:             Async subdevice for async bound of the underlying subdev
>> + * @notifier:                Async notifier that is used to bound the underlying
>> + *                   subdevice to the csi2dc subdevice
>> + * @input_sd:                Reference to the underlying subdevice bound to the
>> + *                   csi2dc subdevice
>> + * @remote_pad:              Pad number of the underlying subdevice that is linked
>> + *                   to the csi2dc subdevice sink pad.
>> + */
>> +struct csi2dc_device {
>> +     void __iomem                    *base;
>> +     struct v4l2_subdev              csi2dc_sd;
>> +     struct device                   *dev;
>> +     struct clk                      *pclk;
>> +     struct clk                      *scck;
>> +
>> +     struct v4l2_mbus_framefmt        format;
>> +
>> +     const struct csi2dc_format      *cur_fmt;
>> +     const struct csi2dc_format      *try_fmt;
>> +
>> +     struct media_pad                pads[CSI2DC_PADS_NUM];
>> +
>> +     bool                            clk_gated;
>> +     bool                            video_pipe;
>> +     u32                             vc;
>> +
>> +     struct v4l2_async_subdev        *asd;
>> +     struct v4l2_async_notifier      notifier;
>> +
>> +     struct v4l2_subdev              *input_sd;
>> +
>> +     u32                             remote_pad;
>> +};
>> +
>> +static void csi2dc_vp_update(struct csi2dc_device *csi2dc)
> 
> Could you move this below closer to the only caller ?
> 
>> +{
>> +     u32 vp;
>> +
>> +     if (!csi2dc->cur_fmt) {
> 
> You should probably initialize this to a default format
> 
>> +             dev_err(csi2dc->dev, "format must be configured first\n");
>> +             return;
>> +     }
>> +
>> +     if (!csi2dc->video_pipe) {
> 
> This is only called internally by the driver at s_stream() time, can
> this happen ? Or rather won't you have a streamon call also when the
> data pipe only is available ? In that case you would error out here

This can happen if the source node is not found in the DT. In that case, 
there is no video pipe available, thus this whole function should be a 
no-op (csi2dc_vp_update) . If no vp, there is no vp update.
Normall without a vp, there should be a dp (data pipe), but I haven't 
tested this and it's not supported in the driver.
It doesn't make any sense to configure the vp registers below if the vp 
is not available.
However, I shouldn't return an error since the vp is not mandatory for 
the function of the hardware. Just that the dp will be implemented at a 
later time (.... or never).

> 
>> +             dev_err(csi2dc->dev, "video pipeline unavailable\n");
>> +             return;
>> +     }
>> +
>> +     vp = CSI2DC_VPCFG_DT(csi2dc->cur_fmt->dt) & CSI2DC_VPCFG_DT_MASK;
>> +     vp |= CSI2DC_VPCFG_VC(csi2dc->vc) & CSI2DC_VPCFG_VC_MASK;
>> +     vp &= ~CSI2DC_VPCFG_DE;
>> +     vp |= CSI2DC_VPCFG_DM(CSI2DC_VPCFG_DM_DECODER8TO12);
>> +     vp &= ~CSI2DC_VPCFG_DP2;
>> +     vp &= ~CSI2DC_VPCFG_RMS;
>> +     vp |= CSI2DC_VPCFG_PA;
>> +
>> +     csi2dc_writel(csi2dc, CSI2DC_VPCFG, vp);
>> +     csi2dc_writel(csi2dc, CSI2DC_VPE, CSI2DC_VPE_ENABLE);
>> +     csi2dc_writel(csi2dc, CSI2DC_PU, CSI2DC_PU_VP);
>> +}
>> +
>> +static inline struct csi2dc_device *
>> +csi2dc_sd_to_csi2dc_device(struct v4l2_subdev *csi2dc_sd)
>> +{
>> +     return container_of(csi2dc_sd, struct csi2dc_device, csi2dc_sd);
>> +}
>> +
>> +static int csi2dc_enum_mbus_code(struct v4l2_subdev *csi2dc_sd,
>> +                              struct v4l2_subdev_state *sd_state,
>> +                              struct v4l2_subdev_mbus_code_enum *code)
>> +{
>> +     if (code->index >= ARRAY_SIZE(csi2dc_formats))
>> +             return -EINVAL;
>> +
>> +     code->code = csi2dc_formats[code->index].mbus_code;
>> +
>> +     return 0;
>> +}
>> +
>> +static int csi2dc_get_fmt(struct v4l2_subdev *csi2dc_sd,
>> +                       struct v4l2_subdev_state *sd_state,
>> +                       struct v4l2_subdev_format *format)
>> +{
>> +     struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
>> +
>> +     format->format = csi2dc->format;
>> +
>> +     return 0;
> 
> You should support try formats by storing the format in the file
> handle state in s_fmt and return it in case which == TRY
> 
> Grep for v4l2_subdev_get_try_format() for usage examples

I did that initially, but could not find any utility for it in my tests. 
So I removed it.
I will try to bring it back, but I have no idea how to test that the try 
format works fine or not

> 
>> +}
>> +
>> +static int csi2dc_set_fmt(struct v4l2_subdev *csi2dc_sd,
>> +                       struct v4l2_subdev_state *sd_state,
>> +                       struct v4l2_subdev_format *req_fmt)
>> +{
>> +     struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
>> +     const struct csi2dc_format *fmt;
>> +     int i;
> 
> unsigned
> 
>> +
>> +     for (i = 0; i < ARRAY_SIZE(csi2dc_formats);  i++) {
>> +             fmt = &csi2dc_formats[i];
>> +             if (req_fmt->format.code == fmt->mbus_code)
>> +                     csi2dc->try_fmt = fmt;
> 
> Shouldn't you break ?
> 
>> +             fmt++;
>> +     }
> 
> And make this a simpler
> 
>          for (i = 0; i < ARRAY_SIZE(csi2dc_formats); i++) {
>                  if (req_fmt->format.code == csi2dc_formats[i].mbus_code)
>                          break;
>          }
> 
>          if (i == ARRAY_SIZE(csi2dc_formats)
>                  i = 0;
> 
>> +
>> +     /* in case we could not find the desired format, default to something */
>> +     if (!csi2dc->try_fmt  ||
>> +         req_fmt->format.code != csi2dc->try_fmt->mbus_code) {
>> +             csi2dc->try_fmt = &csi2dc_formats[0];
>> +
>> +             dev_dbg(csi2dc->dev,
>> +                     "CSI2DC unsupported format 0x%x, defaulting to 0x%x\n",
>> +                     req_fmt->format.code, csi2dc_formats[0].mbus_code);
>> +
>> +             req_fmt->format.code = csi2dc_formats[0].mbus_code;
>> +     }
>> +
>> +     req_fmt->format.colorspace = V4L2_COLORSPACE_SRGB;
>> +     req_fmt->format.field = V4L2_FIELD_NONE;
>> +
>> +     /* save the format for later requests */
> 
> You should support TRY formats
> 
>> +     csi2dc->format = req_fmt->format;
>> +
>> +     /* if we are just trying, we are done */
>> +     if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY)
>> +             return 0;
>> +
>> +     csi2dc->cur_fmt = csi2dc->try_fmt;
> 
>          csi2dc->cur_fmt = &csi2dc_format[i];
> 
> So you can drop the try_fmt from the driver structure as it seems to
> be used as a temporary variable here only.
> 
>> +
>> +     dev_dbg(csi2dc->dev, "new format set: 0x%x\n", req_fmt->format.code);
>> +
>> +     return 0;
>> +}
>> +
>> +static int csi2dc_power(struct csi2dc_device *csi2dc, int on)
>> +{
>> +     int ret = 0;
>> +
>> +     if (on) {
>> +             ret = clk_prepare_enable(csi2dc->pclk);
>> +             if (ret) {
>> +                     dev_err(csi2dc->dev, "failed to enable pclk: %d\n", ret);
>> +                     return ret;
>> +             }
>> +
>> +             ret = clk_prepare_enable(csi2dc->scck);
>> +             if (ret)
>> +                     dev_err(csi2dc->dev,
>> +                             "failed to enable scck: %d\n", ret);
> 
> Shouldn't you bail out here ?

Initially this clock was not mandatory, but you are right, I should bail 
out.

> 
>> +
>> +             /* if powering up, deassert reset line */
>> +             csi2dc_writel(csi2dc, CSI2DC_GCTLR, CSI2DC_GCTLR_SWRST);
>> +     } else {
>> +             clk_disable_unprepare(csi2dc->scck);
>> +
>> +             /* if powering down, assert reset line */
>> +             csi2dc_writel(csi2dc, CSI2DC_GCTLR, !CSI2DC_GCTLR_SWRST);
> 
> Isn't reverse order of activation better ?
> 
>                  csi2dc_writel(..)
>                  clk_disable_unprepare(..)
>                  clk_disable_unprepare(..)
>> +
>> +             clk_disable_unprepare(csi2dc->pclk);
>> +     }
>> +
>> +     return ret;
>> +}
>> +
>> +static int csi2dc_s_stream(struct v4l2_subdev *csi2dc_sd, int enable)
>> +{
>> +     struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
>> +     int ret;
>> +
>> +     if (enable) {
>> +             ret = pm_runtime_resume_and_get(csi2dc->dev);
>> +             if (ret < 0)
>> +                     return ret;
>> +             csi2dc_vp_update(csi2dc);
>> +     } else {
>> +             pm_runtime_put_sync(csi2dc->dev);
>> +     }
>> +
>> +     return v4l2_subdev_call(csi2dc->input_sd, video, s_stream, enable);
> 
> Should the remote subdev be started before and stopped after ?

Do you mean s_power ?

> 
>> +}
>> +
>> +static const struct v4l2_subdev_pad_ops csi2dc_pad_ops = {
>> +     .enum_mbus_code = csi2dc_enum_mbus_code,
>> +     .set_fmt = csi2dc_set_fmt,
>> +     .get_fmt = csi2dc_get_fmt,
>> +};
>> +
>> +static const struct v4l2_subdev_video_ops csi2dc_video_ops = {
>> +     .s_stream = csi2dc_s_stream,
>> +};
>> +
>> +static const struct v4l2_subdev_ops csi2dc_subdev_ops = {
>> +     .pad = &csi2dc_pad_ops,
>> +     .video = &csi2dc_video_ops,
>> +};
>> +
>> +static int csi2dc_get_mbus_config(struct csi2dc_device *csi2dc)
>> +{
>> +     struct v4l2_mbus_config mbus_config = { 0 };
>> +     int ret;
>> +
>> +     ret = v4l2_subdev_call(csi2dc->input_sd, pad, get_mbus_config,
>> +                            csi2dc->remote_pad, &mbus_config);
>> +     if (ret == -ENOIOCTLCMD) {
>> +             dev_dbg(csi2dc->dev,
>> +                     "no remote mbus configuration available\n");
>> +             goto csi2dc_get_mbus_config_defaults;
>> +     }
>> +
>> +     if (ret) {
>> +             dev_err(csi2dc->dev,
>> +                     "failed to get remote mbus configuration\n");
>> +             goto csi2dc_get_mbus_config_defaults;
>> +     }
>> +
>> +     if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_0)
>> +             csi2dc->vc = 0;
>> +     else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_1)
>> +             csi2dc->vc = 1;
>> +     else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_2)
>> +             csi2dc->vc = 2;
>> +     else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_3)
>> +             csi2dc->vc = 3;
>> +
>> +     dev_dbg(csi2dc->dev, "subdev sending on channel %d\n", csi2dc->vc);
>> +
>> +     csi2dc->clk_gated = mbus_config.flags &
>> +                             V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK;
> 
> This should come from the default clock-noncontinuous property in the
> endpoint. It is available in the mbus_configuration only to support
> subdevs that can change it at runtime, and if that's the case it's ok,
> but I think it should be in the endpoint.

I do not completely understand your point. In a previous version which 
you reviewed, I had this as a DT property (so in the endpoint ?), and 
now I converted it to a get_mbus_config flag get operation.
It's not the right way to do it ?
How should it be specified in the endpoint ?

> 
> Speaking of remote subdevs, is there any driver available for IDI
> transmitters ?

I am not aware of any at the moment
> 
>> +
>> +     dev_dbg(csi2dc->dev, "%s clock\n",
>> +             csi2dc->clk_gated ? "gated" : "free running");
>> +
>> +     return 0;
>> +
>> +csi2dc_get_mbus_config_defaults:
>> +     csi2dc->vc = 0; /* Virtual ID 0 by default */
>> +     csi2dc->clk_gated = false; /* Free running clock by default */
>> +
>> +     return 0;
>> +}
>> +
>> +static int csi2dc_async_bound(struct v4l2_async_notifier *notifier,
>> +                           struct v4l2_subdev *subdev,
>> +                           struct v4l2_async_subdev *asd)
>> +{
>> +     struct csi2dc_device *csi2dc = container_of(notifier,
>> +                                             struct csi2dc_device, notifier);
>> +     int pad;
>> +     int ret;
>> +
>> +     csi2dc->input_sd = subdev;
>> +
>> +     pad = media_entity_get_fwnode_pad(&subdev->entity,
> 
> You can use 'ret'

I can, but it is weird to do remote_pad = ret ...

May bring confusion, don't you agree ?

> 
>> +                                       asd->match.fwnode,
>> +                                       MEDIA_PAD_FL_SOURCE);
>> +     if (pad < 0) {
>> +             dev_err(csi2dc->dev, "Failed to find pad for %s\n",
>> +                     subdev->name);
>> +             return pad;
>> +     }
>> +
>> +     csi2dc->remote_pad = pad;
>> +
>> +     csi2dc_get_mbus_config(csi2dc);
> 
> Ideally, as this is not fatal, you could move this at s_stream time to
> fetch the most up-to-date configuration
> 
>> +
>> +     ret = media_create_pad_link(&csi2dc->input_sd->entity,
>> +                                 csi2dc->remote_pad,
>> +                                 &csi2dc->csi2dc_sd.entity, 0,
>> +                                 MEDIA_LNK_FL_ENABLED);
>> +     if (ret < 0) {
> 
>          if (ret)
> 
>> +             dev_err(csi2dc->dev,
>> +                     "Failed to create pad link: %s to %s\n",
>> +                     csi2dc->input_sd->entity.name,
>> +                     csi2dc->csi2dc_sd.entity.name);
>> +             return ret;
>> +     }
>> +
>> +     dev_dbg(csi2dc->dev, "link with %s pad: %d\n",
>> +             csi2dc->input_sd->name, csi2dc->remote_pad);
>> +
>> +     ret = pm_runtime_resume_and_get(csi2dc->dev);
>> +     if (ret < 0)
>> +             return ret;
>> +
>> +     csi2dc_writel(csi2dc, CSI2DC_GCFG,
>> +                   (SAMA7G5_HLC & CSI2DC_GCFG_HLC_MASK) |
>> +                   (csi2dc->clk_gated ? 0 : CSI2DC_GCFG_MIPIFRN));
>> +
>> +     csi2dc_writel(csi2dc, CSI2DC_VPCOL,
>> +                   CSI2DC_VPCOL_COL(0xFFF) & CSI2DC_VPCOL_COL_MASK);
>> +     csi2dc_writel(csi2dc, CSI2DC_VPROW,
>> +                   CSI2DC_VPROW_ROW(0xFFF) & CSI2DC_VPROW_ROW_MASK);
>> +
>> +     pm_runtime_put_sync(csi2dc->dev);
> 
> I would really move access to the HW to s_stream time if possible
> 
>> +
>> +     return ret;
>> +}
>> +
>> +static const struct v4l2_async_notifier_operations csi2dc_async_ops = {
>> +     .bound = csi2dc_async_bound,
>> +};
>> +
>> +static void csi2dc_cleanup_notifier(struct csi2dc_device *csi2dc)
>> +{
>> +     v4l2_async_notifier_unregister(&csi2dc->notifier);
>> +     v4l2_async_notifier_cleanup(&csi2dc->notifier);
>> +}
>> +
>> +static int csi2dc_prepare_notifier(struct csi2dc_device *csi2dc,
>> +                                struct device_node *input_node)
>> +{
>> +     int ret = 0;
>> +
>> +     v4l2_async_notifier_init(&csi2dc->notifier);
>> +
>> +     csi2dc->asd = v4l2_async_notifier_add_fwnode_remote_subdev
> 
> do you need asd in the driver structure ?
> 
>> +                     (&csi2dc->notifier, of_fwnode_handle(input_node),
>> +                     struct v4l2_async_subdev);
>> +
>> +     of_node_put(input_node);
>> +
>> +     if (IS_ERR(csi2dc->asd)) {
>> +             ret = PTR_ERR(csi2dc->asd);
>> +             dev_err(csi2dc->dev,
>> +                     "failed to add async notifier for node %pOF: %d\n",
>> +                     input_node, ret);
>> +             v4l2_async_notifier_cleanup(&csi2dc->notifier);
>> +             return ret;
>> +     }
>> +
>> +     csi2dc->notifier.ops = &csi2dc_async_ops;
>> +
>> +     ret = v4l2_async_subdev_notifier_register(&csi2dc->csi2dc_sd,
>> +                                               &csi2dc->notifier);
>> +
>> +     if (ret) {
>> +             dev_err(csi2dc->dev, "fail to register async notifier: %d\n",
>> +                     ret);
>> +             v4l2_async_notifier_cleanup(&csi2dc->notifier);
>> +     }
>> +
>> +     return ret;
>> +}
>> +
>> +static int csi2dc_of_parse(struct csi2dc_device *csi2dc,
>> +                        struct device_node *of_node)
>> +{
>> +     struct device_node *input_node, *output_node;
>> +     struct v4l2_fwnode_endpoint input_endpoint = { 0 },
>> +                                 output_endpoint = { 0 };
>> +     int ret;
>> +
>> +     output_endpoint.bus_type = V4L2_MBUS_PARALLEL;
>> +
>> +     input_node = of_graph_get_next_endpoint(of_node, NULL);
>> +
>> +     if (!input_node) {
>> +             dev_err(csi2dc->dev,
>> +                     "missing port node at %pOF, input node is mandatory.\n",
>> +                     of_node);
>> +             return -EINVAL;
>> +     }
>> +
>> +     ret = v4l2_fwnode_endpoint_parse(of_fwnode_handle(input_node),
>> +                                      &input_endpoint);
>> +
>> +     if (ret) {
>                  of_node_put(input_node);
> 
>> +             dev_err(csi2dc->dev, "endpoint not defined at %pOF\n", of_node);
>> +             return ret;
>> +     }
>> +
>> +     output_node = of_graph_get_next_endpoint(of_node, input_node);
>> +
>> +     if (output_node)
>> +             ret = v4l2_fwnode_endpoint_parse(of_fwnode_handle(output_node),
>> +                                              &output_endpoint);
> 
>          of_node_put(output_node);
>> +
>> +     if (!output_node || ret) {
>> +             dev_info(csi2dc->dev,
>> +                      "missing output node at %pOF, data pipe available only.\n",
>> +                      of_node);
>> +     } else {
>> +             csi2dc->video_pipe = true;
>> +
>> +             dev_dbg(csi2dc->dev, "block %pOF %d.%d->%d.%d video pipeline\n",
>> +                     of_node, input_endpoint.base.port,
>> +                     input_endpoint.base.id, output_endpoint.base.port,
>> +                     output_endpoint.base.id);
>> +     }
>> +
>> +     of_node_put(output_node);
> 
> Drop this if you put it above
> 
>> +     of_node_put(input_node);
> 
> Should you put input_node before passing it to the function ?
>> +
>> +     /* prepare async notifier for subdevice completion */
>> +     return csi2dc_prepare_notifier(csi2dc, input_node);
>> +}
>> +
>> +static int csi2dc_probe(struct platform_device *pdev)
>> +{
>> +     struct device *dev = &pdev->dev;
>> +     struct csi2dc_device *csi2dc;
>> +     struct resource *res = NULL;
>> +     int ret = 0;
>> +     u32 ver;
>> +
>> +     csi2dc = devm_kzalloc(dev, sizeof(*csi2dc), GFP_KERNEL);
>> +     if (!csi2dc)
>> +             return -ENOMEM;
>> +
>> +     csi2dc->dev = dev;
>> +
>> +     res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
>> +
>> +     csi2dc->base = devm_ioremap_resource(dev, res);
> 
> Should devm_platform_ioremap_resource(pdev, 0) be used here ?
> 
>> +     if (IS_ERR(csi2dc->base)) {
>> +             dev_err(dev, "base address not set\n");
>> +             return PTR_ERR(csi2dc->base);
>> +     }
>> +
>> +     csi2dc->pclk = devm_clk_get(dev, "pclk");
>> +     if (IS_ERR(csi2dc->pclk)) {
>> +             ret = PTR_ERR(csi2dc->pclk);
>> +             dev_err(dev, "failed to get pclk: %d\n", ret);
>> +             return ret;
>> +     }
>> +
>> +     csi2dc->scck = devm_clk_get(dev, "scck");
>> +     if (IS_ERR(csi2dc->scck)) {
>> +             ret = PTR_ERR(csi2dc->scck);
>> +             dev_err(dev, "failed to get scck: %d\n", ret);
>> +             return ret;
>> +     }
>> +
>> +     v4l2_subdev_init(&csi2dc->csi2dc_sd, &csi2dc_subdev_ops);
>> +
>> +     csi2dc->csi2dc_sd.owner = THIS_MODULE;
>> +     csi2dc->csi2dc_sd.dev = dev;
>> +     snprintf(csi2dc->csi2dc_sd.name, sizeof(csi2dc->csi2dc_sd.name),
>> +              "csi2dc");
>> +
>> +     csi2dc->csi2dc_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
>> +     csi2dc->csi2dc_sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
>> +     v4l2_set_subdevdata(&csi2dc->csi2dc_sd, pdev);
> 
> Not used it seems
> 
>> +
>> +     platform_set_drvdata(pdev, csi2dc);
>> +
>> +     ret = csi2dc_of_parse(csi2dc, dev->of_node);
>> +     if (ret)
>> +             goto csi2dc_probe_cleanup_entity;
>> +
>> +     csi2dc->pads[CSI2DC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>> +     if (csi2dc->video_pipe)
>> +             csi2dc->pads[CSI2DC_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
>> +
>> +     ret = media_entity_pads_init(&csi2dc->csi2dc_sd.entity,
>> +                                  csi2dc->video_pipe ? CSI2DC_PADS_NUM : 1,
>> +                                  csi2dc->pads);
>> +     if (ret < 0) {
>> +             dev_err(dev, "media entity init failed\n");
>> +             goto csi2dc_probe_cleanup_entity;
> 
> Should you also clean up the notifier in the error path ?
> 
>> +     }
>> +
>> +     /* turn power on to validate capabilities */
>> +     ret = csi2dc_power(csi2dc, true);
>> +     if (ret < 0)
>> +             goto csi2dc_probe_cleanup_entity;
>> +
>> +     pm_runtime_set_active(dev);
>> +     pm_runtime_enable(dev);
>> +     ver = csi2dc_readl(csi2dc, CSI2DC_VERSION);
>> +     pm_request_idle(dev);
>> +
>> +     /*
>> +      * we must register the subdev after PM runtime has been requested,
>> +      * otherwise we might bound immediately and request pm_runtime_resume
>> +      * before runtime_enable.
>> +      */
>> +     ret = v4l2_async_register_subdev(&csi2dc->csi2dc_sd);
>> +     if (ret) {
>> +             dev_err(csi2dc->dev, "failed to register the subdevice\n");
>> +             goto csi2dc_probe_cleanup_entity;
>> +     }
>> +
>> +     dev_info(dev, "Microchip CSI2DC version %x\n", ver);
>> +
>> +     return 0;
>> +
>> +csi2dc_probe_cleanup_entity:
>> +     media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
>> +
>> +     return ret;
>> +}
>> +
>> +static int csi2dc_remove(struct platform_device *pdev)
>> +{
>> +     struct csi2dc_device *csi2dc = platform_get_drvdata(pdev);
>> +
>> +     pm_runtime_disable(&pdev->dev);
>> +
>> +     v4l2_async_unregister_subdev(&csi2dc->csi2dc_sd);
>> +     csi2dc_cleanup_notifier(csi2dc);
>> +     media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
>> +
>> +     return 0;
>> +}
>> +
>> +static int __maybe_unused csi2dc_runtime_suspend(struct device *dev)
>> +{
>> +     struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
>> +
>> +     return csi2dc_power(csi2dc, false);
>> +}
>> +
>> +static int __maybe_unused csi2dc_runtime_resume(struct device *dev)
>> +{
>> +     struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
>> +
>> +     return csi2dc_power(csi2dc, true);
>> +}
>> +
>> +static const struct dev_pm_ops csi2dc_dev_pm_ops = {
>> +     SET_RUNTIME_PM_OPS(csi2dc_runtime_suspend, csi2dc_runtime_resume, NULL)
>> +};
>> +
>> +static const struct of_device_id csi2dc_of_match[] = {
>> +     { .compatible = "microchip,sama7g5-csi2dc" },
>> +     { }
>> +};
>> +
>> +MODULE_DEVICE_TABLE(of, csi2dc_of_match);
>> +
>> +static struct platform_driver csi2dc_driver = {
>> +     .probe  = csi2dc_probe,
>> +     .remove = csi2dc_remove,
>> +     .driver = {
>> +             .name =                 "microchip-csi2dc",
>> +             .pm =                   &csi2dc_dev_pm_ops,
>> +             .of_match_table =       of_match_ptr(csi2dc_of_match),
>> +     },
>> +};
>> +
>> +module_platform_driver(csi2dc_driver);
>> +
>> +MODULE_AUTHOR("Eugen Hristev <eugen.hristev@microchip.com>");
>> +MODULE_DESCRIPTION("Microchip CSI2 Demux Controller driver");
>> +MODULE_LICENSE("GPL v2");
>> --
>> 2.25.1
>>

Re: [PATCH 03/21] media: atmel: introduce microchip csi2dc driver

[Jacopo Mondi]

Hi Eugen

On Wed, Nov 03, 2021 at 08:31:36AM +0000, Eugen.Hristev@microchip.com wrote:
> On 11/2/21 7:22 PM, Jacopo Mondi wrote:
> > Hi Eugen,
> >
>
> Hi,
>
> Thank you for your review. I will try to understand everything you said
> and come up with a new version. I still have some inline questions
> though, about some things which are still unclear.
>
> > On Fri, Oct 22, 2021 at 10:52:29AM +0300, Eugen Hristev wrote:
> >> Microchip CSI2DC (CSI2 Demultiplexer Controller) is a misc bridge device
> >> that converts a byte stream in IDI Synopsys format (coming from a CSI2HOST)
> >> to a pixel stream that can be captured by a sensor controller.
> >>
> >> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> >> ---
> >> Changes in this revision:
> >> - addressed comments by Jacopo and Laurent as in this thread:
> >> https://www.spinics.net/lists/linux-media/msg181044.html
> >>
> >> Previous change log :
> >> Changes in v5:
> >> - only in bindings
> >>
> >> Changes in v4:
> >> - now using get_mbus_config ops to get data from the subdevice, like the
> >> virtual channel id, and the clock type.
> >> - now having possibility to select any of the RAW10 data modes
> >> - at completion time, select which formats are also available in the subdevice,
> >> and move to the dynamic list accordingly
> >> - changed the pipeline integration, do not advertise subdev ready at probe time.
> >> wait until completion is done, and then start a workqueue that will register
> >> this device as a subdevice for the next element in pipeline.
> >> - moved the s_power code into a different function called now csi2dc_power
> >> that is called with CONFIG_PM functions. This is also called at completion,
> >> to have the device ready in case CONFIG_PM is not selected on the platform.
> >> - merged try_fmt into set_fmt
> >> - driver cleanup, wrapped lines over 80 characters
> >>
> >> Changes in v2:
> >> - moved driver to platform/atmel
> >> - fixed minor things as per Sakari's review
> >> - still some things from v2 review are not yet addressed, to be followed up
> >>
> >>
> >>   drivers/media/platform/atmel/Kconfig          |  15 +
> >>   drivers/media/platform/atmel/Makefile         |   1 +
> >>   .../media/platform/atmel/microchip-csi2dc.c   | 700 ++++++++++++++++++
> >>   3 files changed, 716 insertions(+)
> >>   create mode 100644 drivers/media/platform/atmel/microchip-csi2dc.c
> >>
> >> diff --git a/drivers/media/platform/atmel/Kconfig b/drivers/media/platform/atmel/Kconfig
> >> index dda2f27da317..f83bee373d82 100644
> >> --- a/drivers/media/platform/atmel/Kconfig
> >> +++ b/drivers/media/platform/atmel/Kconfig
> >> @@ -40,3 +40,18 @@ config VIDEO_ATMEL_ISI
> >>        help
> >>          This module makes the ATMEL Image Sensor Interface available
> >>          as a v4l2 device.
> >> +
> >> +config VIDEO_MICROCHIP_CSI2DC
> >> +     tristate "Microchip CSI2 Demux Controller"
> >> +     depends on VIDEO_V4L2 && COMMON_CLK && OF
> >> +     depends on ARCH_AT91 || COMPILE_TEST
> >> +     select MEDIA_CONTROLLER
> >> +     select VIDEO_V4L2_SUBDEV_API
> >> +     select V4L2_FWNODE
> >> +     help
> >> +       CSI2 Demux Controller driver. CSI2DC is a helper chip
> >> +       that converts IDI interface byte stream to a parallel pixel stream.
> >> +       It supports various RAW formats as input.
> >> +
> >> +       To compile this driver as a module, choose M here: the
> >> +       module will be called microchip-csi2dc.
> >> diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
> >> index 46d264ab7948..39f0a7eba702 100644
> >> --- a/drivers/media/platform/atmel/Makefile
> >> +++ b/drivers/media/platform/atmel/Makefile
> >> @@ -6,3 +6,4 @@ obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
> >>   obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o
> >>   obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel-isc.o
> >>   obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel-xisc.o
> >> +obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += microchip-csi2dc.o
> >> diff --git a/drivers/media/platform/atmel/microchip-csi2dc.c b/drivers/media/platform/atmel/microchip-csi2dc.c
> >> new file mode 100644
> >> index 000000000000..277b86988eee
> >> --- /dev/null
> >> +++ b/drivers/media/platform/atmel/microchip-csi2dc.c
> >> @@ -0,0 +1,700 @@
> >> +// SPDX-License-Identifier: GPL-2.0-only
> >> +/*
> >> + * Microchip CSI2 Demux Controller (CSI2DC) driver
> >> + *
> >> + * Copyright (C) 2018 Microchip Technology, Inc.
> >> + *
> >> + * Author: Eugen Hristev <eugen.hristev@microchip.com>
> >> + *
> >> + */
> >> +
> >> +#include <linux/clk.h>
> >> +#include <linux/module.h>
> >> +#include <linux/of.h>
> >
> > Isn't linux/mod_devicetable.h enough ?
> >
> >> +#include <linux/of_graph.h>
> >
> > You should probably move to use the fwnode_graph framwork instead of
> > of_graph. This driver depends on OF so it shouldn't be an issue but I
> > defer this to maintainers
> >
> >> +#include <linux/platform_device.h>
> >> +#include <linux/pm_runtime.h>
> >> +#include <linux/videodev2.h>
> >> +
> >> +#include <media/v4l2-device.h>
> >
> > Do you need this include ?
> >
> >> +#include <media/v4l2-fwnode.h>
> >> +#include <media/v4l2-subdev.h>
> >> +#include <media/videobuf2-dma-contig.h>
> >
> > Is this one needed as well ?
> >
> >> +
> >> +/* Global configuration register */
> >> +#define CSI2DC_GCFG                  0x0
> >> +
> >> +/* MIPI sensor pixel clock is free running */
> >> +#define CSI2DC_GCFG_MIPIFRN          BIT(0)
> >> +/* Output waveform inter-line minimum delay */
> >> +#define CSI2DC_GCFG_HLC(v)           ((v) << 4)
> >> +#define CSI2DC_GCFG_HLC_MASK         GENMASK(7, 4)
> >> +/* SAMA7G5 requires a HLC delay of 15 */
> >> +#define SAMA7G5_HLC                  (15)
> >> +
> >> +/* Global control register */
> >> +#define CSI2DC_GCTLR                 0x04
> >> +#define CSI2DC_GCTLR_SWRST           BIT(0)
> >> +
> >> +/* Global status register */
> >> +#define CSI2DC_GS                    0x08
> >> +
> >> +/* SSP interrupt status register */
> >> +#define CSI2DC_SSPIS                 0x28
> >> +/* Pipe update register */
> >> +#define CSI2DC_PU                    0xC0
> >
> > What about using lowercase for hex values (I know there's not strict
> > rule, so keep what you like the most, but most drivers use lowercase
> >
> >> +/* Video pipe attributes update */
> >> +#define CSI2DC_PU_VP                 BIT(0)
> >> +
> >> +/* Pipe update status register */
> >> +#define CSI2DC_PUS                   0xC4
> >> +
> >> +/* Video pipeline enable register */
> >> +#define CSI2DC_VPE                   0xF8
> >> +#define CSI2DC_VPE_ENABLE            BIT(0)
> >> +
> >> +/* Video pipeline configuration register */
> >> +#define CSI2DC_VPCFG                 0xFC
> >> +/* Data type */
> >> +#define CSI2DC_VPCFG_DT(v)           ((v) << 0)
> >> +#define CSI2DC_VPCFG_DT_MASK         GENMASK(5, 0)
> >> +/* Virtual channel identifier */
> >> +#define CSI2DC_VPCFG_VC(v)           ((v) << 6)
> >> +#define CSI2DC_VPCFG_VC_MASK         GENMASK(7, 6)
> >> +/* Decompression enable */
> >> +#define CSI2DC_VPCFG_DE                      BIT(8)
> >> +/* Decoder mode */
> >> +#define CSI2DC_VPCFG_DM(v)           ((v) << 9)
> >> +#define CSI2DC_VPCFG_DM_DECODER8TO12 0
> >> +/* Decoder predictor 2 selection */
> >> +#define CSI2DC_VPCFG_DP2             BIT(12)
> >> +/* Recommended memory storage */
> >> +#define CSI2DC_VPCFG_RMS             BIT(13)
> >> +/* Post adjustment */
> >> +#define CSI2DC_VPCFG_PA                      BIT(14)
> >> +
> >> +/* Video pipeline column register */
> >> +#define CSI2DC_VPCOL                 0x100
> >> +/* Column number */
> >> +#define CSI2DC_VPCOL_COL(v)          ((v) << 0)
> >> +#define CSI2DC_VPCOL_COL_MASK                GENMASK(15, 0)
> >> +
> >> +/* Video pipeline row register */
> >> +#define CSI2DC_VPROW                 0x104
> >> +/* Row number */
> >> +#define CSI2DC_VPROW_ROW(v)          ((v) << 0)
> >> +#define CSI2DC_VPROW_ROW_MASK                GENMASK(15, 0)
> >> +
> >> +/* Version register */
> >> +#define CSI2DC_VERSION                       0x1FC
> >> +
> >> +/* register read/write helpers */
> >> +#define csi2dc_readl(st, reg)                readl_relaxed((st)->base + (reg))
> >> +#define csi2dc_writel(st, reg, val)  writel_relaxed((val), \
> >> +                                     (st)->base + (reg))
> >> +
> >> +/* supported RAW data types */
> >> +#define CSI2DC_DT_RAW6                       0x28
> >> +#define CSI2DC_DT_RAW7                       0x29
> >> +#define CSI2DC_DT_RAW8                       0x2A
> >> +#define CSI2DC_DT_RAW10                      0x2B
> >> +#define CSI2DC_DT_RAW12                      0x2C
> >> +#define CSI2DC_DT_RAW14                      0x2D
> >> +
> >> +/*
> >> + * struct csi2dc_format - CSI2DC format type struct
> >> + * @mbus_code:               Media bus code for the format
> >> + * @dt:                      Data type constant for this format
> >> + */
> >> +struct csi2dc_format {
> >> +     u32                             mbus_code;
> >> +     u32                             dt;
> >> +};
> >> +
> >> +static const struct csi2dc_format csi2dc_formats[] = {
> >> +     {
> >> +             .mbus_code =            MEDIA_BUS_FMT_SRGGB10_1X10,
> >> +             .dt =                   CSI2DC_DT_RAW10,
> >> +     }, {
> >> +             .mbus_code =            MEDIA_BUS_FMT_SBGGR10_1X10,
> >> +             .dt =                   CSI2DC_DT_RAW10,
> >> +     }, {
> >> +             .mbus_code =            MEDIA_BUS_FMT_SGRBG10_1X10,
> >> +             .dt =                   CSI2DC_DT_RAW10,
> >> +     }, {
> >> +             .mbus_code =            MEDIA_BUS_FMT_SGBRG10_1X10,
> >> +             .dt =                   CSI2DC_DT_RAW10,
> >> +     },
> >> +};
> >
> > How unfortunate we don't have this in the core...
> >
> >> +
> >> +enum mipi_csi_pads {
> >> +     CSI2DC_PAD_SINK                 = 0,
> >> +     CSI2DC_PAD_SOURCE               = 1,
> >> +     CSI2DC_PADS_NUM                 = 2,
> >> +};
> >> +
> >> +/*
> >> + * struct csi2dc_device - CSI2DC device driver data/config struct
> >> + * @base:            Register map base address
> >> + * @csi2dc_sd:               v4l2 subdevice for the csi2dc device
> >> + *                   This is the subdevice that the csi2dc device itself
> >> + *                   registers in v4l2 subsystem
> >> + * @dev:             struct device for this csi2dc device
> >> + * @pclk:            Peripheral clock reference
> >> + *                   Input clock that clocks the hardware block internal
> >> + *                   logic
> >> + * @scck:            Sensor Controller clock reference
> >> + *                   Input clock that is used to generate the pixel clock
> >> + * @format:          Current saved format used in g/s fmt
> >> + * @cur_fmt:         Current state format
> >> + * @try_fmt:         Try format that is being tried
> >> + * @pads:            Media entity pads for the csi2dc subdevice
> >> + * @clk_gated:               Whether the clock is gated or free running
> >> + * @video_pipe:              Whether video pipeline is configured
> >> + * @vc:                      Current set virtual channel
> >> + * @asd:             Async subdevice for async bound of the underlying subdev
> >> + * @notifier:                Async notifier that is used to bound the underlying
> >> + *                   subdevice to the csi2dc subdevice
> >> + * @input_sd:                Reference to the underlying subdevice bound to the
> >> + *                   csi2dc subdevice
> >> + * @remote_pad:              Pad number of the underlying subdevice that is linked
> >> + *                   to the csi2dc subdevice sink pad.
> >> + */
> >> +struct csi2dc_device {
> >> +     void __iomem                    *base;
> >> +     struct v4l2_subdev              csi2dc_sd;
> >> +     struct device                   *dev;
> >> +     struct clk                      *pclk;
> >> +     struct clk                      *scck;
> >> +
> >> +     struct v4l2_mbus_framefmt        format;
> >> +
> >> +     const struct csi2dc_format      *cur_fmt;
> >> +     const struct csi2dc_format      *try_fmt;
> >> +
> >> +     struct media_pad                pads[CSI2DC_PADS_NUM];
> >> +
> >> +     bool                            clk_gated;
> >> +     bool                            video_pipe;
> >> +     u32                             vc;
> >> +
> >> +     struct v4l2_async_subdev        *asd;
> >> +     struct v4l2_async_notifier      notifier;
> >> +
> >> +     struct v4l2_subdev              *input_sd;
> >> +
> >> +     u32                             remote_pad;
> >> +};
> >> +
> >> +static void csi2dc_vp_update(struct csi2dc_device *csi2dc)
> >
> > Could you move this below closer to the only caller ?
> >
> >> +{
> >> +     u32 vp;
> >> +
> >> +     if (!csi2dc->cur_fmt) {
> >
> > You should probably initialize this to a default format
> >
> >> +             dev_err(csi2dc->dev, "format must be configured first\n");
> >> +             return;
> >> +     }
> >> +
> >> +     if (!csi2dc->video_pipe) {
> >
> > This is only called internally by the driver at s_stream() time, can
> > this happen ? Or rather won't you have a streamon call also when the
> > data pipe only is available ? In that case you would error out here
>
> This can happen if the source node is not found in the DT. In that case,
> there is no video pipe available, thus this whole function should be a
> no-op (csi2dc_vp_update) . If no vp, there is no vp update.
> Normall without a vp, there should be a dp (data pipe), but I haven't
> tested this and it's not supported in the driver.
> It doesn't make any sense to configure the vp registers below if the vp
> is not available.
> However, I shouldn't return an error since the vp is not mandatory for
> the function of the hardware. Just that the dp will be implemented at a
> later time (.... or never).
>

I see, so the data pipe is still work in progress... I understand
bindings should allow for that to be later accepted so you cannot
mandate port@1 to be there all the times, but should the driver
instead refuse to operate if no port@1 is provided ?

Out of curiosity what will the interaction model with the driver be in
data pipe mode ? There will be a video device but there won't be an
link to it, how is stream start/stop controlled ?

> >
> >> +             dev_err(csi2dc->dev, "video pipeline unavailable\n");
> >> +             return;
> >> +     }
> >> +
> >> +     vp = CSI2DC_VPCFG_DT(csi2dc->cur_fmt->dt) & CSI2DC_VPCFG_DT_MASK;
> >> +     vp |= CSI2DC_VPCFG_VC(csi2dc->vc) & CSI2DC_VPCFG_VC_MASK;
> >> +     vp &= ~CSI2DC_VPCFG_DE;
> >> +     vp |= CSI2DC_VPCFG_DM(CSI2DC_VPCFG_DM_DECODER8TO12);
> >> +     vp &= ~CSI2DC_VPCFG_DP2;
> >> +     vp &= ~CSI2DC_VPCFG_RMS;
> >> +     vp |= CSI2DC_VPCFG_PA;
> >> +
> >> +     csi2dc_writel(csi2dc, CSI2DC_VPCFG, vp);
> >> +     csi2dc_writel(csi2dc, CSI2DC_VPE, CSI2DC_VPE_ENABLE);
> >> +     csi2dc_writel(csi2dc, CSI2DC_PU, CSI2DC_PU_VP);
> >> +}
> >> +
> >> +static inline struct csi2dc_device *
> >> +csi2dc_sd_to_csi2dc_device(struct v4l2_subdev *csi2dc_sd)
> >> +{
> >> +     return container_of(csi2dc_sd, struct csi2dc_device, csi2dc_sd);
> >> +}
> >> +
> >> +static int csi2dc_enum_mbus_code(struct v4l2_subdev *csi2dc_sd,
> >> +                              struct v4l2_subdev_state *sd_state,
> >> +                              struct v4l2_subdev_mbus_code_enum *code)
> >> +{
> >> +     if (code->index >= ARRAY_SIZE(csi2dc_formats))
> >> +             return -EINVAL;
> >> +
> >> +     code->code = csi2dc_formats[code->index].mbus_code;
> >> +
> >> +     return 0;
> >> +}
> >> +
> >> +static int csi2dc_get_fmt(struct v4l2_subdev *csi2dc_sd,
> >> +                       struct v4l2_subdev_state *sd_state,
> >> +                       struct v4l2_subdev_format *format)
> >> +{
> >> +     struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
> >> +
> >> +     format->format = csi2dc->format;
> >> +
> >> +     return 0;
> >
> > You should support try formats by storing the format in the file
> > handle state in s_fmt and return it in case which == TRY
> >
> > Grep for v4l2_subdev_get_try_format() for usage examples
>
> I did that initially, but could not find any utility for it in my tests.

I understand.. Have you run v4l2-compliance on this sudev ? Doesn't it
test try formats support ?

> So I removed it.
> I will try to bring it back, but I have no idea how to test that the try
> format works fine or not
>

I think also v4l2-ctl supports try formats.

--try-subdev-fmt pad=<pad>,width=<w>,height=<h>,code=<code>,field=<f>,colorspace=<c>,

I'm not sure if it allows you to read it back, but it's easy to hack
it out just to verify it.

> >
> >> +}
> >> +
> >> +static int csi2dc_set_fmt(struct v4l2_subdev *csi2dc_sd,
> >> +                       struct v4l2_subdev_state *sd_state,
> >> +                       struct v4l2_subdev_format *req_fmt)
> >> +{
> >> +     struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
> >> +     const struct csi2dc_format *fmt;
> >> +     int i;
> >
> > unsigned
> >
> >> +
> >> +     for (i = 0; i < ARRAY_SIZE(csi2dc_formats);  i++) {
> >> +             fmt = &csi2dc_formats[i];
> >> +             if (req_fmt->format.code == fmt->mbus_code)
> >> +                     csi2dc->try_fmt = fmt;
> >
> > Shouldn't you break ?
> >
> >> +             fmt++;
> >> +     }
> >
> > And make this a simpler
> >
> >          for (i = 0; i < ARRAY_SIZE(csi2dc_formats); i++) {
> >                  if (req_fmt->format.code == csi2dc_formats[i].mbus_code)
> >                          break;
> >          }
> >
> >          if (i == ARRAY_SIZE(csi2dc_formats)
> >                  i = 0;
> >
> >> +
> >> +     /* in case we could not find the desired format, default to something */
> >> +     if (!csi2dc->try_fmt  ||
> >> +         req_fmt->format.code != csi2dc->try_fmt->mbus_code) {
> >> +             csi2dc->try_fmt = &csi2dc_formats[0];
> >> +
> >> +             dev_dbg(csi2dc->dev,
> >> +                     "CSI2DC unsupported format 0x%x, defaulting to 0x%x\n",
> >> +                     req_fmt->format.code, csi2dc_formats[0].mbus_code);
> >> +
> >> +             req_fmt->format.code = csi2dc_formats[0].mbus_code;
> >> +     }
> >> +
> >> +     req_fmt->format.colorspace = V4L2_COLORSPACE_SRGB;
> >> +     req_fmt->format.field = V4L2_FIELD_NONE;
> >> +
> >> +     /* save the format for later requests */
> >
> > You should support TRY formats
> >
> >> +     csi2dc->format = req_fmt->format;
> >> +
> >> +     /* if we are just trying, we are done */
> >> +     if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY)
> >> +             return 0;
> >> +
> >> +     csi2dc->cur_fmt = csi2dc->try_fmt;
> >
> >          csi2dc->cur_fmt = &csi2dc_format[i];
> >
> > So you can drop the try_fmt from the driver structure as it seems to
> > be used as a temporary variable here only.
> >
> >> +
> >> +     dev_dbg(csi2dc->dev, "new format set: 0x%x\n", req_fmt->format.code);
> >> +
> >> +     return 0;
> >> +}
> >> +
> >> +static int csi2dc_power(struct csi2dc_device *csi2dc, int on)
> >> +{
> >> +     int ret = 0;
> >> +
> >> +     if (on) {
> >> +             ret = clk_prepare_enable(csi2dc->pclk);
> >> +             if (ret) {
> >> +                     dev_err(csi2dc->dev, "failed to enable pclk: %d\n", ret);
> >> +                     return ret;
> >> +             }
> >> +
> >> +             ret = clk_prepare_enable(csi2dc->scck);
> >> +             if (ret)
> >> +                     dev_err(csi2dc->dev,
> >> +                             "failed to enable scck: %d\n", ret);
> >
> > Shouldn't you bail out here ?
>
> Initially this clock was not mandatory, but you are right, I should bail
> out.
>
> >
> >> +
> >> +             /* if powering up, deassert reset line */
> >> +             csi2dc_writel(csi2dc, CSI2DC_GCTLR, CSI2DC_GCTLR_SWRST);
> >> +     } else {
> >> +             clk_disable_unprepare(csi2dc->scck);
> >> +
> >> +             /* if powering down, assert reset line */
> >> +             csi2dc_writel(csi2dc, CSI2DC_GCTLR, !CSI2DC_GCTLR_SWRST);
> >
> > Isn't reverse order of activation better ?
> >
> >                  csi2dc_writel(..)
> >                  clk_disable_unprepare(..)
> >                  clk_disable_unprepare(..)
> >> +
> >> +             clk_disable_unprepare(csi2dc->pclk);
> >> +     }
> >> +
> >> +     return ret;
> >> +}
> >> +
> >> +static int csi2dc_s_stream(struct v4l2_subdev *csi2dc_sd, int enable)
> >> +{
> >> +     struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
> >> +     int ret;
> >> +
> >> +     if (enable) {
> >> +             ret = pm_runtime_resume_and_get(csi2dc->dev);
> >> +             if (ret < 0)
> >> +                     return ret;
> >> +             csi2dc_vp_update(csi2dc);
> >> +     } else {
> >> +             pm_runtime_put_sync(csi2dc->dev);
> >> +     }
> >> +
> >> +     return v4l2_subdev_call(csi2dc->input_sd, video, s_stream, enable);
> >
> > Should the remote subdev be started before and stopped after ?
>
> Do you mean s_power ?
>

Nope, I meant the operation order.

        if (enable) {
                v4l2_subdev_call(...);

                pm_runtime_resume_and_get();
                csi2dc_vp_update();
        } else {
                pm_runtime_put_sync();

                v4l2_subdev_call();
        }

However, my comment is clearly wrong, it should have been the
contrary. Sorry about this

It's right to configure the interface and power if up -before-
starting the remote subdev, but I would stop it before powering it
down. Like in:

        if (enable) {
                pm_runtime_resume_and_get();
                csi2dc_vp_update();

                v4l2_subdev_call(1);
        } else {
                v4l2_subdev_call(0);

                pm_runtime_put_sync();
        }

What do you think ? Stopping the remote before powering the interface
down ensures no data is being put on the bus while the interace is
powered off.

> >
> >> +}
> >> +
> >> +static const struct v4l2_subdev_pad_ops csi2dc_pad_ops = {
> >> +     .enum_mbus_code = csi2dc_enum_mbus_code,
> >> +     .set_fmt = csi2dc_set_fmt,
> >> +     .get_fmt = csi2dc_get_fmt,
> >> +};
> >> +
> >> +static const struct v4l2_subdev_video_ops csi2dc_video_ops = {
> >> +     .s_stream = csi2dc_s_stream,
> >> +};
> >> +
> >> +static const struct v4l2_subdev_ops csi2dc_subdev_ops = {
> >> +     .pad = &csi2dc_pad_ops,
> >> +     .video = &csi2dc_video_ops,
> >> +};
> >> +
> >> +static int csi2dc_get_mbus_config(struct csi2dc_device *csi2dc)
> >> +{
> >> +     struct v4l2_mbus_config mbus_config = { 0 };
> >> +     int ret;
> >> +
> >> +     ret = v4l2_subdev_call(csi2dc->input_sd, pad, get_mbus_config,
> >> +                            csi2dc->remote_pad, &mbus_config);
> >> +     if (ret == -ENOIOCTLCMD) {
> >> +             dev_dbg(csi2dc->dev,
> >> +                     "no remote mbus configuration available\n");
> >> +             goto csi2dc_get_mbus_config_defaults;
> >> +     }
> >> +
> >> +     if (ret) {
> >> +             dev_err(csi2dc->dev,
> >> +                     "failed to get remote mbus configuration\n");
> >> +             goto csi2dc_get_mbus_config_defaults;
> >> +     }
> >> +
> >> +     if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_0)
> >> +             csi2dc->vc = 0;
> >> +     else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_1)
> >> +             csi2dc->vc = 1;
> >> +     else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_2)
> >> +             csi2dc->vc = 2;
> >> +     else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_3)
> >> +             csi2dc->vc = 3;
> >> +
> >> +     dev_dbg(csi2dc->dev, "subdev sending on channel %d\n", csi2dc->vc);
> >> +
> >> +     csi2dc->clk_gated = mbus_config.flags &
> >> +                             V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK;
> >
> > This should come from the default clock-noncontinuous property in the
> > endpoint. It is available in the mbus_configuration only to support
> > subdevs that can change it at runtime, and if that's the case it's ok,
> > but I think it should be in the endpoint.
>
> I do not completely understand your point. In a previous version which
> you reviewed, I had this as a DT property (so in the endpoint ?), and
> now I converted it to a get_mbus_config flag get operation.

I re-read the conversation and I think you're right.

> It's not the right way to do it ?
> How should it be specified in the endpoint ?
>

There's a default property 'clock-noncontinuous' which gets parsed by
v4l2_fwnode_endpoint_parse() and reported to drivers through the
V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK and V4L2_MBUS_CSI2_CONTINUOUS_CLOCK
flags. In your case, as you fetch it at run time it's not required if
not to initialize a default according to the property presence in DT.

Up to you, what you have here it's fine!

> >
> > Speaking of remote subdevs, is there any driver available for IDI
> > transmitters ?
>
> I am not aware of any at the moment
> >
> >> +
> >> +     dev_dbg(csi2dc->dev, "%s clock\n",
> >> +             csi2dc->clk_gated ? "gated" : "free running");
> >> +
> >> +     return 0;
> >> +
> >> +csi2dc_get_mbus_config_defaults:
> >> +     csi2dc->vc = 0; /* Virtual ID 0 by default */
> >> +     csi2dc->clk_gated = false; /* Free running clock by default */
> >> +
> >> +     return 0;
> >> +}
> >> +
> >> +static int csi2dc_async_bound(struct v4l2_async_notifier *notifier,
> >> +                           struct v4l2_subdev *subdev,
> >> +                           struct v4l2_async_subdev *asd)
> >> +{
> >> +     struct csi2dc_device *csi2dc = container_of(notifier,
> >> +                                             struct csi2dc_device, notifier);
> >> +     int pad;
> >> +     int ret;
> >> +
> >> +     csi2dc->input_sd = subdev;
> >> +
> >> +     pad = media_entity_get_fwnode_pad(&subdev->entity,
> >
> > You can use 'ret'
>
> I can, but it is weird to do remote_pad = ret ...
>
> May bring confusion, don't you agree ?
>

Up to you, it's a small detail :)

Thanks!
   j

> >
> >> +                                       asd->match.fwnode,
> >> +                                       MEDIA_PAD_FL_SOURCE);
> >> +     if (pad < 0) {
> >> +             dev_err(csi2dc->dev, "Failed to find pad for %s\n",
> >> +                     subdev->name);
> >> +             return pad;
> >> +     }
> >> +
> >> +     csi2dc->remote_pad = pad;
> >> +
> >> +     csi2dc_get_mbus_config(csi2dc);
> >
> > Ideally, as this is not fatal, you could move this at s_stream time to
> > fetch the most up-to-date configuration
> >
> >> +
> >> +     ret = media_create_pad_link(&csi2dc->input_sd->entity,
> >> +                                 csi2dc->remote_pad,
> >> +                                 &csi2dc->csi2dc_sd.entity, 0,
> >> +                                 MEDIA_LNK_FL_ENABLED);
> >> +     if (ret < 0) {
> >
> >          if (ret)
> >
> >> +             dev_err(csi2dc->dev,
> >> +                     "Failed to create pad link: %s to %s\n",
> >> +                     csi2dc->input_sd->entity.name,
> >> +                     csi2dc->csi2dc_sd.entity.name);
> >> +             return ret;
> >> +     }
> >> +
> >> +     dev_dbg(csi2dc->dev, "link with %s pad: %d\n",
> >> +             csi2dc->input_sd->name, csi2dc->remote_pad);
> >> +
> >> +     ret = pm_runtime_resume_and_get(csi2dc->dev);
> >> +     if (ret < 0)
> >> +             return ret;
> >> +
> >> +     csi2dc_writel(csi2dc, CSI2DC_GCFG,
> >> +                   (SAMA7G5_HLC & CSI2DC_GCFG_HLC_MASK) |
> >> +                   (csi2dc->clk_gated ? 0 : CSI2DC_GCFG_MIPIFRN));
> >> +
> >> +     csi2dc_writel(csi2dc, CSI2DC_VPCOL,
> >> +                   CSI2DC_VPCOL_COL(0xFFF) & CSI2DC_VPCOL_COL_MASK);
> >> +     csi2dc_writel(csi2dc, CSI2DC_VPROW,
> >> +                   CSI2DC_VPROW_ROW(0xFFF) & CSI2DC_VPROW_ROW_MASK);
> >> +
> >> +     pm_runtime_put_sync(csi2dc->dev);
> >
> > I would really move access to the HW to s_stream time if possible
> >
> >> +
> >> +     return ret;
> >> +}
> >> +
> >> +static const struct v4l2_async_notifier_operations csi2dc_async_ops = {
> >> +     .bound = csi2dc_async_bound,
> >> +};
> >> +
> >> +static void csi2dc_cleanup_notifier(struct csi2dc_device *csi2dc)
> >> +{
> >> +     v4l2_async_notifier_unregister(&csi2dc->notifier);
> >> +     v4l2_async_notifier_cleanup(&csi2dc->notifier);
> >> +}
> >> +
> >> +static int csi2dc_prepare_notifier(struct csi2dc_device *csi2dc,
> >> +                                struct device_node *input_node)
> >> +{
> >> +     int ret = 0;
> >> +
> >> +     v4l2_async_notifier_init(&csi2dc->notifier);
> >> +
> >> +     csi2dc->asd = v4l2_async_notifier_add_fwnode_remote_subdev
> >
> > do you need asd in the driver structure ?
> >
> >> +                     (&csi2dc->notifier, of_fwnode_handle(input_node),
> >> +                     struct v4l2_async_subdev);
> >> +
> >> +     of_node_put(input_node);
> >> +
> >> +     if (IS_ERR(csi2dc->asd)) {
> >> +             ret = PTR_ERR(csi2dc->asd);
> >> +             dev_err(csi2dc->dev,
> >> +                     "failed to add async notifier for node %pOF: %d\n",
> >> +                     input_node, ret);
> >> +             v4l2_async_notifier_cleanup(&csi2dc->notifier);
> >> +             return ret;
> >> +     }
> >> +
> >> +     csi2dc->notifier.ops = &csi2dc_async_ops;
> >> +
> >> +     ret = v4l2_async_subdev_notifier_register(&csi2dc->csi2dc_sd,
> >> +                                               &csi2dc->notifier);
> >> +
> >> +     if (ret) {
> >> +             dev_err(csi2dc->dev, "fail to register async notifier: %d\n",
> >> +                     ret);
> >> +             v4l2_async_notifier_cleanup(&csi2dc->notifier);
> >> +     }
> >> +
> >> +     return ret;
> >> +}
> >> +
> >> +static int csi2dc_of_parse(struct csi2dc_device *csi2dc,
> >> +                        struct device_node *of_node)
> >> +{
> >> +     struct device_node *input_node, *output_node;
> >> +     struct v4l2_fwnode_endpoint input_endpoint = { 0 },
> >> +                                 output_endpoint = { 0 };
> >> +     int ret;
> >> +
> >> +     output_endpoint.bus_type = V4L2_MBUS_PARALLEL;
> >> +
> >> +     input_node = of_graph_get_next_endpoint(of_node, NULL);
> >> +
> >> +     if (!input_node) {
> >> +             dev_err(csi2dc->dev,
> >> +                     "missing port node at %pOF, input node is mandatory.\n",
> >> +                     of_node);
> >> +             return -EINVAL;
> >> +     }
> >> +
> >> +     ret = v4l2_fwnode_endpoint_parse(of_fwnode_handle(input_node),
> >> +                                      &input_endpoint);
> >> +
> >> +     if (ret) {
> >                  of_node_put(input_node);
> >
> >> +             dev_err(csi2dc->dev, "endpoint not defined at %pOF\n", of_node);
> >> +             return ret;
> >> +     }
> >> +
> >> +     output_node = of_graph_get_next_endpoint(of_node, input_node);
> >> +
> >> +     if (output_node)
> >> +             ret = v4l2_fwnode_endpoint_parse(of_fwnode_handle(output_node),
> >> +                                              &output_endpoint);
> >
> >          of_node_put(output_node);
> >> +
> >> +     if (!output_node || ret) {
> >> +             dev_info(csi2dc->dev,
> >> +                      "missing output node at %pOF, data pipe available only.\n",
> >> +                      of_node);
> >> +     } else {
> >> +             csi2dc->video_pipe = true;
> >> +
> >> +             dev_dbg(csi2dc->dev, "block %pOF %d.%d->%d.%d video pipeline\n",
> >> +                     of_node, input_endpoint.base.port,
> >> +                     input_endpoint.base.id, output_endpoint.base.port,
> >> +                     output_endpoint.base.id);
> >> +     }
> >> +
> >> +     of_node_put(output_node);
> >
> > Drop this if you put it above
> >
> >> +     of_node_put(input_node);
> >
> > Should you put input_node before passing it to the function ?
> >> +
> >> +     /* prepare async notifier for subdevice completion */
> >> +     return csi2dc_prepare_notifier(csi2dc, input_node);
> >> +}
> >> +
> >> +static int csi2dc_probe(struct platform_device *pdev)
> >> +{
> >> +     struct device *dev = &pdev->dev;
> >> +     struct csi2dc_device *csi2dc;
> >> +     struct resource *res = NULL;
> >> +     int ret = 0;
> >> +     u32 ver;
> >> +
> >> +     csi2dc = devm_kzalloc(dev, sizeof(*csi2dc), GFP_KERNEL);
> >> +     if (!csi2dc)
> >> +             return -ENOMEM;
> >> +
> >> +     csi2dc->dev = dev;
> >> +
> >> +     res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
> >> +
> >> +     csi2dc->base = devm_ioremap_resource(dev, res);
> >
> > Should devm_platform_ioremap_resource(pdev, 0) be used here ?
> >
> >> +     if (IS_ERR(csi2dc->base)) {
> >> +             dev_err(dev, "base address not set\n");
> >> +             return PTR_ERR(csi2dc->base);
> >> +     }
> >> +
> >> +     csi2dc->pclk = devm_clk_get(dev, "pclk");
> >> +     if (IS_ERR(csi2dc->pclk)) {
> >> +             ret = PTR_ERR(csi2dc->pclk);
> >> +             dev_err(dev, "failed to get pclk: %d\n", ret);
> >> +             return ret;
> >> +     }
> >> +
> >> +     csi2dc->scck = devm_clk_get(dev, "scck");
> >> +     if (IS_ERR(csi2dc->scck)) {
> >> +             ret = PTR_ERR(csi2dc->scck);
> >> +             dev_err(dev, "failed to get scck: %d\n", ret);
> >> +             return ret;
> >> +     }
> >> +
> >> +     v4l2_subdev_init(&csi2dc->csi2dc_sd, &csi2dc_subdev_ops);
> >> +
> >> +     csi2dc->csi2dc_sd.owner = THIS_MODULE;
> >> +     csi2dc->csi2dc_sd.dev = dev;
> >> +     snprintf(csi2dc->csi2dc_sd.name, sizeof(csi2dc->csi2dc_sd.name),
> >> +              "csi2dc");
> >> +
> >> +     csi2dc->csi2dc_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> >> +     csi2dc->csi2dc_sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> >> +     v4l2_set_subdevdata(&csi2dc->csi2dc_sd, pdev);
> >
> > Not used it seems
> >
> >> +
> >> +     platform_set_drvdata(pdev, csi2dc);
> >> +
> >> +     ret = csi2dc_of_parse(csi2dc, dev->of_node);
> >> +     if (ret)
> >> +             goto csi2dc_probe_cleanup_entity;
> >> +
> >> +     csi2dc->pads[CSI2DC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> >> +     if (csi2dc->video_pipe)
> >> +             csi2dc->pads[CSI2DC_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
> >> +
> >> +     ret = media_entity_pads_init(&csi2dc->csi2dc_sd.entity,
> >> +                                  csi2dc->video_pipe ? CSI2DC_PADS_NUM : 1,
> >> +                                  csi2dc->pads);
> >> +     if (ret < 0) {
> >> +             dev_err(dev, "media entity init failed\n");
> >> +             goto csi2dc_probe_cleanup_entity;
> >
> > Should you also clean up the notifier in the error path ?
> >
> >> +     }
> >> +
> >> +     /* turn power on to validate capabilities */
> >> +     ret = csi2dc_power(csi2dc, true);
> >> +     if (ret < 0)
> >> +             goto csi2dc_probe_cleanup_entity;
> >> +
> >> +     pm_runtime_set_active(dev);
> >> +     pm_runtime_enable(dev);
> >> +     ver = csi2dc_readl(csi2dc, CSI2DC_VERSION);
> >> +     pm_request_idle(dev);
> >> +
> >> +     /*
> >> +      * we must register the subdev after PM runtime has been requested,
> >> +      * otherwise we might bound immediately and request pm_runtime_resume
> >> +      * before runtime_enable.
> >> +      */
> >> +     ret = v4l2_async_register_subdev(&csi2dc->csi2dc_sd);
> >> +     if (ret) {
> >> +             dev_err(csi2dc->dev, "failed to register the subdevice\n");
> >> +             goto csi2dc_probe_cleanup_entity;
> >> +     }
> >> +
> >> +     dev_info(dev, "Microchip CSI2DC version %x\n", ver);
> >> +
> >> +     return 0;
> >> +
> >> +csi2dc_probe_cleanup_entity:
> >> +     media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
> >> +
> >> +     return ret;
> >> +}
> >> +
> >> +static int csi2dc_remove(struct platform_device *pdev)
> >> +{
> >> +     struct csi2dc_device *csi2dc = platform_get_drvdata(pdev);
> >> +
> >> +     pm_runtime_disable(&pdev->dev);
> >> +
> >> +     v4l2_async_unregister_subdev(&csi2dc->csi2dc_sd);
> >> +     csi2dc_cleanup_notifier(csi2dc);
> >> +     media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
> >> +
> >> +     return 0;
> >> +}
> >> +
> >> +static int __maybe_unused csi2dc_runtime_suspend(struct device *dev)
> >> +{
> >> +     struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
> >> +
> >> +     return csi2dc_power(csi2dc, false);
> >> +}
> >> +
> >> +static int __maybe_unused csi2dc_runtime_resume(struct device *dev)
> >> +{
> >> +     struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
> >> +
> >> +     return csi2dc_power(csi2dc, true);
> >> +}
> >> +
> >> +static const struct dev_pm_ops csi2dc_dev_pm_ops = {
> >> +     SET_RUNTIME_PM_OPS(csi2dc_runtime_suspend, csi2dc_runtime_resume, NULL)
> >> +};
> >> +
> >> +static const struct of_device_id csi2dc_of_match[] = {
> >> +     { .compatible = "microchip,sama7g5-csi2dc" },
> >> +     { }
> >> +};
> >> +
> >> +MODULE_DEVICE_TABLE(of, csi2dc_of_match);
> >> +
> >> +static struct platform_driver csi2dc_driver = {
> >> +     .probe  = csi2dc_probe,
> >> +     .remove = csi2dc_remove,
> >> +     .driver = {
> >> +             .name =                 "microchip-csi2dc",
> >> +             .pm =                   &csi2dc_dev_pm_ops,
> >> +             .of_match_table =       of_match_ptr(csi2dc_of_match),
> >> +     },
> >> +};
> >> +
> >> +module_platform_driver(csi2dc_driver);
> >> +
> >> +MODULE_AUTHOR("Eugen Hristev <eugen.hristev@microchip.com>");
> >> +MODULE_DESCRIPTION("Microchip CSI2 Demux Controller driver");
> >> +MODULE_LICENSE("GPL v2");
> >> --
> >> 2.25.1
> >>
>

Re: [PATCH 03/21] media: atmel: introduce microchip csi2dc driver

[Eugen.Hristev]

On 11/3/21 11:28 AM, Jacopo Mondi wrote:
> Hi Eugen
> 
> On Wed, Nov 03, 2021 at 08:31:36AM +0000, Eugen.Hristev@microchip.com wrote:
>> On 11/2/21 7:22 PM, Jacopo Mondi wrote:
>>> Hi Eugen,
>>>
>>
>> Hi,
>>
>> Thank you for your review. I will try to understand everything you said
>> and come up with a new version. I still have some inline questions
>> though, about some things which are still unclear.
>>
>>> On Fri, Oct 22, 2021 at 10:52:29AM +0300, Eugen Hristev wrote:
>>>> Microchip CSI2DC (CSI2 Demultiplexer Controller) is a misc bridge device
>>>> that converts a byte stream in IDI Synopsys format (coming from a CSI2HOST)
>>>> to a pixel stream that can be captured by a sensor controller.
>>>>
>>>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>>>> ---
>>>> Changes in this revision:
>>>> - addressed comments by Jacopo and Laurent as in this thread:
>>>> https://www.spinics.net/lists/linux-media/msg181044.html
>>>>
>>>> Previous change log :
>>>> Changes in v5:
>>>> - only in bindings
>>>>
>>>> Changes in v4:
>>>> - now using get_mbus_config ops to get data from the subdevice, like the
>>>> virtual channel id, and the clock type.
>>>> - now having possibility to select any of the RAW10 data modes
>>>> - at completion time, select which formats are also available in the subdevice,
>>>> and move to the dynamic list accordingly
>>>> - changed the pipeline integration, do not advertise subdev ready at probe time.
>>>> wait until completion is done, and then start a workqueue that will register
>>>> this device as a subdevice for the next element in pipeline.
>>>> - moved the s_power code into a different function called now csi2dc_power
>>>> that is called with CONFIG_PM functions. This is also called at completion,
>>>> to have the device ready in case CONFIG_PM is not selected on the platform.
>>>> - merged try_fmt into set_fmt
>>>> - driver cleanup, wrapped lines over 80 characters
>>>>
>>>> Changes in v2:
>>>> - moved driver to platform/atmel
>>>> - fixed minor things as per Sakari's review
>>>> - still some things from v2 review are not yet addressed, to be followed up
>>>>
>>>>
>>>>    drivers/media/platform/atmel/Kconfig          |  15 +
>>>>    drivers/media/platform/atmel/Makefile         |   1 +
>>>>    .../media/platform/atmel/microchip-csi2dc.c   | 700 ++++++++++++++++++
>>>>    3 files changed, 716 insertions(+)
>>>>    create mode 100644 drivers/media/platform/atmel/microchip-csi2dc.c
>>>>
>>>> diff --git a/drivers/media/platform/atmel/Kconfig b/drivers/media/platform/atmel/Kconfig
>>>> index dda2f27da317..f83bee373d82 100644
>>>> --- a/drivers/media/platform/atmel/Kconfig
>>>> +++ b/drivers/media/platform/atmel/Kconfig
>>>> @@ -40,3 +40,18 @@ config VIDEO_ATMEL_ISI
>>>>         help
>>>>           This module makes the ATMEL Image Sensor Interface available
>>>>           as a v4l2 device.
>>>> +
>>>> +config VIDEO_MICROCHIP_CSI2DC
>>>> +     tristate "Microchip CSI2 Demux Controller"
>>>> +     depends on VIDEO_V4L2 && COMMON_CLK && OF
>>>> +     depends on ARCH_AT91 || COMPILE_TEST
>>>> +     select MEDIA_CONTROLLER
>>>> +     select VIDEO_V4L2_SUBDEV_API
>>>> +     select V4L2_FWNODE
>>>> +     help
>>>> +       CSI2 Demux Controller driver. CSI2DC is a helper chip
>>>> +       that converts IDI interface byte stream to a parallel pixel stream.
>>>> +       It supports various RAW formats as input.
>>>> +
>>>> +       To compile this driver as a module, choose M here: the
>>>> +       module will be called microchip-csi2dc.
>>>> diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
>>>> index 46d264ab7948..39f0a7eba702 100644
>>>> --- a/drivers/media/platform/atmel/Makefile
>>>> +++ b/drivers/media/platform/atmel/Makefile
>>>> @@ -6,3 +6,4 @@ obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
>>>>    obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o
>>>>    obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel-isc.o
>>>>    obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel-xisc.o
>>>> +obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += microchip-csi2dc.o
>>>> diff --git a/drivers/media/platform/atmel/microchip-csi2dc.c b/drivers/media/platform/atmel/microchip-csi2dc.c
>>>> new file mode 100644
>>>> index 000000000000..277b86988eee
>>>> --- /dev/null
>>>> +++ b/drivers/media/platform/atmel/microchip-csi2dc.c
>>>> @@ -0,0 +1,700 @@
>>>> +// SPDX-License-Identifier: GPL-2.0-only
>>>> +/*
>>>> + * Microchip CSI2 Demux Controller (CSI2DC) driver
>>>> + *
>>>> + * Copyright (C) 2018 Microchip Technology, Inc.
>>>> + *
>>>> + * Author: Eugen Hristev <eugen.hristev@microchip.com>
>>>> + *
>>>> + */
>>>> +
>>>> +#include <linux/clk.h>
>>>> +#include <linux/module.h>
>>>> +#include <linux/of.h>
>>>
>>> Isn't linux/mod_devicetable.h enough ?
>>>
>>>> +#include <linux/of_graph.h>
>>>
>>> You should probably move to use the fwnode_graph framwork instead of
>>> of_graph. This driver depends on OF so it shouldn't be an issue but I
>>> defer this to maintainers
>>>
>>>> +#include <linux/platform_device.h>
>>>> +#include <linux/pm_runtime.h>
>>>> +#include <linux/videodev2.h>
>>>> +
>>>> +#include <media/v4l2-device.h>
>>>
>>> Do you need this include ?
>>>
>>>> +#include <media/v4l2-fwnode.h>
>>>> +#include <media/v4l2-subdev.h>
>>>> +#include <media/videobuf2-dma-contig.h>
>>>
>>> Is this one needed as well ?
>>>
>>>> +
>>>> +/* Global configuration register */
>>>> +#define CSI2DC_GCFG                  0x0
>>>> +
>>>> +/* MIPI sensor pixel clock is free running */
>>>> +#define CSI2DC_GCFG_MIPIFRN          BIT(0)
>>>> +/* Output waveform inter-line minimum delay */
>>>> +#define CSI2DC_GCFG_HLC(v)           ((v) << 4)
>>>> +#define CSI2DC_GCFG_HLC_MASK         GENMASK(7, 4)
>>>> +/* SAMA7G5 requires a HLC delay of 15 */
>>>> +#define SAMA7G5_HLC                  (15)
>>>> +
>>>> +/* Global control register */
>>>> +#define CSI2DC_GCTLR                 0x04
>>>> +#define CSI2DC_GCTLR_SWRST           BIT(0)
>>>> +
>>>> +/* Global status register */
>>>> +#define CSI2DC_GS                    0x08
>>>> +
>>>> +/* SSP interrupt status register */
>>>> +#define CSI2DC_SSPIS                 0x28
>>>> +/* Pipe update register */
>>>> +#define CSI2DC_PU                    0xC0
>>>
>>> What about using lowercase for hex values (I know there's not strict
>>> rule, so keep what you like the most, but most drivers use lowercase
>>>
>>>> +/* Video pipe attributes update */
>>>> +#define CSI2DC_PU_VP                 BIT(0)
>>>> +
>>>> +/* Pipe update status register */
>>>> +#define CSI2DC_PUS                   0xC4
>>>> +
>>>> +/* Video pipeline enable register */
>>>> +#define CSI2DC_VPE                   0xF8
>>>> +#define CSI2DC_VPE_ENABLE            BIT(0)
>>>> +
>>>> +/* Video pipeline configuration register */
>>>> +#define CSI2DC_VPCFG                 0xFC
>>>> +/* Data type */
>>>> +#define CSI2DC_VPCFG_DT(v)           ((v) << 0)
>>>> +#define CSI2DC_VPCFG_DT_MASK         GENMASK(5, 0)
>>>> +/* Virtual channel identifier */
>>>> +#define CSI2DC_VPCFG_VC(v)           ((v) << 6)
>>>> +#define CSI2DC_VPCFG_VC_MASK         GENMASK(7, 6)
>>>> +/* Decompression enable */
>>>> +#define CSI2DC_VPCFG_DE                      BIT(8)
>>>> +/* Decoder mode */
>>>> +#define CSI2DC_VPCFG_DM(v)           ((v) << 9)
>>>> +#define CSI2DC_VPCFG_DM_DECODER8TO12 0
>>>> +/* Decoder predictor 2 selection */
>>>> +#define CSI2DC_VPCFG_DP2             BIT(12)
>>>> +/* Recommended memory storage */
>>>> +#define CSI2DC_VPCFG_RMS             BIT(13)
>>>> +/* Post adjustment */
>>>> +#define CSI2DC_VPCFG_PA                      BIT(14)
>>>> +
>>>> +/* Video pipeline column register */
>>>> +#define CSI2DC_VPCOL                 0x100
>>>> +/* Column number */
>>>> +#define CSI2DC_VPCOL_COL(v)          ((v) << 0)
>>>> +#define CSI2DC_VPCOL_COL_MASK                GENMASK(15, 0)
>>>> +
>>>> +/* Video pipeline row register */
>>>> +#define CSI2DC_VPROW                 0x104
>>>> +/* Row number */
>>>> +#define CSI2DC_VPROW_ROW(v)          ((v) << 0)
>>>> +#define CSI2DC_VPROW_ROW_MASK                GENMASK(15, 0)
>>>> +
>>>> +/* Version register */
>>>> +#define CSI2DC_VERSION                       0x1FC
>>>> +
>>>> +/* register read/write helpers */
>>>> +#define csi2dc_readl(st, reg)                readl_relaxed((st)->base + (reg))
>>>> +#define csi2dc_writel(st, reg, val)  writel_relaxed((val), \
>>>> +                                     (st)->base + (reg))
>>>> +
>>>> +/* supported RAW data types */
>>>> +#define CSI2DC_DT_RAW6                       0x28
>>>> +#define CSI2DC_DT_RAW7                       0x29
>>>> +#define CSI2DC_DT_RAW8                       0x2A
>>>> +#define CSI2DC_DT_RAW10                      0x2B
>>>> +#define CSI2DC_DT_RAW12                      0x2C
>>>> +#define CSI2DC_DT_RAW14                      0x2D
>>>> +
>>>> +/*
>>>> + * struct csi2dc_format - CSI2DC format type struct
>>>> + * @mbus_code:               Media bus code for the format
>>>> + * @dt:                      Data type constant for this format
>>>> + */
>>>> +struct csi2dc_format {
>>>> +     u32                             mbus_code;
>>>> +     u32                             dt;
>>>> +};
>>>> +
>>>> +static const struct csi2dc_format csi2dc_formats[] = {
>>>> +     {
>>>> +             .mbus_code =            MEDIA_BUS_FMT_SRGGB10_1X10,
>>>> +             .dt =                   CSI2DC_DT_RAW10,
>>>> +     }, {
>>>> +             .mbus_code =            MEDIA_BUS_FMT_SBGGR10_1X10,
>>>> +             .dt =                   CSI2DC_DT_RAW10,
>>>> +     }, {
>>>> +             .mbus_code =            MEDIA_BUS_FMT_SGRBG10_1X10,
>>>> +             .dt =                   CSI2DC_DT_RAW10,
>>>> +     }, {
>>>> +             .mbus_code =            MEDIA_BUS_FMT_SGBRG10_1X10,
>>>> +             .dt =                   CSI2DC_DT_RAW10,
>>>> +     },
>>>> +};
>>>
>>> How unfortunate we don't have this in the core...
>>>
>>>> +
>>>> +enum mipi_csi_pads {
>>>> +     CSI2DC_PAD_SINK                 = 0,
>>>> +     CSI2DC_PAD_SOURCE               = 1,
>>>> +     CSI2DC_PADS_NUM                 = 2,
>>>> +};
>>>> +
>>>> +/*
>>>> + * struct csi2dc_device - CSI2DC device driver data/config struct
>>>> + * @base:            Register map base address
>>>> + * @csi2dc_sd:               v4l2 subdevice for the csi2dc device
>>>> + *                   This is the subdevice that the csi2dc device itself
>>>> + *                   registers in v4l2 subsystem
>>>> + * @dev:             struct device for this csi2dc device
>>>> + * @pclk:            Peripheral clock reference
>>>> + *                   Input clock that clocks the hardware block internal
>>>> + *                   logic
>>>> + * @scck:            Sensor Controller clock reference
>>>> + *                   Input clock that is used to generate the pixel clock
>>>> + * @format:          Current saved format used in g/s fmt
>>>> + * @cur_fmt:         Current state format
>>>> + * @try_fmt:         Try format that is being tried
>>>> + * @pads:            Media entity pads for the csi2dc subdevice
>>>> + * @clk_gated:               Whether the clock is gated or free running
>>>> + * @video_pipe:              Whether video pipeline is configured
>>>> + * @vc:                      Current set virtual channel
>>>> + * @asd:             Async subdevice for async bound of the underlying subdev
>>>> + * @notifier:                Async notifier that is used to bound the underlying
>>>> + *                   subdevice to the csi2dc subdevice
>>>> + * @input_sd:                Reference to the underlying subdevice bound to the
>>>> + *                   csi2dc subdevice
>>>> + * @remote_pad:              Pad number of the underlying subdevice that is linked
>>>> + *                   to the csi2dc subdevice sink pad.
>>>> + */
>>>> +struct csi2dc_device {
>>>> +     void __iomem                    *base;
>>>> +     struct v4l2_subdev              csi2dc_sd;
>>>> +     struct device                   *dev;
>>>> +     struct clk                      *pclk;
>>>> +     struct clk                      *scck;
>>>> +
>>>> +     struct v4l2_mbus_framefmt        format;
>>>> +
>>>> +     const struct csi2dc_format      *cur_fmt;
>>>> +     const struct csi2dc_format      *try_fmt;
>>>> +
>>>> +     struct media_pad                pads[CSI2DC_PADS_NUM];
>>>> +
>>>> +     bool                            clk_gated;
>>>> +     bool                            video_pipe;
>>>> +     u32                             vc;
>>>> +
>>>> +     struct v4l2_async_subdev        *asd;
>>>> +     struct v4l2_async_notifier      notifier;
>>>> +
>>>> +     struct v4l2_subdev              *input_sd;
>>>> +
>>>> +     u32                             remote_pad;
>>>> +};
>>>> +
>>>> +static void csi2dc_vp_update(struct csi2dc_device *csi2dc)
>>>
>>> Could you move this below closer to the only caller ?
>>>
>>>> +{
>>>> +     u32 vp;
>>>> +
>>>> +     if (!csi2dc->cur_fmt) {
>>>
>>> You should probably initialize this to a default format
>>>
>>>> +             dev_err(csi2dc->dev, "format must be configured first\n");
>>>> +             return;
>>>> +     }
>>>> +
>>>> +     if (!csi2dc->video_pipe) {
>>>
>>> This is only called internally by the driver at s_stream() time, can
>>> this happen ? Or rather won't you have a streamon call also when the
>>> data pipe only is available ? In that case you would error out here
>>
>> This can happen if the source node is not found in the DT. In that case,
>> there is no video pipe available, thus this whole function should be a
>> no-op (csi2dc_vp_update) . If no vp, there is no vp update.
>> Normall without a vp, there should be a dp (data pipe), but I haven't
>> tested this and it's not supported in the driver.
>> It doesn't make any sense to configure the vp registers below if the vp
>> is not available.
>> However, I shouldn't return an error since the vp is not mandatory for
>> the function of the hardware. Just that the dp will be implemented at a
>> later time (.... or never).
>>
> 
> I see, so the data pipe is still work in progress... I understand
> bindings should allow for that to be later accepted so you cannot
> mandate port@1 to be there all the times, but should the driver
> instead refuse to operate if no port@1 is provided ?
> 

It will not refuse to operate, but won't operate anything basically.
And if I make it refuse to operate, it will violate the bindings, since 
port@1 is not mandatory

> Out of curiosity what will the interaction model with the driver be in
> data pipe mode ? There will be a video device but there won't be an
> link to it, how is stream start/stop controlled ?

My idea is that on data pipe, the csi2dc should register it's own video 
device and act basically as a dma engine entity that will use an 
external DMA controller to just write data to DRAM.

It is useful if we do not want to convert the MIPI/IDI packets into 
pixels, but rather dump the to DRAM and have a piece of software further 
use them.

However, it's not a clear use case about this functionality at this 
moment. One possibility is to be able to obtain stream meta-data from 
the MIPI / IDI interface, like for example synchronization data, to 
synchronize with an audio capture device.

> 
>>>
>>>> +             dev_err(csi2dc->dev, "video pipeline unavailable\n");
>>>> +             return;
>>>> +     }
>>>> +
>>>> +     vp = CSI2DC_VPCFG_DT(csi2dc->cur_fmt->dt) & CSI2DC_VPCFG_DT_MASK;
>>>> +     vp |= CSI2DC_VPCFG_VC(csi2dc->vc) & CSI2DC_VPCFG_VC_MASK;
>>>> +     vp &= ~CSI2DC_VPCFG_DE;
>>>> +     vp |= CSI2DC_VPCFG_DM(CSI2DC_VPCFG_DM_DECODER8TO12);
>>>> +     vp &= ~CSI2DC_VPCFG_DP2;
>>>> +     vp &= ~CSI2DC_VPCFG_RMS;
>>>> +     vp |= CSI2DC_VPCFG_PA;
>>>> +
>>>> +     csi2dc_writel(csi2dc, CSI2DC_VPCFG, vp);
>>>> +     csi2dc_writel(csi2dc, CSI2DC_VPE, CSI2DC_VPE_ENABLE);
>>>> +     csi2dc_writel(csi2dc, CSI2DC_PU, CSI2DC_PU_VP);
>>>> +}
>>>> +
>>>> +static inline struct csi2dc_device *
>>>> +csi2dc_sd_to_csi2dc_device(struct v4l2_subdev *csi2dc_sd)
>>>> +{
>>>> +     return container_of(csi2dc_sd, struct csi2dc_device, csi2dc_sd);
>>>> +}
>>>> +
>>>> +static int csi2dc_enum_mbus_code(struct v4l2_subdev *csi2dc_sd,
>>>> +                              struct v4l2_subdev_state *sd_state,
>>>> +                              struct v4l2_subdev_mbus_code_enum *code)
>>>> +{
>>>> +     if (code->index >= ARRAY_SIZE(csi2dc_formats))
>>>> +             return -EINVAL;
>>>> +
>>>> +     code->code = csi2dc_formats[code->index].mbus_code;
>>>> +
>>>> +     return 0;
>>>> +}
>>>> +
>>>> +static int csi2dc_get_fmt(struct v4l2_subdev *csi2dc_sd,
>>>> +                       struct v4l2_subdev_state *sd_state,
>>>> +                       struct v4l2_subdev_format *format)
>>>> +{
>>>> +     struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
>>>> +
>>>> +     format->format = csi2dc->format;
>>>> +
>>>> +     return 0;
>>>
>>> You should support try formats by storing the format in the file
>>> handle state in s_fmt and return it in case which == TRY
>>>
>>> Grep for v4l2_subdev_get_try_format() for usage examples
>>
>> I did that initially, but could not find any utility for it in my tests.
> 
> I understand.. Have you run v4l2-compliance on this sudev ? Doesn't it
> test try formats support ?
> 
>> So I removed it.
>> I will try to bring it back, but I have no idea how to test that the try
>> format works fine or not
>>
> 
> I think also v4l2-ctl supports try formats.
> 
> --try-subdev-fmt pad=<pad>,width=<w>,height=<h>,code=<code>,field=<f>,colorspace=<c>,
> 
> I'm not sure if it allows you to read it back, but it's easy to hack
> it out just to verify it.
> 

Ok, I will try it

>>>
>>>> +}
>>>> +
>>>> +static int csi2dc_set_fmt(struct v4l2_subdev *csi2dc_sd,
>>>> +                       struct v4l2_subdev_state *sd_state,
>>>> +                       struct v4l2_subdev_format *req_fmt)
>>>> +{
>>>> +     struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
>>>> +     const struct csi2dc_format *fmt;
>>>> +     int i;
>>>
>>> unsigned
>>>
>>>> +
>>>> +     for (i = 0; i < ARRAY_SIZE(csi2dc_formats);  i++) {
>>>> +             fmt = &csi2dc_formats[i];
>>>> +             if (req_fmt->format.code == fmt->mbus_code)
>>>> +                     csi2dc->try_fmt = fmt;
>>>
>>> Shouldn't you break ?
>>>
>>>> +             fmt++;
>>>> +     }
>>>
>>> And make this a simpler
>>>
>>>           for (i = 0; i < ARRAY_SIZE(csi2dc_formats); i++) {
>>>                   if (req_fmt->format.code == csi2dc_formats[i].mbus_code)
>>>                           break;
>>>           }
>>>
>>>           if (i == ARRAY_SIZE(csi2dc_formats)
>>>                   i = 0;
>>>
>>>> +
>>>> +     /* in case we could not find the desired format, default to something */
>>>> +     if (!csi2dc->try_fmt  ||
>>>> +         req_fmt->format.code != csi2dc->try_fmt->mbus_code) {
>>>> +             csi2dc->try_fmt = &csi2dc_formats[0];
>>>> +
>>>> +             dev_dbg(csi2dc->dev,
>>>> +                     "CSI2DC unsupported format 0x%x, defaulting to 0x%x\n",
>>>> +                     req_fmt->format.code, csi2dc_formats[0].mbus_code);
>>>> +
>>>> +             req_fmt->format.code = csi2dc_formats[0].mbus_code;
>>>> +     }
>>>> +
>>>> +     req_fmt->format.colorspace = V4L2_COLORSPACE_SRGB;
>>>> +     req_fmt->format.field = V4L2_FIELD_NONE;
>>>> +
>>>> +     /* save the format for later requests */
>>>
>>> You should support TRY formats
>>>
>>>> +     csi2dc->format = req_fmt->format;
>>>> +
>>>> +     /* if we are just trying, we are done */
>>>> +     if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY)
>>>> +             return 0;
>>>> +
>>>> +     csi2dc->cur_fmt = csi2dc->try_fmt;
>>>
>>>           csi2dc->cur_fmt = &csi2dc_format[i];
>>>
>>> So you can drop the try_fmt from the driver structure as it seems to
>>> be used as a temporary variable here only.
>>>
>>>> +
>>>> +     dev_dbg(csi2dc->dev, "new format set: 0x%x\n", req_fmt->format.code);
>>>> +
>>>> +     return 0;
>>>> +}
>>>> +
>>>> +static int csi2dc_power(struct csi2dc_device *csi2dc, int on)
>>>> +{
>>>> +     int ret = 0;
>>>> +
>>>> +     if (on) {
>>>> +             ret = clk_prepare_enable(csi2dc->pclk);
>>>> +             if (ret) {
>>>> +                     dev_err(csi2dc->dev, "failed to enable pclk: %d\n", ret);
>>>> +                     return ret;
>>>> +             }
>>>> +
>>>> +             ret = clk_prepare_enable(csi2dc->scck);
>>>> +             if (ret)
>>>> +                     dev_err(csi2dc->dev,
>>>> +                             "failed to enable scck: %d\n", ret);
>>>
>>> Shouldn't you bail out here ?
>>
>> Initially this clock was not mandatory, but you are right, I should bail
>> out.
>>
>>>
>>>> +
>>>> +             /* if powering up, deassert reset line */
>>>> +             csi2dc_writel(csi2dc, CSI2DC_GCTLR, CSI2DC_GCTLR_SWRST);
>>>> +     } else {
>>>> +             clk_disable_unprepare(csi2dc->scck);
>>>> +
>>>> +             /* if powering down, assert reset line */
>>>> +             csi2dc_writel(csi2dc, CSI2DC_GCTLR, !CSI2DC_GCTLR_SWRST);
>>>
>>> Isn't reverse order of activation better ?
>>>
>>>                   csi2dc_writel(..)
>>>                   clk_disable_unprepare(..)
>>>                   clk_disable_unprepare(..)
>>>> +
>>>> +             clk_disable_unprepare(csi2dc->pclk);
>>>> +     }
>>>> +
>>>> +     return ret;
>>>> +}
>>>> +
>>>> +static int csi2dc_s_stream(struct v4l2_subdev *csi2dc_sd, int enable)
>>>> +{
>>>> +     struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
>>>> +     int ret;
>>>> +
>>>> +     if (enable) {
>>>> +             ret = pm_runtime_resume_and_get(csi2dc->dev);
>>>> +             if (ret < 0)
>>>> +                     return ret;
>>>> +             csi2dc_vp_update(csi2dc);
>>>> +     } else {
>>>> +             pm_runtime_put_sync(csi2dc->dev);
>>>> +     }
>>>> +
>>>> +     return v4l2_subdev_call(csi2dc->input_sd, video, s_stream, enable);
>>>
>>> Should the remote subdev be started before and stopped after ?
>>
>> Do you mean s_power ?
>>
> 
> Nope, I meant the operation order.
> 
>          if (enable) {
>                  v4l2_subdev_call(...);
> 
>                  pm_runtime_resume_and_get();
>                  csi2dc_vp_update();
>          } else {
>                  pm_runtime_put_sync();
> 
>                  v4l2_subdev_call();
>          }
> 
> However, my comment is clearly wrong, it should have been the
> contrary. Sorry about this
> 
> It's right to configure the interface and power if up -before-
> starting the remote subdev, but I would stop it before powering it
> down. Like in:
> 
>          if (enable) {
>                  pm_runtime_resume_and_get();
>                  csi2dc_vp_update();
> 
>                  v4l2_subdev_call(1);
>          } else {
>                  v4l2_subdev_call(0);
> 
>                  pm_runtime_put_sync();
>          }
> 
> What do you think ? Stopping the remote before powering the interface
> down ensures no data is being put on the bus while the interace is
> powered off.
> 

In this case it might not work like this. If I start the sensor 
streaming before the vp update , then, there will be frames received on 
video pipe update, which means that the csi2dc will not detect the phy 
STOP state reached, and it will fail initialization.
I believe the csi2dc has to be started before the sensor's first frame 
such that the interface is prepared for receiving. The PHY must be in 
stop state for a certain amount of time.
I will try to see if it works the other way around, to double check to 
make sure.
But if it doesn't , then the VP must be configured while the whole 
pipeline is in stop state.

>>>
>>>> +}
>>>> +
>>>> +static const struct v4l2_subdev_pad_ops csi2dc_pad_ops = {
>>>> +     .enum_mbus_code = csi2dc_enum_mbus_code,
>>>> +     .set_fmt = csi2dc_set_fmt,
>>>> +     .get_fmt = csi2dc_get_fmt,
>>>> +};
>>>> +
>>>> +static const struct v4l2_subdev_video_ops csi2dc_video_ops = {
>>>> +     .s_stream = csi2dc_s_stream,
>>>> +};
>>>> +
>>>> +static const struct v4l2_subdev_ops csi2dc_subdev_ops = {
>>>> +     .pad = &csi2dc_pad_ops,
>>>> +     .video = &csi2dc_video_ops,
>>>> +};
>>>> +
>>>> +static int csi2dc_get_mbus_config(struct csi2dc_device *csi2dc)
>>>> +{
>>>> +     struct v4l2_mbus_config mbus_config = { 0 };
>>>> +     int ret;
>>>> +
>>>> +     ret = v4l2_subdev_call(csi2dc->input_sd, pad, get_mbus_config,
>>>> +                            csi2dc->remote_pad, &mbus_config);
>>>> +     if (ret == -ENOIOCTLCMD) {
>>>> +             dev_dbg(csi2dc->dev,
>>>> +                     "no remote mbus configuration available\n");
>>>> +             goto csi2dc_get_mbus_config_defaults;
>>>> +     }
>>>> +
>>>> +     if (ret) {
>>>> +             dev_err(csi2dc->dev,
>>>> +                     "failed to get remote mbus configuration\n");
>>>> +             goto csi2dc_get_mbus_config_defaults;
>>>> +     }
>>>> +
>>>> +     if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_0)
>>>> +             csi2dc->vc = 0;
>>>> +     else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_1)
>>>> +             csi2dc->vc = 1;
>>>> +     else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_2)
>>>> +             csi2dc->vc = 2;
>>>> +     else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_3)
>>>> +             csi2dc->vc = 3;
>>>> +
>>>> +     dev_dbg(csi2dc->dev, "subdev sending on channel %d\n", csi2dc->vc);
>>>> +
>>>> +     csi2dc->clk_gated = mbus_config.flags &
>>>> +                             V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK;
>>>
>>> This should come from the default clock-noncontinuous property in the
>>> endpoint. It is available in the mbus_configuration only to support
>>> subdevs that can change it at runtime, and if that's the case it's ok,
>>> but I think it should be in the endpoint.
>>
>> I do not completely understand your point. In a previous version which
>> you reviewed, I had this as a DT property (so in the endpoint ?), and
>> now I converted it to a get_mbus_config flag get operation.
> 
> I re-read the conversation and I think you're right.
> 
>> It's not the right way to do it ?
>> How should it be specified in the endpoint ?
>>
> 
> There's a default property 'clock-noncontinuous' which gets parsed by
> v4l2_fwnode_endpoint_parse() and reported to drivers through the
> V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK and V4L2_MBUS_CSI2_CONTINUOUS_CLOCK
> flags. In your case, as you fetch it at run time it's not required if
> not to initialize a default according to the property presence in DT.
> 
> Up to you, what you have here it's fine!
> 

Speaking of this, I have a parallel sensor that does not implement 
get_mbus_config .

I tried to implement this in the sensor driver, to see if the csi2dc 
receives correct information from the sensor subdev (to see if it's 
parallel or serial). And it works fine.
I sent a patch for this, but Sakari says that the csi2dc can obtain all 
this information from DT, and I guess he is right. What would be the 
correct way to do it then ?

Could you comment on this patch please :

https://lore.kernel.org/linux-media/YYJDcIiBXo%2FXlKCX@paasikivi.fi.intel.com/T/#m990b903ed6d56c3c34232c54c251490ad33e92c9


Thanks again for reviewing !

>>>
>>> Speaking of remote subdevs, is there any driver available for IDI
>>> transmitters ?
>>
>> I am not aware of any at the moment
>>>
>>>> +
>>>> +     dev_dbg(csi2dc->dev, "%s clock\n",
>>>> +             csi2dc->clk_gated ? "gated" : "free running");
>>>> +
>>>> +     return 0;
>>>> +
>>>> +csi2dc_get_mbus_config_defaults:
>>>> +     csi2dc->vc = 0; /* Virtual ID 0 by default */
>>>> +     csi2dc->clk_gated = false; /* Free running clock by default */
>>>> +
>>>> +     return 0;
>>>> +}
>>>> +
>>>> +static int csi2dc_async_bound(struct v4l2_async_notifier *notifier,
>>>> +                           struct v4l2_subdev *subdev,
>>>> +                           struct v4l2_async_subdev *asd)
>>>> +{
>>>> +     struct csi2dc_device *csi2dc = container_of(notifier,
>>>> +                                             struct csi2dc_device, notifier);
>>>> +     int pad;
>>>> +     int ret;
>>>> +
>>>> +     csi2dc->input_sd = subdev;
>>>> +
>>>> +     pad = media_entity_get_fwnode_pad(&subdev->entity,
>>>
>>> You can use 'ret'
>>
>> I can, but it is weird to do remote_pad = ret ...
>>
>> May bring confusion, don't you agree ?
>>
> 
> Up to you, it's a small detail :)
> 
> Thanks!
>     j
> 
>>>
>>>> +                                       asd->match.fwnode,
>>>> +                                       MEDIA_PAD_FL_SOURCE);
>>>> +     if (pad < 0) {
>>>> +             dev_err(csi2dc->dev, "Failed to find pad for %s\n",
>>>> +                     subdev->name);
>>>> +             return pad;
>>>> +     }
>>>> +
>>>> +     csi2dc->remote_pad = pad;
>>>> +
>>>> +     csi2dc_get_mbus_config(csi2dc);
>>>
>>> Ideally, as this is not fatal, you could move this at s_stream time to
>>> fetch the most up-to-date configuration
>>>
>>>> +
>>>> +     ret = media_create_pad_link(&csi2dc->input_sd->entity,
>>>> +                                 csi2dc->remote_pad,
>>>> +                                 &csi2dc->csi2dc_sd.entity, 0,
>>>> +                                 MEDIA_LNK_FL_ENABLED);
>>>> +     if (ret < 0) {
>>>
>>>           if (ret)
>>>
>>>> +             dev_err(csi2dc->dev,
>>>> +                     "Failed to create pad link: %s to %s\n",
>>>> +                     csi2dc->input_sd->entity.name,
>>>> +                     csi2dc->csi2dc_sd.entity.name);
>>>> +             return ret;
>>>> +     }
>>>> +
>>>> +     dev_dbg(csi2dc->dev, "link with %s pad: %d\n",
>>>> +             csi2dc->input_sd->name, csi2dc->remote_pad);
>>>> +
>>>> +     ret = pm_runtime_resume_and_get(csi2dc->dev);
>>>> +     if (ret < 0)
>>>> +             return ret;
>>>> +
>>>> +     csi2dc_writel(csi2dc, CSI2DC_GCFG,
>>>> +                   (SAMA7G5_HLC & CSI2DC_GCFG_HLC_MASK) |
>>>> +                   (csi2dc->clk_gated ? 0 : CSI2DC_GCFG_MIPIFRN));
>>>> +
>>>> +     csi2dc_writel(csi2dc, CSI2DC_VPCOL,
>>>> +                   CSI2DC_VPCOL_COL(0xFFF) & CSI2DC_VPCOL_COL_MASK);
>>>> +     csi2dc_writel(csi2dc, CSI2DC_VPROW,
>>>> +                   CSI2DC_VPROW_ROW(0xFFF) & CSI2DC_VPROW_ROW_MASK);
>>>> +
>>>> +     pm_runtime_put_sync(csi2dc->dev);
>>>
>>> I would really move access to the HW to s_stream time if possible
>>>
>>>> +
>>>> +     return ret;
>>>> +}
>>>> +
>>>> +static const struct v4l2_async_notifier_operations csi2dc_async_ops = {
>>>> +     .bound = csi2dc_async_bound,
>>>> +};
>>>> +
>>>> +static void csi2dc_cleanup_notifier(struct csi2dc_device *csi2dc)
>>>> +{
>>>> +     v4l2_async_notifier_unregister(&csi2dc->notifier);
>>>> +     v4l2_async_notifier_cleanup(&csi2dc->notifier);
>>>> +}
>>>> +
>>>> +static int csi2dc_prepare_notifier(struct csi2dc_device *csi2dc,
>>>> +                                struct device_node *input_node)
>>>> +{
>>>> +     int ret = 0;
>>>> +
>>>> +     v4l2_async_notifier_init(&csi2dc->notifier);
>>>> +
>>>> +     csi2dc->asd = v4l2_async_notifier_add_fwnode_remote_subdev
>>>
>>> do you need asd in the driver structure ?
>>>
>>>> +                     (&csi2dc->notifier, of_fwnode_handle(input_node),
>>>> +                     struct v4l2_async_subdev);
>>>> +
>>>> +     of_node_put(input_node);
>>>> +
>>>> +     if (IS_ERR(csi2dc->asd)) {
>>>> +             ret = PTR_ERR(csi2dc->asd);
>>>> +             dev_err(csi2dc->dev,
>>>> +                     "failed to add async notifier for node %pOF: %d\n",
>>>> +                     input_node, ret);
>>>> +             v4l2_async_notifier_cleanup(&csi2dc->notifier);
>>>> +             return ret;
>>>> +     }
>>>> +
>>>> +     csi2dc->notifier.ops = &csi2dc_async_ops;
>>>> +
>>>> +     ret = v4l2_async_subdev_notifier_register(&csi2dc->csi2dc_sd,
>>>> +                                               &csi2dc->notifier);
>>>> +
>>>> +     if (ret) {
>>>> +             dev_err(csi2dc->dev, "fail to register async notifier: %d\n",
>>>> +                     ret);
>>>> +             v4l2_async_notifier_cleanup(&csi2dc->notifier);
>>>> +     }
>>>> +
>>>> +     return ret;
>>>> +}
>>>> +
>>>> +static int csi2dc_of_parse(struct csi2dc_device *csi2dc,
>>>> +                        struct device_node *of_node)
>>>> +{
>>>> +     struct device_node *input_node, *output_node;
>>>> +     struct v4l2_fwnode_endpoint input_endpoint = { 0 },
>>>> +                                 output_endpoint = { 0 };
>>>> +     int ret;
>>>> +
>>>> +     output_endpoint.bus_type = V4L2_MBUS_PARALLEL;
>>>> +
>>>> +     input_node = of_graph_get_next_endpoint(of_node, NULL);
>>>> +
>>>> +     if (!input_node) {
>>>> +             dev_err(csi2dc->dev,
>>>> +                     "missing port node at %pOF, input node is mandatory.\n",
>>>> +                     of_node);
>>>> +             return -EINVAL;
>>>> +     }
>>>> +
>>>> +     ret = v4l2_fwnode_endpoint_parse(of_fwnode_handle(input_node),
>>>> +                                      &input_endpoint);
>>>> +
>>>> +     if (ret) {
>>>                   of_node_put(input_node);
>>>
>>>> +             dev_err(csi2dc->dev, "endpoint not defined at %pOF\n", of_node);
>>>> +             return ret;
>>>> +     }
>>>> +
>>>> +     output_node = of_graph_get_next_endpoint(of_node, input_node);
>>>> +
>>>> +     if (output_node)
>>>> +             ret = v4l2_fwnode_endpoint_parse(of_fwnode_handle(output_node),
>>>> +                                              &output_endpoint);
>>>
>>>           of_node_put(output_node);
>>>> +
>>>> +     if (!output_node || ret) {
>>>> +             dev_info(csi2dc->dev,
>>>> +                      "missing output node at %pOF, data pipe available only.\n",
>>>> +                      of_node);
>>>> +     } else {
>>>> +             csi2dc->video_pipe = true;
>>>> +
>>>> +             dev_dbg(csi2dc->dev, "block %pOF %d.%d->%d.%d video pipeline\n",
>>>> +                     of_node, input_endpoint.base.port,
>>>> +                     input_endpoint.base.id, output_endpoint.base.port,
>>>> +                     output_endpoint.base.id);
>>>> +     }
>>>> +
>>>> +     of_node_put(output_node);
>>>
>>> Drop this if you put it above
>>>
>>>> +     of_node_put(input_node);
>>>
>>> Should you put input_node before passing it to the function ?
>>>> +
>>>> +     /* prepare async notifier for subdevice completion */
>>>> +     return csi2dc_prepare_notifier(csi2dc, input_node);
>>>> +}
>>>> +
>>>> +static int csi2dc_probe(struct platform_device *pdev)
>>>> +{
>>>> +     struct device *dev = &pdev->dev;
>>>> +     struct csi2dc_device *csi2dc;
>>>> +     struct resource *res = NULL;
>>>> +     int ret = 0;
>>>> +     u32 ver;
>>>> +
>>>> +     csi2dc = devm_kzalloc(dev, sizeof(*csi2dc), GFP_KERNEL);
>>>> +     if (!csi2dc)
>>>> +             return -ENOMEM;
>>>> +
>>>> +     csi2dc->dev = dev;
>>>> +
>>>> +     res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
>>>> +
>>>> +     csi2dc->base = devm_ioremap_resource(dev, res);
>>>
>>> Should devm_platform_ioremap_resource(pdev, 0) be used here ?
>>>
>>>> +     if (IS_ERR(csi2dc->base)) {
>>>> +             dev_err(dev, "base address not set\n");
>>>> +             return PTR_ERR(csi2dc->base);
>>>> +     }
>>>> +
>>>> +     csi2dc->pclk = devm_clk_get(dev, "pclk");
>>>> +     if (IS_ERR(csi2dc->pclk)) {
>>>> +             ret = PTR_ERR(csi2dc->pclk);
>>>> +             dev_err(dev, "failed to get pclk: %d\n", ret);
>>>> +             return ret;
>>>> +     }
>>>> +
>>>> +     csi2dc->scck = devm_clk_get(dev, "scck");
>>>> +     if (IS_ERR(csi2dc->scck)) {
>>>> +             ret = PTR_ERR(csi2dc->scck);
>>>> +             dev_err(dev, "failed to get scck: %d\n", ret);
>>>> +             return ret;
>>>> +     }
>>>> +
>>>> +     v4l2_subdev_init(&csi2dc->csi2dc_sd, &csi2dc_subdev_ops);
>>>> +
>>>> +     csi2dc->csi2dc_sd.owner = THIS_MODULE;
>>>> +     csi2dc->csi2dc_sd.dev = dev;
>>>> +     snprintf(csi2dc->csi2dc_sd.name, sizeof(csi2dc->csi2dc_sd.name),
>>>> +              "csi2dc");
>>>> +
>>>> +     csi2dc->csi2dc_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
>>>> +     csi2dc->csi2dc_sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
>>>> +     v4l2_set_subdevdata(&csi2dc->csi2dc_sd, pdev);
>>>
>>> Not used it seems
>>>
>>>> +
>>>> +     platform_set_drvdata(pdev, csi2dc);
>>>> +
>>>> +     ret = csi2dc_of_parse(csi2dc, dev->of_node);
>>>> +     if (ret)
>>>> +             goto csi2dc_probe_cleanup_entity;
>>>> +
>>>> +     csi2dc->pads[CSI2DC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>>>> +     if (csi2dc->video_pipe)
>>>> +             csi2dc->pads[CSI2DC_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
>>>> +
>>>> +     ret = media_entity_pads_init(&csi2dc->csi2dc_sd.entity,
>>>> +                                  csi2dc->video_pipe ? CSI2DC_PADS_NUM : 1,
>>>> +                                  csi2dc->pads);
>>>> +     if (ret < 0) {
>>>> +             dev_err(dev, "media entity init failed\n");
>>>> +             goto csi2dc_probe_cleanup_entity;
>>>
>>> Should you also clean up the notifier in the error path ?
>>>
>>>> +     }
>>>> +
>>>> +     /* turn power on to validate capabilities */
>>>> +     ret = csi2dc_power(csi2dc, true);
>>>> +     if (ret < 0)
>>>> +             goto csi2dc_probe_cleanup_entity;
>>>> +
>>>> +     pm_runtime_set_active(dev);
>>>> +     pm_runtime_enable(dev);
>>>> +     ver = csi2dc_readl(csi2dc, CSI2DC_VERSION);
>>>> +     pm_request_idle(dev);
>>>> +
>>>> +     /*
>>>> +      * we must register the subdev after PM runtime has been requested,
>>>> +      * otherwise we might bound immediately and request pm_runtime_resume
>>>> +      * before runtime_enable.
>>>> +      */
>>>> +     ret = v4l2_async_register_subdev(&csi2dc->csi2dc_sd);
>>>> +     if (ret) {
>>>> +             dev_err(csi2dc->dev, "failed to register the subdevice\n");
>>>> +             goto csi2dc_probe_cleanup_entity;
>>>> +     }
>>>> +
>>>> +     dev_info(dev, "Microchip CSI2DC version %x\n", ver);
>>>> +
>>>> +     return 0;
>>>> +
>>>> +csi2dc_probe_cleanup_entity:
>>>> +     media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
>>>> +
>>>> +     return ret;
>>>> +}
>>>> +
>>>> +static int csi2dc_remove(struct platform_device *pdev)
>>>> +{
>>>> +     struct csi2dc_device *csi2dc = platform_get_drvdata(pdev);
>>>> +
>>>> +     pm_runtime_disable(&pdev->dev);
>>>> +
>>>> +     v4l2_async_unregister_subdev(&csi2dc->csi2dc_sd);
>>>> +     csi2dc_cleanup_notifier(csi2dc);
>>>> +     media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
>>>> +
>>>> +     return 0;
>>>> +}
>>>> +
>>>> +static int __maybe_unused csi2dc_runtime_suspend(struct device *dev)
>>>> +{
>>>> +     struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
>>>> +
>>>> +     return csi2dc_power(csi2dc, false);
>>>> +}
>>>> +
>>>> +static int __maybe_unused csi2dc_runtime_resume(struct device *dev)
>>>> +{
>>>> +     struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
>>>> +
>>>> +     return csi2dc_power(csi2dc, true);
>>>> +}
>>>> +
>>>> +static const struct dev_pm_ops csi2dc_dev_pm_ops = {
>>>> +     SET_RUNTIME_PM_OPS(csi2dc_runtime_suspend, csi2dc_runtime_resume, NULL)
>>>> +};
>>>> +
>>>> +static const struct of_device_id csi2dc_of_match[] = {
>>>> +     { .compatible = "microchip,sama7g5-csi2dc" },
>>>> +     { }
>>>> +};
>>>> +
>>>> +MODULE_DEVICE_TABLE(of, csi2dc_of_match);
>>>> +
>>>> +static struct platform_driver csi2dc_driver = {
>>>> +     .probe  = csi2dc_probe,
>>>> +     .remove = csi2dc_remove,
>>>> +     .driver = {
>>>> +             .name =                 "microchip-csi2dc",
>>>> +             .pm =                   &csi2dc_dev_pm_ops,
>>>> +             .of_match_table =       of_match_ptr(csi2dc_of_match),
>>>> +     },
>>>> +};
>>>> +
>>>> +module_platform_driver(csi2dc_driver);
>>>> +
>>>> +MODULE_AUTHOR("Eugen Hristev <eugen.hristev@microchip.com>");
>>>> +MODULE_DESCRIPTION("Microchip CSI2 Demux Controller driver");
>>>> +MODULE_LICENSE("GPL v2");
>>>> --
>>>> 2.25.1
>>>>
>>

Re: [PATCH 03/21] media: atmel: introduce microchip csi2dc driver

[Jacopo Mondi]

Hi Eugen

On Wed, Nov 03, 2021 at 09:59:35AM +0000, Eugen.Hristev@microchip.com wrote:
> On 11/3/21 11:28 AM, Jacopo Mondi wrote:
> > Hi Eugen
> >
> > On Wed, Nov 03, 2021 at 08:31:36AM +0000, Eugen.Hristev@microchip.com wrote:
> >> On 11/2/21 7:22 PM, Jacopo Mondi wrote:
> >>> Hi Eugen,
> >>>
> >>
> >> Hi,
> >>
> >> Thank you for your review. I will try to understand everything you said
> >> and come up with a new version. I still have some inline questions
> >> though, about some things which are still unclear.
> >>
> >>> On Fri, Oct 22, 2021 at 10:52:29AM +0300, Eugen Hristev wrote:
> >>>> Microchip CSI2DC (CSI2 Demultiplexer Controller) is a misc bridge device
> >>>> that converts a byte stream in IDI Synopsys format (coming from a CSI2HOST)
> >>>> to a pixel stream that can be captured by a sensor controller.
> >>>>
> >>>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> >>>> ---
> >>>> Changes in this revision:
> >>>> - addressed comments by Jacopo and Laurent as in this thread:
> >>>> https://www.spinics.net/lists/linux-media/msg181044.html
> >>>>
> >>>> Previous change log :
> >>>> Changes in v5:
> >>>> - only in bindings
> >>>>
> >>>> Changes in v4:
> >>>> - now using get_mbus_config ops to get data from the subdevice, like the
> >>>> virtual channel id, and the clock type.
> >>>> - now having possibility to select any of the RAW10 data modes
> >>>> - at completion time, select which formats are also available in the subdevice,
> >>>> and move to the dynamic list accordingly
> >>>> - changed the pipeline integration, do not advertise subdev ready at probe time.
> >>>> wait until completion is done, and then start a workqueue that will register
> >>>> this device as a subdevice for the next element in pipeline.
> >>>> - moved the s_power code into a different function called now csi2dc_power
> >>>> that is called with CONFIG_PM functions. This is also called at completion,
> >>>> to have the device ready in case CONFIG_PM is not selected on the platform.
> >>>> - merged try_fmt into set_fmt
> >>>> - driver cleanup, wrapped lines over 80 characters
> >>>>
> >>>> Changes in v2:
> >>>> - moved driver to platform/atmel
> >>>> - fixed minor things as per Sakari's review
> >>>> - still some things from v2 review are not yet addressed, to be followed up
> >>>>
> >>>>
> >>>>    drivers/media/platform/atmel/Kconfig          |  15 +
> >>>>    drivers/media/platform/atmel/Makefile         |   1 +
> >>>>    .../media/platform/atmel/microchip-csi2dc.c   | 700 ++++++++++++++++++
> >>>>    3 files changed, 716 insertions(+)
> >>>>    create mode 100644 drivers/media/platform/atmel/microchip-csi2dc.c
> >>>>
> >>>> diff --git a/drivers/media/platform/atmel/Kconfig b/drivers/media/platform/atmel/Kconfig
> >>>> index dda2f27da317..f83bee373d82 100644
> >>>> --- a/drivers/media/platform/atmel/Kconfig
> >>>> +++ b/drivers/media/platform/atmel/Kconfig
> >>>> @@ -40,3 +40,18 @@ config VIDEO_ATMEL_ISI
> >>>>         help
> >>>>           This module makes the ATMEL Image Sensor Interface available
> >>>>           as a v4l2 device.
> >>>> +
> >>>> +config VIDEO_MICROCHIP_CSI2DC
> >>>> +     tristate "Microchip CSI2 Demux Controller"
> >>>> +     depends on VIDEO_V4L2 && COMMON_CLK && OF
> >>>> +     depends on ARCH_AT91 || COMPILE_TEST
> >>>> +     select MEDIA_CONTROLLER
> >>>> +     select VIDEO_V4L2_SUBDEV_API
> >>>> +     select V4L2_FWNODE
> >>>> +     help
> >>>> +       CSI2 Demux Controller driver. CSI2DC is a helper chip
> >>>> +       that converts IDI interface byte stream to a parallel pixel stream.
> >>>> +       It supports various RAW formats as input.
> >>>> +
> >>>> +       To compile this driver as a module, choose M here: the
> >>>> +       module will be called microchip-csi2dc.
> >>>> diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
> >>>> index 46d264ab7948..39f0a7eba702 100644
> >>>> --- a/drivers/media/platform/atmel/Makefile
> >>>> +++ b/drivers/media/platform/atmel/Makefile
> >>>> @@ -6,3 +6,4 @@ obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
> >>>>    obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o
> >>>>    obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel-isc.o
> >>>>    obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel-xisc.o
> >>>> +obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += microchip-csi2dc.o
> >>>> diff --git a/drivers/media/platform/atmel/microchip-csi2dc.c b/drivers/media/platform/atmel/microchip-csi2dc.c
> >>>> new file mode 100644
> >>>> index 000000000000..277b86988eee
> >>>> --- /dev/null
> >>>> +++ b/drivers/media/platform/atmel/microchip-csi2dc.c
> >>>> @@ -0,0 +1,700 @@
> >>>> +// SPDX-License-Identifier: GPL-2.0-only
> >>>> +/*
> >>>> + * Microchip CSI2 Demux Controller (CSI2DC) driver
> >>>> + *
> >>>> + * Copyright (C) 2018 Microchip Technology, Inc.
> >>>> + *
> >>>> + * Author: Eugen Hristev <eugen.hristev@microchip.com>
> >>>> + *
> >>>> + */
> >>>> +
> >>>> +#include <linux/clk.h>
> >>>> +#include <linux/module.h>
> >>>> +#include <linux/of.h>
> >>>
> >>> Isn't linux/mod_devicetable.h enough ?
> >>>
> >>>> +#include <linux/of_graph.h>
> >>>
> >>> You should probably move to use the fwnode_graph framwork instead of
> >>> of_graph. This driver depends on OF so it shouldn't be an issue but I
> >>> defer this to maintainers
> >>>
> >>>> +#include <linux/platform_device.h>
> >>>> +#include <linux/pm_runtime.h>
> >>>> +#include <linux/videodev2.h>
> >>>> +
> >>>> +#include <media/v4l2-device.h>
> >>>
> >>> Do you need this include ?
> >>>
> >>>> +#include <media/v4l2-fwnode.h>
> >>>> +#include <media/v4l2-subdev.h>
> >>>> +#include <media/videobuf2-dma-contig.h>
> >>>
> >>> Is this one needed as well ?
> >>>
> >>>> +
> >>>> +/* Global configuration register */
> >>>> +#define CSI2DC_GCFG                  0x0
> >>>> +
> >>>> +/* MIPI sensor pixel clock is free running */
> >>>> +#define CSI2DC_GCFG_MIPIFRN          BIT(0)
> >>>> +/* Output waveform inter-line minimum delay */
> >>>> +#define CSI2DC_GCFG_HLC(v)           ((v) << 4)
> >>>> +#define CSI2DC_GCFG_HLC_MASK         GENMASK(7, 4)
> >>>> +/* SAMA7G5 requires a HLC delay of 15 */
> >>>> +#define SAMA7G5_HLC                  (15)
> >>>> +
> >>>> +/* Global control register */
> >>>> +#define CSI2DC_GCTLR                 0x04
> >>>> +#define CSI2DC_GCTLR_SWRST           BIT(0)
> >>>> +
> >>>> +/* Global status register */
> >>>> +#define CSI2DC_GS                    0x08
> >>>> +
> >>>> +/* SSP interrupt status register */
> >>>> +#define CSI2DC_SSPIS                 0x28
> >>>> +/* Pipe update register */
> >>>> +#define CSI2DC_PU                    0xC0
> >>>
> >>> What about using lowercase for hex values (I know there's not strict
> >>> rule, so keep what you like the most, but most drivers use lowercase
> >>>
> >>>> +/* Video pipe attributes update */
> >>>> +#define CSI2DC_PU_VP                 BIT(0)
> >>>> +
> >>>> +/* Pipe update status register */
> >>>> +#define CSI2DC_PUS                   0xC4
> >>>> +
> >>>> +/* Video pipeline enable register */
> >>>> +#define CSI2DC_VPE                   0xF8
> >>>> +#define CSI2DC_VPE_ENABLE            BIT(0)
> >>>> +
> >>>> +/* Video pipeline configuration register */
> >>>> +#define CSI2DC_VPCFG                 0xFC
> >>>> +/* Data type */
> >>>> +#define CSI2DC_VPCFG_DT(v)           ((v) << 0)
> >>>> +#define CSI2DC_VPCFG_DT_MASK         GENMASK(5, 0)
> >>>> +/* Virtual channel identifier */
> >>>> +#define CSI2DC_VPCFG_VC(v)           ((v) << 6)
> >>>> +#define CSI2DC_VPCFG_VC_MASK         GENMASK(7, 6)
> >>>> +/* Decompression enable */
> >>>> +#define CSI2DC_VPCFG_DE                      BIT(8)
> >>>> +/* Decoder mode */
> >>>> +#define CSI2DC_VPCFG_DM(v)           ((v) << 9)
> >>>> +#define CSI2DC_VPCFG_DM_DECODER8TO12 0
> >>>> +/* Decoder predictor 2 selection */
> >>>> +#define CSI2DC_VPCFG_DP2             BIT(12)
> >>>> +/* Recommended memory storage */
> >>>> +#define CSI2DC_VPCFG_RMS             BIT(13)
> >>>> +/* Post adjustment */
> >>>> +#define CSI2DC_VPCFG_PA                      BIT(14)
> >>>> +
> >>>> +/* Video pipeline column register */
> >>>> +#define CSI2DC_VPCOL                 0x100
> >>>> +/* Column number */
> >>>> +#define CSI2DC_VPCOL_COL(v)          ((v) << 0)
> >>>> +#define CSI2DC_VPCOL_COL_MASK                GENMASK(15, 0)
> >>>> +
> >>>> +/* Video pipeline row register */
> >>>> +#define CSI2DC_VPROW                 0x104
> >>>> +/* Row number */
> >>>> +#define CSI2DC_VPROW_ROW(v)          ((v) << 0)
> >>>> +#define CSI2DC_VPROW_ROW_MASK                GENMASK(15, 0)
> >>>> +
> >>>> +/* Version register */
> >>>> +#define CSI2DC_VERSION                       0x1FC
> >>>> +
> >>>> +/* register read/write helpers */
> >>>> +#define csi2dc_readl(st, reg)                readl_relaxed((st)->base + (reg))
> >>>> +#define csi2dc_writel(st, reg, val)  writel_relaxed((val), \
> >>>> +                                     (st)->base + (reg))
> >>>> +
> >>>> +/* supported RAW data types */
> >>>> +#define CSI2DC_DT_RAW6                       0x28
> >>>> +#define CSI2DC_DT_RAW7                       0x29
> >>>> +#define CSI2DC_DT_RAW8                       0x2A
> >>>> +#define CSI2DC_DT_RAW10                      0x2B
> >>>> +#define CSI2DC_DT_RAW12                      0x2C
> >>>> +#define CSI2DC_DT_RAW14                      0x2D
> >>>> +
> >>>> +/*
> >>>> + * struct csi2dc_format - CSI2DC format type struct
> >>>> + * @mbus_code:               Media bus code for the format
> >>>> + * @dt:                      Data type constant for this format
> >>>> + */
> >>>> +struct csi2dc_format {
> >>>> +     u32                             mbus_code;
> >>>> +     u32                             dt;
> >>>> +};
> >>>> +
> >>>> +static const struct csi2dc_format csi2dc_formats[] = {
> >>>> +     {
> >>>> +             .mbus_code =            MEDIA_BUS_FMT_SRGGB10_1X10,
> >>>> +             .dt =                   CSI2DC_DT_RAW10,
> >>>> +     }, {
> >>>> +             .mbus_code =            MEDIA_BUS_FMT_SBGGR10_1X10,
> >>>> +             .dt =                   CSI2DC_DT_RAW10,
> >>>> +     }, {
> >>>> +             .mbus_code =            MEDIA_BUS_FMT_SGRBG10_1X10,
> >>>> +             .dt =                   CSI2DC_DT_RAW10,
> >>>> +     }, {
> >>>> +             .mbus_code =            MEDIA_BUS_FMT_SGBRG10_1X10,
> >>>> +             .dt =                   CSI2DC_DT_RAW10,
> >>>> +     },
> >>>> +};
> >>>
> >>> How unfortunate we don't have this in the core...
> >>>
> >>>> +
> >>>> +enum mipi_csi_pads {
> >>>> +     CSI2DC_PAD_SINK                 = 0,
> >>>> +     CSI2DC_PAD_SOURCE               = 1,
> >>>> +     CSI2DC_PADS_NUM                 = 2,
> >>>> +};
> >>>> +
> >>>> +/*
> >>>> + * struct csi2dc_device - CSI2DC device driver data/config struct
> >>>> + * @base:            Register map base address
> >>>> + * @csi2dc_sd:               v4l2 subdevice for the csi2dc device
> >>>> + *                   This is the subdevice that the csi2dc device itself
> >>>> + *                   registers in v4l2 subsystem
> >>>> + * @dev:             struct device for this csi2dc device
> >>>> + * @pclk:            Peripheral clock reference
> >>>> + *                   Input clock that clocks the hardware block internal
> >>>> + *                   logic
> >>>> + * @scck:            Sensor Controller clock reference
> >>>> + *                   Input clock that is used to generate the pixel clock
> >>>> + * @format:          Current saved format used in g/s fmt
> >>>> + * @cur_fmt:         Current state format
> >>>> + * @try_fmt:         Try format that is being tried
> >>>> + * @pads:            Media entity pads for the csi2dc subdevice
> >>>> + * @clk_gated:               Whether the clock is gated or free running
> >>>> + * @video_pipe:              Whether video pipeline is configured
> >>>> + * @vc:                      Current set virtual channel
> >>>> + * @asd:             Async subdevice for async bound of the underlying subdev
> >>>> + * @notifier:                Async notifier that is used to bound the underlying
> >>>> + *                   subdevice to the csi2dc subdevice
> >>>> + * @input_sd:                Reference to the underlying subdevice bound to the
> >>>> + *                   csi2dc subdevice
> >>>> + * @remote_pad:              Pad number of the underlying subdevice that is linked
> >>>> + *                   to the csi2dc subdevice sink pad.
> >>>> + */
> >>>> +struct csi2dc_device {
> >>>> +     void __iomem                    *base;
> >>>> +     struct v4l2_subdev              csi2dc_sd;
> >>>> +     struct device                   *dev;
> >>>> +     struct clk                      *pclk;
> >>>> +     struct clk                      *scck;
> >>>> +
> >>>> +     struct v4l2_mbus_framefmt        format;
> >>>> +
> >>>> +     const struct csi2dc_format      *cur_fmt;
> >>>> +     const struct csi2dc_format      *try_fmt;
> >>>> +
> >>>> +     struct media_pad                pads[CSI2DC_PADS_NUM];
> >>>> +
> >>>> +     bool                            clk_gated;
> >>>> +     bool                            video_pipe;
> >>>> +     u32                             vc;
> >>>> +
> >>>> +     struct v4l2_async_subdev        *asd;
> >>>> +     struct v4l2_async_notifier      notifier;
> >>>> +
> >>>> +     struct v4l2_subdev              *input_sd;
> >>>> +
> >>>> +     u32                             remote_pad;
> >>>> +};
> >>>> +
> >>>> +static void csi2dc_vp_update(struct csi2dc_device *csi2dc)
> >>>
> >>> Could you move this below closer to the only caller ?
> >>>
> >>>> +{
> >>>> +     u32 vp;
> >>>> +
> >>>> +     if (!csi2dc->cur_fmt) {
> >>>
> >>> You should probably initialize this to a default format
> >>>
> >>>> +             dev_err(csi2dc->dev, "format must be configured first\n");
> >>>> +             return;
> >>>> +     }
> >>>> +
> >>>> +     if (!csi2dc->video_pipe) {
> >>>
> >>> This is only called internally by the driver at s_stream() time, can
> >>> this happen ? Or rather won't you have a streamon call also when the
> >>> data pipe only is available ? In that case you would error out here
> >>
> >> This can happen if the source node is not found in the DT. In that case,
> >> there is no video pipe available, thus this whole function should be a
> >> no-op (csi2dc_vp_update) . If no vp, there is no vp update.
> >> Normall without a vp, there should be a dp (data pipe), but I haven't
> >> tested this and it's not supported in the driver.
> >> It doesn't make any sense to configure the vp registers below if the vp
> >> is not available.
> >> However, I shouldn't return an error since the vp is not mandatory for
> >> the function of the hardware. Just that the dp will be implemented at a
> >> later time (.... or never).
> >>
> >
> > I see, so the data pipe is still work in progress... I understand
> > bindings should allow for that to be later accepted so you cannot
> > mandate port@1 to be there all the times, but should the driver
> > instead refuse to operate if no port@1 is provided ?
> >
>
> It will not refuse to operate, but won't operate anything basically.
> And if I make it refuse to operate, it will violate the bindings, since
> port@1 is not mandatory

I don't think that's a problem. I would say the it would also be fine
to have port@1 mandatory as if it becomes optional in future once the
data-pipe is implemented, old DTS will still be compatible.

However I also think it would be fine for the driver to not support
all the options listed in the bindings. What's importante is that
anything that gets into the bindings does not break
retro-compatibility in case it gets extended.

To make a counter example, if your bindings define port@1 as optional
but you later make it mandatory for some reason, old dts without
port@1 won't work anymore and that's a problem. In this case if you
make it mandatory and later demote it to be optional, old DTS will all
have port@1 and will work in video-pipe mode while new ones can
optionally swtich to data-pipe by not providing port@1.

Anyway, not a big deal, I think what you have here is fine but I won't
bother too much about checking for if (!csi2dc->video_pipe) in the
driver as if I understood you correctly the driver cannot currently
be operated in data-pipe mode.

>
> > Out of curiosity what will the interaction model with the driver be in
> > data pipe mode ? There will be a video device but there won't be an
> > link to it, how is stream start/stop controlled ?
>
> My idea is that on data pipe, the csi2dc should register it's own video
> device and act basically as a dma engine entity that will use an
> external DMA controller to just write data to DRAM.
>
> It is useful if we do not want to convert the MIPI/IDI packets into
> pixels, but rather dump the to DRAM and have a piece of software further
> use them.
>
> However, it's not a clear use case about this functionality at this
> moment. One possibility is to be able to obtain stream meta-data from
> the MIPI / IDI interface, like for example synchronization data, to
> synchronize with an audio capture device.
>
> >
> >>>
> >>>> +             dev_err(csi2dc->dev, "video pipeline unavailable\n");
> >>>> +             return;
> >>>> +     }
> >>>> +
> >>>> +     vp = CSI2DC_VPCFG_DT(csi2dc->cur_fmt->dt) & CSI2DC_VPCFG_DT_MASK;
> >>>> +     vp |= CSI2DC_VPCFG_VC(csi2dc->vc) & CSI2DC_VPCFG_VC_MASK;
> >>>> +     vp &= ~CSI2DC_VPCFG_DE;
> >>>> +     vp |= CSI2DC_VPCFG_DM(CSI2DC_VPCFG_DM_DECODER8TO12);
> >>>> +     vp &= ~CSI2DC_VPCFG_DP2;
> >>>> +     vp &= ~CSI2DC_VPCFG_RMS;
> >>>> +     vp |= CSI2DC_VPCFG_PA;
> >>>> +
> >>>> +     csi2dc_writel(csi2dc, CSI2DC_VPCFG, vp);
> >>>> +     csi2dc_writel(csi2dc, CSI2DC_VPE, CSI2DC_VPE_ENABLE);
> >>>> +     csi2dc_writel(csi2dc, CSI2DC_PU, CSI2DC_PU_VP);
> >>>> +}
> >>>> +
> >>>> +static inline struct csi2dc_device *
> >>>> +csi2dc_sd_to_csi2dc_device(struct v4l2_subdev *csi2dc_sd)
> >>>> +{
> >>>> +     return container_of(csi2dc_sd, struct csi2dc_device, csi2dc_sd);
> >>>> +}
> >>>> +
> >>>> +static int csi2dc_enum_mbus_code(struct v4l2_subdev *csi2dc_sd,
> >>>> +                              struct v4l2_subdev_state *sd_state,
> >>>> +                              struct v4l2_subdev_mbus_code_enum *code)
> >>>> +{
> >>>> +     if (code->index >= ARRAY_SIZE(csi2dc_formats))
> >>>> +             return -EINVAL;
> >>>> +
> >>>> +     code->code = csi2dc_formats[code->index].mbus_code;
> >>>> +
> >>>> +     return 0;
> >>>> +}
> >>>> +
> >>>> +static int csi2dc_get_fmt(struct v4l2_subdev *csi2dc_sd,
> >>>> +                       struct v4l2_subdev_state *sd_state,
> >>>> +                       struct v4l2_subdev_format *format)
> >>>> +{
> >>>> +     struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
> >>>> +
> >>>> +     format->format = csi2dc->format;
> >>>> +
> >>>> +     return 0;
> >>>
> >>> You should support try formats by storing the format in the file
> >>> handle state in s_fmt and return it in case which == TRY
> >>>
> >>> Grep for v4l2_subdev_get_try_format() for usage examples
> >>
> >> I did that initially, but could not find any utility for it in my tests.
> >
> > I understand.. Have you run v4l2-compliance on this sudev ? Doesn't it
> > test try formats support ?
> >
> >> So I removed it.
> >> I will try to bring it back, but I have no idea how to test that the try
> >> format works fine or not
> >>
> >
> > I think also v4l2-ctl supports try formats.
> >
> > --try-subdev-fmt pad=<pad>,width=<w>,height=<h>,code=<code>,field=<f>,colorspace=<c>,
> >
> > I'm not sure if it allows you to read it back, but it's easy to hack
> > it out just to verify it.
> >
>
> Ok, I will try it
>
> >>>
> >>>> +}
> >>>> +
> >>>> +static int csi2dc_set_fmt(struct v4l2_subdev *csi2dc_sd,
> >>>> +                       struct v4l2_subdev_state *sd_state,
> >>>> +                       struct v4l2_subdev_format *req_fmt)
> >>>> +{
> >>>> +     struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
> >>>> +     const struct csi2dc_format *fmt;
> >>>> +     int i;
> >>>
> >>> unsigned
> >>>
> >>>> +
> >>>> +     for (i = 0; i < ARRAY_SIZE(csi2dc_formats);  i++) {
> >>>> +             fmt = &csi2dc_formats[i];
> >>>> +             if (req_fmt->format.code == fmt->mbus_code)
> >>>> +                     csi2dc->try_fmt = fmt;
> >>>
> >>> Shouldn't you break ?
> >>>
> >>>> +             fmt++;
> >>>> +     }
> >>>
> >>> And make this a simpler
> >>>
> >>>           for (i = 0; i < ARRAY_SIZE(csi2dc_formats); i++) {
> >>>                   if (req_fmt->format.code == csi2dc_formats[i].mbus_code)
> >>>                           break;
> >>>           }
> >>>
> >>>           if (i == ARRAY_SIZE(csi2dc_formats)
> >>>                   i = 0;
> >>>
> >>>> +
> >>>> +     /* in case we could not find the desired format, default to something */
> >>>> +     if (!csi2dc->try_fmt  ||
> >>>> +         req_fmt->format.code != csi2dc->try_fmt->mbus_code) {
> >>>> +             csi2dc->try_fmt = &csi2dc_formats[0];
> >>>> +
> >>>> +             dev_dbg(csi2dc->dev,
> >>>> +                     "CSI2DC unsupported format 0x%x, defaulting to 0x%x\n",
> >>>> +                     req_fmt->format.code, csi2dc_formats[0].mbus_code);
> >>>> +
> >>>> +             req_fmt->format.code = csi2dc_formats[0].mbus_code;
> >>>> +     }
> >>>> +
> >>>> +     req_fmt->format.colorspace = V4L2_COLORSPACE_SRGB;
> >>>> +     req_fmt->format.field = V4L2_FIELD_NONE;
> >>>> +
> >>>> +     /* save the format for later requests */
> >>>
> >>> You should support TRY formats
> >>>
> >>>> +     csi2dc->format = req_fmt->format;
> >>>> +
> >>>> +     /* if we are just trying, we are done */
> >>>> +     if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY)
> >>>> +             return 0;
> >>>> +
> >>>> +     csi2dc->cur_fmt = csi2dc->try_fmt;
> >>>
> >>>           csi2dc->cur_fmt = &csi2dc_format[i];
> >>>
> >>> So you can drop the try_fmt from the driver structure as it seems to
> >>> be used as a temporary variable here only.
> >>>
> >>>> +
> >>>> +     dev_dbg(csi2dc->dev, "new format set: 0x%x\n", req_fmt->format.code);
> >>>> +
> >>>> +     return 0;
> >>>> +}
> >>>> +
> >>>> +static int csi2dc_power(struct csi2dc_device *csi2dc, int on)
> >>>> +{
> >>>> +     int ret = 0;
> >>>> +
> >>>> +     if (on) {
> >>>> +             ret = clk_prepare_enable(csi2dc->pclk);
> >>>> +             if (ret) {
> >>>> +                     dev_err(csi2dc->dev, "failed to enable pclk: %d\n", ret);
> >>>> +                     return ret;
> >>>> +             }
> >>>> +
> >>>> +             ret = clk_prepare_enable(csi2dc->scck);
> >>>> +             if (ret)
> >>>> +                     dev_err(csi2dc->dev,
> >>>> +                             "failed to enable scck: %d\n", ret);
> >>>
> >>> Shouldn't you bail out here ?
> >>
> >> Initially this clock was not mandatory, but you are right, I should bail
> >> out.
> >>
> >>>
> >>>> +
> >>>> +             /* if powering up, deassert reset line */
> >>>> +             csi2dc_writel(csi2dc, CSI2DC_GCTLR, CSI2DC_GCTLR_SWRST);
> >>>> +     } else {
> >>>> +             clk_disable_unprepare(csi2dc->scck);
> >>>> +
> >>>> +             /* if powering down, assert reset line */
> >>>> +             csi2dc_writel(csi2dc, CSI2DC_GCTLR, !CSI2DC_GCTLR_SWRST);
> >>>
> >>> Isn't reverse order of activation better ?
> >>>
> >>>                   csi2dc_writel(..)
> >>>                   clk_disable_unprepare(..)
> >>>                   clk_disable_unprepare(..)
> >>>> +
> >>>> +             clk_disable_unprepare(csi2dc->pclk);
> >>>> +     }
> >>>> +
> >>>> +     return ret;
> >>>> +}
> >>>> +
> >>>> +static int csi2dc_s_stream(struct v4l2_subdev *csi2dc_sd, int enable)
> >>>> +{
> >>>> +     struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
> >>>> +     int ret;
> >>>> +
> >>>> +     if (enable) {
> >>>> +             ret = pm_runtime_resume_and_get(csi2dc->dev);
> >>>> +             if (ret < 0)
> >>>> +                     return ret;
> >>>> +             csi2dc_vp_update(csi2dc);
> >>>> +     } else {
> >>>> +             pm_runtime_put_sync(csi2dc->dev);
> >>>> +     }
> >>>> +
> >>>> +     return v4l2_subdev_call(csi2dc->input_sd, video, s_stream, enable);
> >>>
> >>> Should the remote subdev be started before and stopped after ?
> >>
> >> Do you mean s_power ?
> >>
> >
> > Nope, I meant the operation order.
> >
> >          if (enable) {
> >                  v4l2_subdev_call(...);
> >
> >                  pm_runtime_resume_and_get();
> >                  csi2dc_vp_update();
> >          } else {
> >                  pm_runtime_put_sync();
> >
> >                  v4l2_subdev_call();
> >          }
> >
> > However, my comment is clearly wrong, it should have been the
> > contrary. Sorry about this
> >
> > It's right to configure the interface and power if up -before-
> > starting the remote subdev, but I would stop it before powering it
> > down. Like in:
> >
> >          if (enable) {
> >                  pm_runtime_resume_and_get();
> >                  csi2dc_vp_update();
> >
> >                  v4l2_subdev_call(1);
> >          } else {
> >                  v4l2_subdev_call(0);
> >
> >                  pm_runtime_put_sync();
> >          }
> >
> > What do you think ? Stopping the remote before powering the interface
> > down ensures no data is being put on the bus while the interace is
> > powered off.
> >
>
> In this case it might not work like this. If I start the sensor
> streaming before the vp update , then, there will be frames received on
> video pipe update, which means that the csi2dc will not detect the phy
> STOP state reached, and it will fail initialization.
> I believe the csi2dc has to be started before the sensor's first frame
> such that the interface is prepared for receiving. The PHY must be in
> stop state for a certain amount of time.
> I will try to see if it works the other way around, to double check to
> make sure.
> But if it doesn't , then the VP must be configured while the whole
> pipeline is in stop state.
>

I agree, as I've said my first comment was wrong and the sensor has to
be started after the csi2dc. However in my last comment I was
suggesting to stop it before the csi2dc to avoid having data being
sent on the bus while the csi2dc gets powered down.

> >>>
> >>>> +}
> >>>> +
> >>>> +static const struct v4l2_subdev_pad_ops csi2dc_pad_ops = {
> >>>> +     .enum_mbus_code = csi2dc_enum_mbus_code,
> >>>> +     .set_fmt = csi2dc_set_fmt,
> >>>> +     .get_fmt = csi2dc_get_fmt,
> >>>> +};
> >>>> +
> >>>> +static const struct v4l2_subdev_video_ops csi2dc_video_ops = {
> >>>> +     .s_stream = csi2dc_s_stream,
> >>>> +};
> >>>> +
> >>>> +static const struct v4l2_subdev_ops csi2dc_subdev_ops = {
> >>>> +     .pad = &csi2dc_pad_ops,
> >>>> +     .video = &csi2dc_video_ops,
> >>>> +};
> >>>> +
> >>>> +static int csi2dc_get_mbus_config(struct csi2dc_device *csi2dc)
> >>>> +{
> >>>> +     struct v4l2_mbus_config mbus_config = { 0 };
> >>>> +     int ret;
> >>>> +
> >>>> +     ret = v4l2_subdev_call(csi2dc->input_sd, pad, get_mbus_config,
> >>>> +                            csi2dc->remote_pad, &mbus_config);
> >>>> +     if (ret == -ENOIOCTLCMD) {
> >>>> +             dev_dbg(csi2dc->dev,
> >>>> +                     "no remote mbus configuration available\n");
> >>>> +             goto csi2dc_get_mbus_config_defaults;
> >>>> +     }
> >>>> +
> >>>> +     if (ret) {
> >>>> +             dev_err(csi2dc->dev,
> >>>> +                     "failed to get remote mbus configuration\n");
> >>>> +             goto csi2dc_get_mbus_config_defaults;
> >>>> +     }
> >>>> +
> >>>> +     if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_0)
> >>>> +             csi2dc->vc = 0;
> >>>> +     else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_1)
> >>>> +             csi2dc->vc = 1;
> >>>> +     else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_2)
> >>>> +             csi2dc->vc = 2;
> >>>> +     else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_3)
> >>>> +             csi2dc->vc = 3;
> >>>> +
> >>>> +     dev_dbg(csi2dc->dev, "subdev sending on channel %d\n", csi2dc->vc);
> >>>> +
> >>>> +     csi2dc->clk_gated = mbus_config.flags &
> >>>> +                             V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK;
> >>>
> >>> This should come from the default clock-noncontinuous property in the
> >>> endpoint. It is available in the mbus_configuration only to support
> >>> subdevs that can change it at runtime, and if that's the case it's ok,
> >>> but I think it should be in the endpoint.
> >>
> >> I do not completely understand your point. In a previous version which
> >> you reviewed, I had this as a DT property (so in the endpoint ?), and
> >> now I converted it to a get_mbus_config flag get operation.
> >
> > I re-read the conversation and I think you're right.
> >
> >> It's not the right way to do it ?
> >> How should it be specified in the endpoint ?
> >>
> >
> > There's a default property 'clock-noncontinuous' which gets parsed by
> > v4l2_fwnode_endpoint_parse() and reported to drivers through the
> > V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK and V4L2_MBUS_CSI2_CONTINUOUS_CLOCK
> > flags. In your case, as you fetch it at run time it's not required if
> > not to initialize a default according to the property presence in DT.
> >
> > Up to you, what you have here it's fine!
> >
>
> Speaking of this, I have a parallel sensor that does not implement
> get_mbus_config .
>
> I tried to implement this in the sensor driver, to see if the csi2dc
> receives correct information from the sensor subdev (to see if it's
> parallel or serial). And it works fine.
> I sent a patch for this, but Sakari says that the csi2dc can obtain all
> this information from DT, and I guess he is right. What would be the
> correct way to do it then ?

Good question, and probably Sakari is in a better position than me to
set a direction.

My take is that get_mbus_config should be used to retrieve from the remote
subdevice configuration parameters that can change at runtime, or to
express properties that cannot be defined in bindings as they're not
strictly media bus configuration parameters (like the CSI-2 VC, as
you're doing here). To give you another example, I used
get_mbus_config to retrieve at run-time the number of CSI-2 data lanes
currently in use, as I have a transmitter that if configured to output
low resolution data needs to reduce the number of data lanes to stay
in the D-PHY defined clock frequency lower bound (I think that's the
use case that lead to resurect get_mbus_config() as a pad operation,
as in the past there was a g_mbus_config() video operation that got
deprecated along with the soc_camera framework).

The noncontinuous clock option cannot be changed at runtime afaict and
it's indeed an mbus configuration parameter for which a standard
DT-property is defined so it makes sense to have it in DTS. The clear
win is that your receiver driver can operate with sensor drivers not
instrumented with get_mbus_config() as it will get the info from DTS.

In your case you require get_mbus_config() for VC handling, something
everyone hopes is temporary as long as the in-development multiplexed
stream support has not landed, but if the operation is not implemented
you fallback to a default which is indeed the best option for VC, but
for clock-continuous you could avoid that by using the standard DTS
mechanism and retain compatibility with subdevices that do not
implement get_mbus_config.

I know in v3 you have been suggested to use get_mbus_config and I'm
sorry if it drove you off-road, but I think the best option here is:

- get the clock continuous property from DTS
- as you need get_mbus_config() for VC, inspect if the remote
  reports an option for clock continuous and use it if that's the case
- In case the subdev does not implement get_mbus_config() or does not
  report clock information use the value parsed from DTS instead of an
  arbitrary default

This should allow you to operate with sensor drivers not instrumented
with get_mbus_config() (as long as they send data on the VC you use as
a defaul fallback :)

>
> Could you comment on this patch please :
>
> https://lore.kernel.org/linux-media/YYJDcIiBXo%2FXlKCX@paasikivi.fi.intel.com/T/#m990b903ed6d56c3c34232c54c251490ad33e92c9
>

Afaict all the mbus configuration for a parallel bus can be fully
described in DTS, and for the above reasons of being compatible with
most subdev drivers I would fetch it from there.

>
> Thanks again for reviewing !
>

You're welcome!

> >>>
> >>> Speaking of remote subdevs, is there any driver available for IDI
> >>> transmitters ?
> >>
> >> I am not aware of any at the moment
> >>>
> >>>> +
> >>>> +     dev_dbg(csi2dc->dev, "%s clock\n",
> >>>> +             csi2dc->clk_gated ? "gated" : "free running");
> >>>> +
> >>>> +     return 0;
> >>>> +
> >>>> +csi2dc_get_mbus_config_defaults:
> >>>> +     csi2dc->vc = 0; /* Virtual ID 0 by default */
> >>>> +     csi2dc->clk_gated = false; /* Free running clock by default */
> >>>> +
> >>>> +     return 0;
> >>>> +}
> >>>> +
> >>>> +static int csi2dc_async_bound(struct v4l2_async_notifier *notifier,
> >>>> +                           struct v4l2_subdev *subdev,
> >>>> +                           struct v4l2_async_subdev *asd)
> >>>> +{
> >>>> +     struct csi2dc_device *csi2dc = container_of(notifier,
> >>>> +                                             struct csi2dc_device, notifier);
> >>>> +     int pad;
> >>>> +     int ret;
> >>>> +
> >>>> +     csi2dc->input_sd = subdev;
> >>>> +
> >>>> +     pad = media_entity_get_fwnode_pad(&subdev->entity,
> >>>
> >>> You can use 'ret'
> >>
> >> I can, but it is weird to do remote_pad = ret ...
> >>
> >> May bring confusion, don't you agree ?
> >>
> >
> > Up to you, it's a small detail :)
> >
> > Thanks!
> >     j
> >
> >>>
> >>>> +                                       asd->match.fwnode,
> >>>> +                                       MEDIA_PAD_FL_SOURCE);
> >>>> +     if (pad < 0) {
> >>>> +             dev_err(csi2dc->dev, "Failed to find pad for %s\n",
> >>>> +                     subdev->name);
> >>>> +             return pad;
> >>>> +     }
> >>>> +
> >>>> +     csi2dc->remote_pad = pad;
> >>>> +
> >>>> +     csi2dc_get_mbus_config(csi2dc);
> >>>
> >>> Ideally, as this is not fatal, you could move this at s_stream time to
> >>> fetch the most up-to-date configuration
> >>>
> >>>> +
> >>>> +     ret = media_create_pad_link(&csi2dc->input_sd->entity,
> >>>> +                                 csi2dc->remote_pad,
> >>>> +                                 &csi2dc->csi2dc_sd.entity, 0,
> >>>> +                                 MEDIA_LNK_FL_ENABLED);
> >>>> +     if (ret < 0) {
> >>>
> >>>           if (ret)
> >>>
> >>>> +             dev_err(csi2dc->dev,
> >>>> +                     "Failed to create pad link: %s to %s\n",
> >>>> +                     csi2dc->input_sd->entity.name,
> >>>> +                     csi2dc->csi2dc_sd.entity.name);
> >>>> +             return ret;
> >>>> +     }
> >>>> +
> >>>> +     dev_dbg(csi2dc->dev, "link with %s pad: %d\n",
> >>>> +             csi2dc->input_sd->name, csi2dc->remote_pad);
> >>>> +
> >>>> +     ret = pm_runtime_resume_and_get(csi2dc->dev);
> >>>> +     if (ret < 0)
> >>>> +             return ret;
> >>>> +
> >>>> +     csi2dc_writel(csi2dc, CSI2DC_GCFG,
> >>>> +                   (SAMA7G5_HLC & CSI2DC_GCFG_HLC_MASK) |
> >>>> +                   (csi2dc->clk_gated ? 0 : CSI2DC_GCFG_MIPIFRN));
> >>>> +
> >>>> +     csi2dc_writel(csi2dc, CSI2DC_VPCOL,
> >>>> +                   CSI2DC_VPCOL_COL(0xFFF) & CSI2DC_VPCOL_COL_MASK);
> >>>> +     csi2dc_writel(csi2dc, CSI2DC_VPROW,
> >>>> +                   CSI2DC_VPROW_ROW(0xFFF) & CSI2DC_VPROW_ROW_MASK);
> >>>> +
> >>>> +     pm_runtime_put_sync(csi2dc->dev);
> >>>
> >>> I would really move access to the HW to s_stream time if possible
> >>>
> >>>> +
> >>>> +     return ret;
> >>>> +}
> >>>> +
> >>>> +static const struct v4l2_async_notifier_operations csi2dc_async_ops = {
> >>>> +     .bound = csi2dc_async_bound,
> >>>> +};
> >>>> +
> >>>> +static void csi2dc_cleanup_notifier(struct csi2dc_device *csi2dc)
> >>>> +{
> >>>> +     v4l2_async_notifier_unregister(&csi2dc->notifier);
> >>>> +     v4l2_async_notifier_cleanup(&csi2dc->notifier);
> >>>> +}
> >>>> +
> >>>> +static int csi2dc_prepare_notifier(struct csi2dc_device *csi2dc,
> >>>> +                                struct device_node *input_node)
> >>>> +{
> >>>> +     int ret = 0;
> >>>> +
> >>>> +     v4l2_async_notifier_init(&csi2dc->notifier);
> >>>> +
> >>>> +     csi2dc->asd = v4l2_async_notifier_add_fwnode_remote_subdev
> >>>
> >>> do you need asd in the driver structure ?
> >>>
> >>>> +                     (&csi2dc->notifier, of_fwnode_handle(input_node),
> >>>> +                     struct v4l2_async_subdev);
> >>>> +
> >>>> +     of_node_put(input_node);
> >>>> +
> >>>> +     if (IS_ERR(csi2dc->asd)) {
> >>>> +             ret = PTR_ERR(csi2dc->asd);
> >>>> +             dev_err(csi2dc->dev,
> >>>> +                     "failed to add async notifier for node %pOF: %d\n",
> >>>> +                     input_node, ret);
> >>>> +             v4l2_async_notifier_cleanup(&csi2dc->notifier);
> >>>> +             return ret;
> >>>> +     }
> >>>> +
> >>>> +     csi2dc->notifier.ops = &csi2dc_async_ops;
> >>>> +
> >>>> +     ret = v4l2_async_subdev_notifier_register(&csi2dc->csi2dc_sd,
> >>>> +                                               &csi2dc->notifier);
> >>>> +
> >>>> +     if (ret) {
> >>>> +             dev_err(csi2dc->dev, "fail to register async notifier: %d\n",
> >>>> +                     ret);
> >>>> +             v4l2_async_notifier_cleanup(&csi2dc->notifier);
> >>>> +     }
> >>>> +
> >>>> +     return ret;
> >>>> +}
> >>>> +
> >>>> +static int csi2dc_of_parse(struct csi2dc_device *csi2dc,
> >>>> +                        struct device_node *of_node)
> >>>> +{
> >>>> +     struct device_node *input_node, *output_node;
> >>>> +     struct v4l2_fwnode_endpoint input_endpoint = { 0 },
> >>>> +                                 output_endpoint = { 0 };
> >>>> +     int ret;
> >>>> +
> >>>> +     output_endpoint.bus_type = V4L2_MBUS_PARALLEL;
> >>>> +
> >>>> +     input_node = of_graph_get_next_endpoint(of_node, NULL);
> >>>> +
> >>>> +     if (!input_node) {
> >>>> +             dev_err(csi2dc->dev,
> >>>> +                     "missing port node at %pOF, input node is mandatory.\n",
> >>>> +                     of_node);
> >>>> +             return -EINVAL;
> >>>> +     }
> >>>> +
> >>>> +     ret = v4l2_fwnode_endpoint_parse(of_fwnode_handle(input_node),
> >>>> +                                      &input_endpoint);
> >>>> +
> >>>> +     if (ret) {
> >>>                   of_node_put(input_node);
> >>>
> >>>> +             dev_err(csi2dc->dev, "endpoint not defined at %pOF\n", of_node);
> >>>> +             return ret;
> >>>> +     }
> >>>> +
> >>>> +     output_node = of_graph_get_next_endpoint(of_node, input_node);
> >>>> +
> >>>> +     if (output_node)
> >>>> +             ret = v4l2_fwnode_endpoint_parse(of_fwnode_handle(output_node),
> >>>> +                                              &output_endpoint);
> >>>
> >>>           of_node_put(output_node);
> >>>> +
> >>>> +     if (!output_node || ret) {
> >>>> +             dev_info(csi2dc->dev,
> >>>> +                      "missing output node at %pOF, data pipe available only.\n",
> >>>> +                      of_node);
> >>>> +     } else {
> >>>> +             csi2dc->video_pipe = true;
> >>>> +
> >>>> +             dev_dbg(csi2dc->dev, "block %pOF %d.%d->%d.%d video pipeline\n",
> >>>> +                     of_node, input_endpoint.base.port,
> >>>> +                     input_endpoint.base.id, output_endpoint.base.port,
> >>>> +                     output_endpoint.base.id);
> >>>> +     }
> >>>> +
> >>>> +     of_node_put(output_node);
> >>>
> >>> Drop this if you put it above
> >>>
> >>>> +     of_node_put(input_node);
> >>>
> >>> Should you put input_node before passing it to the function ?
> >>>> +
> >>>> +     /* prepare async notifier for subdevice completion */
> >>>> +     return csi2dc_prepare_notifier(csi2dc, input_node);
> >>>> +}
> >>>> +
> >>>> +static int csi2dc_probe(struct platform_device *pdev)
> >>>> +{
> >>>> +     struct device *dev = &pdev->dev;
> >>>> +     struct csi2dc_device *csi2dc;
> >>>> +     struct resource *res = NULL;
> >>>> +     int ret = 0;
> >>>> +     u32 ver;
> >>>> +
> >>>> +     csi2dc = devm_kzalloc(dev, sizeof(*csi2dc), GFP_KERNEL);
> >>>> +     if (!csi2dc)
> >>>> +             return -ENOMEM;
> >>>> +
> >>>> +     csi2dc->dev = dev;
> >>>> +
> >>>> +     res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
> >>>> +
> >>>> +     csi2dc->base = devm_ioremap_resource(dev, res);
> >>>
> >>> Should devm_platform_ioremap_resource(pdev, 0) be used here ?
> >>>
> >>>> +     if (IS_ERR(csi2dc->base)) {
> >>>> +             dev_err(dev, "base address not set\n");
> >>>> +             return PTR_ERR(csi2dc->base);
> >>>> +     }
> >>>> +
> >>>> +     csi2dc->pclk = devm_clk_get(dev, "pclk");
> >>>> +     if (IS_ERR(csi2dc->pclk)) {
> >>>> +             ret = PTR_ERR(csi2dc->pclk);
> >>>> +             dev_err(dev, "failed to get pclk: %d\n", ret);
> >>>> +             return ret;
> >>>> +     }
> >>>> +
> >>>> +     csi2dc->scck = devm_clk_get(dev, "scck");
> >>>> +     if (IS_ERR(csi2dc->scck)) {
> >>>> +             ret = PTR_ERR(csi2dc->scck);
> >>>> +             dev_err(dev, "failed to get scck: %d\n", ret);
> >>>> +             return ret;
> >>>> +     }
> >>>> +
> >>>> +     v4l2_subdev_init(&csi2dc->csi2dc_sd, &csi2dc_subdev_ops);
> >>>> +
> >>>> +     csi2dc->csi2dc_sd.owner = THIS_MODULE;
> >>>> +     csi2dc->csi2dc_sd.dev = dev;
> >>>> +     snprintf(csi2dc->csi2dc_sd.name, sizeof(csi2dc->csi2dc_sd.name),
> >>>> +              "csi2dc");
> >>>> +
> >>>> +     csi2dc->csi2dc_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> >>>> +     csi2dc->csi2dc_sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> >>>> +     v4l2_set_subdevdata(&csi2dc->csi2dc_sd, pdev);
> >>>
> >>> Not used it seems
> >>>
> >>>> +
> >>>> +     platform_set_drvdata(pdev, csi2dc);
> >>>> +
> >>>> +     ret = csi2dc_of_parse(csi2dc, dev->of_node);
> >>>> +     if (ret)
> >>>> +             goto csi2dc_probe_cleanup_entity;
> >>>> +
> >>>> +     csi2dc->pads[CSI2DC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> >>>> +     if (csi2dc->video_pipe)
> >>>> +             csi2dc->pads[CSI2DC_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
> >>>> +
> >>>> +     ret = media_entity_pads_init(&csi2dc->csi2dc_sd.entity,
> >>>> +                                  csi2dc->video_pipe ? CSI2DC_PADS_NUM : 1,
> >>>> +                                  csi2dc->pads);
> >>>> +     if (ret < 0) {
> >>>> +             dev_err(dev, "media entity init failed\n");
> >>>> +             goto csi2dc_probe_cleanup_entity;
> >>>
> >>> Should you also clean up the notifier in the error path ?
> >>>
> >>>> +     }
> >>>> +
> >>>> +     /* turn power on to validate capabilities */
> >>>> +     ret = csi2dc_power(csi2dc, true);
> >>>> +     if (ret < 0)
> >>>> +             goto csi2dc_probe_cleanup_entity;
> >>>> +
> >>>> +     pm_runtime_set_active(dev);
> >>>> +     pm_runtime_enable(dev);
> >>>> +     ver = csi2dc_readl(csi2dc, CSI2DC_VERSION);
> >>>> +     pm_request_idle(dev);
> >>>> +
> >>>> +     /*
> >>>> +      * we must register the subdev after PM runtime has been requested,
> >>>> +      * otherwise we might bound immediately and request pm_runtime_resume
> >>>> +      * before runtime_enable.
> >>>> +      */
> >>>> +     ret = v4l2_async_register_subdev(&csi2dc->csi2dc_sd);
> >>>> +     if (ret) {
> >>>> +             dev_err(csi2dc->dev, "failed to register the subdevice\n");
> >>>> +             goto csi2dc_probe_cleanup_entity;
> >>>> +     }
> >>>> +
> >>>> +     dev_info(dev, "Microchip CSI2DC version %x\n", ver);
> >>>> +
> >>>> +     return 0;
> >>>> +
> >>>> +csi2dc_probe_cleanup_entity:
> >>>> +     media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
> >>>> +
> >>>> +     return ret;
> >>>> +}
> >>>> +
> >>>> +static int csi2dc_remove(struct platform_device *pdev)
> >>>> +{
> >>>> +     struct csi2dc_device *csi2dc = platform_get_drvdata(pdev);
> >>>> +
> >>>> +     pm_runtime_disable(&pdev->dev);
> >>>> +
> >>>> +     v4l2_async_unregister_subdev(&csi2dc->csi2dc_sd);
> >>>> +     csi2dc_cleanup_notifier(csi2dc);
> >>>> +     media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
> >>>> +
> >>>> +     return 0;
> >>>> +}
> >>>> +
> >>>> +static int __maybe_unused csi2dc_runtime_suspend(struct device *dev)
> >>>> +{
> >>>> +     struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
> >>>> +
> >>>> +     return csi2dc_power(csi2dc, false);
> >>>> +}
> >>>> +
> >>>> +static int __maybe_unused csi2dc_runtime_resume(struct device *dev)
> >>>> +{
> >>>> +     struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
> >>>> +
> >>>> +     return csi2dc_power(csi2dc, true);
> >>>> +}
> >>>> +
> >>>> +static const struct dev_pm_ops csi2dc_dev_pm_ops = {
> >>>> +     SET_RUNTIME_PM_OPS(csi2dc_runtime_suspend, csi2dc_runtime_resume, NULL)
> >>>> +};
> >>>> +
> >>>> +static const struct of_device_id csi2dc_of_match[] = {
> >>>> +     { .compatible = "microchip,sama7g5-csi2dc" },
> >>>> +     { }
> >>>> +};
> >>>> +
> >>>> +MODULE_DEVICE_TABLE(of, csi2dc_of_match);
> >>>> +
> >>>> +static struct platform_driver csi2dc_driver = {
> >>>> +     .probe  = csi2dc_probe,
> >>>> +     .remove = csi2dc_remove,
> >>>> +     .driver = {
> >>>> +             .name =                 "microchip-csi2dc",
> >>>> +             .pm =                   &csi2dc_dev_pm_ops,
> >>>> +             .of_match_table =       of_match_ptr(csi2dc_of_match),
> >>>> +     },
> >>>> +};
> >>>> +
> >>>> +module_platform_driver(csi2dc_driver);
> >>>> +
> >>>> +MODULE_AUTHOR("Eugen Hristev <eugen.hristev@microchip.com>");
> >>>> +MODULE_DESCRIPTION("Microchip CSI2 Demux Controller driver");
> >>>> +MODULE_LICENSE("GPL v2");
> >>>> --
> >>>> 2.25.1
> >>>>
> >>
>

Re: [PATCH 05/21] media: atmel: atmel-isc: split the clock code into separate source file

[Jacopo Mondi]

Hi Eugen

On Fri, Oct 22, 2021 at 10:52:31AM +0300, Eugen Hristev wrote:
> The atmel-isc-base is getting crowded. Split the clock functions into
> atmel-isc-clk.c.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> ---
>  drivers/media/platform/atmel/Makefile         |   2 +-
>  drivers/media/platform/atmel/atmel-isc-base.c | 294 ----------------
>  drivers/media/platform/atmel/atmel-isc-clk.c  | 316 ++++++++++++++++++
>  3 files changed, 317 insertions(+), 295 deletions(-)
>  create mode 100644 drivers/media/platform/atmel/atmel-isc-clk.c
>
> diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
> index 39f0a7eba702..1f6fe7427769 100644
> --- a/drivers/media/platform/atmel/Makefile
> +++ b/drivers/media/platform/atmel/Makefile
> @@ -3,7 +3,7 @@ atmel-isc-objs = atmel-sama5d2-isc.o
>  atmel-xisc-objs = atmel-sama7g5-isc.o
>
>  obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
> -obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o
> +obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o atmel-isc-clk.o
>  obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel-isc.o
>  obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel-xisc.o
>  obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += microchip-csi2dc.o
> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> index ebf264b980f9..f532fd03e807 100644
> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> @@ -8,9 +8,6 @@
>   * Author: Eugen Hristev <eugen.hristev@microchip.com>
>   *
>   */
> -
> -#include <linux/clk.h>
> -#include <linux/clkdev.h>
>  #include <linux/clk-provider.h>

Do you still need clk-provider here ?

>  #include <linux/delay.h>
>  #include <linux/interrupt.h>
> @@ -100,297 +97,6 @@ static inline void isc_reset_awb_ctrls(struct isc_device *isc)
>  	}
>  }
>
> -static int isc_wait_clk_stable(struct clk_hw *hw)
> -{
> -	struct isc_clk *isc_clk = to_isc_clk(hw);
> -	struct regmap *regmap = isc_clk->regmap;
> -	unsigned long timeout = jiffies + usecs_to_jiffies(1000);
> -	unsigned int status;
> -
> -	while (time_before(jiffies, timeout)) {
> -		regmap_read(regmap, ISC_CLKSR, &status);
> -		if (!(status & ISC_CLKSR_SIP))
> -			return 0;
> -
> -		usleep_range(10, 250);
> -	}
> -
> -	return -ETIMEDOUT;
> -}
> -
> -static int isc_clk_prepare(struct clk_hw *hw)
> -{
> -	struct isc_clk *isc_clk = to_isc_clk(hw);
> -	int ret;
> -
> -	ret = pm_runtime_resume_and_get(isc_clk->dev);
> -	if (ret < 0)
> -		return ret;
> -
> -	return isc_wait_clk_stable(hw);
> -}
> -
> -static void isc_clk_unprepare(struct clk_hw *hw)
> -{
> -	struct isc_clk *isc_clk = to_isc_clk(hw);
> -
> -	isc_wait_clk_stable(hw);
> -
> -	pm_runtime_put_sync(isc_clk->dev);
> -}
> -
> -static int isc_clk_enable(struct clk_hw *hw)
> -{
> -	struct isc_clk *isc_clk = to_isc_clk(hw);
> -	u32 id = isc_clk->id;
> -	struct regmap *regmap = isc_clk->regmap;
> -	unsigned long flags;
> -	unsigned int status;
> -
> -	dev_dbg(isc_clk->dev, "ISC CLK: %s, id = %d, div = %d, parent id = %d\n",
> -		__func__, id, isc_clk->div, isc_clk->parent_id);
> -
> -	spin_lock_irqsave(&isc_clk->lock, flags);
> -	regmap_update_bits(regmap, ISC_CLKCFG,
> -			   ISC_CLKCFG_DIV_MASK(id) | ISC_CLKCFG_SEL_MASK(id),
> -			   (isc_clk->div << ISC_CLKCFG_DIV_SHIFT(id)) |
> -			   (isc_clk->parent_id << ISC_CLKCFG_SEL_SHIFT(id)));
> -
> -	regmap_write(regmap, ISC_CLKEN, ISC_CLK(id));
> -	spin_unlock_irqrestore(&isc_clk->lock, flags);
> -
> -	regmap_read(regmap, ISC_CLKSR, &status);
> -	if (status & ISC_CLK(id))
> -		return 0;
> -	else
> -		return -EINVAL;
> -}
> -
> -static void isc_clk_disable(struct clk_hw *hw)
> -{
> -	struct isc_clk *isc_clk = to_isc_clk(hw);
> -	u32 id = isc_clk->id;
> -	unsigned long flags;
> -
> -	spin_lock_irqsave(&isc_clk->lock, flags);
> -	regmap_write(isc_clk->regmap, ISC_CLKDIS, ISC_CLK(id));
> -	spin_unlock_irqrestore(&isc_clk->lock, flags);
> -}
> -
> -static int isc_clk_is_enabled(struct clk_hw *hw)
> -{
> -	struct isc_clk *isc_clk = to_isc_clk(hw);
> -	u32 status;
> -	int ret;
> -
> -	ret = pm_runtime_resume_and_get(isc_clk->dev);
> -	if (ret < 0)
> -		return 0;
> -
> -	regmap_read(isc_clk->regmap, ISC_CLKSR, &status);
> -
> -	pm_runtime_put_sync(isc_clk->dev);
> -
> -	return status & ISC_CLK(isc_clk->id) ? 1 : 0;
> -}
> -
> -static unsigned long
> -isc_clk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
> -{
> -	struct isc_clk *isc_clk = to_isc_clk(hw);
> -
> -	return DIV_ROUND_CLOSEST(parent_rate, isc_clk->div + 1);
> -}
> -
> -static int isc_clk_determine_rate(struct clk_hw *hw,
> -				   struct clk_rate_request *req)
> -{
> -	struct isc_clk *isc_clk = to_isc_clk(hw);
> -	long best_rate = -EINVAL;
> -	int best_diff = -1;
> -	unsigned int i, div;
> -
> -	for (i = 0; i < clk_hw_get_num_parents(hw); i++) {
> -		struct clk_hw *parent;
> -		unsigned long parent_rate;
> -
> -		parent = clk_hw_get_parent_by_index(hw, i);
> -		if (!parent)
> -			continue;
> -
> -		parent_rate = clk_hw_get_rate(parent);
> -		if (!parent_rate)
> -			continue;
> -
> -		for (div = 1; div < ISC_CLK_MAX_DIV + 2; div++) {
> -			unsigned long rate;
> -			int diff;
> -
> -			rate = DIV_ROUND_CLOSEST(parent_rate, div);
> -			diff = abs(req->rate - rate);
> -
> -			if (best_diff < 0 || best_diff > diff) {
> -				best_rate = rate;
> -				best_diff = diff;
> -				req->best_parent_rate = parent_rate;
> -				req->best_parent_hw = parent;
> -			}
> -
> -			if (!best_diff || rate < req->rate)
> -				break;
> -		}
> -
> -		if (!best_diff)
> -			break;
> -	}
> -
> -	dev_dbg(isc_clk->dev,
> -		"ISC CLK: %s, best_rate = %ld, parent clk: %s @ %ld\n",
> -		__func__, best_rate,
> -		__clk_get_name((req->best_parent_hw)->clk),
> -		req->best_parent_rate);
> -
> -	if (best_rate < 0)
> -		return best_rate;
> -
> -	req->rate = best_rate;
> -
> -	return 0;
> -}
> -
> -static int isc_clk_set_parent(struct clk_hw *hw, u8 index)
> -{
> -	struct isc_clk *isc_clk = to_isc_clk(hw);
> -
> -	if (index >= clk_hw_get_num_parents(hw))
> -		return -EINVAL;
> -
> -	isc_clk->parent_id = index;
> -
> -	return 0;
> -}
> -
> -static u8 isc_clk_get_parent(struct clk_hw *hw)
> -{
> -	struct isc_clk *isc_clk = to_isc_clk(hw);
> -
> -	return isc_clk->parent_id;
> -}
> -
> -static int isc_clk_set_rate(struct clk_hw *hw,
> -			     unsigned long rate,
> -			     unsigned long parent_rate)
> -{
> -	struct isc_clk *isc_clk = to_isc_clk(hw);
> -	u32 div;
> -
> -	if (!rate)
> -		return -EINVAL;
> -
> -	div = DIV_ROUND_CLOSEST(parent_rate, rate);
> -	if (div > (ISC_CLK_MAX_DIV + 1) || !div)
> -		return -EINVAL;
> -
> -	isc_clk->div = div - 1;
> -
> -	return 0;
> -}
> -
> -static const struct clk_ops isc_clk_ops = {
> -	.prepare	= isc_clk_prepare,
> -	.unprepare	= isc_clk_unprepare,
> -	.enable		= isc_clk_enable,
> -	.disable	= isc_clk_disable,
> -	.is_enabled	= isc_clk_is_enabled,
> -	.recalc_rate	= isc_clk_recalc_rate,
> -	.determine_rate	= isc_clk_determine_rate,
> -	.set_parent	= isc_clk_set_parent,
> -	.get_parent	= isc_clk_get_parent,
> -	.set_rate	= isc_clk_set_rate,
> -};
> -
> -static int isc_clk_register(struct isc_device *isc, unsigned int id)
> -{
> -	struct regmap *regmap = isc->regmap;
> -	struct device_node *np = isc->dev->of_node;
> -	struct isc_clk *isc_clk;
> -	struct clk_init_data init;
> -	const char *clk_name = np->name;
> -	const char *parent_names[3];
> -	int num_parents;
> -
> -	if (id == ISC_ISPCK && !isc->ispck_required)
> -		return 0;
> -
> -	num_parents = of_clk_get_parent_count(np);
> -	if (num_parents < 1 || num_parents > 3)
> -		return -EINVAL;
> -
> -	if (num_parents > 2 && id == ISC_ISPCK)
> -		num_parents = 2;
> -
> -	of_clk_parent_fill(np, parent_names, num_parents);
> -
> -	if (id == ISC_MCK)
> -		of_property_read_string(np, "clock-output-names", &clk_name);
> -	else
> -		clk_name = "isc-ispck";
> -
> -	init.parent_names	= parent_names;
> -	init.num_parents	= num_parents;
> -	init.name		= clk_name;
> -	init.ops		= &isc_clk_ops;
> -	init.flags		= CLK_SET_RATE_GATE | CLK_SET_PARENT_GATE;
> -
> -	isc_clk = &isc->isc_clks[id];
> -	isc_clk->hw.init	= &init;
> -	isc_clk->regmap		= regmap;
> -	isc_clk->id		= id;
> -	isc_clk->dev		= isc->dev;
> -	spin_lock_init(&isc_clk->lock);
> -
> -	isc_clk->clk = clk_register(isc->dev, &isc_clk->hw);
> -	if (IS_ERR(isc_clk->clk)) {
> -		dev_err(isc->dev, "%s: clock register fail\n", clk_name);
> -		return PTR_ERR(isc_clk->clk);
> -	} else if (id == ISC_MCK)
> -		of_clk_add_provider(np, of_clk_src_simple_get, isc_clk->clk);
> -
> -	return 0;
> -}
> -
> -int isc_clk_init(struct isc_device *isc)
> -{
> -	unsigned int i;
> -	int ret;
> -
> -	for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++)
> -		isc->isc_clks[i].clk = ERR_PTR(-EINVAL);
> -
> -	for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
> -		ret = isc_clk_register(isc, i);
> -		if (ret)
> -			return ret;
> -	}
> -
> -	return 0;
> -}
> -EXPORT_SYMBOL_GPL(isc_clk_init);
> -
> -void isc_clk_cleanup(struct isc_device *isc)
> -{
> -	unsigned int i;
> -
> -	of_clk_del_provider(isc->dev->of_node);
> -
> -	for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
> -		struct isc_clk *isc_clk = &isc->isc_clks[i];
> -
> -		if (!IS_ERR(isc_clk->clk))
> -			clk_unregister(isc_clk->clk);
> -	}
> -}
> -EXPORT_SYMBOL_GPL(isc_clk_cleanup);
>
>  static int isc_queue_setup(struct vb2_queue *vq,
>  			    unsigned int *nbuffers, unsigned int *nplanes,
> diff --git a/drivers/media/platform/atmel/atmel-isc-clk.c b/drivers/media/platform/atmel/atmel-isc-clk.c
> new file mode 100644
> index 000000000000..d650caade396
> --- /dev/null
> +++ b/drivers/media/platform/atmel/atmel-isc-clk.c
> @@ -0,0 +1,316 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Microchip Image Sensor Controller (ISC) common clock driver setup
> + *
> + * Copyright (C) 2016-2019 Microchip Technology, Inc.

Time flies!

> + *
> + * Author: Songjun Wu
> + * Author: Eugen Hristev <eugen.hristev@microchip.com>
> + *
> + */
> +#include <linux/clk.h>
> +#include <linux/clkdev.h>
> +#include <linux/clk-provider.h>
> +#include <linux/platform_device.h>

Is this needed ?

> +#include <linux/pm_runtime.h>
> +#include <linux/regmap.h>
> +
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-device.h>
> +#include <media/videobuf2-dma-contig.h>

Is any one of these needed ?

Removing them highlights how these includes should probably be
moved to atmel-isc.h which fails to compile if not preceeded by these
inclusions.

I think you can merge this with the previous patch that adds the file
entry to MAINTAINERS

Thanks
   j

> +
> +#include "atmel-isc-regs.h"
> +#include "atmel-isc.h"
> +
> +static int isc_wait_clk_stable(struct clk_hw *hw)
> +{
> +	struct isc_clk *isc_clk = to_isc_clk(hw);
> +	struct regmap *regmap = isc_clk->regmap;
> +	unsigned long timeout = jiffies + usecs_to_jiffies(1000);
> +	unsigned int status;
> +
> +	while (time_before(jiffies, timeout)) {
> +		regmap_read(regmap, ISC_CLKSR, &status);
> +		if (!(status & ISC_CLKSR_SIP))
> +			return 0;
> +
> +		usleep_range(10, 250);
> +	}
> +
> +	return -ETIMEDOUT;
> +}
> +
> +static int isc_clk_prepare(struct clk_hw *hw)
> +{
> +	struct isc_clk *isc_clk = to_isc_clk(hw);
> +	int ret;
> +
> +	ret = pm_runtime_resume_and_get(isc_clk->dev);
> +	if (ret < 0)
> +		return ret;
> +
> +	return isc_wait_clk_stable(hw);
> +}
> +
> +static void isc_clk_unprepare(struct clk_hw *hw)
> +{
> +	struct isc_clk *isc_clk = to_isc_clk(hw);
> +
> +	isc_wait_clk_stable(hw);
> +
> +	pm_runtime_put_sync(isc_clk->dev);
> +}
> +
> +static int isc_clk_enable(struct clk_hw *hw)
> +{
> +	struct isc_clk *isc_clk = to_isc_clk(hw);
> +	u32 id = isc_clk->id;
> +	struct regmap *regmap = isc_clk->regmap;
> +	unsigned long flags;
> +	unsigned int status;
> +
> +	dev_dbg(isc_clk->dev, "ISC CLK: %s, id = %d, div = %d, parent id = %d\n",
> +		__func__, id, isc_clk->div, isc_clk->parent_id);
> +
> +	spin_lock_irqsave(&isc_clk->lock, flags);
> +	regmap_update_bits(regmap, ISC_CLKCFG,
> +			   ISC_CLKCFG_DIV_MASK(id) | ISC_CLKCFG_SEL_MASK(id),
> +			   (isc_clk->div << ISC_CLKCFG_DIV_SHIFT(id)) |
> +			   (isc_clk->parent_id << ISC_CLKCFG_SEL_SHIFT(id)));
> +
> +	regmap_write(regmap, ISC_CLKEN, ISC_CLK(id));
> +	spin_unlock_irqrestore(&isc_clk->lock, flags);
> +
> +	regmap_read(regmap, ISC_CLKSR, &status);
> +	if (status & ISC_CLK(id))
> +		return 0;
> +	else
> +		return -EINVAL;
> +}
> +
> +static void isc_clk_disable(struct clk_hw *hw)
> +{
> +	struct isc_clk *isc_clk = to_isc_clk(hw);
> +	u32 id = isc_clk->id;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&isc_clk->lock, flags);
> +	regmap_write(isc_clk->regmap, ISC_CLKDIS, ISC_CLK(id));
> +	spin_unlock_irqrestore(&isc_clk->lock, flags);
> +}
> +
> +static int isc_clk_is_enabled(struct clk_hw *hw)
> +{
> +	struct isc_clk *isc_clk = to_isc_clk(hw);
> +	u32 status;
> +	int ret;
> +
> +	ret = pm_runtime_resume_and_get(isc_clk->dev);
> +	if (ret < 0)
> +		return 0;
> +
> +	regmap_read(isc_clk->regmap, ISC_CLKSR, &status);
> +
> +	pm_runtime_put_sync(isc_clk->dev);
> +
> +	return status & ISC_CLK(isc_clk->id) ? 1 : 0;
> +}
> +
> +static unsigned long
> +isc_clk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
> +{
> +	struct isc_clk *isc_clk = to_isc_clk(hw);
> +
> +	return DIV_ROUND_CLOSEST(parent_rate, isc_clk->div + 1);
> +}
> +
> +static int isc_clk_determine_rate(struct clk_hw *hw,
> +				  struct clk_rate_request *req)
> +{
> +	struct isc_clk *isc_clk = to_isc_clk(hw);
> +	long best_rate = -EINVAL;
> +	int best_diff = -1;
> +	unsigned int i, div;
> +
> +	for (i = 0; i < clk_hw_get_num_parents(hw); i++) {
> +		struct clk_hw *parent;
> +		unsigned long parent_rate;
> +
> +		parent = clk_hw_get_parent_by_index(hw, i);
> +		if (!parent)
> +			continue;
> +
> +		parent_rate = clk_hw_get_rate(parent);
> +		if (!parent_rate)
> +			continue;
> +
> +		for (div = 1; div < ISC_CLK_MAX_DIV + 2; div++) {
> +			unsigned long rate;
> +			int diff;
> +
> +			rate = DIV_ROUND_CLOSEST(parent_rate, div);
> +			diff = abs(req->rate - rate);
> +
> +			if (best_diff < 0 || best_diff > diff) {
> +				best_rate = rate;
> +				best_diff = diff;
> +				req->best_parent_rate = parent_rate;
> +				req->best_parent_hw = parent;
> +			}
> +
> +			if (!best_diff || rate < req->rate)
> +				break;
> +		}
> +
> +		if (!best_diff)
> +			break;
> +	}
> +
> +	dev_dbg(isc_clk->dev,
> +		"ISC CLK: %s, best_rate = %ld, parent clk: %s @ %ld\n",
> +		__func__, best_rate,
> +		__clk_get_name((req->best_parent_hw)->clk),
> +		req->best_parent_rate);
> +
> +	if (best_rate < 0)
> +		return best_rate;
> +
> +	req->rate = best_rate;
> +
> +	return 0;
> +}
> +
> +static int isc_clk_set_parent(struct clk_hw *hw, u8 index)
> +{
> +	struct isc_clk *isc_clk = to_isc_clk(hw);
> +
> +	if (index >= clk_hw_get_num_parents(hw))
> +		return -EINVAL;
> +
> +	isc_clk->parent_id = index;
> +
> +	return 0;
> +}
> +
> +static u8 isc_clk_get_parent(struct clk_hw *hw)
> +{
> +	struct isc_clk *isc_clk = to_isc_clk(hw);
> +
> +	return isc_clk->parent_id;
> +}
> +
> +static int isc_clk_set_rate(struct clk_hw *hw,
> +			    unsigned long rate,
> +			    unsigned long parent_rate)
> +{
> +	struct isc_clk *isc_clk = to_isc_clk(hw);
> +	u32 div;
> +
> +	if (!rate)
> +		return -EINVAL;
> +
> +	div = DIV_ROUND_CLOSEST(parent_rate, rate);
> +	if (div > (ISC_CLK_MAX_DIV + 1) || !div)
> +		return -EINVAL;
> +
> +	isc_clk->div = div - 1;
> +
> +	return 0;
> +}
> +
> +static const struct clk_ops isc_clk_ops = {
> +	.prepare	= isc_clk_prepare,
> +	.unprepare	= isc_clk_unprepare,
> +	.enable		= isc_clk_enable,
> +	.disable	= isc_clk_disable,
> +	.is_enabled	= isc_clk_is_enabled,
> +	.recalc_rate	= isc_clk_recalc_rate,
> +	.determine_rate	= isc_clk_determine_rate,
> +	.set_parent	= isc_clk_set_parent,
> +	.get_parent	= isc_clk_get_parent,
> +	.set_rate	= isc_clk_set_rate,
> +};
> +
> +static int isc_clk_register(struct isc_device *isc, unsigned int id)
> +{
> +	struct regmap *regmap = isc->regmap;
> +	struct device_node *np = isc->dev->of_node;
> +	struct isc_clk *isc_clk;
> +	struct clk_init_data init;
> +	const char *clk_name = np->name;
> +	const char *parent_names[3];
> +	int num_parents;
> +
> +	if (id == ISC_ISPCK && !isc->ispck_required)
> +		return 0;
> +
> +	num_parents = of_clk_get_parent_count(np);
> +	if (num_parents < 1 || num_parents > 3)
> +		return -EINVAL;
> +
> +	if (num_parents > 2 && id == ISC_ISPCK)
> +		num_parents = 2;
> +
> +	of_clk_parent_fill(np, parent_names, num_parents);
> +
> +	if (id == ISC_MCK)
> +		of_property_read_string(np, "clock-output-names", &clk_name);
> +	else
> +		clk_name = "isc-ispck";
> +
> +	init.parent_names	= parent_names;
> +	init.num_parents	= num_parents;
> +	init.name		= clk_name;
> +	init.ops		= &isc_clk_ops;
> +	init.flags		= CLK_SET_RATE_GATE | CLK_SET_PARENT_GATE;
> +
> +	isc_clk = &isc->isc_clks[id];
> +	isc_clk->hw.init	= &init;
> +	isc_clk->regmap		= regmap;
> +	isc_clk->id		= id;
> +	isc_clk->dev		= isc->dev;
> +	spin_lock_init(&isc_clk->lock);
> +
> +	isc_clk->clk = clk_register(isc->dev, &isc_clk->hw);
> +	if (IS_ERR(isc_clk->clk)) {
> +		dev_err(isc->dev, "%s: clock register fail\n", clk_name);
> +		return PTR_ERR(isc_clk->clk);
> +	} else if (id == ISC_MCK) {
> +		of_clk_add_provider(np, of_clk_src_simple_get, isc_clk->clk);
> +	}
> +
> +	return 0;
> +}
> +
> +int isc_clk_init(struct isc_device *isc)
> +{
> +	unsigned int i;
> +	int ret;
> +
> +	for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++)
> +		isc->isc_clks[i].clk = ERR_PTR(-EINVAL);
> +
> +	for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
> +		ret = isc_clk_register(isc, i);
> +		if (ret)
> +			return ret;
> +	}
> +
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(isc_clk_init);
> +
> +void isc_clk_cleanup(struct isc_device *isc)
> +{
> +	unsigned int i;
> +
> +	of_clk_del_provider(isc->dev->of_node);
> +
> +	for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
> +		struct isc_clk *isc_clk = &isc->isc_clks[i];
> +
> +		if (!IS_ERR(isc_clk->clk))
> +			clk_unregister(isc_clk->clk);
> +	}
> +}
> +EXPORT_SYMBOL_GPL(isc_clk_cleanup);
> --
> 2.25.1
>

Re: [PATCH 05/21] media: atmel: atmel-isc: split the clock code into separate source file

[Eugen.Hristev]

On 11/5/21 11:02 AM, Jacopo Mondi wrote:
> Hi Eugen
> 
> On Fri, Oct 22, 2021 at 10:52:31AM +0300, Eugen Hristev wrote:
>> The atmel-isc-base is getting crowded. Split the clock functions into
>> atmel-isc-clk.c.
>>
>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>> ---
>>   drivers/media/platform/atmel/Makefile         |   2 +-
>>   drivers/media/platform/atmel/atmel-isc-base.c | 294 ----------------
>>   drivers/media/platform/atmel/atmel-isc-clk.c  | 316 ++++++++++++++++++
>>   3 files changed, 317 insertions(+), 295 deletions(-)
>>   create mode 100644 drivers/media/platform/atmel/atmel-isc-clk.c
>>
>> diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
>> index 39f0a7eba702..1f6fe7427769 100644
>> --- a/drivers/media/platform/atmel/Makefile
>> +++ b/drivers/media/platform/atmel/Makefile
>> @@ -3,7 +3,7 @@ atmel-isc-objs = atmel-sama5d2-isc.o
>>   atmel-xisc-objs = atmel-sama7g5-isc.o
>>
>>   obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
>> -obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o
>> +obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o atmel-isc-clk.o
>>   obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel-isc.o
>>   obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel-xisc.o
>>   obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += microchip-csi2dc.o
>> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
>> index ebf264b980f9..f532fd03e807 100644
>> --- a/drivers/media/platform/atmel/atmel-isc-base.c
>> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
>> @@ -8,9 +8,6 @@
>>    * Author: Eugen Hristev <eugen.hristev@microchip.com>
>>    *
>>    */
>> -
>> -#include <linux/clk.h>
>> -#include <linux/clkdev.h>
>>   #include <linux/clk-provider.h>
> 
> Do you still need clk-provider here ?

Yes it's needed in atmel-isc.h

> 
>>   #include <linux/delay.h>
>>   #include <linux/interrupt.h>
>> @@ -100,297 +97,6 @@ static inline void isc_reset_awb_ctrls(struct isc_device *isc)
>>        }
>>   }
>>
>> -static int isc_wait_clk_stable(struct clk_hw *hw)
>> -{
>> -     struct isc_clk *isc_clk = to_isc_clk(hw);
>> -     struct regmap *regmap = isc_clk->regmap;
>> -     unsigned long timeout = jiffies + usecs_to_jiffies(1000);
>> -     unsigned int status;
>> -
>> -     while (time_before(jiffies, timeout)) {
>> -             regmap_read(regmap, ISC_CLKSR, &status);
>> -             if (!(status & ISC_CLKSR_SIP))
>> -                     return 0;
>> -
>> -             usleep_range(10, 250);
>> -     }
>> -
>> -     return -ETIMEDOUT;
>> -}
>> -
>> -static int isc_clk_prepare(struct clk_hw *hw)
>> -{
>> -     struct isc_clk *isc_clk = to_isc_clk(hw);
>> -     int ret;
>> -
>> -     ret = pm_runtime_resume_and_get(isc_clk->dev);
>> -     if (ret < 0)
>> -             return ret;
>> -
>> -     return isc_wait_clk_stable(hw);
>> -}
>> -
>> -static void isc_clk_unprepare(struct clk_hw *hw)
>> -{
>> -     struct isc_clk *isc_clk = to_isc_clk(hw);
>> -
>> -     isc_wait_clk_stable(hw);
>> -
>> -     pm_runtime_put_sync(isc_clk->dev);
>> -}
>> -
>> -static int isc_clk_enable(struct clk_hw *hw)
>> -{
>> -     struct isc_clk *isc_clk = to_isc_clk(hw);
>> -     u32 id = isc_clk->id;
>> -     struct regmap *regmap = isc_clk->regmap;
>> -     unsigned long flags;
>> -     unsigned int status;
>> -
>> -     dev_dbg(isc_clk->dev, "ISC CLK: %s, id = %d, div = %d, parent id = %d\n",
>> -             __func__, id, isc_clk->div, isc_clk->parent_id);
>> -
>> -     spin_lock_irqsave(&isc_clk->lock, flags);
>> -     regmap_update_bits(regmap, ISC_CLKCFG,
>> -                        ISC_CLKCFG_DIV_MASK(id) | ISC_CLKCFG_SEL_MASK(id),
>> -                        (isc_clk->div << ISC_CLKCFG_DIV_SHIFT(id)) |
>> -                        (isc_clk->parent_id << ISC_CLKCFG_SEL_SHIFT(id)));
>> -
>> -     regmap_write(regmap, ISC_CLKEN, ISC_CLK(id));
>> -     spin_unlock_irqrestore(&isc_clk->lock, flags);
>> -
>> -     regmap_read(regmap, ISC_CLKSR, &status);
>> -     if (status & ISC_CLK(id))
>> -             return 0;
>> -     else
>> -             return -EINVAL;
>> -}
>> -
>> -static void isc_clk_disable(struct clk_hw *hw)
>> -{
>> -     struct isc_clk *isc_clk = to_isc_clk(hw);
>> -     u32 id = isc_clk->id;
>> -     unsigned long flags;
>> -
>> -     spin_lock_irqsave(&isc_clk->lock, flags);
>> -     regmap_write(isc_clk->regmap, ISC_CLKDIS, ISC_CLK(id));
>> -     spin_unlock_irqrestore(&isc_clk->lock, flags);
>> -}
>> -
>> -static int isc_clk_is_enabled(struct clk_hw *hw)
>> -{
>> -     struct isc_clk *isc_clk = to_isc_clk(hw);
>> -     u32 status;
>> -     int ret;
>> -
>> -     ret = pm_runtime_resume_and_get(isc_clk->dev);
>> -     if (ret < 0)
>> -             return 0;
>> -
>> -     regmap_read(isc_clk->regmap, ISC_CLKSR, &status);
>> -
>> -     pm_runtime_put_sync(isc_clk->dev);
>> -
>> -     return status & ISC_CLK(isc_clk->id) ? 1 : 0;
>> -}
>> -
>> -static unsigned long
>> -isc_clk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
>> -{
>> -     struct isc_clk *isc_clk = to_isc_clk(hw);
>> -
>> -     return DIV_ROUND_CLOSEST(parent_rate, isc_clk->div + 1);
>> -}
>> -
>> -static int isc_clk_determine_rate(struct clk_hw *hw,
>> -                                struct clk_rate_request *req)
>> -{
>> -     struct isc_clk *isc_clk = to_isc_clk(hw);
>> -     long best_rate = -EINVAL;
>> -     int best_diff = -1;
>> -     unsigned int i, div;
>> -
>> -     for (i = 0; i < clk_hw_get_num_parents(hw); i++) {
>> -             struct clk_hw *parent;
>> -             unsigned long parent_rate;
>> -
>> -             parent = clk_hw_get_parent_by_index(hw, i);
>> -             if (!parent)
>> -                     continue;
>> -
>> -             parent_rate = clk_hw_get_rate(parent);
>> -             if (!parent_rate)
>> -                     continue;
>> -
>> -             for (div = 1; div < ISC_CLK_MAX_DIV + 2; div++) {
>> -                     unsigned long rate;
>> -                     int diff;
>> -
>> -                     rate = DIV_ROUND_CLOSEST(parent_rate, div);
>> -                     diff = abs(req->rate - rate);
>> -
>> -                     if (best_diff < 0 || best_diff > diff) {
>> -                             best_rate = rate;
>> -                             best_diff = diff;
>> -                             req->best_parent_rate = parent_rate;
>> -                             req->best_parent_hw = parent;
>> -                     }
>> -
>> -                     if (!best_diff || rate < req->rate)
>> -                             break;
>> -             }
>> -
>> -             if (!best_diff)
>> -                     break;
>> -     }
>> -
>> -     dev_dbg(isc_clk->dev,
>> -             "ISC CLK: %s, best_rate = %ld, parent clk: %s @ %ld\n",
>> -             __func__, best_rate,
>> -             __clk_get_name((req->best_parent_hw)->clk),
>> -             req->best_parent_rate);
>> -
>> -     if (best_rate < 0)
>> -             return best_rate;
>> -
>> -     req->rate = best_rate;
>> -
>> -     return 0;
>> -}
>> -
>> -static int isc_clk_set_parent(struct clk_hw *hw, u8 index)
>> -{
>> -     struct isc_clk *isc_clk = to_isc_clk(hw);
>> -
>> -     if (index >= clk_hw_get_num_parents(hw))
>> -             return -EINVAL;
>> -
>> -     isc_clk->parent_id = index;
>> -
>> -     return 0;
>> -}
>> -
>> -static u8 isc_clk_get_parent(struct clk_hw *hw)
>> -{
>> -     struct isc_clk *isc_clk = to_isc_clk(hw);
>> -
>> -     return isc_clk->parent_id;
>> -}
>> -
>> -static int isc_clk_set_rate(struct clk_hw *hw,
>> -                          unsigned long rate,
>> -                          unsigned long parent_rate)
>> -{
>> -     struct isc_clk *isc_clk = to_isc_clk(hw);
>> -     u32 div;
>> -
>> -     if (!rate)
>> -             return -EINVAL;
>> -
>> -     div = DIV_ROUND_CLOSEST(parent_rate, rate);
>> -     if (div > (ISC_CLK_MAX_DIV + 1) || !div)
>> -             return -EINVAL;
>> -
>> -     isc_clk->div = div - 1;
>> -
>> -     return 0;
>> -}
>> -
>> -static const struct clk_ops isc_clk_ops = {
>> -     .prepare        = isc_clk_prepare,
>> -     .unprepare      = isc_clk_unprepare,
>> -     .enable         = isc_clk_enable,
>> -     .disable        = isc_clk_disable,
>> -     .is_enabled     = isc_clk_is_enabled,
>> -     .recalc_rate    = isc_clk_recalc_rate,
>> -     .determine_rate = isc_clk_determine_rate,
>> -     .set_parent     = isc_clk_set_parent,
>> -     .get_parent     = isc_clk_get_parent,
>> -     .set_rate       = isc_clk_set_rate,
>> -};
>> -
>> -static int isc_clk_register(struct isc_device *isc, unsigned int id)
>> -{
>> -     struct regmap *regmap = isc->regmap;
>> -     struct device_node *np = isc->dev->of_node;
>> -     struct isc_clk *isc_clk;
>> -     struct clk_init_data init;
>> -     const char *clk_name = np->name;
>> -     const char *parent_names[3];
>> -     int num_parents;
>> -
>> -     if (id == ISC_ISPCK && !isc->ispck_required)
>> -             return 0;
>> -
>> -     num_parents = of_clk_get_parent_count(np);
>> -     if (num_parents < 1 || num_parents > 3)
>> -             return -EINVAL;
>> -
>> -     if (num_parents > 2 && id == ISC_ISPCK)
>> -             num_parents = 2;
>> -
>> -     of_clk_parent_fill(np, parent_names, num_parents);
>> -
>> -     if (id == ISC_MCK)
>> -             of_property_read_string(np, "clock-output-names", &clk_name);
>> -     else
>> -             clk_name = "isc-ispck";
>> -
>> -     init.parent_names       = parent_names;
>> -     init.num_parents        = num_parents;
>> -     init.name               = clk_name;
>> -     init.ops                = &isc_clk_ops;
>> -     init.flags              = CLK_SET_RATE_GATE | CLK_SET_PARENT_GATE;
>> -
>> -     isc_clk = &isc->isc_clks[id];
>> -     isc_clk->hw.init        = &init;
>> -     isc_clk->regmap         = regmap;
>> -     isc_clk->id             = id;
>> -     isc_clk->dev            = isc->dev;
>> -     spin_lock_init(&isc_clk->lock);
>> -
>> -     isc_clk->clk = clk_register(isc->dev, &isc_clk->hw);
>> -     if (IS_ERR(isc_clk->clk)) {
>> -             dev_err(isc->dev, "%s: clock register fail\n", clk_name);
>> -             return PTR_ERR(isc_clk->clk);
>> -     } else if (id == ISC_MCK)
>> -             of_clk_add_provider(np, of_clk_src_simple_get, isc_clk->clk);
>> -
>> -     return 0;
>> -}
>> -
>> -int isc_clk_init(struct isc_device *isc)
>> -{
>> -     unsigned int i;
>> -     int ret;
>> -
>> -     for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++)
>> -             isc->isc_clks[i].clk = ERR_PTR(-EINVAL);
>> -
>> -     for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
>> -             ret = isc_clk_register(isc, i);
>> -             if (ret)
>> -                     return ret;
>> -     }
>> -
>> -     return 0;
>> -}
>> -EXPORT_SYMBOL_GPL(isc_clk_init);
>> -
>> -void isc_clk_cleanup(struct isc_device *isc)
>> -{
>> -     unsigned int i;
>> -
>> -     of_clk_del_provider(isc->dev->of_node);
>> -
>> -     for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
>> -             struct isc_clk *isc_clk = &isc->isc_clks[i];
>> -
>> -             if (!IS_ERR(isc_clk->clk))
>> -                     clk_unregister(isc_clk->clk);
>> -     }
>> -}
>> -EXPORT_SYMBOL_GPL(isc_clk_cleanup);
>>
>>   static int isc_queue_setup(struct vb2_queue *vq,
>>                            unsigned int *nbuffers, unsigned int *nplanes,
>> diff --git a/drivers/media/platform/atmel/atmel-isc-clk.c b/drivers/media/platform/atmel/atmel-isc-clk.c
>> new file mode 100644
>> index 000000000000..d650caade396
>> --- /dev/null
>> +++ b/drivers/media/platform/atmel/atmel-isc-clk.c
>> @@ -0,0 +1,316 @@
>> +// SPDX-License-Identifier: GPL-2.0-only
>> +/*
>> + * Microchip Image Sensor Controller (ISC) common clock driver setup
>> + *
>> + * Copyright (C) 2016-2019 Microchip Technology, Inc.
> 
> Time flies!

I think I will change this to 2016 only, to keep just the original 
driver's first year of inclusion

> 
>> + *
>> + * Author: Songjun Wu
>> + * Author: Eugen Hristev <eugen.hristev@microchip.com>
>> + *
>> + */
>> +#include <linux/clk.h>
>> +#include <linux/clkdev.h>
>> +#include <linux/clk-provider.h>
>> +#include <linux/platform_device.h>
> 
> Is this needed ?
> 
>> +#include <linux/pm_runtime.h>
>> +#include <linux/regmap.h>
>> +
>> +#include <media/v4l2-ctrls.h>
>> +#include <media/v4l2-device.h>
>> +#include <media/videobuf2-dma-contig.h>
> 
> Is any one of these needed ?
> 
> Removing them highlights how these includes should probably be
> moved to atmel-isc.h which fails to compile if not preceeded by these
> inclusions.

Yes, this is the reason why the headers are there. I will try to move 
them to atmel-isc.h

> 
> I think you can merge this with the previous patch that adds the file
> entry to MAINTAINERS

I usually avoid to do that, because of the usual mess with merging 
patches on MAINTAINERS. So rather make it easy and clean with a simple 
patch directly to MAINTAINERS.

But if you think it's better to have it together with the driver change, 
I can squash it

Thank you for having a look at this,
Eugen
> 
> Thanks
>     j
> 
>> +
>> +#include "atmel-isc-regs.h"
>> +#include "atmel-isc.h"
>> +
>> +static int isc_wait_clk_stable(struct clk_hw *hw)
>> +{
>> +     struct isc_clk *isc_clk = to_isc_clk(hw);
>> +     struct regmap *regmap = isc_clk->regmap;
>> +     unsigned long timeout = jiffies + usecs_to_jiffies(1000);
>> +     unsigned int status;
>> +
>> +     while (time_before(jiffies, timeout)) {
>> +             regmap_read(regmap, ISC_CLKSR, &status);
>> +             if (!(status & ISC_CLKSR_SIP))
>> +                     return 0;
>> +
>> +             usleep_range(10, 250);
>> +     }
>> +
>> +     return -ETIMEDOUT;
>> +}
>> +
>> +static int isc_clk_prepare(struct clk_hw *hw)
>> +{
>> +     struct isc_clk *isc_clk = to_isc_clk(hw);
>> +     int ret;
>> +
>> +     ret = pm_runtime_resume_and_get(isc_clk->dev);
>> +     if (ret < 0)
>> +             return ret;
>> +
>> +     return isc_wait_clk_stable(hw);
>> +}
>> +
>> +static void isc_clk_unprepare(struct clk_hw *hw)
>> +{
>> +     struct isc_clk *isc_clk = to_isc_clk(hw);
>> +
>> +     isc_wait_clk_stable(hw);
>> +
>> +     pm_runtime_put_sync(isc_clk->dev);
>> +}
>> +
>> +static int isc_clk_enable(struct clk_hw *hw)
>> +{
>> +     struct isc_clk *isc_clk = to_isc_clk(hw);
>> +     u32 id = isc_clk->id;
>> +     struct regmap *regmap = isc_clk->regmap;
>> +     unsigned long flags;
>> +     unsigned int status;
>> +
>> +     dev_dbg(isc_clk->dev, "ISC CLK: %s, id = %d, div = %d, parent id = %d\n",
>> +             __func__, id, isc_clk->div, isc_clk->parent_id);
>> +
>> +     spin_lock_irqsave(&isc_clk->lock, flags);
>> +     regmap_update_bits(regmap, ISC_CLKCFG,
>> +                        ISC_CLKCFG_DIV_MASK(id) | ISC_CLKCFG_SEL_MASK(id),
>> +                        (isc_clk->div << ISC_CLKCFG_DIV_SHIFT(id)) |
>> +                        (isc_clk->parent_id << ISC_CLKCFG_SEL_SHIFT(id)));
>> +
>> +     regmap_write(regmap, ISC_CLKEN, ISC_CLK(id));
>> +     spin_unlock_irqrestore(&isc_clk->lock, flags);
>> +
>> +     regmap_read(regmap, ISC_CLKSR, &status);
>> +     if (status & ISC_CLK(id))
>> +             return 0;
>> +     else
>> +             return -EINVAL;
>> +}
>> +
>> +static void isc_clk_disable(struct clk_hw *hw)
>> +{
>> +     struct isc_clk *isc_clk = to_isc_clk(hw);
>> +     u32 id = isc_clk->id;
>> +     unsigned long flags;
>> +
>> +     spin_lock_irqsave(&isc_clk->lock, flags);
>> +     regmap_write(isc_clk->regmap, ISC_CLKDIS, ISC_CLK(id));
>> +     spin_unlock_irqrestore(&isc_clk->lock, flags);
>> +}
>> +
>> +static int isc_clk_is_enabled(struct clk_hw *hw)
>> +{
>> +     struct isc_clk *isc_clk = to_isc_clk(hw);
>> +     u32 status;
>> +     int ret;
>> +
>> +     ret = pm_runtime_resume_and_get(isc_clk->dev);
>> +     if (ret < 0)
>> +             return 0;
>> +
>> +     regmap_read(isc_clk->regmap, ISC_CLKSR, &status);
>> +
>> +     pm_runtime_put_sync(isc_clk->dev);
>> +
>> +     return status & ISC_CLK(isc_clk->id) ? 1 : 0;
>> +}
>> +
>> +static unsigned long
>> +isc_clk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
>> +{
>> +     struct isc_clk *isc_clk = to_isc_clk(hw);
>> +
>> +     return DIV_ROUND_CLOSEST(parent_rate, isc_clk->div + 1);
>> +}
>> +
>> +static int isc_clk_determine_rate(struct clk_hw *hw,
>> +                               struct clk_rate_request *req)
>> +{
>> +     struct isc_clk *isc_clk = to_isc_clk(hw);
>> +     long best_rate = -EINVAL;
>> +     int best_diff = -1;
>> +     unsigned int i, div;
>> +
>> +     for (i = 0; i < clk_hw_get_num_parents(hw); i++) {
>> +             struct clk_hw *parent;
>> +             unsigned long parent_rate;
>> +
>> +             parent = clk_hw_get_parent_by_index(hw, i);
>> +             if (!parent)
>> +                     continue;
>> +
>> +             parent_rate = clk_hw_get_rate(parent);
>> +             if (!parent_rate)
>> +                     continue;
>> +
>> +             for (div = 1; div < ISC_CLK_MAX_DIV + 2; div++) {
>> +                     unsigned long rate;
>> +                     int diff;
>> +
>> +                     rate = DIV_ROUND_CLOSEST(parent_rate, div);
>> +                     diff = abs(req->rate - rate);
>> +
>> +                     if (best_diff < 0 || best_diff > diff) {
>> +                             best_rate = rate;
>> +                             best_diff = diff;
>> +                             req->best_parent_rate = parent_rate;
>> +                             req->best_parent_hw = parent;
>> +                     }
>> +
>> +                     if (!best_diff || rate < req->rate)
>> +                             break;
>> +             }
>> +
>> +             if (!best_diff)
>> +                     break;
>> +     }
>> +
>> +     dev_dbg(isc_clk->dev,
>> +             "ISC CLK: %s, best_rate = %ld, parent clk: %s @ %ld\n",
>> +             __func__, best_rate,
>> +             __clk_get_name((req->best_parent_hw)->clk),
>> +             req->best_parent_rate);
>> +
>> +     if (best_rate < 0)
>> +             return best_rate;
>> +
>> +     req->rate = best_rate;
>> +
>> +     return 0;
>> +}
>> +
>> +static int isc_clk_set_parent(struct clk_hw *hw, u8 index)
>> +{
>> +     struct isc_clk *isc_clk = to_isc_clk(hw);
>> +
>> +     if (index >= clk_hw_get_num_parents(hw))
>> +             return -EINVAL;
>> +
>> +     isc_clk->parent_id = index;
>> +
>> +     return 0;
>> +}
>> +
>> +static u8 isc_clk_get_parent(struct clk_hw *hw)
>> +{
>> +     struct isc_clk *isc_clk = to_isc_clk(hw);
>> +
>> +     return isc_clk->parent_id;
>> +}
>> +
>> +static int isc_clk_set_rate(struct clk_hw *hw,
>> +                         unsigned long rate,
>> +                         unsigned long parent_rate)
>> +{
>> +     struct isc_clk *isc_clk = to_isc_clk(hw);
>> +     u32 div;
>> +
>> +     if (!rate)
>> +             return -EINVAL;
>> +
>> +     div = DIV_ROUND_CLOSEST(parent_rate, rate);
>> +     if (div > (ISC_CLK_MAX_DIV + 1) || !div)
>> +             return -EINVAL;
>> +
>> +     isc_clk->div = div - 1;
>> +
>> +     return 0;
>> +}
>> +
>> +static const struct clk_ops isc_clk_ops = {
>> +     .prepare        = isc_clk_prepare,
>> +     .unprepare      = isc_clk_unprepare,
>> +     .enable         = isc_clk_enable,
>> +     .disable        = isc_clk_disable,
>> +     .is_enabled     = isc_clk_is_enabled,
>> +     .recalc_rate    = isc_clk_recalc_rate,
>> +     .determine_rate = isc_clk_determine_rate,
>> +     .set_parent     = isc_clk_set_parent,
>> +     .get_parent     = isc_clk_get_parent,
>> +     .set_rate       = isc_clk_set_rate,
>> +};
>> +
>> +static int isc_clk_register(struct isc_device *isc, unsigned int id)
>> +{
>> +     struct regmap *regmap = isc->regmap;
>> +     struct device_node *np = isc->dev->of_node;
>> +     struct isc_clk *isc_clk;
>> +     struct clk_init_data init;
>> +     const char *clk_name = np->name;
>> +     const char *parent_names[3];
>> +     int num_parents;
>> +
>> +     if (id == ISC_ISPCK && !isc->ispck_required)
>> +             return 0;
>> +
>> +     num_parents = of_clk_get_parent_count(np);
>> +     if (num_parents < 1 || num_parents > 3)
>> +             return -EINVAL;
>> +
>> +     if (num_parents > 2 && id == ISC_ISPCK)
>> +             num_parents = 2;
>> +
>> +     of_clk_parent_fill(np, parent_names, num_parents);
>> +
>> +     if (id == ISC_MCK)
>> +             of_property_read_string(np, "clock-output-names", &clk_name);
>> +     else
>> +             clk_name = "isc-ispck";
>> +
>> +     init.parent_names       = parent_names;
>> +     init.num_parents        = num_parents;
>> +     init.name               = clk_name;
>> +     init.ops                = &isc_clk_ops;
>> +     init.flags              = CLK_SET_RATE_GATE | CLK_SET_PARENT_GATE;
>> +
>> +     isc_clk = &isc->isc_clks[id];
>> +     isc_clk->hw.init        = &init;
>> +     isc_clk->regmap         = regmap;
>> +     isc_clk->id             = id;
>> +     isc_clk->dev            = isc->dev;
>> +     spin_lock_init(&isc_clk->lock);
>> +
>> +     isc_clk->clk = clk_register(isc->dev, &isc_clk->hw);
>> +     if (IS_ERR(isc_clk->clk)) {
>> +             dev_err(isc->dev, "%s: clock register fail\n", clk_name);
>> +             return PTR_ERR(isc_clk->clk);
>> +     } else if (id == ISC_MCK) {
>> +             of_clk_add_provider(np, of_clk_src_simple_get, isc_clk->clk);
>> +     }
>> +
>> +     return 0;
>> +}
>> +
>> +int isc_clk_init(struct isc_device *isc)
>> +{
>> +     unsigned int i;
>> +     int ret;
>> +
>> +     for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++)
>> +             isc->isc_clks[i].clk = ERR_PTR(-EINVAL);
>> +
>> +     for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
>> +             ret = isc_clk_register(isc, i);
>> +             if (ret)
>> +                     return ret;
>> +     }
>> +
>> +     return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(isc_clk_init);
>> +
>> +void isc_clk_cleanup(struct isc_device *isc)
>> +{
>> +     unsigned int i;
>> +
>> +     of_clk_del_provider(isc->dev->of_node);
>> +
>> +     for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
>> +             struct isc_clk *isc_clk = &isc->isc_clks[i];
>> +
>> +             if (!IS_ERR(isc_clk->clk))
>> +                     clk_unregister(isc_clk->clk);
>> +     }
>> +}
>> +EXPORT_SYMBOL_GPL(isc_clk_cleanup);
>> --
>> 2.25.1
>>

Re: [PATCH 06/21] media: atmel: atmel-isc: replace video device name with module name

[Jacopo Mondi]

Hi Eugen

On Fri, Oct 22, 2021 at 10:52:32AM +0300, Eugen Hristev wrote:
> To have consistency with future media controller development,
> replace the video device name with KBUILD_MODNAME.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>

Makes sense to me
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>

Thanks
   j

> ---
>  drivers/media/platform/atmel/atmel-isc-base.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> index f532fd03e807..f3e89a892373 100644
> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> @@ -1887,7 +1887,7 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
>  	}
>
>  	/* Register video device */
> -	strscpy(vdev->name, "microchip-isc", sizeof(vdev->name));
> +	strscpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name));
>  	vdev->release		= video_device_release_empty;
>  	vdev->fops		= &isc_fops;
>  	vdev->ioctl_ops		= &isc_ioctl_ops;
> --
> 2.25.1
>

Re: [PATCH 07/21] media: atmel: atmel-sama7g5-isc: fix ispck leftover

[Jacopo Mondi]

Hi Eugen,

On Fri, Oct 22, 2021 at 10:52:33AM +0300, Eugen Hristev wrote:
> The ispck is not used for sama7g5 variant of the ISC.
> Calls to ispck have to be removed also from module insert/removal.
>
> Fixes: d7f26849ed7c ("media: atmel: fix the ispck initialization")
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>

Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>

Thanks
   j

> ---
>  drivers/media/platform/atmel/atmel-sama7g5-isc.c | 6 ------
>  1 file changed, 6 deletions(-)
>
> diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> index 9c05acafd072..6a5d3f7ce75e 100644
> --- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> +++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> @@ -555,7 +555,6 @@ static int microchip_xisc_remove(struct platform_device *pdev)
>
>  	v4l2_device_unregister(&isc->v4l2_dev);
>
> -	clk_disable_unprepare(isc->ispck);
>  	clk_disable_unprepare(isc->hclock);
>
>  	isc_clk_cleanup(isc);
> @@ -567,7 +566,6 @@ static int __maybe_unused xisc_runtime_suspend(struct device *dev)
>  {
>  	struct isc_device *isc = dev_get_drvdata(dev);
>
> -	clk_disable_unprepare(isc->ispck);
>  	clk_disable_unprepare(isc->hclock);
>
>  	return 0;
> @@ -582,10 +580,6 @@ static int __maybe_unused xisc_runtime_resume(struct device *dev)
>  	if (ret)
>  		return ret;
>
> -	ret = clk_prepare_enable(isc->ispck);
> -	if (ret)
> -		clk_disable_unprepare(isc->hclock);
> -
>  	return ret;
>  }
>
> --
> 2.25.1
>

Re: [PATCH 08/21] media: atmel: atmel-isc-base: use streaming status when queueing buffers

[Jacopo Mondi]

Hi Eugen

On Fri, Oct 22, 2021 at 10:52:34AM +0300, Eugen Hristev wrote:
> During experiments with libcamera, it looks like vb2_is_streaming returns
> true before our start streaming is called.
> Order of operations is streamon -> queue -> start_streaming
> ISC would have started the DMA immediately when a buffer is being added
> to the vbqueue if the queue is streaming.
> It is more safe to start the DMA after the start streaming of the driver is
> called.
> Thus, even if vb2queue is streaming, add the buffer to the dma queue of the
> driver instead of actually starting the DMA process, if the start streaming
> has not been called yet.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>

I cannot really comment on the issue itself, but seeing that isc->stop
is mostly used in negated form (!isc->stop) I suggest to make it
isc->streaming ?

Thanks
   j

> ---
>  drivers/media/platform/atmel/atmel-isc-base.c | 17 ++++++++++-------
>  1 file changed, 10 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> index f3e89a892373..6f14cc549543 100644
> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> @@ -442,12 +442,14 @@ static void isc_buffer_queue(struct vb2_buffer *vb)
>  	unsigned long flags;
>
>  	spin_lock_irqsave(&isc->dma_queue_lock, flags);
> -	if (!isc->cur_frm && list_empty(&isc->dma_queue) &&
> -		vb2_is_streaming(vb->vb2_queue)) {
> +
> +	if (!isc->cur_frm && list_empty(&isc->dma_queue) && !isc->stop) {
>  		isc->cur_frm = buf;
>  		isc_start_dma(isc);
> -	} else
> +	} else {
>  		list_add_tail(&buf->list, &isc->dma_queue);
> +	}
> +
>  	spin_unlock_irqrestore(&isc->dma_queue_lock, flags);
>  }
>
> @@ -1015,7 +1017,7 @@ static int isc_s_fmt_vid_cap(struct file *file, void *priv,
>  {
>  	struct isc_device *isc = video_drvdata(file);
>
> -	if (vb2_is_streaming(&isc->vb2_vidq))
> +	if (!isc->stop)
>  		return -EBUSY;
>
>  	return isc_set_fmt(isc, f);
> @@ -1537,7 +1539,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
>
>  		isc_update_awb_ctrls(isc);
>
> -		if (vb2_is_streaming(&isc->vb2_vidq)) {
> +		if (!isc->stop) {
>  			/*
>  			 * If we are streaming, we can update profile to
>  			 * have the new settings in place.
> @@ -1553,8 +1555,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
>  		}
>
>  		/* if we have autowhitebalance on, start histogram procedure */
> -		if (ctrls->awb == ISC_WB_AUTO &&
> -		    vb2_is_streaming(&isc->vb2_vidq) &&
> +		if (ctrls->awb == ISC_WB_AUTO && !isc->stop &&
>  		    ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
>  			isc_set_histogram(isc, true);
>
> @@ -1830,6 +1831,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
>  	struct vb2_queue *q = &isc->vb2_vidq;
>  	int ret = 0;
>
> +	isc->stop = true;
> +
>  	INIT_WORK(&isc->awb_work, isc_awb_work);
>
>  	ret = v4l2_device_register_subdev_nodes(&isc->v4l2_dev);
> --
> 2.25.1
>

Re: [PATCH 09/21] media: atmel: atmel-isc-base: remove frameintervals VIDIOC

[Jacopo Mondi]

Hi Eugen

On Fri, Oct 22, 2021 at 10:52:35AM +0300, Eugen Hristev wrote:
> VIDIOC_ENUM_FRAMEINTERVALS is not recommended for a top video driver.
> The frame rate is defined by the sensor subdevice, thus it can be queried
> directly by anyone interested in the frame intervals.

This change makes sense in the context of moving towards media
controller, but I wonder how does this impact existing userspace that
relies on this.

Anyway
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>

Thanks
  j
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> ---
>  drivers/media/platform/atmel/atmel-isc-base.c | 37 -------------------
>  1 file changed, 37 deletions(-)
>
> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> index 6f14cc549543..8537ad73d160 100644
> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> @@ -1108,42 +1108,6 @@ static int isc_enum_framesizes(struct file *file, void *fh,
>  	return 0;
>  }
>
> -static int isc_enum_frameintervals(struct file *file, void *fh,
> -				    struct v4l2_frmivalenum *fival)
> -{
> -	struct isc_device *isc = video_drvdata(file);
> -	struct v4l2_subdev_frame_interval_enum fie = {
> -		.code = isc->config.sd_format->mbus_code,
> -		.index = fival->index,
> -		.width = fival->width,
> -		.height = fival->height,
> -		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
> -	};
> -	int ret = -EINVAL;
> -	unsigned int i;
> -
> -	for (i = 0; i < isc->num_user_formats; i++)
> -		if (isc->user_formats[i]->fourcc == fival->pixel_format)
> -			ret = 0;
> -
> -	for (i = 0; i < isc->controller_formats_size; i++)
> -		if (isc->controller_formats[i].fourcc == fival->pixel_format)
> -			ret = 0;
> -
> -	if (ret)
> -		return ret;
> -
> -	ret = v4l2_subdev_call(isc->current_subdev->sd, pad,
> -			       enum_frame_interval, NULL, &fie);
> -	if (ret)
> -		return ret;
> -
> -	fival->type = V4L2_FRMIVAL_TYPE_DISCRETE;
> -	fival->discrete = fie.interval;
> -
> -	return 0;
> -}
> -
>  static const struct v4l2_ioctl_ops isc_ioctl_ops = {
>  	.vidioc_querycap		= isc_querycap,
>  	.vidioc_enum_fmt_vid_cap	= isc_enum_fmt_vid_cap,
> @@ -1168,7 +1132,6 @@ static const struct v4l2_ioctl_ops isc_ioctl_ops = {
>  	.vidioc_g_parm			= isc_g_parm,
>  	.vidioc_s_parm			= isc_s_parm,
>  	.vidioc_enum_framesizes		= isc_enum_framesizes,
> -	.vidioc_enum_frameintervals	= isc_enum_frameintervals,
>
>  	.vidioc_log_status		= v4l2_ctrl_log_status,
>  	.vidioc_subscribe_event		= v4l2_ctrl_subscribe_event,
> --
> 2.25.1
>

Re: [PATCH 11/21] media: atmel: atmel-isc-base: implement mbus_code support in enumfmt

[Jacopo Mondi]

Hi Eugen,

On Fri, Oct 22, 2021 at 10:52:37AM +0300, Eugen Hristev wrote:
> If enumfmt is called with an mbus_code, the enumfmt handler should only
> return the formats that are supported for this mbus_code.
> To make it more easy to understand the formats, changed the report order
> to report first the native formats, and after that the formats that the ISC
> can convert to.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>

Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>

Thanks
   j

> ---
>  drivers/media/platform/atmel/atmel-isc-base.c | 51 ++++++++++++++++---
>  1 file changed, 43 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> index 2dd2511c7be1..1f7fbe5e4d79 100644
> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> @@ -499,21 +499,56 @@ static int isc_enum_fmt_vid_cap(struct file *file, void *priv,
>  	u32 index = f->index;
>  	u32 i, supported_index;
>
> -	if (index < isc->controller_formats_size) {
> -		f->pixelformat = isc->controller_formats[index].fourcc;
> -		return 0;
> +	supported_index = 0;
> +
> +	for (i = 0; i < isc->formats_list_size; i++) {
> +		if (!isc->formats_list[i].sd_support)
> +			continue;
> +		/*
> +		 * If specific mbus_code is requested, provide only
> +		 * supported formats with this mbus code
> +		 */
> +		if (f->mbus_code && f->mbus_code !=
> +		    isc->formats_list[i].mbus_code)
> +			continue;
> +		if (supported_index == index) {
> +			f->pixelformat = isc->formats_list[i].fourcc;
> +			return 0;
> +		}
> +		supported_index++;
>  	}
>
> -	index -= isc->controller_formats_size;
> +	/*
> +	 * If the sensor does not support this mbus_code whatsoever,
> +	 * there is no reason to advertise any of our output formats
> +	 */
> +	if (supported_index == 0)
> +		return -EINVAL;
> +
> +	/*
> +	 * If the sensor uses a format that is not raw, then we cannot
> +	 * convert it to any of the formats that we usually can with a
> +	 * RAW sensor. Thus, do not advertise them.
> +	 */
> +	if (!isc->config.sd_format ||
> +	    !ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
> +		return -EINVAL;
>
> +	/*
> +	 * Iterate again through the formats that we can convert to.
> +	 * However, to avoid duplicates, skip the formats that
> +	 * the sensor already supports directly
> +	 */
> +	index -= supported_index;
>  	supported_index = 0;
>
> -	for (i = 0; i < isc->formats_list_size; i++) {
> -		if (!ISC_IS_FORMAT_RAW(isc->formats_list[i].mbus_code) ||
> -		    !isc->formats_list[i].sd_support)
> +	for (i = 0; i < isc->controller_formats_size; i++) {
> +		/* if this format is already supported by sensor, skip it */
> +		if (find_format_by_fourcc(isc, isc->controller_formats[i].fourcc))
>  			continue;
>  		if (supported_index == index) {
> -			f->pixelformat = isc->formats_list[i].fourcc;
> +			f->pixelformat =
> +				isc->controller_formats[i].fourcc;
>  			return 0;
>  		}
>  		supported_index++;
> --
> 2.25.1
>

Re: [PATCH 11/21] media: atmel: atmel-isc-base: implement mbus_code support in enumfmt

[Jacopo Mondi]

Hi Eugen

On Fri, Nov 05, 2021 at 10:25:59AM +0100, Jacopo Mondi wrote:
> Hi Eugen,
>
> On Fri, Oct 22, 2021 at 10:52:37AM +0300, Eugen Hristev wrote:
> > If enumfmt is called with an mbus_code, the enumfmt handler should only
> > return the formats that are supported for this mbus_code.
> > To make it more easy to understand the formats, changed the report order
> > to report first the native formats, and after that the formats that the ISC
> > can convert to.
> >
> > Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>
> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
>

Too soon! Sorry...

> Thanks
>    j
>
> > ---
> >  drivers/media/platform/atmel/atmel-isc-base.c | 51 ++++++++++++++++---
> >  1 file changed, 43 insertions(+), 8 deletions(-)
> >
> > diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> > index 2dd2511c7be1..1f7fbe5e4d79 100644
> > --- a/drivers/media/platform/atmel/atmel-isc-base.c
> > +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> > @@ -499,21 +499,56 @@ static int isc_enum_fmt_vid_cap(struct file *file, void *priv,
> >  	u32 index = f->index;
> >  	u32 i, supported_index;
> >
> > -	if (index < isc->controller_formats_size) {
> > -		f->pixelformat = isc->controller_formats[index].fourcc;
> > -		return 0;
> > +	supported_index = 0;
> > +
> > +	for (i = 0; i < isc->formats_list_size; i++) {
> > +		if (!isc->formats_list[i].sd_support)


> > +			continue;
> > +		/*
> > +		 * If specific mbus_code is requested, provide only
> > +		 * supported formats with this mbus code
> > +		 */
> > +		if (f->mbus_code && f->mbus_code !=
> > +		    isc->formats_list[i].mbus_code)
> > +			continue;
> > +		if (supported_index == index) {
> > +			f->pixelformat = isc->formats_list[i].fourcc;
> > +			return 0;
> > +		}
> > +		supported_index++;
> >  	}
> >
> > -	index -= isc->controller_formats_size;
> > +	/*
> > +	 * If the sensor does not support this mbus_code whatsoever,
> > +	 * there is no reason to advertise any of our output formats
> > +	 */
> > +	if (supported_index == 0)
> > +		return -EINVAL;

Shouldn't you also return -EINVAL if index > supported_index ?

In that case, I'm not gettng what the rest of the function is for ?

> > +
> > +	/*
> > +	 * If the sensor uses a format that is not raw, then we cannot
> > +	 * convert it to any of the formats that we usually can with a
> > +	 * RAW sensor. Thus, do not advertise them.
> > +	 */
> > +	if (!isc->config.sd_format ||
> > +	    !ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
> > +		return -EINVAL;
> >
> > +	/*
> > +	 * Iterate again through the formats that we can convert to.
> > +	 * However, to avoid duplicates, skip the formats that
> > +	 * the sensor already supports directly
> > +	 */
> > +	index -= supported_index;
> >  	supported_index = 0;
> >
> > -	for (i = 0; i < isc->formats_list_size; i++) {
> > -		if (!ISC_IS_FORMAT_RAW(isc->formats_list[i].mbus_code) ||
> > -		    !isc->formats_list[i].sd_support)
> > +	for (i = 0; i < isc->controller_formats_size; i++) {
> > +		/* if this format is already supported by sensor, skip it */
> > +		if (find_format_by_fourcc(isc, isc->controller_formats[i].fourcc))
> >  			continue;
> >  		if (supported_index == index) {
> > -			f->pixelformat = isc->formats_list[i].fourcc;
> > +			f->pixelformat =
> > +				isc->controller_formats[i].fourcc;
> >  			return 0;
> >  		}
> >  		supported_index++;
> > --
> > 2.25.1
> >

Re: [PATCH 10/21] media: atmel: atmel-isc-base: report frame sizes as full supported range

[Jacopo Mondi]

Hi Eugen

On Fri, Oct 22, 2021 at 10:52:36AM +0300, Eugen Hristev wrote:
> The ISC supports a full broad range of frame sizes.
> Until now, the subdevice was queried for possible frame sizes and these
> were reported to the user space.
> However, the ISC should not care about which frame sizes the subdev supports,
> as long as this frame size is supported.
> Thus, report a continuous range from smallest frame size up to the max
> resolution.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>

Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>

Thanks
   j

> ---
>  drivers/media/platform/atmel/atmel-isc-base.c | 22 +++++++++----------
>  1 file changed, 10 insertions(+), 12 deletions(-)
>
> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> index 8537ad73d160..2dd2511c7be1 100644
> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> @@ -1077,14 +1077,12 @@ static int isc_enum_framesizes(struct file *file, void *fh,
>  			       struct v4l2_frmsizeenum *fsize)
>  {
>  	struct isc_device *isc = video_drvdata(file);
> -	struct v4l2_subdev_frame_size_enum fse = {
> -		.code = isc->config.sd_format->mbus_code,
> -		.index = fsize->index,
> -		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
> -	};
>  	int ret = -EINVAL;
>  	int i;
>
> +	if (fsize->index)
> +		return -EINVAL;
> +
>  	for (i = 0; i < isc->num_user_formats; i++)
>  		if (isc->user_formats[i]->fourcc == fsize->pixel_format)
>  			ret = 0;
> @@ -1096,14 +1094,14 @@ static int isc_enum_framesizes(struct file *file, void *fh,
>  	if (ret)
>  		return ret;
>
> -	ret = v4l2_subdev_call(isc->current_subdev->sd, pad, enum_frame_size,
> -			       NULL, &fse);
> -	if (ret)
> -		return ret;
> +	fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
>
> -	fsize->type = V4L2_FRMSIZE_TYPE_DISCRETE;
> -	fsize->discrete.width = fse.max_width;
> -	fsize->discrete.height = fse.max_height;
> +	fsize->stepwise.min_width = 16;
> +	fsize->stepwise.max_width = isc->max_width;
> +	fsize->stepwise.min_height = 16;
> +	fsize->stepwise.max_height = isc->max_height;
> +	fsize->stepwise.step_width = 1;
> +	fsize->stepwise.step_height = 1;
>
>  	return 0;
>  }
> --
> 2.25.1
>

Re: [PATCH 11/21] media: atmel: atmel-isc-base: implement mbus_code support in enumfmt

[Eugen.Hristev]

On 11/5/21 11:51 AM, Jacopo Mondi wrote:
> Hi Eugen
> 
> On Fri, Nov 05, 2021 at 10:25:59AM +0100, Jacopo Mondi wrote:
>> Hi Eugen,
>>
>> On Fri, Oct 22, 2021 at 10:52:37AM +0300, Eugen Hristev wrote:
>>> If enumfmt is called with an mbus_code, the enumfmt handler should only
>>> return the formats that are supported for this mbus_code.
>>> To make it more easy to understand the formats, changed the report order
>>> to report first the native formats, and after that the formats that the ISC
>>> can convert to.
>>>
>>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>>
>> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
>>
> 
> Too soon! Sorry...
> 
>> Thanks
>>     j
>>
>>> ---
>>>   drivers/media/platform/atmel/atmel-isc-base.c | 51 ++++++++++++++++---
>>>   1 file changed, 43 insertions(+), 8 deletions(-)
>>>
>>> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
>>> index 2dd2511c7be1..1f7fbe5e4d79 100644
>>> --- a/drivers/media/platform/atmel/atmel-isc-base.c
>>> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
>>> @@ -499,21 +499,56 @@ static int isc_enum_fmt_vid_cap(struct file *file, void *priv,
>>>      u32 index = f->index;
>>>      u32 i, supported_index;
>>>
>>> -   if (index < isc->controller_formats_size) {
>>> -           f->pixelformat = isc->controller_formats[index].fourcc;
>>> -           return 0;
>>> +   supported_index = 0;
>>> +

Hi Jacopo,

This for loop below first iterates through the formats that were 
identified as directly supported by the subdevice.
As we are able in the ISC to just dump what the subdevice can stream, we 
advertise all these formats here.
(if the userspace requires one specific mbus code, we only advertise the 
corresponding format)

>>> +   for (i = 0; i < isc->formats_list_size; i++) {
>>> +           if (!isc->formats_list[i].sd_support)
> 
> 
>>> +                   continue;
>>> +           /*
>>> +            * If specific mbus_code is requested, provide only
>>> +            * supported formats with this mbus code
>>> +            */
>>> +           if (f->mbus_code && f->mbus_code !=
>>> +               isc->formats_list[i].mbus_code)
>>> +                   continue;
>>> +           if (supported_index == index) {
>>> +                   f->pixelformat = isc->formats_list[i].fourcc;
>>> +                   return 0;
>>> +           }
>>> +           supported_index++;
>>>      }
>>>
>>> -   index -= isc->controller_formats_size;
>>> +   /*
>>> +    * If the sensor does not support this mbus_code whatsoever,
>>> +    * there is no reason to advertise any of our output formats
>>> +    */
>>> +   if (supported_index == 0)
>>> +           return -EINVAL;
> 
> Shouldn't you also return -EINVAL if index > supported_index ?

No. If we could not find any format that the sensor can use 
(supported_index == 0), and that our ISC can also use, then we don't 
support anything, thus, return -EINVAL regardless of the index.

> 
> In that case, I'm not gettng what the rest of the function is for ?
> 

This is the case when we identified one supported format for both the 
sensor and the ISC (case where supported_index found earlier is >= 1)

>>> +
>>> +   /*
>>> +    * If the sensor uses a format that is not raw, then we cannot
>>> +    * convert it to any of the formats that we usually can with a
>>> +    * RAW sensor. Thus, do not advertise them.
>>> +    */
>>> +   if (!isc->config.sd_format ||
>>> +       !ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
>>> +           return -EINVAL;
>>>

Next, if the current format from the subdev is not raw, we are done.
But, if it's raw, we can use it to convert to a plethora of other 
formats. So we have to enumerate them below :

With the previous checks, I am making sure that the ISC can really 
convert to these formats, otherwise they are badly reported

Hope this makes things more clear, but please ask if something looks strange

>>> +   /*
>>> +    * Iterate again through the formats that we can convert to.
>>> +    * However, to avoid duplicates, skip the formats that
>>> +    * the sensor already supports directly
>>> +    */
>>> +   index -= supported_index;
>>>      supported_index = 0;
>>>
>>> -   for (i = 0; i < isc->formats_list_size; i++) {
>>> -           if (!ISC_IS_FORMAT_RAW(isc->formats_list[i].mbus_code) ||
>>> -               !isc->formats_list[i].sd_support)
>>> +   for (i = 0; i < isc->controller_formats_size; i++) {
>>> +           /* if this format is already supported by sensor, skip it */
>>> +           if (find_format_by_fourcc(isc, isc->controller_formats[i].fourcc))
>>>                      continue;
>>>              if (supported_index == index) {
>>> -                   f->pixelformat = isc->formats_list[i].fourcc;
>>> +                   f->pixelformat =
>>> +                           isc->controller_formats[i].fourcc;
>>>                      return 0;
>>>              }
>>>              supported_index++;
>>> --
>>> 2.25.1
>>>

Re: [PATCH 11/21] media: atmel: atmel-isc-base: implement mbus_code support in enumfmt

[Jacopo Mondi]

Hi Eugen

On Fri, Nov 05, 2021 at 11:03:25AM +0000, Eugen.Hristev@microchip.com wrote:
> On 11/5/21 11:51 AM, Jacopo Mondi wrote:
> > Hi Eugen
> >
> > On Fri, Nov 05, 2021 at 10:25:59AM +0100, Jacopo Mondi wrote:
> >> Hi Eugen,
> >>
> >> On Fri, Oct 22, 2021 at 10:52:37AM +0300, Eugen Hristev wrote:
> >>> If enumfmt is called with an mbus_code, the enumfmt handler should only
> >>> return the formats that are supported for this mbus_code.
> >>> To make it more easy to understand the formats, changed the report order
> >>> to report first the native formats, and after that the formats that the ISC
> >>> can convert to.
> >>>
> >>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> >>
> >> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
> >>
> >
> > Too soon! Sorry...
> >
> >> Thanks
> >>     j
> >>
> >>> ---
> >>>   drivers/media/platform/atmel/atmel-isc-base.c | 51 ++++++++++++++++---
> >>>   1 file changed, 43 insertions(+), 8 deletions(-)
> >>>
> >>> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> >>> index 2dd2511c7be1..1f7fbe5e4d79 100644
> >>> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> >>> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> >>> @@ -499,21 +499,56 @@ static int isc_enum_fmt_vid_cap(struct file *file, void *priv,
> >>>      u32 index = f->index;
> >>>      u32 i, supported_index;
> >>>
> >>> -   if (index < isc->controller_formats_size) {
> >>> -           f->pixelformat = isc->controller_formats[index].fourcc;
> >>> -           return 0;
> >>> +   supported_index = 0;
> >>> +
>
> Hi Jacopo,
>
> This for loop below first iterates through the formats that were
> identified as directly supported by the subdevice.
> As we are able in the ISC to just dump what the subdevice can stream, we
> advertise all these formats here.
> (if the userspace requires one specific mbus code, we only advertise the
> corresponding format)
>
> >>> +   for (i = 0; i < isc->formats_list_size; i++) {
> >>> +           if (!isc->formats_list[i].sd_support)
> >
> >
> >>> +                   continue;
> >>> +           /*
> >>> +            * If specific mbus_code is requested, provide only
> >>> +            * supported formats with this mbus code
> >>> +            */
> >>> +           if (f->mbus_code && f->mbus_code !=
> >>> +               isc->formats_list[i].mbus_code)
> >>> +                   continue;
> >>> +           if (supported_index == index) {
> >>> +                   f->pixelformat = isc->formats_list[i].fourcc;
> >>> +                   return 0;
> >>> +           }
> >>> +           supported_index++;
> >>>      }
> >>>
> >>> -   index -= isc->controller_formats_size;
> >>> +   /*
> >>> +    * If the sensor does not support this mbus_code whatsoever,
> >>> +    * there is no reason to advertise any of our output formats
> >>> +    */
> >>> +   if (supported_index == 0)
> >>> +           return -EINVAL;
> >
> > Shouldn't you also return -EINVAL if index > supported_index ?
>
> No. If we could not find any format that the sensor can use
> (supported_index == 0), and that our ISC can also use, then we don't
> support anything, thus, return -EINVAL regardless of the index.
>
> >
> > In that case, I'm not gettng what the rest of the function is for ?
> >
>
> This is the case when we identified one supported format for both the
> sensor and the ISC (case where supported_index found earlier is >= 1)
>
> >>> +
> >>> +   /*
> >>> +    * If the sensor uses a format that is not raw, then we cannot
> >>> +    * convert it to any of the formats that we usually can with a
> >>> +    * RAW sensor. Thus, do not advertise them.
> >>> +    */
> >>> +   if (!isc->config.sd_format ||
> >>> +       !ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
> >>> +           return -EINVAL;
> >>>
>
> Next, if the current format from the subdev is not raw, we are done.

Ok, I think here it's were I disconnect.

I don't think you should care about the -current- format on the
subdev, as once you move to MC the ISC formats should be enumerated
regardless of the how the remote subdev is configured. Rather, you
should consider if IS_RAW(f->mbus_code) and from there enumerate the
output formats you can generate from raw bayer (in addition to the
ones that can be produced by dumping the sensor produced formats)

> But, if it's raw, we can use it to convert to a plethora of other
> formats. So we have to enumerate them below :
>
> With the previous checks, I am making sure that the ISC can really
> convert to these formats, otherwise they are badly reported
>
> Hope this makes things more clear, but please ask if something looks strange
>

I think our disconnection comes from the way the supported formats are
defined in ISC and I think their definition could be reworkd to
simplify the format selection logic.

The driver defines three lists of formats:

- isc->controller_formats

       static const struct isc_format sama7g5_controller_formats[] = {
	{
		.fourcc		= V4L2_PIX_FMT_ARGB444,
	},
	{
		.fourcc		= V4L2_PIX_FMT_ARGB555,
	},
        ...

        };

 These are listed with the output fourcc only, and if I get
 try_configure_pipeline() right, they can be generated from any Bayer
 RAW format. Is this right ?

- isc->formats_list

        static struct isc_format sama7g5_formats_list[] = {
                {
                        .fourcc		= V4L2_PIX_FMT_SBGGR8,
                        .mbus_code	= MEDIA_BUS_FMT_SBGGR8_1X8,
                        .pfe_cfg0_bps	= ISC_PFE_CFG0_BPS_EIGHT,
                        .cfa_baycfg	= ISC_BAY_CFG_BGBG,
                },
                {
                        .fourcc		= V4L2_PIX_FMT_SGBRG8,
                        .mbus_code	= MEDIA_BUS_FMT_SGBRG8_1X8,
                        .pfe_cfg0_bps	= ISC_PFE_CFG0_BPS_EIGHT,
                        .cfa_baycfg	= ISC_BAY_CFG_GBGB,
                },

         ...

         };

  These are formats the ISC can output by dumping the sensor output,
  hence they require a specific format to be output from the sensor

- isc->user_formats

        static int isc_formats_init() {

                ...

                fmt = &isc->formats_list[0];
                for (i = 0, j = 0; i < list_size; i++) {
                        if (fmt->sd_support)
                                isc->user_formats[j++] = fmt;
                        fmt++;
                }

      }

  this list is obtained at runtime by restricting the formats_list to
  what the sensor can actually output. I think these, if you move to
  MC should be removed but I'm not 100% sure it is possible with the
  current implementation of set_fmt().

Do you think controller_formats and formats_list should be unified ?

If my understanding that all controller_formats can be generated from
RAW Bayer formats you could model struct isc_format as

        struct isc_format {
                u32	fourcc;
                bool    processed;
                u32     mbus_codes
                u32	cfa_baycfg;
                u32	pfe_cfg0_bps;
        };

and have in example:

	{
		.fourcc		= V4L2_PIX_FMT_ARGB444,
                .processed      = true,
                .mbus_codes     = nullptr,
                ....
	},
        {
                .fourcc		= V4L2_PIX_FMT_SBGGR8,
                .processed      = false,
                .mbus_codes	= { MEDIA_BUS_FMT_SBGGR8_1X8 }
                .pfe_cfg0_bps	= ISC_PFE_CFG0_BPS_EIGHT,
                .cfa_baycfg	= ISC_BAY_CFG_BGBG,
        },

and when enumerating and configuring formats check that

        if (isc_format[i].processed &&
            (f->mbus_code && IS_RAW(f->mbus)code))

or

        if (!isc_format[i].processed &&
            (f->mbus_code == isc_format[i].mbus_code

if you have other restrictions as in example V4L2_PIX_FMT_YUV420
requiring a packed YUV format, you can implement more complex
validations to match processed formats, like you do in try_validate_formats()

Also, and a bit unrelated to enum_fmt, if I got this right
at format configuration time you match the ISC format with
the sensor format. I think this should be moved to .link_validate() op time.

The MC core calls .link_validate when streaming is started on each
entity part of a pipeline, and there you could make sure that the ISC output format can be produced using
the sensor format (and sizes). This will greatly simplify set_fmt() as
there you will have just to make sure the supplied format is in the
list of the ISC supported ones and that's it. If userspace fails to
configure the pipeline correctly (in example by setting a non RAW
Bayer sensor on the format and requesting a RAW Bayer format from ISC,
you will fail at s_stream() time by returning an EPIPE.

Hope all of this makes a bit of sense :)

> >>> +   /*
> >>> +    * Iterate again through the formats that we can convert to.
> >>> +    * However, to avoid duplicates, skip the formats that
> >>> +    * the sensor already supports directly
> >>> +    */
> >>> +   index -= supported_index;
> >>>      supported_index = 0;
> >>>
> >>> -   for (i = 0; i < isc->formats_list_size; i++) {
> >>> -           if (!ISC_IS_FORMAT_RAW(isc->formats_list[i].mbus_code) ||
> >>> -               !isc->formats_list[i].sd_support)
> >>> +   for (i = 0; i < isc->controller_formats_size; i++) {
> >>> +           /* if this format is already supported by sensor, skip it */
> >>> +           if (find_format_by_fourcc(isc, isc->controller_formats[i].fourcc))
> >>>                      continue;
> >>>              if (supported_index == index) {
> >>> -                   f->pixelformat = isc->formats_list[i].fourcc;
> >>> +                   f->pixelformat =
> >>> +                           isc->controller_formats[i].fourcc;
> >>>                      return 0;
> >>>              }
> >>>              supported_index++;
> >>> --
> >>> 2.25.1
> >>>
>

Re: [PATCH 11/21] media: atmel: atmel-isc-base: implement mbus_code support in enumfmt

[Eugen.Hristev]

On 11/8/21 1:20 PM, Jacopo Mondi wrote:
> Hi Eugen
> 
> On Fri, Nov 05, 2021 at 11:03:25AM +0000, Eugen.Hristev@microchip.com wrote:
>> On 11/5/21 11:51 AM, Jacopo Mondi wrote:
>>> Hi Eugen
>>>
>>> On Fri, Nov 05, 2021 at 10:25:59AM +0100, Jacopo Mondi wrote:
>>>> Hi Eugen,
>>>>
>>>> On Fri, Oct 22, 2021 at 10:52:37AM +0300, Eugen Hristev wrote:
>>>>> If enumfmt is called with an mbus_code, the enumfmt handler should only
>>>>> return the formats that are supported for this mbus_code.
>>>>> To make it more easy to understand the formats, changed the report order
>>>>> to report first the native formats, and after that the formats that the ISC
>>>>> can convert to.
>>>>>
>>>>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>>>>
>>>> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
>>>>
>>>
>>> Too soon! Sorry...
>>>
>>>> Thanks
>>>>      j
>>>>
>>>>> ---
>>>>>    drivers/media/platform/atmel/atmel-isc-base.c | 51 ++++++++++++++++---
>>>>>    1 file changed, 43 insertions(+), 8 deletions(-)
>>>>>
>>>>> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
>>>>> index 2dd2511c7be1..1f7fbe5e4d79 100644
>>>>> --- a/drivers/media/platform/atmel/atmel-isc-base.c
>>>>> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
>>>>> @@ -499,21 +499,56 @@ static int isc_enum_fmt_vid_cap(struct file *file, void *priv,
>>>>>       u32 index = f->index;
>>>>>       u32 i, supported_index;
>>>>>
>>>>> -   if (index < isc->controller_formats_size) {
>>>>> -           f->pixelformat = isc->controller_formats[index].fourcc;
>>>>> -           return 0;
>>>>> +   supported_index = 0;
>>>>> +
>>
>> Hi Jacopo,
>>
>> This for loop below first iterates through the formats that were
>> identified as directly supported by the subdevice.
>> As we are able in the ISC to just dump what the subdevice can stream, we
>> advertise all these formats here.
>> (if the userspace requires one specific mbus code, we only advertise the
>> corresponding format)
>>
>>>>> +   for (i = 0; i < isc->formats_list_size; i++) {
>>>>> +           if (!isc->formats_list[i].sd_support)
>>>
>>>
>>>>> +                   continue;
>>>>> +           /*
>>>>> +            * If specific mbus_code is requested, provide only
>>>>> +            * supported formats with this mbus code
>>>>> +            */
>>>>> +           if (f->mbus_code && f->mbus_code !=
>>>>> +               isc->formats_list[i].mbus_code)
>>>>> +                   continue;
>>>>> +           if (supported_index == index) {
>>>>> +                   f->pixelformat = isc->formats_list[i].fourcc;
>>>>> +                   return 0;
>>>>> +           }
>>>>> +           supported_index++;
>>>>>       }
>>>>>
>>>>> -   index -= isc->controller_formats_size;
>>>>> +   /*
>>>>> +    * If the sensor does not support this mbus_code whatsoever,
>>>>> +    * there is no reason to advertise any of our output formats
>>>>> +    */
>>>>> +   if (supported_index == 0)
>>>>> +           return -EINVAL;
>>>
>>> Shouldn't you also return -EINVAL if index > supported_index ?
>>
>> No. If we could not find any format that the sensor can use
>> (supported_index == 0), and that our ISC can also use, then we don't
>> support anything, thus, return -EINVAL regardless of the index.
>>
>>>
>>> In that case, I'm not gettng what the rest of the function is for ?
>>>
>>
>> This is the case when we identified one supported format for both the
>> sensor and the ISC (case where supported_index found earlier is >= 1)
>>
>>>>> +
>>>>> +   /*
>>>>> +    * If the sensor uses a format that is not raw, then we cannot
>>>>> +    * convert it to any of the formats that we usually can with a
>>>>> +    * RAW sensor. Thus, do not advertise them.
>>>>> +    */
>>>>> +   if (!isc->config.sd_format ||
>>>>> +       !ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
>>>>> +           return -EINVAL;
>>>>>
>>
>> Next, if the current format from the subdev is not raw, we are done.
> 
> Ok, I think here it's were I disconnect.
> 
> I don't think you should care about the -current- format on the
> subdev, as once you move to MC the ISC formats should be enumerated
> regardless of the how the remote subdev is configured. Rather, you
> should consider if IS_RAW(f->mbus_code) and from there enumerate the
> output formats you can generate from raw bayer (in addition to the
> ones that can be produced by dumping the sensor produced formats)

Actually , in some words, this is what I am doing.
I am following Laurent's advice:
enumerate the formats supported for a given mbus code.

In consequence, if the mbus code given is a raw mbus code , I can 
advertise all my ISC formats, and if the mbus code is not raw, I don't .

So I am doing what you are saying, namely three cases:

If there is no mbus code given as a parameter to this function, I 
advertise all my formats, raw, non-raw, etc.

If there is raw mbus code given, I advertise this specific raw mbus code 
if the sensor supports it, and the ISC supports it, and in addition, all 
the formats ISC can convert from raw to.

If the mbus code given is not raw, then, I can only advertise this 
specific non-raw format, since there is nothing more I can do with it.
It wouldn't make much sense to still advertise my rgb,yuv formats, since 
they cannot be used at all, and my later try_validate_formats will bail out


> 
>> But, if it's raw, we can use it to convert to a plethora of other
>> formats. So we have to enumerate them below :
>>
>> With the previous checks, I am making sure that the ISC can really
>> convert to these formats, otherwise they are badly reported
>>
>> Hope this makes things more clear, but please ask if something looks strange
>>
> 
> I think our disconnection comes from the way the supported formats are
> defined in ISC and I think their definition could be reworkd to
> simplify the format selection logic.
> 
> The driver defines three lists of formats:
> 
> - isc->controller_formats
> 
>         static const struct isc_format sama7g5_controller_formats[] = {
>          {
>                  .fourcc         = V4L2_PIX_FMT_ARGB444,
>          },
>          {
>                  .fourcc         = V4L2_PIX_FMT_ARGB555,
>          },
>          ...
> 
>          };
> 
>   These are listed with the output fourcc only, and if I get
>   try_configure_pipeline() right, they can be generated from any Bayer
>   RAW format. Is this right ?
> 
> - isc->formats_list
> 
>          static struct isc_format sama7g5_formats_list[] = {
>                  {
>                          .fourcc         = V4L2_PIX_FMT_SBGGR8,
>                          .mbus_code      = MEDIA_BUS_FMT_SBGGR8_1X8,
>                          .pfe_cfg0_bps   = ISC_PFE_CFG0_BPS_EIGHT,
>                          .cfa_baycfg     = ISC_BAY_CFG_BGBG,
>                  },
>                  {
>                          .fourcc         = V4L2_PIX_FMT_SGBRG8,
>                          .mbus_code      = MEDIA_BUS_FMT_SGBRG8_1X8,
>                          .pfe_cfg0_bps   = ISC_PFE_CFG0_BPS_EIGHT,
>                          .cfa_baycfg     = ISC_BAY_CFG_GBGB,
>                  },
> 
>           ...
> 
>           };
> 
>    These are formats the ISC can output by dumping the sensor output,
>    hence they require a specific format to be output from the sensor
> 
> - isc->user_formats
> 
>          static int isc_formats_init() {
> 
>                  ...
> 
>                  fmt = &isc->formats_list[0];
>                  for (i = 0, j = 0; i < list_size; i++) {
>                          if (fmt->sd_support)
>                                  isc->user_formats[j++] = fmt;
>                          fmt++;
>                  }
> 
>        }
> 
>    this list is obtained at runtime by restricting the formats_list to
>    what the sensor can actually output. I think these, if you move to
>    MC should be removed but I'm not 100% sure it is possible with the
>    current implementation of set_fmt().
> 
> Do you think controller_formats and formats_list should be unified ?
> 
> If my understanding that all controller_formats can be generated from
> RAW Bayer formats you could model struct isc_format as
> 
>          struct isc_format {
>                  u32     fourcc;
>                  bool    processed;
>                  u32     mbus_codes
>                  u32     cfa_baycfg;
>                  u32     pfe_cfg0_bps;
>          };
> 
> and have in example:
> 
>          {
>                  .fourcc         = V4L2_PIX_FMT_ARGB444,
>                  .processed      = true,
>                  .mbus_codes     = nullptr,
>                  ....
>          },
>          {
>                  .fourcc         = V4L2_PIX_FMT_SBGGR8,
>                  .processed      = false,
>                  .mbus_codes     = { MEDIA_BUS_FMT_SBGGR8_1X8 }
>                  .pfe_cfg0_bps   = ISC_PFE_CFG0_BPS_EIGHT,
>                  .cfa_baycfg     = ISC_BAY_CFG_BGBG,
>          },
> 
> and when enumerating and configuring formats check that
> 
>          if (isc_format[i].processed &&
>              (f->mbus_code && IS_RAW(f->mbus)code))
> 
> or
> 
>          if (!isc_format[i].processed &&
>              (f->mbus_code == isc_format[i].mbus_code
> 
> if you have other restrictions as in example V4L2_PIX_FMT_YUV420
> requiring a packed YUV format, you can implement more complex
> validations to match processed formats, like you do in try_validate_formats()
> 
> Also, and a bit unrelated to enum_fmt, if I got this right
> at format configuration time you match the ISC format with
> the sensor format. I think this should be moved to .link_validate() op time.
> 
> The MC core calls .link_validate when streaming is started on each
> entity part of a pipeline, and there you could make sure that the ISC output format can be produced using
> the sensor format (and sizes). This will greatly simplify set_fmt() as
> there you will have just to make sure the supplied format is in the
> list of the ISC supported ones and that's it. If userspace fails to
> configure the pipeline correctly (in example by setting a non RAW
> Bayer sensor on the format and requesting a RAW Bayer format from ISC,
> you will fail at s_stream() time by returning an EPIPE.
> 
> Hope all of this makes a bit of sense :)

About this last part you are telling me: I have to tell you what am I 
doing right now: with this patch series, including this patch, I am 
adding support for mc handling in this driver, but! the driver is still 
completely compatible with 'the old way' like setting fmt-video for 
/dev/video0 and everything is propagated down the pipeline.

I am doing the conversion to mc-only type of driver in multiple steps.
This series adds the 'new way' while having the 'old way' still there.
At the moment I am working on another patch on top of this series that 
will basically remove most of my format propagation logic from 
/dev/video0 to the subdevice, and after this patch that is on the way, 
the 'old way' is gone. The sensor will *have to* be configured through 
userspace because ISC will never call set_fmt on it at all.

So the purpose of the patch you are reviewing now is to have the mbus 
code parameter in the enum_fmt_vid_cap in the way the current driver 
handles things.

So if you agree, I will let the other patch speak for itself and rework 
the logic completely .
I am currently trying to do it at streamon() time like Laurent told me, 
but I can try to have it at validate link as well, to see how it works.

I will add that patch to the series when it's ready and I have a v2 of 
this series as well. So in baby steps, I am working towards the goal. I 
am not sure if this means that you could agree to this patch as-is, or I 
have to integrate it completely into a bigger patch that will also fix 
everything up including the format logic.

Thanks again for your review and ideas

Eugen
> 
>>>>> +   /*
>>>>> +    * Iterate again through the formats that we can convert to.
>>>>> +    * However, to avoid duplicates, skip the formats that
>>>>> +    * the sensor already supports directly
>>>>> +    */
>>>>> +   index -= supported_index;
>>>>>       supported_index = 0;
>>>>>
>>>>> -   for (i = 0; i < isc->formats_list_size; i++) {
>>>>> -           if (!ISC_IS_FORMAT_RAW(isc->formats_list[i].mbus_code) ||
>>>>> -               !isc->formats_list[i].sd_support)
>>>>> +   for (i = 0; i < isc->controller_formats_size; i++) {
>>>>> +           /* if this format is already supported by sensor, skip it */
>>>>> +           if (find_format_by_fourcc(isc, isc->controller_formats[i].fourcc))
>>>>>                       continue;
>>>>>               if (supported_index == index) {
>>>>> -                   f->pixelformat = isc->formats_list[i].fourcc;
>>>>> +                   f->pixelformat =
>>>>> +                           isc->controller_formats[i].fourcc;
>>>>>                       return 0;
>>>>>               }
>>>>>               supported_index++;
>>>>> --
>>>>> 2.25.1
>>>>>
>>

Re: [PATCH 11/21] media: atmel: atmel-isc-base: implement mbus_code support in enumfmt

[Eugen.Hristev]

On 11/8/21 4:08 PM, Eugen Hristev - M18282 wrote:
> On 11/8/21 1:20 PM, Jacopo Mondi wrote:
>> Hi Eugen
>>
>> On Fri, Nov 05, 2021 at 11:03:25AM +0000, Eugen.Hristev@microchip.com wrote:
>>> On 11/5/21 11:51 AM, Jacopo Mondi wrote:
>>>> Hi Eugen
>>>>
>>>> On Fri, Nov 05, 2021 at 10:25:59AM +0100, Jacopo Mondi wrote:
>>>>> Hi Eugen,
>>>>>
>>>>> On Fri, Oct 22, 2021 at 10:52:37AM +0300, Eugen Hristev wrote:
>>>>>> If enumfmt is called with an mbus_code, the enumfmt handler should only
>>>>>> return the formats that are supported for this mbus_code.
>>>>>> To make it more easy to understand the formats, changed the report order
>>>>>> to report first the native formats, and after that the formats that the ISC
>>>>>> can convert to.
>>>>>>
>>>>>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>>>>>
>>>>> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
>>>>>
>>>>
>>>> Too soon! Sorry...
>>>>
>>>>> Thanks
>>>>>       j
>>>>>
>>>>>> ---
>>>>>>     drivers/media/platform/atmel/atmel-isc-base.c | 51 ++++++++++++++++---
>>>>>>     1 file changed, 43 insertions(+), 8 deletions(-)
>>>>>>
>>>>>> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
>>>>>> index 2dd2511c7be1..1f7fbe5e4d79 100644
>>>>>> --- a/drivers/media/platform/atmel/atmel-isc-base.c
>>>>>> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
>>>>>> @@ -499,21 +499,56 @@ static int isc_enum_fmt_vid_cap(struct file *file, void *priv,
>>>>>>        u32 index = f->index;
>>>>>>        u32 i, supported_index;
>>>>>>
>>>>>> -   if (index < isc->controller_formats_size) {
>>>>>> -           f->pixelformat = isc->controller_formats[index].fourcc;
>>>>>> -           return 0;
>>>>>> +   supported_index = 0;
>>>>>> +
>>>
>>> Hi Jacopo,
>>>
>>> This for loop below first iterates through the formats that were
>>> identified as directly supported by the subdevice.
>>> As we are able in the ISC to just dump what the subdevice can stream, we
>>> advertise all these formats here.
>>> (if the userspace requires one specific mbus code, we only advertise the
>>> corresponding format)
>>>
>>>>>> +   for (i = 0; i < isc->formats_list_size; i++) {
>>>>>> +           if (!isc->formats_list[i].sd_support)
>>>>
>>>>
>>>>>> +                   continue;
>>>>>> +           /*
>>>>>> +            * If specific mbus_code is requested, provide only
>>>>>> +            * supported formats with this mbus code
>>>>>> +            */
>>>>>> +           if (f->mbus_code && f->mbus_code !=
>>>>>> +               isc->formats_list[i].mbus_code)
>>>>>> +                   continue;
>>>>>> +           if (supported_index == index) {
>>>>>> +                   f->pixelformat = isc->formats_list[i].fourcc;
>>>>>> +                   return 0;
>>>>>> +           }
>>>>>> +           supported_index++;
>>>>>>        }
>>>>>>
>>>>>> -   index -= isc->controller_formats_size;
>>>>>> +   /*
>>>>>> +    * If the sensor does not support this mbus_code whatsoever,
>>>>>> +    * there is no reason to advertise any of our output formats
>>>>>> +    */
>>>>>> +   if (supported_index == 0)
>>>>>> +           return -EINVAL;
>>>>
>>>> Shouldn't you also return -EINVAL if index > supported_index ?
>>>
>>> No. If we could not find any format that the sensor can use
>>> (supported_index == 0), and that our ISC can also use, then we don't
>>> support anything, thus, return -EINVAL regardless of the index.
>>>
>>>>
>>>> In that case, I'm not gettng what the rest of the function is for ?
>>>>
>>>
>>> This is the case when we identified one supported format for both the
>>> sensor and the ISC (case where supported_index found earlier is >= 1)
>>>
>>>>>> +
>>>>>> +   /*
>>>>>> +    * If the sensor uses a format that is not raw, then we cannot
>>>>>> +    * convert it to any of the formats that we usually can with a
>>>>>> +    * RAW sensor. Thus, do not advertise them.
>>>>>> +    */
>>>>>> +   if (!isc->config.sd_format ||
>>>>>> +       !ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
>>>>>> +           return -EINVAL;
>>>>>>
>>>
>>> Next, if the current format from the subdev is not raw, we are done.
>>
>> Ok, I think here it's were I disconnect.
>>
>> I don't think you should care about the -current- format on the
>> subdev, as once you move to MC the ISC formats should be enumerated
>> regardless of the how the remote subdev is configured. Rather, you
>> should consider if IS_RAW(f->mbus_code) and from there enumerate the
>> output formats you can generate from raw bayer (in addition to the
>> ones that can be produced by dumping the sensor produced formats)
> 
> Actually , in some words, this is what I am doing.
> I am following Laurent's advice:
> enumerate the formats supported for a given mbus code.
> 
> In consequence, if the mbus code given is a raw mbus code , I can
> advertise all my ISC formats, and if the mbus code is not raw, I don't .
> 
> So I am doing what you are saying, namely three cases:
> 
> If there is no mbus code given as a parameter to this function, I
> advertise all my formats, raw, non-raw, etc.
> 
> If there is raw mbus code given, I advertise this specific raw mbus code
> if the sensor supports it, and the ISC supports it, and in addition, all
> the formats ISC can convert from raw to.
> 
> If the mbus code given is not raw, then, I can only advertise this
> specific non-raw format, since there is nothing more I can do with it.
> It wouldn't make much sense to still advertise my rgb,yuv formats, since
> they cannot be used at all, and my later try_validate_formats will bail out
> 
> 
>>
>>> But, if it's raw, we can use it to convert to a plethora of other
>>> formats. So we have to enumerate them below :
>>>
>>> With the previous checks, I am making sure that the ISC can really
>>> convert to these formats, otherwise they are badly reported
>>>
>>> Hope this makes things more clear, but please ask if something looks strange
>>>
>>
>> I think our disconnection comes from the way the supported formats are
>> defined in ISC and I think their definition could be reworkd to
>> simplify the format selection logic.
>>
>> The driver defines three lists of formats:
>>
>> - isc->controller_formats
>>
>>          static const struct isc_format sama7g5_controller_formats[] = {
>>           {
>>                   .fourcc         = V4L2_PIX_FMT_ARGB444,
>>           },
>>           {
>>                   .fourcc         = V4L2_PIX_FMT_ARGB555,
>>           },
>>           ...
>>
>>           };
>>
>>    These are listed with the output fourcc only, and if I get
>>    try_configure_pipeline() right, they can be generated from any Bayer
>>    RAW format. Is this right ?
>>
>> - isc->formats_list
>>
>>           static struct isc_format sama7g5_formats_list[] = {
>>                   {
>>                           .fourcc         = V4L2_PIX_FMT_SBGGR8,
>>                           .mbus_code      = MEDIA_BUS_FMT_SBGGR8_1X8,
>>                           .pfe_cfg0_bps   = ISC_PFE_CFG0_BPS_EIGHT,
>>                           .cfa_baycfg     = ISC_BAY_CFG_BGBG,
>>                   },
>>                   {
>>                           .fourcc         = V4L2_PIX_FMT_SGBRG8,
>>                           .mbus_code      = MEDIA_BUS_FMT_SGBRG8_1X8,
>>                           .pfe_cfg0_bps   = ISC_PFE_CFG0_BPS_EIGHT,
>>                           .cfa_baycfg     = ISC_BAY_CFG_GBGB,
>>                   },
>>
>>            ...
>>
>>            };
>>
>>     These are formats the ISC can output by dumping the sensor output,
>>     hence they require a specific format to be output from the sensor
>>
>> - isc->user_formats
>>
>>           static int isc_formats_init() {
>>
>>                   ...
>>
>>                   fmt = &isc->formats_list[0];
>>                   for (i = 0, j = 0; i < list_size; i++) {
>>                           if (fmt->sd_support)
>>                                   isc->user_formats[j++] = fmt;
>>                           fmt++;
>>                   }
>>
>>         }
>>
>>     this list is obtained at runtime by restricting the formats_list to
>>     what the sensor can actually output. I think these, if you move to
>>     MC should be removed but I'm not 100% sure it is possible with the
>>     current implementation of set_fmt().
>>
>> Do you think controller_formats and formats_list should be unified ?
>>
>> If my understanding that all controller_formats can be generated from
>> RAW Bayer formats you could model struct isc_format as
>>
>>           struct isc_format {
>>                   u32     fourcc;
>>                   bool    processed;
>>                   u32     mbus_codes
>>                   u32     cfa_baycfg;
>>                   u32     pfe_cfg0_bps;
>>           };
>>
>> and have in example:
>>
>>           {
>>                   .fourcc         = V4L2_PIX_FMT_ARGB444,
>>                   .processed      = true,
>>                   .mbus_codes     = nullptr,
>>                   ....
>>           },
>>           {
>>                   .fourcc         = V4L2_PIX_FMT_SBGGR8,
>>                   .processed      = false,
>>                   .mbus_codes     = { MEDIA_BUS_FMT_SBGGR8_1X8 }
>>                   .pfe_cfg0_bps   = ISC_PFE_CFG0_BPS_EIGHT,
>>                   .cfa_baycfg     = ISC_BAY_CFG_BGBG,
>>           },
>>
>> and when enumerating and configuring formats check that
>>
>>           if (isc_format[i].processed &&
>>               (f->mbus_code && IS_RAW(f->mbus)code))
>>
>> or
>>
>>           if (!isc_format[i].processed &&
>>               (f->mbus_code == isc_format[i].mbus_code
>>
>> if you have other restrictions as in example V4L2_PIX_FMT_YUV420
>> requiring a packed YUV format, you can implement more complex
>> validations to match processed formats, like you do in try_validate_formats()
>>
>> Also, and a bit unrelated to enum_fmt, if I got this right
>> at format configuration time you match the ISC format with
>> the sensor format. I think this should be moved to .link_validate() op time.
>>
>> The MC core calls .link_validate when streaming is started on each
>> entity part of a pipeline, and there you could make sure that the ISC output format can be produced using
>> the sensor format (and sizes). This will greatly simplify set_fmt() as
>> there you will have just to make sure the supplied format is in the
>> list of the ISC supported ones and that's it. If userspace fails to
>> configure the pipeline correctly (in example by setting a non RAW
>> Bayer sensor on the format and requesting a RAW Bayer format from ISC,
>> you will fail at s_stream() time by returning an EPIPE.

Hi Jacopo,

I tried to look over the link_validate call.
It looks like this is only called by media_pipeline_start, which most 
drivers use in start_streaming() .
However, I need the format validation to be done before that, at 
streamon() time, because after streamon() , the vb2 queue will be filled 
with buffers, and I need to know exactly which format I will be using.
So, do you think it's fine to keep the format validation at streamon() 
time instead of calling media_pipeline_start in start_streaming ?

Currently I am not calling media_pipeline_start at all.
Do you think it's required?

Or maybe I am missing something and it should be done in a different way ?

Thanks !

>>
>> Hope all of this makes a bit of sense :)
> 
> About this last part you are telling me: I have to tell you what am I
> doing right now: with this patch series, including this patch, I am
> adding support for mc handling in this driver, but! the driver is still
> completely compatible with 'the old way' like setting fmt-video for
> /dev/video0 and everything is propagated down the pipeline.
> 
> I am doing the conversion to mc-only type of driver in multiple steps.
> This series adds the 'new way' while having the 'old way' still there.
> At the moment I am working on another patch on top of this series that
> will basically remove most of my format propagation logic from
> /dev/video0 to the subdevice, and after this patch that is on the way,
> the 'old way' is gone. The sensor will *have to* be configured through
> userspace because ISC will never call set_fmt on it at all.
> 
> So the purpose of the patch you are reviewing now is to have the mbus
> code parameter in the enum_fmt_vid_cap in the way the current driver
> handles things.
> 
> So if you agree, I will let the other patch speak for itself and rework
> the logic completely .
> I am currently trying to do it at streamon() time like Laurent told me,
> but I can try to have it at validate link as well, to see how it works.
> 
> I will add that patch to the series when it's ready and I have a v2 of
> this series as well. So in baby steps, I am working towards the goal. I
> am not sure if this means that you could agree to this patch as-is, or I
> have to integrate it completely into a bigger patch that will also fix
> everything up including the format logic.
> 
> Thanks again for your review and ideas
> 
> Eugen
>>
>>>>>> +   /*
>>>>>> +    * Iterate again through the formats that we can convert to.
>>>>>> +    * However, to avoid duplicates, skip the formats that
>>>>>> +    * the sensor already supports directly
>>>>>> +    */
>>>>>> +   index -= supported_index;
>>>>>>        supported_index = 0;
>>>>>>
>>>>>> -   for (i = 0; i < isc->formats_list_size; i++) {
>>>>>> -           if (!ISC_IS_FORMAT_RAW(isc->formats_list[i].mbus_code) ||
>>>>>> -               !isc->formats_list[i].sd_support)
>>>>>> +   for (i = 0; i < isc->controller_formats_size; i++) {
>>>>>> +           /* if this format is already supported by sensor, skip it */
>>>>>> +           if (find_format_by_fourcc(isc, isc->controller_formats[i].fourcc))
>>>>>>                        continue;
>>>>>>                if (supported_index == index) {
>>>>>> -                   f->pixelformat = isc->formats_list[i].fourcc;
>>>>>> +                   f->pixelformat =
>>>>>> +                           isc->controller_formats[i].fourcc;
>>>>>>                        return 0;
>>>>>>                }
>>>>>>                supported_index++;
>>>>>> --
>>>>>> 2.25.1
>>>>>>
>>>
> 

Re: [PATCH 03/21] media: atmel: introduce microchip csi2dc driver

[Eugen.Hristev]

On 11/3/21 12:59 PM, Jacopo Mondi wrote:
> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> 
> Hi Eugen
> 
> On Wed, Nov 03, 2021 at 09:59:35AM +0000, Eugen.Hristev@microchip.com wrote:
>> On 11/3/21 11:28 AM, Jacopo Mondi wrote:
>>> Hi Eugen
>>>
>>> On Wed, Nov 03, 2021 at 08:31:36AM +0000, Eugen.Hristev@microchip.com wrote:
>>>> On 11/2/21 7:22 PM, Jacopo Mondi wrote:
>>>>> Hi Eugen,
>>>>>
>>>>
>>>> Hi,
>>>>
>>>> Thank you for your review. I will try to understand everything you said
>>>> and come up with a new version. I still have some inline questions
>>>> though, about some things which are still unclear.
>>>>
>>>>> On Fri, Oct 22, 2021 at 10:52:29AM +0300, Eugen Hristev wrote:
>>>>>> Microchip CSI2DC (CSI2 Demultiplexer Controller) is a misc bridge device
>>>>>> that converts a byte stream in IDI Synopsys format (coming from a CSI2HOST)
>>>>>> to a pixel stream that can be captured by a sensor controller.
>>>>>>
>>>>>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>>>>>> ---
>>>>>> Changes in this revision:
>>>>>> - addressed comments by Jacopo and Laurent as in this thread:
>>>>>> https://www.spinics.net/lists/linux-media/msg181044.html
>>>>>>
>>>>>> Previous change log :
>>>>>> Changes in v5:
>>>>>> - only in bindings
>>>>>>
>>>>>> Changes in v4:
>>>>>> - now using get_mbus_config ops to get data from the subdevice, like the
>>>>>> virtual channel id, and the clock type.
>>>>>> - now having possibility to select any of the RAW10 data modes
>>>>>> - at completion time, select which formats are also available in the subdevice,
>>>>>> and move to the dynamic list accordingly
>>>>>> - changed the pipeline integration, do not advertise subdev ready at probe time.
>>>>>> wait until completion is done, and then start a workqueue that will register
>>>>>> this device as a subdevice for the next element in pipeline.
>>>>>> - moved the s_power code into a different function called now csi2dc_power
>>>>>> that is called with CONFIG_PM functions. This is also called at completion,
>>>>>> to have the device ready in case CONFIG_PM is not selected on the platform.
>>>>>> - merged try_fmt into set_fmt
>>>>>> - driver cleanup, wrapped lines over 80 characters
>>>>>>
>>>>>> Changes in v2:
>>>>>> - moved driver to platform/atmel
>>>>>> - fixed minor things as per Sakari's review
>>>>>> - still some things from v2 review are not yet addressed, to be followed up
>>>>>>
>>>>>>
>>>>>>     drivers/media/platform/atmel/Kconfig          |  15 +
>>>>>>     drivers/media/platform/atmel/Makefile         |   1 +
>>>>>>     .../media/platform/atmel/microchip-csi2dc.c   | 700 ++++++++++++++++++
>>>>>>     3 files changed, 716 insertions(+)
>>>>>>     create mode 100644 drivers/media/platform/atmel/microchip-csi2dc.c
>>>>>>
>>>>>> diff --git a/drivers/media/platform/atmel/Kconfig b/drivers/media/platform/atmel/Kconfig
>>>>>> index dda2f27da317..f83bee373d82 100644
>>>>>> --- a/drivers/media/platform/atmel/Kconfig
>>>>>> +++ b/drivers/media/platform/atmel/Kconfig
>>>>>> @@ -40,3 +40,18 @@ config VIDEO_ATMEL_ISI
>>>>>>          help
>>>>>>            This module makes the ATMEL Image Sensor Interface available
>>>>>>            as a v4l2 device.
>>>>>> +
>>>>>> +config VIDEO_MICROCHIP_CSI2DC
>>>>>> +     tristate "Microchip CSI2 Demux Controller"
>>>>>> +     depends on VIDEO_V4L2 && COMMON_CLK && OF
>>>>>> +     depends on ARCH_AT91 || COMPILE_TEST
>>>>>> +     select MEDIA_CONTROLLER
>>>>>> +     select VIDEO_V4L2_SUBDEV_API
>>>>>> +     select V4L2_FWNODE
>>>>>> +     help
>>>>>> +       CSI2 Demux Controller driver. CSI2DC is a helper chip
>>>>>> +       that converts IDI interface byte stream to a parallel pixel stream.
>>>>>> +       It supports various RAW formats as input.
>>>>>> +
>>>>>> +       To compile this driver as a module, choose M here: the
>>>>>> +       module will be called microchip-csi2dc.
>>>>>> diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
>>>>>> index 46d264ab7948..39f0a7eba702 100644
>>>>>> --- a/drivers/media/platform/atmel/Makefile
>>>>>> +++ b/drivers/media/platform/atmel/Makefile
>>>>>> @@ -6,3 +6,4 @@ obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
>>>>>>     obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o
>>>>>>     obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel-isc.o
>>>>>>     obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel-xisc.o
>>>>>> +obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += microchip-csi2dc.o
>>>>>> diff --git a/drivers/media/platform/atmel/microchip-csi2dc.c b/drivers/media/platform/atmel/microchip-csi2dc.c
>>>>>> new file mode 100644
>>>>>> index 000000000000..277b86988eee
>>>>>> --- /dev/null
>>>>>> +++ b/drivers/media/platform/atmel/microchip-csi2dc.c
>>>>>> @@ -0,0 +1,700 @@
>>>>>> +// SPDX-License-Identifier: GPL-2.0-only
>>>>>> +/*
>>>>>> + * Microchip CSI2 Demux Controller (CSI2DC) driver
>>>>>> + *
>>>>>> + * Copyright (C) 2018 Microchip Technology, Inc.
>>>>>> + *
>>>>>> + * Author: Eugen Hristev <eugen.hristev@microchip.com>
>>>>>> + *
>>>>>> + */
>>>>>> +
>>>>>> +#include <linux/clk.h>
>>>>>> +#include <linux/module.h>
>>>>>> +#include <linux/of.h>
>>>>>
>>>>> Isn't linux/mod_devicetable.h enough ?
>>>>>
>>>>>> +#include <linux/of_graph.h>
>>>>>
>>>>> You should probably move to use the fwnode_graph framwork instead of
>>>>> of_graph. This driver depends on OF so it shouldn't be an issue but I
>>>>> defer this to maintainers
>>>>>
>>>>>> +#include <linux/platform_device.h>
>>>>>> +#include <linux/pm_runtime.h>
>>>>>> +#include <linux/videodev2.h>
>>>>>> +
>>>>>> +#include <media/v4l2-device.h>
>>>>>
>>>>> Do you need this include ?
>>>>>
>>>>>> +#include <media/v4l2-fwnode.h>
>>>>>> +#include <media/v4l2-subdev.h>
>>>>>> +#include <media/videobuf2-dma-contig.h>
>>>>>
>>>>> Is this one needed as well ?
>>>>>
>>>>>> +
>>>>>> +/* Global configuration register */
>>>>>> +#define CSI2DC_GCFG                  0x0
>>>>>> +
>>>>>> +/* MIPI sensor pixel clock is free running */
>>>>>> +#define CSI2DC_GCFG_MIPIFRN          BIT(0)
>>>>>> +/* Output waveform inter-line minimum delay */
>>>>>> +#define CSI2DC_GCFG_HLC(v)           ((v) << 4)
>>>>>> +#define CSI2DC_GCFG_HLC_MASK         GENMASK(7, 4)
>>>>>> +/* SAMA7G5 requires a HLC delay of 15 */
>>>>>> +#define SAMA7G5_HLC                  (15)
>>>>>> +
>>>>>> +/* Global control register */
>>>>>> +#define CSI2DC_GCTLR                 0x04
>>>>>> +#define CSI2DC_GCTLR_SWRST           BIT(0)
>>>>>> +
>>>>>> +/* Global status register */
>>>>>> +#define CSI2DC_GS                    0x08
>>>>>> +
>>>>>> +/* SSP interrupt status register */
>>>>>> +#define CSI2DC_SSPIS                 0x28
>>>>>> +/* Pipe update register */
>>>>>> +#define CSI2DC_PU                    0xC0
>>>>>
>>>>> What about using lowercase for hex values (I know there's not strict
>>>>> rule, so keep what you like the most, but most drivers use lowercase
>>>>>
>>>>>> +/* Video pipe attributes update */
>>>>>> +#define CSI2DC_PU_VP                 BIT(0)
>>>>>> +
>>>>>> +/* Pipe update status register */
>>>>>> +#define CSI2DC_PUS                   0xC4
>>>>>> +
>>>>>> +/* Video pipeline enable register */
>>>>>> +#define CSI2DC_VPE                   0xF8
>>>>>> +#define CSI2DC_VPE_ENABLE            BIT(0)
>>>>>> +
>>>>>> +/* Video pipeline configuration register */
>>>>>> +#define CSI2DC_VPCFG                 0xFC
>>>>>> +/* Data type */
>>>>>> +#define CSI2DC_VPCFG_DT(v)           ((v) << 0)
>>>>>> +#define CSI2DC_VPCFG_DT_MASK         GENMASK(5, 0)
>>>>>> +/* Virtual channel identifier */
>>>>>> +#define CSI2DC_VPCFG_VC(v)           ((v) << 6)
>>>>>> +#define CSI2DC_VPCFG_VC_MASK         GENMASK(7, 6)
>>>>>> +/* Decompression enable */
>>>>>> +#define CSI2DC_VPCFG_DE                      BIT(8)
>>>>>> +/* Decoder mode */
>>>>>> +#define CSI2DC_VPCFG_DM(v)           ((v) << 9)
>>>>>> +#define CSI2DC_VPCFG_DM_DECODER8TO12 0
>>>>>> +/* Decoder predictor 2 selection */
>>>>>> +#define CSI2DC_VPCFG_DP2             BIT(12)
>>>>>> +/* Recommended memory storage */
>>>>>> +#define CSI2DC_VPCFG_RMS             BIT(13)
>>>>>> +/* Post adjustment */
>>>>>> +#define CSI2DC_VPCFG_PA                      BIT(14)
>>>>>> +
>>>>>> +/* Video pipeline column register */
>>>>>> +#define CSI2DC_VPCOL                 0x100
>>>>>> +/* Column number */
>>>>>> +#define CSI2DC_VPCOL_COL(v)          ((v) << 0)
>>>>>> +#define CSI2DC_VPCOL_COL_MASK                GENMASK(15, 0)
>>>>>> +
>>>>>> +/* Video pipeline row register */
>>>>>> +#define CSI2DC_VPROW                 0x104
>>>>>> +/* Row number */
>>>>>> +#define CSI2DC_VPROW_ROW(v)          ((v) << 0)
>>>>>> +#define CSI2DC_VPROW_ROW_MASK                GENMASK(15, 0)
>>>>>> +
>>>>>> +/* Version register */
>>>>>> +#define CSI2DC_VERSION                       0x1FC
>>>>>> +
>>>>>> +/* register read/write helpers */
>>>>>> +#define csi2dc_readl(st, reg)                readl_relaxed((st)->base + (reg))
>>>>>> +#define csi2dc_writel(st, reg, val)  writel_relaxed((val), \
>>>>>> +                                     (st)->base + (reg))
>>>>>> +
>>>>>> +/* supported RAW data types */
>>>>>> +#define CSI2DC_DT_RAW6                       0x28
>>>>>> +#define CSI2DC_DT_RAW7                       0x29
>>>>>> +#define CSI2DC_DT_RAW8                       0x2A
>>>>>> +#define CSI2DC_DT_RAW10                      0x2B
>>>>>> +#define CSI2DC_DT_RAW12                      0x2C
>>>>>> +#define CSI2DC_DT_RAW14                      0x2D
>>>>>> +
>>>>>> +/*
>>>>>> + * struct csi2dc_format - CSI2DC format type struct
>>>>>> + * @mbus_code:               Media bus code for the format
>>>>>> + * @dt:                      Data type constant for this format
>>>>>> + */
>>>>>> +struct csi2dc_format {
>>>>>> +     u32                             mbus_code;
>>>>>> +     u32                             dt;
>>>>>> +};
>>>>>> +
>>>>>> +static const struct csi2dc_format csi2dc_formats[] = {
>>>>>> +     {
>>>>>> +             .mbus_code =            MEDIA_BUS_FMT_SRGGB10_1X10,
>>>>>> +             .dt =                   CSI2DC_DT_RAW10,
>>>>>> +     }, {
>>>>>> +             .mbus_code =            MEDIA_BUS_FMT_SBGGR10_1X10,
>>>>>> +             .dt =                   CSI2DC_DT_RAW10,
>>>>>> +     }, {
>>>>>> +             .mbus_code =            MEDIA_BUS_FMT_SGRBG10_1X10,
>>>>>> +             .dt =                   CSI2DC_DT_RAW10,
>>>>>> +     }, {
>>>>>> +             .mbus_code =            MEDIA_BUS_FMT_SGBRG10_1X10,
>>>>>> +             .dt =                   CSI2DC_DT_RAW10,
>>>>>> +     },
>>>>>> +};
>>>>>
>>>>> How unfortunate we don't have this in the core...
>>>>>
>>>>>> +
>>>>>> +enum mipi_csi_pads {
>>>>>> +     CSI2DC_PAD_SINK                 = 0,
>>>>>> +     CSI2DC_PAD_SOURCE               = 1,
>>>>>> +     CSI2DC_PADS_NUM                 = 2,
>>>>>> +};
>>>>>> +
>>>>>> +/*
>>>>>> + * struct csi2dc_device - CSI2DC device driver data/config struct
>>>>>> + * @base:            Register map base address
>>>>>> + * @csi2dc_sd:               v4l2 subdevice for the csi2dc device
>>>>>> + *                   This is the subdevice that the csi2dc device itself
>>>>>> + *                   registers in v4l2 subsystem
>>>>>> + * @dev:             struct device for this csi2dc device
>>>>>> + * @pclk:            Peripheral clock reference
>>>>>> + *                   Input clock that clocks the hardware block internal
>>>>>> + *                   logic
>>>>>> + * @scck:            Sensor Controller clock reference
>>>>>> + *                   Input clock that is used to generate the pixel clock
>>>>>> + * @format:          Current saved format used in g/s fmt
>>>>>> + * @cur_fmt:         Current state format
>>>>>> + * @try_fmt:         Try format that is being tried
>>>>>> + * @pads:            Media entity pads for the csi2dc subdevice
>>>>>> + * @clk_gated:               Whether the clock is gated or free running
>>>>>> + * @video_pipe:              Whether video pipeline is configured
>>>>>> + * @vc:                      Current set virtual channel
>>>>>> + * @asd:             Async subdevice for async bound of the underlying subdev
>>>>>> + * @notifier:                Async notifier that is used to bound the underlying
>>>>>> + *                   subdevice to the csi2dc subdevice
>>>>>> + * @input_sd:                Reference to the underlying subdevice bound to the
>>>>>> + *                   csi2dc subdevice
>>>>>> + * @remote_pad:              Pad number of the underlying subdevice that is linked
>>>>>> + *                   to the csi2dc subdevice sink pad.
>>>>>> + */
>>>>>> +struct csi2dc_device {
>>>>>> +     void __iomem                    *base;
>>>>>> +     struct v4l2_subdev              csi2dc_sd;
>>>>>> +     struct device                   *dev;
>>>>>> +     struct clk                      *pclk;
>>>>>> +     struct clk                      *scck;
>>>>>> +
>>>>>> +     struct v4l2_mbus_framefmt        format;
>>>>>> +
>>>>>> +     const struct csi2dc_format      *cur_fmt;
>>>>>> +     const struct csi2dc_format      *try_fmt;
>>>>>> +
>>>>>> +     struct media_pad                pads[CSI2DC_PADS_NUM];
>>>>>> +
>>>>>> +     bool                            clk_gated;
>>>>>> +     bool                            video_pipe;
>>>>>> +     u32                             vc;
>>>>>> +
>>>>>> +     struct v4l2_async_subdev        *asd;
>>>>>> +     struct v4l2_async_notifier      notifier;
>>>>>> +
>>>>>> +     struct v4l2_subdev              *input_sd;
>>>>>> +
>>>>>> +     u32                             remote_pad;
>>>>>> +};
>>>>>> +
>>>>>> +static void csi2dc_vp_update(struct csi2dc_device *csi2dc)
>>>>>
>>>>> Could you move this below closer to the only caller ?
>>>>>
>>>>>> +{
>>>>>> +     u32 vp;
>>>>>> +
>>>>>> +     if (!csi2dc->cur_fmt) {
>>>>>
>>>>> You should probably initialize this to a default format
>>>>>
>>>>>> +             dev_err(csi2dc->dev, "format must be configured first\n");
>>>>>> +             return;
>>>>>> +     }
>>>>>> +
>>>>>> +     if (!csi2dc->video_pipe) {
>>>>>
>>>>> This is only called internally by the driver at s_stream() time, can
>>>>> this happen ? Or rather won't you have a streamon call also when the
>>>>> data pipe only is available ? In that case you would error out here
>>>>
>>>> This can happen if the source node is not found in the DT. In that case,
>>>> there is no video pipe available, thus this whole function should be a
>>>> no-op (csi2dc_vp_update) . If no vp, there is no vp update.
>>>> Normall without a vp, there should be a dp (data pipe), but I haven't
>>>> tested this and it's not supported in the driver.
>>>> It doesn't make any sense to configure the vp registers below if the vp
>>>> is not available.
>>>> However, I shouldn't return an error since the vp is not mandatory for
>>>> the function of the hardware. Just that the dp will be implemented at a
>>>> later time (.... or never).
>>>>
>>>
>>> I see, so the data pipe is still work in progress... I understand
>>> bindings should allow for that to be later accepted so you cannot
>>> mandate port@1 to be there all the times, but should the driver
>>> instead refuse to operate if no port@1 is provided ?
>>>
>>
>> It will not refuse to operate, but won't operate anything basically.
>> And if I make it refuse to operate, it will violate the bindings, since
>> port@1 is not mandatory
> 
> I don't think that's a problem. I would say the it would also be fine
> to have port@1 mandatory as if it becomes optional in future once the
> data-pipe is implemented, old DTS will still be compatible.
> 
> However I also think it would be fine for the driver to not support
> all the options listed in the bindings. What's importante is that
> anything that gets into the bindings does not break
> retro-compatibility in case it gets extended.
> 
> To make a counter example, if your bindings define port@1 as optional
> but you later make it mandatory for some reason, old dts without
> port@1 won't work anymore and that's a problem. In this case if you
> make it mandatory and later demote it to be optional, old DTS will all
> have port@1 and will work in video-pipe mode while new ones can
> optionally swtich to data-pipe by not providing port@1.
> 
> Anyway, not a big deal, I think what you have here is fine but I won't
> bother too much about checking for if (!csi2dc->video_pipe) in the
> driver as if I understood you correctly the driver cannot currently
> be operated in data-pipe mode.
> 
>>
>>> Out of curiosity what will the interaction model with the driver be in
>>> data pipe mode ? There will be a video device but there won't be an
>>> link to it, how is stream start/stop controlled ?
>>
>> My idea is that on data pipe, the csi2dc should register it's own video
>> device and act basically as a dma engine entity that will use an
>> external DMA controller to just write data to DRAM.
>>
>> It is useful if we do not want to convert the MIPI/IDI packets into
>> pixels, but rather dump the to DRAM and have a piece of software further
>> use them.
>>
>> However, it's not a clear use case about this functionality at this
>> moment. One possibility is to be able to obtain stream meta-data from
>> the MIPI / IDI interface, like for example synchronization data, to
>> synchronize with an audio capture device.
>>
>>>
>>>>>
>>>>>> +             dev_err(csi2dc->dev, "video pipeline unavailable\n");
>>>>>> +             return;
>>>>>> +     }
>>>>>> +
>>>>>> +     vp = CSI2DC_VPCFG_DT(csi2dc->cur_fmt->dt) & CSI2DC_VPCFG_DT_MASK;
>>>>>> +     vp |= CSI2DC_VPCFG_VC(csi2dc->vc) & CSI2DC_VPCFG_VC_MASK;
>>>>>> +     vp &= ~CSI2DC_VPCFG_DE;
>>>>>> +     vp |= CSI2DC_VPCFG_DM(CSI2DC_VPCFG_DM_DECODER8TO12);
>>>>>> +     vp &= ~CSI2DC_VPCFG_DP2;
>>>>>> +     vp &= ~CSI2DC_VPCFG_RMS;
>>>>>> +     vp |= CSI2DC_VPCFG_PA;
>>>>>> +
>>>>>> +     csi2dc_writel(csi2dc, CSI2DC_VPCFG, vp);
>>>>>> +     csi2dc_writel(csi2dc, CSI2DC_VPE, CSI2DC_VPE_ENABLE);
>>>>>> +     csi2dc_writel(csi2dc, CSI2DC_PU, CSI2DC_PU_VP);
>>>>>> +}
>>>>>> +
>>>>>> +static inline struct csi2dc_device *
>>>>>> +csi2dc_sd_to_csi2dc_device(struct v4l2_subdev *csi2dc_sd)
>>>>>> +{
>>>>>> +     return container_of(csi2dc_sd, struct csi2dc_device, csi2dc_sd);
>>>>>> +}
>>>>>> +
>>>>>> +static int csi2dc_enum_mbus_code(struct v4l2_subdev *csi2dc_sd,
>>>>>> +                              struct v4l2_subdev_state *sd_state,
>>>>>> +                              struct v4l2_subdev_mbus_code_enum *code)
>>>>>> +{
>>>>>> +     if (code->index >= ARRAY_SIZE(csi2dc_formats))
>>>>>> +             return -EINVAL;
>>>>>> +
>>>>>> +     code->code = csi2dc_formats[code->index].mbus_code;
>>>>>> +
>>>>>> +     return 0;
>>>>>> +}
>>>>>> +
>>>>>> +static int csi2dc_get_fmt(struct v4l2_subdev *csi2dc_sd,
>>>>>> +                       struct v4l2_subdev_state *sd_state,
>>>>>> +                       struct v4l2_subdev_format *format)
>>>>>> +{
>>>>>> +     struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
>>>>>> +
>>>>>> +     format->format = csi2dc->format;
>>>>>> +
>>>>>> +     return 0;
>>>>>
>>>>> You should support try formats by storing the format in the file
>>>>> handle state in s_fmt and return it in case which == TRY
>>>>>
>>>>> Grep for v4l2_subdev_get_try_format() for usage examples
>>>>
>>>> I did that initially, but could not find any utility for it in my tests.
>>>
>>> I understand.. Have you run v4l2-compliance on this sudev ? Doesn't it
>>> test try formats support ?

Hi Jacopo,

Coming back to this, I tried to run v4l2-compliance, but,


Format ioctls:
         test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
         test VIDIOC_G/S_PARM: OK (Not Supported)
         test VIDIOC_G_FBUF: OK (Not Supported)
         test VIDIOC_G_FMT: OK (Not Supported)
         test VIDIOC_TRY_FMT: OK (Not Supported)
         test VIDIOC_S_FMT: OK (Not Supported)
         test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
         test Cropping: OK (Not Supported)
         test Composing: OK (Not Supported)
         test Scaling: OK (Not Supported)


It looks like it;s not attempted.
I have another subdev and for that one it works, it tries.
Maybe this subdev is configured in a different way (bridge) ?
Or why is it that v4l2-compliance does not try to use try formats for it ?
(that's one reason why I did not find issues for formats, as it looks 
g/s fmt is not tested either )

Thanks!
Eugen
>>>
>>>> So I removed it.
>>>> I will try to bring it back, but I have no idea how to test that the try
>>>> format works fine or not
>>>>
>>>
>>> I think also v4l2-ctl supports try formats.
>>>
>>> --try-subdev-fmt pad=<pad>,width=<w>,height=<h>,code=<code>,field=<f>,colorspace=<c>,
>>>
>>> I'm not sure if it allows you to read it back, but it's easy to hack
>>> it out just to verify it.
>>>
>>
>> Ok, I will try it
>>
>>>>>
>>>>>> +}
>>>>>> +
>>>>>> +static int csi2dc_set_fmt(struct v4l2_subdev *csi2dc_sd,
>>>>>> +                       struct v4l2_subdev_state *sd_state,
>>>>>> +                       struct v4l2_subdev_format *req_fmt)
>>>>>> +{
>>>>>> +     struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
>>>>>> +     const struct csi2dc_format *fmt;
>>>>>> +     int i;
>>>>>
>>>>> unsigned
>>>>>
>>>>>> +
>>>>>> +     for (i = 0; i < ARRAY_SIZE(csi2dc_formats);  i++) {
>>>>>> +             fmt = &csi2dc_formats[i];
>>>>>> +             if (req_fmt->format.code == fmt->mbus_code)
>>>>>> +                     csi2dc->try_fmt = fmt;
>>>>>
>>>>> Shouldn't you break ?
>>>>>
>>>>>> +             fmt++;
>>>>>> +     }
>>>>>
>>>>> And make this a simpler
>>>>>
>>>>>            for (i = 0; i < ARRAY_SIZE(csi2dc_formats); i++) {
>>>>>                    if (req_fmt->format.code == csi2dc_formats[i].mbus_code)
>>>>>                            break;
>>>>>            }
>>>>>
>>>>>            if (i == ARRAY_SIZE(csi2dc_formats)
>>>>>                    i = 0;
>>>>>
>>>>>> +
>>>>>> +     /* in case we could not find the desired format, default to something */
>>>>>> +     if (!csi2dc->try_fmt  ||
>>>>>> +         req_fmt->format.code != csi2dc->try_fmt->mbus_code) {
>>>>>> +             csi2dc->try_fmt = &csi2dc_formats[0];
>>>>>> +
>>>>>> +             dev_dbg(csi2dc->dev,
>>>>>> +                     "CSI2DC unsupported format 0x%x, defaulting to 0x%x\n",
>>>>>> +                     req_fmt->format.code, csi2dc_formats[0].mbus_code);
>>>>>> +
>>>>>> +             req_fmt->format.code = csi2dc_formats[0].mbus_code;
>>>>>> +     }
>>>>>> +
>>>>>> +     req_fmt->format.colorspace = V4L2_COLORSPACE_SRGB;
>>>>>> +     req_fmt->format.field = V4L2_FIELD_NONE;
>>>>>> +
>>>>>> +     /* save the format for later requests */
>>>>>
>>>>> You should support TRY formats
>>>>>
>>>>>> +     csi2dc->format = req_fmt->format;
>>>>>> +
>>>>>> +     /* if we are just trying, we are done */
>>>>>> +     if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY)
>>>>>> +             return 0;
>>>>>> +
>>>>>> +     csi2dc->cur_fmt = csi2dc->try_fmt;
>>>>>
>>>>>            csi2dc->cur_fmt = &csi2dc_format[i];
>>>>>
>>>>> So you can drop the try_fmt from the driver structure as it seems to
>>>>> be used as a temporary variable here only.
>>>>>
>>>>>> +
>>>>>> +     dev_dbg(csi2dc->dev, "new format set: 0x%x\n", req_fmt->format.code);
>>>>>> +
>>>>>> +     return 0;
>>>>>> +}
>>>>>> +
>>>>>> +static int csi2dc_power(struct csi2dc_device *csi2dc, int on)
>>>>>> +{
>>>>>> +     int ret = 0;
>>>>>> +
>>>>>> +     if (on) {
>>>>>> +             ret = clk_prepare_enable(csi2dc->pclk);
>>>>>> +             if (ret) {
>>>>>> +                     dev_err(csi2dc->dev, "failed to enable pclk: %d\n", ret);
>>>>>> +                     return ret;
>>>>>> +             }
>>>>>> +
>>>>>> +             ret = clk_prepare_enable(csi2dc->scck);
>>>>>> +             if (ret)
>>>>>> +                     dev_err(csi2dc->dev,
>>>>>> +                             "failed to enable scck: %d\n", ret);
>>>>>
>>>>> Shouldn't you bail out here ?
>>>>
>>>> Initially this clock was not mandatory, but you are right, I should bail
>>>> out.
>>>>
>>>>>
>>>>>> +
>>>>>> +             /* if powering up, deassert reset line */
>>>>>> +             csi2dc_writel(csi2dc, CSI2DC_GCTLR, CSI2DC_GCTLR_SWRST);
>>>>>> +     } else {
>>>>>> +             clk_disable_unprepare(csi2dc->scck);
>>>>>> +
>>>>>> +             /* if powering down, assert reset line */
>>>>>> +             csi2dc_writel(csi2dc, CSI2DC_GCTLR, !CSI2DC_GCTLR_SWRST);
>>>>>
>>>>> Isn't reverse order of activation better ?
>>>>>
>>>>>                    csi2dc_writel(..)
>>>>>                    clk_disable_unprepare(..)
>>>>>                    clk_disable_unprepare(..)
>>>>>> +
>>>>>> +             clk_disable_unprepare(csi2dc->pclk);
>>>>>> +     }
>>>>>> +
>>>>>> +     return ret;
>>>>>> +}
>>>>>> +
>>>>>> +static int csi2dc_s_stream(struct v4l2_subdev *csi2dc_sd, int enable)
>>>>>> +{
>>>>>> +     struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
>>>>>> +     int ret;
>>>>>> +
>>>>>> +     if (enable) {
>>>>>> +             ret = pm_runtime_resume_and_get(csi2dc->dev);
>>>>>> +             if (ret < 0)
>>>>>> +                     return ret;
>>>>>> +             csi2dc_vp_update(csi2dc);
>>>>>> +     } else {
>>>>>> +             pm_runtime_put_sync(csi2dc->dev);
>>>>>> +     }
>>>>>> +
>>>>>> +     return v4l2_subdev_call(csi2dc->input_sd, video, s_stream, enable);
>>>>>
>>>>> Should the remote subdev be started before and stopped after ?
>>>>
>>>> Do you mean s_power ?
>>>>
>>>
>>> Nope, I meant the operation order.
>>>
>>>           if (enable) {
>>>                   v4l2_subdev_call(...);
>>>
>>>                   pm_runtime_resume_and_get();
>>>                   csi2dc_vp_update();
>>>           } else {
>>>                   pm_runtime_put_sync();
>>>
>>>                   v4l2_subdev_call();
>>>           }
>>>
>>> However, my comment is clearly wrong, it should have been the
>>> contrary. Sorry about this
>>>
>>> It's right to configure the interface and power if up -before-
>>> starting the remote subdev, but I would stop it before powering it
>>> down. Like in:
>>>
>>>           if (enable) {
>>>                   pm_runtime_resume_and_get();
>>>                   csi2dc_vp_update();
>>>
>>>                   v4l2_subdev_call(1);
>>>           } else {
>>>                   v4l2_subdev_call(0);
>>>
>>>                   pm_runtime_put_sync();
>>>           }
>>>
>>> What do you think ? Stopping the remote before powering the interface
>>> down ensures no data is being put on the bus while the interace is
>>> powered off.
>>>
>>
>> In this case it might not work like this. If I start the sensor
>> streaming before the vp update , then, there will be frames received on
>> video pipe update, which means that the csi2dc will not detect the phy
>> STOP state reached, and it will fail initialization.
>> I believe the csi2dc has to be started before the sensor's first frame
>> such that the interface is prepared for receiving. The PHY must be in
>> stop state for a certain amount of time.
>> I will try to see if it works the other way around, to double check to
>> make sure.
>> But if it doesn't , then the VP must be configured while the whole
>> pipeline is in stop state.
>>
> 
> I agree, as I've said my first comment was wrong and the sensor has to
> be started after the csi2dc. However in my last comment I was
> suggesting to stop it before the csi2dc to avoid having data being
> sent on the bus while the csi2dc gets powered down.
> 
>>>>>
>>>>>> +}
>>>>>> +
>>>>>> +static const struct v4l2_subdev_pad_ops csi2dc_pad_ops = {
>>>>>> +     .enum_mbus_code = csi2dc_enum_mbus_code,
>>>>>> +     .set_fmt = csi2dc_set_fmt,
>>>>>> +     .get_fmt = csi2dc_get_fmt,
>>>>>> +};
>>>>>> +
>>>>>> +static const struct v4l2_subdev_video_ops csi2dc_video_ops = {
>>>>>> +     .s_stream = csi2dc_s_stream,
>>>>>> +};
>>>>>> +
>>>>>> +static const struct v4l2_subdev_ops csi2dc_subdev_ops = {
>>>>>> +     .pad = &csi2dc_pad_ops,
>>>>>> +     .video = &csi2dc_video_ops,
>>>>>> +};
>>>>>> +
>>>>>> +static int csi2dc_get_mbus_config(struct csi2dc_device *csi2dc)
>>>>>> +{
>>>>>> +     struct v4l2_mbus_config mbus_config = { 0 };
>>>>>> +     int ret;
>>>>>> +
>>>>>> +     ret = v4l2_subdev_call(csi2dc->input_sd, pad, get_mbus_config,
>>>>>> +                            csi2dc->remote_pad, &mbus_config);
>>>>>> +     if (ret == -ENOIOCTLCMD) {
>>>>>> +             dev_dbg(csi2dc->dev,
>>>>>> +                     "no remote mbus configuration available\n");
>>>>>> +             goto csi2dc_get_mbus_config_defaults;
>>>>>> +     }
>>>>>> +
>>>>>> +     if (ret) {
>>>>>> +             dev_err(csi2dc->dev,
>>>>>> +                     "failed to get remote mbus configuration\n");
>>>>>> +             goto csi2dc_get_mbus_config_defaults;
>>>>>> +     }
>>>>>> +
>>>>>> +     if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_0)
>>>>>> +             csi2dc->vc = 0;
>>>>>> +     else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_1)
>>>>>> +             csi2dc->vc = 1;
>>>>>> +     else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_2)
>>>>>> +             csi2dc->vc = 2;
>>>>>> +     else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_3)
>>>>>> +             csi2dc->vc = 3;
>>>>>> +
>>>>>> +     dev_dbg(csi2dc->dev, "subdev sending on channel %d\n", csi2dc->vc);
>>>>>> +
>>>>>> +     csi2dc->clk_gated = mbus_config.flags &
>>>>>> +                             V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK;
>>>>>
>>>>> This should come from the default clock-noncontinuous property in the
>>>>> endpoint. It is available in the mbus_configuration only to support
>>>>> subdevs that can change it at runtime, and if that's the case it's ok,
>>>>> but I think it should be in the endpoint.
>>>>
>>>> I do not completely understand your point. In a previous version which
>>>> you reviewed, I had this as a DT property (so in the endpoint ?), and
>>>> now I converted it to a get_mbus_config flag get operation.
>>>
>>> I re-read the conversation and I think you're right.
>>>
>>>> It's not the right way to do it ?
>>>> How should it be specified in the endpoint ?
>>>>
>>>
>>> There's a default property 'clock-noncontinuous' which gets parsed by
>>> v4l2_fwnode_endpoint_parse() and reported to drivers through the
>>> V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK and V4L2_MBUS_CSI2_CONTINUOUS_CLOCK
>>> flags. In your case, as you fetch it at run time it's not required if
>>> not to initialize a default according to the property presence in DT.
>>>
>>> Up to you, what you have here it's fine!
>>>
>>
>> Speaking of this, I have a parallel sensor that does not implement
>> get_mbus_config .
>>
>> I tried to implement this in the sensor driver, to see if the csi2dc
>> receives correct information from the sensor subdev (to see if it's
>> parallel or serial). And it works fine.
>> I sent a patch for this, but Sakari says that the csi2dc can obtain all
>> this information from DT, and I guess he is right. What would be the
>> correct way to do it then ?
> 
> Good question, and probably Sakari is in a better position than me to
> set a direction.
> 
> My take is that get_mbus_config should be used to retrieve from the remote
> subdevice configuration parameters that can change at runtime, or to
> express properties that cannot be defined in bindings as they're not
> strictly media bus configuration parameters (like the CSI-2 VC, as
> you're doing here). To give you another example, I used
> get_mbus_config to retrieve at run-time the number of CSI-2 data lanes
> currently in use, as I have a transmitter that if configured to output
> low resolution data needs to reduce the number of data lanes to stay
> in the D-PHY defined clock frequency lower bound (I think that's the
> use case that lead to resurect get_mbus_config() as a pad operation,
> as in the past there was a g_mbus_config() video operation that got
> deprecated along with the soc_camera framework).
> 
> The noncontinuous clock option cannot be changed at runtime afaict and
> it's indeed an mbus configuration parameter for which a standard
> DT-property is defined so it makes sense to have it in DTS. The clear
> win is that your receiver driver can operate with sensor drivers not
> instrumented with get_mbus_config() as it will get the info from DTS.
> 
> In your case you require get_mbus_config() for VC handling, something
> everyone hopes is temporary as long as the in-development multiplexed
> stream support has not landed, but if the operation is not implemented
> you fallback to a default which is indeed the best option for VC, but
> for clock-continuous you could avoid that by using the standard DTS
> mechanism and retain compatibility with subdevices that do not
> implement get_mbus_config.
> 
> I know in v3 you have been suggested to use get_mbus_config and I'm
> sorry if it drove you off-road, but I think the best option here is:
> 
> - get the clock continuous property from DTS
> - as you need get_mbus_config() for VC, inspect if the remote
>    reports an option for clock continuous and use it if that's the case
> - In case the subdev does not implement get_mbus_config() or does not
>    report clock information use the value parsed from DTS instead of an
>    arbitrary default
> 
> This should allow you to operate with sensor drivers not instrumented
> with get_mbus_config() (as long as they send data on the VC you use as
> a defaul fallback :)
> 
>>
>> Could you comment on this patch please :
>>
>> https://lore.kernel.org/linux-media/YYJDcIiBXo%2FXlKCX@paasikivi.fi.intel.com/T/#m990b903ed6d56c3c34232c54c251490ad33e92c9
>>
> 
> Afaict all the mbus configuration for a parallel bus can be fully
> described in DTS, and for the above reasons of being compatible with
> most subdev drivers I would fetch it from there.
> 
>>
>> Thanks again for reviewing !
>>
> 
> You're welcome!
> 
>>>>>
>>>>> Speaking of remote subdevs, is there any driver available for IDI
>>>>> transmitters ?
>>>>
>>>> I am not aware of any at the moment
>>>>>
>>>>>> +
>>>>>> +     dev_dbg(csi2dc->dev, "%s clock\n",
>>>>>> +             csi2dc->clk_gated ? "gated" : "free running");
>>>>>> +
>>>>>> +     return 0;
>>>>>> +
>>>>>> +csi2dc_get_mbus_config_defaults:
>>>>>> +     csi2dc->vc = 0; /* Virtual ID 0 by default */
>>>>>> +     csi2dc->clk_gated = false; /* Free running clock by default */
>>>>>> +
>>>>>> +     return 0;
>>>>>> +}
>>>>>> +
>>>>>> +static int csi2dc_async_bound(struct v4l2_async_notifier *notifier,
>>>>>> +                           struct v4l2_subdev *subdev,
>>>>>> +                           struct v4l2_async_subdev *asd)
>>>>>> +{
>>>>>> +     struct csi2dc_device *csi2dc = container_of(notifier,
>>>>>> +                                             struct csi2dc_device, notifier);
>>>>>> +     int pad;
>>>>>> +     int ret;
>>>>>> +
>>>>>> +     csi2dc->input_sd = subdev;
>>>>>> +
>>>>>> +     pad = media_entity_get_fwnode_pad(&subdev->entity,
>>>>>
>>>>> You can use 'ret'
>>>>
>>>> I can, but it is weird to do remote_pad = ret ...
>>>>
>>>> May bring confusion, don't you agree ?
>>>>
>>>
>>> Up to you, it's a small detail :)
>>>
>>> Thanks!
>>>      j
>>>
>>>>>
>>>>>> +                                       asd->match.fwnode,
>>>>>> +                                       MEDIA_PAD_FL_SOURCE);
>>>>>> +     if (pad < 0) {
>>>>>> +             dev_err(csi2dc->dev, "Failed to find pad for %s\n",
>>>>>> +                     subdev->name);
>>>>>> +             return pad;
>>>>>> +     }
>>>>>> +
>>>>>> +     csi2dc->remote_pad = pad;
>>>>>> +
>>>>>> +     csi2dc_get_mbus_config(csi2dc);
>>>>>
>>>>> Ideally, as this is not fatal, you could move this at s_stream time to
>>>>> fetch the most up-to-date configuration
>>>>>
>>>>>> +
>>>>>> +     ret = media_create_pad_link(&csi2dc->input_sd->entity,
>>>>>> +                                 csi2dc->remote_pad,
>>>>>> +                                 &csi2dc->csi2dc_sd.entity, 0,
>>>>>> +                                 MEDIA_LNK_FL_ENABLED);
>>>>>> +     if (ret < 0) {
>>>>>
>>>>>            if (ret)
>>>>>
>>>>>> +             dev_err(csi2dc->dev,
>>>>>> +                     "Failed to create pad link: %s to %s\n",
>>>>>> +                     csi2dc->input_sd->entity.name,
>>>>>> +                     csi2dc->csi2dc_sd.entity.name);
>>>>>> +             return ret;
>>>>>> +     }
>>>>>> +
>>>>>> +     dev_dbg(csi2dc->dev, "link with %s pad: %d\n",
>>>>>> +             csi2dc->input_sd->name, csi2dc->remote_pad);
>>>>>> +
>>>>>> +     ret = pm_runtime_resume_and_get(csi2dc->dev);
>>>>>> +     if (ret < 0)
>>>>>> +             return ret;
>>>>>> +
>>>>>> +     csi2dc_writel(csi2dc, CSI2DC_GCFG,
>>>>>> +                   (SAMA7G5_HLC & CSI2DC_GCFG_HLC_MASK) |
>>>>>> +                   (csi2dc->clk_gated ? 0 : CSI2DC_GCFG_MIPIFRN));
>>>>>> +
>>>>>> +     csi2dc_writel(csi2dc, CSI2DC_VPCOL,
>>>>>> +                   CSI2DC_VPCOL_COL(0xFFF) & CSI2DC_VPCOL_COL_MASK);
>>>>>> +     csi2dc_writel(csi2dc, CSI2DC_VPROW,
>>>>>> +                   CSI2DC_VPROW_ROW(0xFFF) & CSI2DC_VPROW_ROW_MASK);
>>>>>> +
>>>>>> +     pm_runtime_put_sync(csi2dc->dev);
>>>>>
>>>>> I would really move access to the HW to s_stream time if possible
>>>>>
>>>>>> +
>>>>>> +     return ret;
>>>>>> +}
>>>>>> +
>>>>>> +static const struct v4l2_async_notifier_operations csi2dc_async_ops = {
>>>>>> +     .bound = csi2dc_async_bound,
>>>>>> +};
>>>>>> +
>>>>>> +static void csi2dc_cleanup_notifier(struct csi2dc_device *csi2dc)
>>>>>> +{
>>>>>> +     v4l2_async_notifier_unregister(&csi2dc->notifier);
>>>>>> +     v4l2_async_notifier_cleanup(&csi2dc->notifier);
>>>>>> +}
>>>>>> +
>>>>>> +static int csi2dc_prepare_notifier(struct csi2dc_device *csi2dc,
>>>>>> +                                struct device_node *input_node)
>>>>>> +{
>>>>>> +     int ret = 0;
>>>>>> +
>>>>>> +     v4l2_async_notifier_init(&csi2dc->notifier);
>>>>>> +
>>>>>> +     csi2dc->asd = v4l2_async_notifier_add_fwnode_remote_subdev
>>>>>
>>>>> do you need asd in the driver structure ?
>>>>>
>>>>>> +                     (&csi2dc->notifier, of_fwnode_handle(input_node),
>>>>>> +                     struct v4l2_async_subdev);
>>>>>> +
>>>>>> +     of_node_put(input_node);
>>>>>> +
>>>>>> +     if (IS_ERR(csi2dc->asd)) {
>>>>>> +             ret = PTR_ERR(csi2dc->asd);
>>>>>> +             dev_err(csi2dc->dev,
>>>>>> +                     "failed to add async notifier for node %pOF: %d\n",
>>>>>> +                     input_node, ret);
>>>>>> +             v4l2_async_notifier_cleanup(&csi2dc->notifier);
>>>>>> +             return ret;
>>>>>> +     }
>>>>>> +
>>>>>> +     csi2dc->notifier.ops = &csi2dc_async_ops;
>>>>>> +
>>>>>> +     ret = v4l2_async_subdev_notifier_register(&csi2dc->csi2dc_sd,
>>>>>> +                                               &csi2dc->notifier);
>>>>>> +
>>>>>> +     if (ret) {
>>>>>> +             dev_err(csi2dc->dev, "fail to register async notifier: %d\n",
>>>>>> +                     ret);
>>>>>> +             v4l2_async_notifier_cleanup(&csi2dc->notifier);
>>>>>> +     }
>>>>>> +
>>>>>> +     return ret;
>>>>>> +}
>>>>>> +
>>>>>> +static int csi2dc_of_parse(struct csi2dc_device *csi2dc,
>>>>>> +                        struct device_node *of_node)
>>>>>> +{
>>>>>> +     struct device_node *input_node, *output_node;
>>>>>> +     struct v4l2_fwnode_endpoint input_endpoint = { 0 },
>>>>>> +                                 output_endpoint = { 0 };
>>>>>> +     int ret;
>>>>>> +
>>>>>> +     output_endpoint.bus_type = V4L2_MBUS_PARALLEL;
>>>>>> +
>>>>>> +     input_node = of_graph_get_next_endpoint(of_node, NULL);
>>>>>> +
>>>>>> +     if (!input_node) {
>>>>>> +             dev_err(csi2dc->dev,
>>>>>> +                     "missing port node at %pOF, input node is mandatory.\n",
>>>>>> +                     of_node);
>>>>>> +             return -EINVAL;
>>>>>> +     }
>>>>>> +
>>>>>> +     ret = v4l2_fwnode_endpoint_parse(of_fwnode_handle(input_node),
>>>>>> +                                      &input_endpoint);
>>>>>> +
>>>>>> +     if (ret) {
>>>>>                    of_node_put(input_node);
>>>>>
>>>>>> +             dev_err(csi2dc->dev, "endpoint not defined at %pOF\n", of_node);
>>>>>> +             return ret;
>>>>>> +     }
>>>>>> +
>>>>>> +     output_node = of_graph_get_next_endpoint(of_node, input_node);
>>>>>> +
>>>>>> +     if (output_node)
>>>>>> +             ret = v4l2_fwnode_endpoint_parse(of_fwnode_handle(output_node),
>>>>>> +                                              &output_endpoint);
>>>>>
>>>>>            of_node_put(output_node);
>>>>>> +
>>>>>> +     if (!output_node || ret) {
>>>>>> +             dev_info(csi2dc->dev,
>>>>>> +                      "missing output node at %pOF, data pipe available only.\n",
>>>>>> +                      of_node);
>>>>>> +     } else {
>>>>>> +             csi2dc->video_pipe = true;
>>>>>> +
>>>>>> +             dev_dbg(csi2dc->dev, "block %pOF %d.%d->%d.%d video pipeline\n",
>>>>>> +                     of_node, input_endpoint.base.port,
>>>>>> +                     input_endpoint.base.id, output_endpoint.base.port,
>>>>>> +                     output_endpoint.base.id);
>>>>>> +     }
>>>>>> +
>>>>>> +     of_node_put(output_node);
>>>>>
>>>>> Drop this if you put it above
>>>>>
>>>>>> +     of_node_put(input_node);
>>>>>
>>>>> Should you put input_node before passing it to the function ?
>>>>>> +
>>>>>> +     /* prepare async notifier for subdevice completion */
>>>>>> +     return csi2dc_prepare_notifier(csi2dc, input_node);
>>>>>> +}
>>>>>> +
>>>>>> +static int csi2dc_probe(struct platform_device *pdev)
>>>>>> +{
>>>>>> +     struct device *dev = &pdev->dev;
>>>>>> +     struct csi2dc_device *csi2dc;
>>>>>> +     struct resource *res = NULL;
>>>>>> +     int ret = 0;
>>>>>> +     u32 ver;
>>>>>> +
>>>>>> +     csi2dc = devm_kzalloc(dev, sizeof(*csi2dc), GFP_KERNEL);
>>>>>> +     if (!csi2dc)
>>>>>> +             return -ENOMEM;
>>>>>> +
>>>>>> +     csi2dc->dev = dev;
>>>>>> +
>>>>>> +     res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
>>>>>> +
>>>>>> +     csi2dc->base = devm_ioremap_resource(dev, res);
>>>>>
>>>>> Should devm_platform_ioremap_resource(pdev, 0) be used here ?
>>>>>
>>>>>> +     if (IS_ERR(csi2dc->base)) {
>>>>>> +             dev_err(dev, "base address not set\n");
>>>>>> +             return PTR_ERR(csi2dc->base);
>>>>>> +     }
>>>>>> +
>>>>>> +     csi2dc->pclk = devm_clk_get(dev, "pclk");
>>>>>> +     if (IS_ERR(csi2dc->pclk)) {
>>>>>> +             ret = PTR_ERR(csi2dc->pclk);
>>>>>> +             dev_err(dev, "failed to get pclk: %d\n", ret);
>>>>>> +             return ret;
>>>>>> +     }
>>>>>> +
>>>>>> +     csi2dc->scck = devm_clk_get(dev, "scck");
>>>>>> +     if (IS_ERR(csi2dc->scck)) {
>>>>>> +             ret = PTR_ERR(csi2dc->scck);
>>>>>> +             dev_err(dev, "failed to get scck: %d\n", ret);
>>>>>> +             return ret;
>>>>>> +     }
>>>>>> +
>>>>>> +     v4l2_subdev_init(&csi2dc->csi2dc_sd, &csi2dc_subdev_ops);
>>>>>> +
>>>>>> +     csi2dc->csi2dc_sd.owner = THIS_MODULE;
>>>>>> +     csi2dc->csi2dc_sd.dev = dev;
>>>>>> +     snprintf(csi2dc->csi2dc_sd.name, sizeof(csi2dc->csi2dc_sd.name),
>>>>>> +              "csi2dc");
>>>>>> +
>>>>>> +     csi2dc->csi2dc_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
>>>>>> +     csi2dc->csi2dc_sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
>>>>>> +     v4l2_set_subdevdata(&csi2dc->csi2dc_sd, pdev);
>>>>>
>>>>> Not used it seems
>>>>>
>>>>>> +
>>>>>> +     platform_set_drvdata(pdev, csi2dc);
>>>>>> +
>>>>>> +     ret = csi2dc_of_parse(csi2dc, dev->of_node);
>>>>>> +     if (ret)
>>>>>> +             goto csi2dc_probe_cleanup_entity;
>>>>>> +
>>>>>> +     csi2dc->pads[CSI2DC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>>>>>> +     if (csi2dc->video_pipe)
>>>>>> +             csi2dc->pads[CSI2DC_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
>>>>>> +
>>>>>> +     ret = media_entity_pads_init(&csi2dc->csi2dc_sd.entity,
>>>>>> +                                  csi2dc->video_pipe ? CSI2DC_PADS_NUM : 1,
>>>>>> +                                  csi2dc->pads);
>>>>>> +     if (ret < 0) {
>>>>>> +             dev_err(dev, "media entity init failed\n");
>>>>>> +             goto csi2dc_probe_cleanup_entity;
>>>>>
>>>>> Should you also clean up the notifier in the error path ?
>>>>>
>>>>>> +     }
>>>>>> +
>>>>>> +     /* turn power on to validate capabilities */
>>>>>> +     ret = csi2dc_power(csi2dc, true);
>>>>>> +     if (ret < 0)
>>>>>> +             goto csi2dc_probe_cleanup_entity;
>>>>>> +
>>>>>> +     pm_runtime_set_active(dev);
>>>>>> +     pm_runtime_enable(dev);
>>>>>> +     ver = csi2dc_readl(csi2dc, CSI2DC_VERSION);
>>>>>> +     pm_request_idle(dev);
>>>>>> +
>>>>>> +     /*
>>>>>> +      * we must register the subdev after PM runtime has been requested,
>>>>>> +      * otherwise we might bound immediately and request pm_runtime_resume
>>>>>> +      * before runtime_enable.
>>>>>> +      */
>>>>>> +     ret = v4l2_async_register_subdev(&csi2dc->csi2dc_sd);
>>>>>> +     if (ret) {
>>>>>> +             dev_err(csi2dc->dev, "failed to register the subdevice\n");
>>>>>> +             goto csi2dc_probe_cleanup_entity;
>>>>>> +     }
>>>>>> +
>>>>>> +     dev_info(dev, "Microchip CSI2DC version %x\n", ver);
>>>>>> +
>>>>>> +     return 0;
>>>>>> +
>>>>>> +csi2dc_probe_cleanup_entity:
>>>>>> +     media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
>>>>>> +
>>>>>> +     return ret;
>>>>>> +}
>>>>>> +
>>>>>> +static int csi2dc_remove(struct platform_device *pdev)
>>>>>> +{
>>>>>> +     struct csi2dc_device *csi2dc = platform_get_drvdata(pdev);
>>>>>> +
>>>>>> +     pm_runtime_disable(&pdev->dev);
>>>>>> +
>>>>>> +     v4l2_async_unregister_subdev(&csi2dc->csi2dc_sd);
>>>>>> +     csi2dc_cleanup_notifier(csi2dc);
>>>>>> +     media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
>>>>>> +
>>>>>> +     return 0;
>>>>>> +}
>>>>>> +
>>>>>> +static int __maybe_unused csi2dc_runtime_suspend(struct device *dev)
>>>>>> +{
>>>>>> +     struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
>>>>>> +
>>>>>> +     return csi2dc_power(csi2dc, false);
>>>>>> +}
>>>>>> +
>>>>>> +static int __maybe_unused csi2dc_runtime_resume(struct device *dev)
>>>>>> +{
>>>>>> +     struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
>>>>>> +
>>>>>> +     return csi2dc_power(csi2dc, true);
>>>>>> +}
>>>>>> +
>>>>>> +static const struct dev_pm_ops csi2dc_dev_pm_ops = {
>>>>>> +     SET_RUNTIME_PM_OPS(csi2dc_runtime_suspend, csi2dc_runtime_resume, NULL)
>>>>>> +};
>>>>>> +
>>>>>> +static const struct of_device_id csi2dc_of_match[] = {
>>>>>> +     { .compatible = "microchip,sama7g5-csi2dc" },
>>>>>> +     { }
>>>>>> +};
>>>>>> +
>>>>>> +MODULE_DEVICE_TABLE(of, csi2dc_of_match);
>>>>>> +
>>>>>> +static struct platform_driver csi2dc_driver = {
>>>>>> +     .probe  = csi2dc_probe,
>>>>>> +     .remove = csi2dc_remove,
>>>>>> +     .driver = {
>>>>>> +             .name =                 "microchip-csi2dc",
>>>>>> +             .pm =                   &csi2dc_dev_pm_ops,
>>>>>> +             .of_match_table =       of_match_ptr(csi2dc_of_match),
>>>>>> +     },
>>>>>> +};
>>>>>> +
>>>>>> +module_platform_driver(csi2dc_driver);
>>>>>> +
>>>>>> +MODULE_AUTHOR("Eugen Hristev <eugen.hristev@microchip.com>");
>>>>>> +MODULE_DESCRIPTION("Microchip CSI2 Demux Controller driver");
>>>>>> +MODULE_LICENSE("GPL v2");
>>>>>> --
>>>>>> 2.25.1
>>>>>>
>>>>
>>