[PATCH v6 00/25] rcar-vin: Add Gen3 with media controller

[PATCH v6 00/25] rcar-vin: Add Gen3 with media controller

[Niklas Söderlund]

Hi,

This series adds Gen3 VIN support to rcar-vin driver for Renesas r8a7795 
and r8a7796. It is based on the media-tree.

The driver is tested on both Renesas H3 (r8a7795, ES1.x + ES2.0) and 
M3-W (r8a7796) together with the rcar-csi2 driver (posted separately and 
not yet upstream) and the Salvator-X onboard ADV7482.

It is possible to capture both CVBS and HDMI video streams, 
v4l2-compliance passes with no errors and media-ctl can be used to 
change the routing and formats for the different entities in the media 
graph.

Gen2 compatibility is verified on Koelsch and no problems where found, video
can be captured just like before and v4l2-compliance passes without errors or
warnings just like before this series.

I have started on a very basic test suite for the VIN driver at:

  https://git.ragnatech.se/vin-tests

And as before the state of the driver and information about how to test it can
be found on the elinux wiki:

  http://elinux.org/R-Car/Tests:rcar-vin

* Changes since v5
- Extract and make use of common format checking for both Gen2 and Gen3.
- Assign pad at declaration time in rvin_get_sd_format()
- Always call pm_runtime_{get_sync,put}() and v4l2_pipeline_pm_use() 
  when opening/closing a video device, remove the check of  
  v4l2_fh_is_singular_file().
- Make rvin_set_chsel() return void instead of int since it always
  return 0.
- Simplify the VIN group allocator functions.
- Make the group notifier callbacks and setup more robust.
- Moved the video device registration back to probe time.
- Add H3 ES2.0 support.
- Fix handling of single field formats (top, bottom, alternate) as this 
  was obviously wrong before but hidden by the Gen2 scaler support.
- Added review tags from Kieran.

* Changes since v4 (Not posted to ML)
- Updated to the new fwnode functions.
- Moved the registration of the video devices to the async notification 
  callback.

* Changes since v3
- Only add neighboring subdevices to the async notifier. Instead of 
  parsing the whole OF graph depend on incremental async subnotifier to
  discover the whole pipeline. This is needed to support arbitrarily
  long graphs and support the new ADV7482 prototype driver which Kieran
  is working on.
- Fix warning from lockdep, reported by Kieran.
- Fix commit messages from feedback from Sergei, thanks.
- Fix chip info an OF device ids sorting order, thanks Geert.
- Use subdev->of_node instead of subdev->dev->of_node, thanks Kieran.

* Changes since v2
- Do not try to control the subdevices in the media graph from the rcar-vin
  driver. Have user-space configure to format in the pipeline instead.
- Add link validation before starting the stream.
- Rework on how the subdevices are and the video node behave by defining
  specific V4L2 operations for the MC mode of operation, this simplified the
  driver quit a bit, thanks Laurent!
- Add a new 'renesas,id' DT property which is needed to to be able to keep the
  VIN to CSI-2 routing table inside the driver. Previously this information was
  taken from the CSI-2 DT node which is obviusly the wrong way to do things.
  Thanks Laurent for pointing this out.
- Fixed a memory leek in the group allocator function.
- Return -EMLINK instead of -EBUSY if a MC link is not possible given the
  current routing setup.
- Add comments to clarify that the 4 channels from the CSI-2 node is not
  directly related to CSI-2 virtual channels, the CSI-2 node can output any VC
  on any of its output channels.

* Changes since v1
- Remove unneeded casts as pointed out by Geert.
- Fix spelling and DT documentation as pointed out by Geert and Sergei, thanks!
- Refresh patch 2/32 with an updated version, thanks Sakari for pointing this
  out.
- Add Sakaris Ack to patch 1/32.
- Rebase on top of v4.9-rc1 instead of v4.9-rc3 to ease integration testing
  together with renesas-drivers tree.


Niklas Söderlund (25):
  rcar-vin: add Gen3 devicetree bindings documentation
  rcar-vin: register the video device at probe time
  rcar-vin: move chip information to own struct
  rcar-vin: move max width and height information to chip information
  rcar-vin: change name of video device
  rcar-vin: move functions regarding scaling
  rcar-vin: all Gen2 boards can scale simplify logic
  rcar-vin: do not reset crop and compose when setting format
  rcar-vin: do not allow changing scaling and composing while streaming
  rcar-vin: read subdevice format for crop only when needed
  rcar-vin: fix handling of single field frames (top, bottom and
    alternate fields)
  rcar-vin: move media bus configuration to struct rvin_info
  rcar-vin: enable Gen3 hardware configuration
  rcar-vin: add functions to manipulate Gen3 CHSEL value
  rcar-vin: add flag to switch to media controller mode
  rcar-vin: break out format alignment and checking
  rcar-vin: use different v4l2 operations in media controller mode
  rcar-vin: prepare for media controller mode initialization
  rcar-vin: add group allocator functions
  rcar-vin: add chsel information to rvin_info
  rcar-vin: parse Gen3 OF and setup media graph
  rcar-vin: add link notify for Gen3
  rcar-vin: extend {start,stop}_streaming to work with media controller
  rcar-vin: enable support for r8a7795
  rcar-vin: enable support for r8a7796

 .../devicetree/bindings/media/rcar_vin.txt         |  107 +-
 drivers/media/platform/rcar-vin/Kconfig            |    2 +-
 drivers/media/platform/rcar-vin/rcar-core.c        | 1021 +++++++++++++++++++-
 drivers/media/platform/rcar-vin/rcar-dma.c         | 1000 +++++++++++--------
 drivers/media/platform/rcar-vin/rcar-v4l2.c        |  443 +++++----
 drivers/media/platform/rcar-vin/rcar-vin.h         |  119 ++-
 6 files changed, 2044 insertions(+), 648 deletions(-)

-- 
2.14.0

[PATCH v6 01/25] rcar-vin: add Gen3 devicetree bindings documentation

[Niklas Söderlund]

Document the devicetree bindings for the CSI-2 inputs available on Gen3.

There is a need to add a custom property 'renesas,id' and to define
which CSI-2 input is described in which endpoint under the port@1 node.
This information is needed since there are a set of predefined routes
between each VIN and CSI-2 block. This routing table will be kept
inside the driver but in order for it to act on it it must know which
VIN and CSI-2 is which.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 .../devicetree/bindings/media/rcar_vin.txt         | 106 +++++++++++++++++++--
 1 file changed, 96 insertions(+), 10 deletions(-)

diff --git a/Documentation/devicetree/bindings/media/rcar_vin.txt b/Documentation/devicetree/bindings/media/rcar_vin.txt
index 6e4ef8caf759e5d3..be38ad89d71ad05d 100644
--- a/Documentation/devicetree/bindings/media/rcar_vin.txt
+++ b/Documentation/devicetree/bindings/media/rcar_vin.txt
@@ -2,8 +2,12 @@ Renesas R-Car Video Input driver (rcar_vin)
 -------------------------------------------
 
 The rcar_vin device provides video input capabilities for the Renesas R-Car
-family of devices. The current blocks are always slaves and suppot one input
-channel which can be either RGB, YUYV or BT656.
+family of devices.
+
+On Gen2 the current blocks are always slaves and support one input channel
+which can be either RGB, YUYV or BT656. On Gen3 the current blocks are
+always slaves and support multiple input channels which can be either RGB,
+YUVU, BT656 or CSI-2.
 
  - compatible: Must be one or more of the following
    - "renesas,vin-r8a7795" for the R8A7795 device
@@ -28,7 +32,7 @@ channel which can be either RGB, YUYV or BT656.
 Additionally, an alias named vinX will need to be created to specify
 which video input device this is.
 
-The per-board settings:
+The per-board settings Gen2:
  - port sub-node describing a single endpoint connected to the vin
    as described in video-interfaces.txt[1]. Only the first one will
    be considered as each vin interface has one input port.
@@ -36,13 +40,21 @@ The per-board settings:
    These settings are used to work out video input format and widths
    into the system.
 
+The per-board settings Gen3:
+- renesas,id - ID number of the VIN
+- Port 0 - Digital video source (same as port node on Gen2)
+- Port 1 - CSI-2 video sources
+        - Endpoint 0 - sub-node describing the endpoint which is CSI20
+        - Endpoint 1 - sub-node describing the endpoint which is CSI21
+        - Endpoint 2 - sub-node describing the endpoint which is CSI40
+        - Endpoint 3 - sub-node describing the endpoint which is CSI41
 
-Device node example
--------------------
+Device node example Gen2
+------------------------
 
-	aliases {
-	       vin0 = &vin0;
-	};
+        aliases {
+                vin0 = &vin0;
+        };
 
         vin0: vin@0xe6ef0000 {
                 compatible = "renesas,vin-r8a7790", "renesas,rcar-gen2-vin";
@@ -52,8 +64,8 @@ Device node example
                 status = "disabled";
         };
 
-Board setup example (vin1 composite video input)
-------------------------------------------------
+Board setup example Gen2 (vin1 composite video input)
+-----------------------------------------------------
 
 &i2c2   {
         status = "ok";
@@ -92,6 +104,80 @@ Board setup example (vin1 composite video input)
         };
 };
 
+Device node example Gen3
+------------------------
+
+        vin0: video@e6ef0000 {
+                compatible = "renesas,vin-r8a7795";
+                reg = <0 0xe6ef0000 0 0x1000>;
+                interrupts = <GIC_SPI 188 IRQ_TYPE_LEVEL_HIGH>;
+                clocks = <&cpg CPG_MOD 811>;
+                power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
+                status = "disabled";
+
+                renesas,id = <0>;
+
+                ports {
+                        #address-cells = <1>;
+                        #size-cells = <0>;
+
+                        port@1 {
+                                #address-cells = <1>;
+                                #size-cells = <0>;
+
+                                reg = <1>;
+
+                                vin0csi20: endpoint@0 {
+                                        reg = <0>;
+                                        remote-endpoint= <&csi20vin0>;
+                                };
+                                vin0csi21: endpoint@1 {
+                                        reg = <1>;
+                                        remote-endpoint= <&csi21vin0>;
+                                };
+                                vin0csi40: endpoint@2 {
+                                        reg = <2>;
+                                        remote-endpoint= <&csi40vin0>;
+                                };
+                        };
+                };
+        };
+
+        csi20: csi2@fea80000 {
+                compatible = "renesas,r8a7795-csi2", "renesas,rcar-gen3-csi2";
+                reg = <0 0xfea80000 0 0x10000>;
+                interrupts = <GIC_SPI 184 IRQ_TYPE_LEVEL_HIGH>;
+                clocks = <&cpg CPG_MOD 714>;
+                power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
+                status = "disabled";
+
+                ports {
+                        #address-cells = <1>;
+                        #size-cells = <0>;
+
+                        port@0 {
+                        #address-cells = <1>;
+                        #size-cells = <0>;
+
+                                reg = <0>;
+                                csi20_in: endpoint@0 {
+                                        clock-lanes = <0>;
+                                        data-lanes = <1>;
+                                        remote-endpoint = <&adv7482_txb>;
+                                };
+                        };
 
+                        port@1 {
+                                #address-cells = <1>;
+                                #size-cells = <0>;
+
+                                reg = <1>;
+
+                                csi20vin0: endpoint@0 {
+                                        remote-endpoint = <&vin0csi20>;
+                                };
+                        };
+                };
+        };
 
 [1] video-interfaces.txt common video media interface
-- 
2.14.0

[PATCH v6 02/25] rcar-vin: register the video device at probe time

[Niklas Söderlund]

The driver registers the video device from the async complete callback
and unregistered in the async unbind callback. This creates problems if
if the subdevice is bound, unbound and later rebound. The second time
video_register_device() is called it fails:

   kobject (eb3be918): tried to init an initialized object, something is seriously wrong.

To prevent this register the video device at prob time and don't allow
user-space to open the video device if the subdevice have not yet been
bound.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/rcar-vin/rcar-core.c | 42 +++++++++++++++++++++++++++--
 drivers/media/platform/rcar-vin/rcar-v4l2.c | 42 +++++------------------------
 drivers/media/platform/rcar-vin/rcar-vin.h  |  1 +
 3 files changed, 47 insertions(+), 38 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index 77dff047c41c803e..aefbe8e3ccddb3e4 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -74,6 +74,7 @@ static bool rvin_mbus_supported(struct rvin_graph_entity *entity)
 static int rvin_digital_notify_complete(struct v4l2_async_notifier *notifier)
 {
 	struct rvin_dev *vin = notifier_to_vin(notifier);
+	struct v4l2_subdev *sd = vin_to_source(vin);
 	int ret;
 
 	/* Verify subdevices mbus format */
@@ -92,7 +93,35 @@ static int rvin_digital_notify_complete(struct v4l2_async_notifier *notifier)
 		return ret;
 	}
 
-	return rvin_v4l2_probe(vin);
+	/* Add the controls */
+	/*
+	 * Currently the subdev with the largest number of controls (13) is
+	 * ov6550. So let's pick 16 as a hint for the control handler. Note
+	 * that this is a hint only: too large and you waste some memory, too
+	 * small and there is a (very) small performance hit when looking up
+	 * controls in the internal hash.
+	 */
+	ret = v4l2_ctrl_handler_init(&vin->ctrl_handler, 16);
+	if (ret < 0)
+		return ret;
+
+	ret = v4l2_ctrl_add_handler(&vin->ctrl_handler, sd->ctrl_handler, NULL);
+	if (ret < 0)
+		return ret;
+
+	ret = v4l2_subdev_call(sd, video, g_tvnorms, &vin->vdev.tvnorms);
+	if (ret < 0 && ret != -ENOIOCTLCMD && ret != -ENODEV)
+		return ret;
+
+	if (vin->vdev.tvnorms == 0) {
+		/* Disable the STD API if there are no tvnorms defined */
+		v4l2_disable_ioctl(&vin->vdev, VIDIOC_G_STD);
+		v4l2_disable_ioctl(&vin->vdev, VIDIOC_S_STD);
+		v4l2_disable_ioctl(&vin->vdev, VIDIOC_QUERYSTD);
+		v4l2_disable_ioctl(&vin->vdev, VIDIOC_ENUMSTD);
+	}
+
+	return rvin_reset_format(vin);
 }
 
 static void rvin_digital_notify_unbind(struct v4l2_async_notifier *notifier,
@@ -102,7 +131,7 @@ static void rvin_digital_notify_unbind(struct v4l2_async_notifier *notifier,
 	struct rvin_dev *vin = notifier_to_vin(notifier);
 
 	vin_dbg(vin, "unbind digital subdev %s\n", subdev->name);
-	rvin_v4l2_remove(vin);
+	v4l2_ctrl_handler_free(&vin->ctrl_handler);
 	vin->digital.subdev = NULL;
 }
 
@@ -231,6 +260,10 @@ static int rvin_digital_graph_init(struct rvin_dev *vin)
 	vin->notifier.unbind = rvin_digital_notify_unbind;
 	vin->notifier.complete = rvin_digital_notify_complete;
 
+	ret = rvin_v4l2_probe(vin);
+	if (ret)
+		return ret;
+
 	ret = v4l2_async_notifier_register(&vin->v4l2_dev, &vin->notifier);
 	if (ret < 0) {
 		vin_err(vin, "Notifier registration failed\n");
@@ -314,6 +347,11 @@ static int rcar_vin_remove(struct platform_device *pdev)
 
 	v4l2_async_notifier_unregister(&vin->notifier);
 
+	/* Checks internaly if handlers have been init or not */
+	v4l2_ctrl_handler_free(&vin->ctrl_handler);
+
+	rvin_v4l2_remove(vin);
+
 	rvin_dma_remove(vin);
 
 	return 0;
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index dd37ea8116804df3..81ff59c3b4744075 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -103,7 +103,7 @@ static void rvin_reset_crop_compose(struct rvin_dev *vin)
 	vin->compose.height = vin->format.height;
 }
 
-static int rvin_reset_format(struct rvin_dev *vin)
+int rvin_reset_format(struct rvin_dev *vin)
 {
 	struct v4l2_subdev_format fmt = {
 		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
@@ -781,6 +781,11 @@ static int rvin_open(struct file *file)
 
 	mutex_lock(&vin->lock);
 
+	if (!vin->digital.subdev) {
+		ret = -ENODEV;
+		goto unlock;
+	}
+
 	file->private_data = vin;
 
 	ret = v4l2_fh_open(file);
@@ -844,9 +849,6 @@ void rvin_v4l2_remove(struct rvin_dev *vin)
 	v4l2_info(&vin->v4l2_dev, "Removing %s\n",
 		  video_device_node_name(&vin->vdev));
 
-	/* Checks internaly if handlers have been init or not */
-	v4l2_ctrl_handler_free(&vin->ctrl_handler);
-
 	/* Checks internaly if vdev have been init or not */
 	video_unregister_device(&vin->vdev);
 }
@@ -869,41 +871,10 @@ static void rvin_notify(struct v4l2_subdev *sd,
 int rvin_v4l2_probe(struct rvin_dev *vin)
 {
 	struct video_device *vdev = &vin->vdev;
-	struct v4l2_subdev *sd = vin_to_source(vin);
 	int ret;
 
-	v4l2_set_subdev_hostdata(sd, vin);
-
 	vin->v4l2_dev.notify = rvin_notify;
 
-	ret = v4l2_subdev_call(sd, video, g_tvnorms, &vin->vdev.tvnorms);
-	if (ret < 0 && ret != -ENOIOCTLCMD && ret != -ENODEV)
-		return ret;
-
-	if (vin->vdev.tvnorms == 0) {
-		/* Disable the STD API if there are no tvnorms defined */
-		v4l2_disable_ioctl(&vin->vdev, VIDIOC_G_STD);
-		v4l2_disable_ioctl(&vin->vdev, VIDIOC_S_STD);
-		v4l2_disable_ioctl(&vin->vdev, VIDIOC_QUERYSTD);
-		v4l2_disable_ioctl(&vin->vdev, VIDIOC_ENUMSTD);
-	}
-
-	/* Add the controls */
-	/*
-	 * Currently the subdev with the largest number of controls (13) is
-	 * ov6550. So let's pick 16 as a hint for the control handler. Note
-	 * that this is a hint only: too large and you waste some memory, too
-	 * small and there is a (very) small performance hit when looking up
-	 * controls in the internal hash.
-	 */
-	ret = v4l2_ctrl_handler_init(&vin->ctrl_handler, 16);
-	if (ret < 0)
-		return ret;
-
-	ret = v4l2_ctrl_add_handler(&vin->ctrl_handler, sd->ctrl_handler, NULL);
-	if (ret < 0)
-		return ret;
-
 	/* video node */
 	vdev->fops = &rvin_fops;
 	vdev->v4l2_dev = &vin->v4l2_dev;
@@ -917,7 +888,6 @@ int rvin_v4l2_probe(struct rvin_dev *vin)
 		V4L2_CAP_READWRITE;
 
 	vin->format.pixelformat	= RVIN_DEFAULT_FORMAT;
-	rvin_reset_format(vin);
 
 	ret = video_register_device(&vin->vdev, VFL_TYPE_GRABBER, -1);
 	if (ret) {
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 9bfb5a7c4dc4f215..9d0d4a5001b6ccd8 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -158,6 +158,7 @@ void rvin_dma_remove(struct rvin_dev *vin);
 
 int rvin_v4l2_probe(struct rvin_dev *vin);
 void rvin_v4l2_remove(struct rvin_dev *vin);
+int rvin_reset_format(struct rvin_dev *vin);
 
 const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
 
-- 
2.14.0

[PATCH v6 03/25] rcar-vin: move chip information to own struct

[Niklas Söderlund]

When Gen3 support is added to the driver more then chip id will be
different for the different Soc. To avoid a lot of if statements in the
code create a struct chip_info to contain this information.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
---
 drivers/media/platform/rcar-vin/rcar-core.c | 49 ++++++++++++++++++++++++-----
 drivers/media/platform/rcar-vin/rcar-v4l2.c |  3 +-
 drivers/media/platform/rcar-vin/rcar-vin.h  | 12 +++++--
 3 files changed, 53 insertions(+), 11 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index aefbe8e3ccddb3e4..dae38de706b66b64 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -277,14 +277,47 @@ static int rvin_digital_graph_init(struct rvin_dev *vin)
  * Platform Device Driver
  */
 
+static const struct rvin_info rcar_info_h1 = {
+	.chip = RCAR_H1,
+};
+
+static const struct rvin_info rcar_info_m1 = {
+	.chip = RCAR_M1,
+};
+
+static const struct rvin_info rcar_info_gen2 = {
+	.chip = RCAR_GEN2,
+};
+
 static const struct of_device_id rvin_of_id_table[] = {
-	{ .compatible = "renesas,vin-r8a7794", .data = (void *)RCAR_GEN2 },
-	{ .compatible = "renesas,vin-r8a7793", .data = (void *)RCAR_GEN2 },
-	{ .compatible = "renesas,vin-r8a7791", .data = (void *)RCAR_GEN2 },
-	{ .compatible = "renesas,vin-r8a7790", .data = (void *)RCAR_GEN2 },
-	{ .compatible = "renesas,vin-r8a7779", .data = (void *)RCAR_H1 },
-	{ .compatible = "renesas,vin-r8a7778", .data = (void *)RCAR_M1 },
-	{ .compatible = "renesas,rcar-gen2-vin", .data = (void *)RCAR_GEN2 },
+	{
+		.compatible = "renesas,vin-r8a7794",
+		.data = &rcar_info_gen2,
+	},
+	{
+		.compatible = "renesas,vin-r8a7793",
+		.data = &rcar_info_gen2,
+	},
+	{
+		.compatible = "renesas,vin-r8a7791",
+		.data = &rcar_info_gen2,
+	},
+	{
+		.compatible = "renesas,vin-r8a7790",
+		.data = &rcar_info_gen2,
+	},
+	{
+		.compatible = "renesas,vin-r8a7779",
+		.data = &rcar_info_h1,
+	},
+	{
+		.compatible = "renesas,vin-r8a7778",
+		.data = &rcar_info_m1,
+	},
+	{
+		.compatible = "renesas,rcar-gen2-vin",
+		.data = &rcar_info_gen2,
+	},
 	{ },
 };
 MODULE_DEVICE_TABLE(of, rvin_of_id_table);
@@ -305,7 +338,7 @@ static int rcar_vin_probe(struct platform_device *pdev)
 		return -ENODEV;
 
 	vin->dev = &pdev->dev;
-	vin->chip = (enum chip_id)match->data;
+	vin->info = match->data;
 
 	mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
 	if (mem == NULL)
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index 81ff59c3b4744075..02a08cf5acfce1ce 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -266,7 +266,8 @@ static int __rvin_try_format(struct rvin_dev *vin,
 	pix->sizeimage = max_t(u32, pix->sizeimage,
 			       rvin_format_sizeimage(pix));
 
-	if (vin->chip == RCAR_M1 && pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
+	if (vin->info->chip == RCAR_M1 &&
+	    pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
 		vin_err(vin, "pixel format XBGR32 not supported on M1\n");
 		return -EINVAL;
 	}
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 9d0d4a5001b6ccd8..13466dfd72292fc0 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -88,11 +88,19 @@ struct rvin_graph_entity {
 	unsigned int sink_pad;
 };
 
+/**
+ * struct rvin_info - Information about the particular VIN implementation
+ * @chip:		type of VIN chip
+ */
+struct rvin_info {
+	enum chip_id chip;
+};
+
 /**
  * struct rvin_dev - Renesas VIN device structure
  * @dev:		(OF) device
  * @base:		device I/O register space remapped to virtual memory
- * @chip:		type of VIN chip
+ * @info:		info about VIN instance
  *
  * @vdev:		V4L2 video device associated with VIN
  * @v4l2_dev:		V4L2 device
@@ -120,7 +128,7 @@ struct rvin_graph_entity {
 struct rvin_dev {
 	struct device *dev;
 	void __iomem *base;
-	enum chip_id chip;
+	const struct rvin_info *info;
 
 	struct video_device vdev;
 	struct v4l2_device v4l2_dev;
-- 
2.14.0

[PATCH v6 04/25] rcar-vin: move max width and height information to chip information

[Niklas Söderlund]

On Gen3 the max supported width and height will be different from Gen2.
Move the limits to the struct rvin_info to prepare for Gen3 support.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
---
 drivers/media/platform/rcar-vin/rcar-core.c | 6 ++++++
 drivers/media/platform/rcar-vin/rcar-v4l2.c | 6 ++----
 drivers/media/platform/rcar-vin/rcar-vin.h  | 6 ++++++
 3 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index dae38de706b66b64..4dc148e7835439ab 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -279,14 +279,20 @@ static int rvin_digital_graph_init(struct rvin_dev *vin)
 
 static const struct rvin_info rcar_info_h1 = {
 	.chip = RCAR_H1,
+	.max_width = 2048,
+	.max_height = 2048,
 };
 
 static const struct rvin_info rcar_info_m1 = {
 	.chip = RCAR_M1,
+	.max_width = 2048,
+	.max_height = 2048,
 };
 
 static const struct rvin_info rcar_info_gen2 = {
 	.chip = RCAR_GEN2,
+	.max_width = 2048,
+	.max_height = 2048,
 };
 
 static const struct of_device_id rvin_of_id_table[] = {
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index 02a08cf5acfce1ce..3c4dd08261a0d3f5 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -23,8 +23,6 @@
 #include "rcar-vin.h"
 
 #define RVIN_DEFAULT_FORMAT	V4L2_PIX_FMT_YUYV
-#define RVIN_MAX_WIDTH		2048
-#define RVIN_MAX_HEIGHT		2048
 
 /* -----------------------------------------------------------------------------
  * Format Conversions
@@ -258,8 +256,8 @@ static int __rvin_try_format(struct rvin_dev *vin,
 	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
 
 	/* Limit to VIN capabilities */
-	v4l_bound_align_image(&pix->width, 2, RVIN_MAX_WIDTH, walign,
-			      &pix->height, 4, RVIN_MAX_HEIGHT, 2, 0);
+	v4l_bound_align_image(&pix->width, 2, vin->info->max_width, walign,
+			      &pix->height, 4, vin->info->max_height, 2, 0);
 
 	pix->bytesperline = max_t(u32, pix->bytesperline,
 				  rvin_format_bytesperline(pix));
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 13466dfd72292fc0..2d8b362012ea46a3 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -91,9 +91,15 @@ struct rvin_graph_entity {
 /**
  * struct rvin_info - Information about the particular VIN implementation
  * @chip:		type of VIN chip
+ *
+ * max_width:		max input width the VIN supports
+ * max_height:		max input height the VIN supports
  */
 struct rvin_info {
 	enum chip_id chip;
+
+	unsigned int max_width;
+	unsigned int max_height;
 };
 
 /**
-- 
2.14.0

[PATCH v6 05/25] rcar-vin: change name of video device

[Niklas Söderlund]

The rcar-vin driver needs to be part of a media controller to support
Gen3. Give each VIN instance a unique name so it can be referenced from
userspace.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
---
 drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index 3c4dd08261a0d3f5..ba88774bd5379a98 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -878,7 +878,8 @@ int rvin_v4l2_probe(struct rvin_dev *vin)
 	vdev->fops = &rvin_fops;
 	vdev->v4l2_dev = &vin->v4l2_dev;
 	vdev->queue = &vin->queue;
-	strlcpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name));
+	snprintf(vdev->name, sizeof(vdev->name), "%s %s", KBUILD_MODNAME,
+		 dev_name(vin->dev));
 	vdev->release = video_device_release_empty;
 	vdev->ioctl_ops = &rvin_ioctl_ops;
 	vdev->lock = &vin->lock;
-- 
2.14.0

[PATCH v6 06/25] rcar-vin: move functions regarding scaling

[Niklas Söderlund]

In preparation of refactoring the scaling code move the code regarding
scaling to to the top of the file to avoid the need to add forward
declarations. No code is changed in this commit only whole functions
moved inside the same file.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/rcar-vin/rcar-dma.c | 806 +++++++++++++++--------------
 1 file changed, 405 insertions(+), 401 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index b136844499f677cf..03a79de197d19e43 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -138,305 +138,6 @@ static u32 rvin_read(struct rvin_dev *vin, u32 offset)
 	return ioread32(vin->base + offset);
 }
 
-static int rvin_setup(struct rvin_dev *vin)
-{
-	u32 vnmc, dmr, dmr2, interrupts;
-	v4l2_std_id std;
-	bool progressive = false, output_is_yuv = false, input_is_yuv = false;
-
-	switch (vin->format.field) {
-	case V4L2_FIELD_TOP:
-		vnmc = VNMC_IM_ODD;
-		break;
-	case V4L2_FIELD_BOTTOM:
-		vnmc = VNMC_IM_EVEN;
-		break;
-	case V4L2_FIELD_INTERLACED:
-		/* Default to TB */
-		vnmc = VNMC_IM_FULL;
-		/* Use BT if video standard can be read and is 60 Hz format */
-		if (!v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
-			if (std & V4L2_STD_525_60)
-				vnmc = VNMC_IM_FULL | VNMC_FOC;
-		}
-		break;
-	case V4L2_FIELD_INTERLACED_TB:
-		vnmc = VNMC_IM_FULL;
-		break;
-	case V4L2_FIELD_INTERLACED_BT:
-		vnmc = VNMC_IM_FULL | VNMC_FOC;
-		break;
-	case V4L2_FIELD_ALTERNATE:
-	case V4L2_FIELD_NONE:
-		if (vin->continuous) {
-			vnmc = VNMC_IM_ODD_EVEN;
-			progressive = true;
-		} else {
-			vnmc = VNMC_IM_ODD;
-		}
-		break;
-	default:
-		vnmc = VNMC_IM_ODD;
-		break;
-	}
-
-	/*
-	 * Input interface
-	 */
-	switch (vin->digital.code) {
-	case MEDIA_BUS_FMT_YUYV8_1X16:
-		/* BT.601/BT.1358 16bit YCbCr422 */
-		vnmc |= VNMC_INF_YUV16;
-		input_is_yuv = true;
-		break;
-	case MEDIA_BUS_FMT_UYVY8_2X8:
-		/* BT.656 8bit YCbCr422 or BT.601 8bit YCbCr422 */
-		vnmc |= vin->digital.mbus_cfg.type == V4L2_MBUS_BT656 ?
-			VNMC_INF_YUV8_BT656 : VNMC_INF_YUV8_BT601;
-		input_is_yuv = true;
-		break;
-	case MEDIA_BUS_FMT_RGB888_1X24:
-		vnmc |= VNMC_INF_RGB888;
-		break;
-	case MEDIA_BUS_FMT_UYVY10_2X10:
-		/* BT.656 10bit YCbCr422 or BT.601 10bit YCbCr422 */
-		vnmc |= vin->digital.mbus_cfg.type == V4L2_MBUS_BT656 ?
-			VNMC_INF_YUV10_BT656 : VNMC_INF_YUV10_BT601;
-		input_is_yuv = true;
-		break;
-	default:
-		break;
-	}
-
-	/* Enable VSYNC Field Toogle mode after one VSYNC input */
-	dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
-
-	/* Hsync Signal Polarity Select */
-	if (!(vin->digital.mbus_cfg.flags & V4L2_MBUS_HSYNC_ACTIVE_LOW))
-		dmr2 |= VNDMR2_HPS;
-
-	/* Vsync Signal Polarity Select */
-	if (!(vin->digital.mbus_cfg.flags & V4L2_MBUS_VSYNC_ACTIVE_LOW))
-		dmr2 |= VNDMR2_VPS;
-
-	/*
-	 * Output format
-	 */
-	switch (vin->format.pixelformat) {
-	case V4L2_PIX_FMT_NV16:
-		rvin_write(vin,
-			   ALIGN(vin->format.width * vin->format.height, 0x80),
-			   VNUVAOF_REG);
-		dmr = VNDMR_DTMD_YCSEP;
-		output_is_yuv = true;
-		break;
-	case V4L2_PIX_FMT_YUYV:
-		dmr = VNDMR_BPSM;
-		output_is_yuv = true;
-		break;
-	case V4L2_PIX_FMT_UYVY:
-		dmr = 0;
-		output_is_yuv = true;
-		break;
-	case V4L2_PIX_FMT_XRGB555:
-		dmr = VNDMR_DTMD_ARGB1555;
-		break;
-	case V4L2_PIX_FMT_RGB565:
-		dmr = 0;
-		break;
-	case V4L2_PIX_FMT_XBGR32:
-		/* Note: not supported on M1 */
-		dmr = VNDMR_EXRGB;
-		break;
-	default:
-		vin_err(vin, "Invalid pixelformat (0x%x)\n",
-			vin->format.pixelformat);
-		return -EINVAL;
-	}
-
-	/* Always update on field change */
-	vnmc |= VNMC_VUP;
-
-	/* If input and output use the same colorspace, use bypass mode */
-	if (input_is_yuv == output_is_yuv)
-		vnmc |= VNMC_BPS;
-
-	/* Progressive or interlaced mode */
-	interrupts = progressive ? VNIE_FIE : VNIE_EFE;
-
-	/* Ack interrupts */
-	rvin_write(vin, interrupts, VNINTS_REG);
-	/* Enable interrupts */
-	rvin_write(vin, interrupts, VNIE_REG);
-	/* Start capturing */
-	rvin_write(vin, dmr, VNDMR_REG);
-	rvin_write(vin, dmr2, VNDMR2_REG);
-
-	/* Enable module */
-	rvin_write(vin, vnmc | VNMC_ME, VNMC_REG);
-
-	return 0;
-}
-
-static void rvin_disable_interrupts(struct rvin_dev *vin)
-{
-	rvin_write(vin, 0, VNIE_REG);
-}
-
-static u32 rvin_get_interrupt_status(struct rvin_dev *vin)
-{
-	return rvin_read(vin, VNINTS_REG);
-}
-
-static void rvin_ack_interrupt(struct rvin_dev *vin)
-{
-	rvin_write(vin, rvin_read(vin, VNINTS_REG), VNINTS_REG);
-}
-
-static bool rvin_capture_active(struct rvin_dev *vin)
-{
-	return rvin_read(vin, VNMS_REG) & VNMS_CA;
-}
-
-static int rvin_get_active_slot(struct rvin_dev *vin, u32 vnms)
-{
-	if (vin->continuous)
-		return (vnms & VNMS_FBS_MASK) >> VNMS_FBS_SHIFT;
-
-	return 0;
-}
-
-static enum v4l2_field rvin_get_active_field(struct rvin_dev *vin, u32 vnms)
-{
-	if (vin->format.field == V4L2_FIELD_ALTERNATE) {
-		/* If FS is set it's a Even field */
-		if (vnms & VNMS_FS)
-			return V4L2_FIELD_BOTTOM;
-		return V4L2_FIELD_TOP;
-	}
-
-	return vin->format.field;
-}
-
-static void rvin_set_slot_addr(struct rvin_dev *vin, int slot, dma_addr_t addr)
-{
-	const struct rvin_video_format *fmt;
-	int offsetx, offsety;
-	dma_addr_t offset;
-
-	fmt = rvin_format_from_pixel(vin->format.pixelformat);
-
-	/*
-	 * There is no HW support for composition do the beast we can
-	 * by modifying the buffer offset
-	 */
-	offsetx = vin->compose.left * fmt->bpp;
-	offsety = vin->compose.top * vin->format.bytesperline;
-	offset = addr + offsetx + offsety;
-
-	/*
-	 * The address needs to be 128 bytes aligned. Driver should never accept
-	 * settings that do not satisfy this in the first place...
-	 */
-	if (WARN_ON((offsetx | offsety | offset) & HW_BUFFER_MASK))
-		return;
-
-	rvin_write(vin, offset, VNMB_REG(slot));
-}
-
-/* Moves a buffer from the queue to the HW slots */
-static bool rvin_fill_hw_slot(struct rvin_dev *vin, int slot)
-{
-	struct rvin_buffer *buf;
-	struct vb2_v4l2_buffer *vbuf;
-	dma_addr_t phys_addr_top;
-
-	if (vin->queue_buf[slot] != NULL)
-		return true;
-
-	if (list_empty(&vin->buf_list))
-		return false;
-
-	vin_dbg(vin, "Filling HW slot: %d\n", slot);
-
-	/* Keep track of buffer we give to HW */
-	buf = list_entry(vin->buf_list.next, struct rvin_buffer, list);
-	vbuf = &buf->vb;
-	list_del_init(to_buf_list(vbuf));
-	vin->queue_buf[slot] = vbuf;
-
-	/* Setup DMA */
-	phys_addr_top = vb2_dma_contig_plane_dma_addr(&vbuf->vb2_buf, 0);
-	rvin_set_slot_addr(vin, slot, phys_addr_top);
-
-	return true;
-}
-
-static bool rvin_fill_hw(struct rvin_dev *vin)
-{
-	int slot, limit;
-
-	limit = vin->continuous ? HW_BUFFER_NUM : 1;
-
-	for (slot = 0; slot < limit; slot++)
-		if (!rvin_fill_hw_slot(vin, slot))
-			return false;
-	return true;
-}
-
-static void rvin_capture_on(struct rvin_dev *vin)
-{
-	vin_dbg(vin, "Capture on in %s mode\n",
-		vin->continuous ? "continuous" : "single");
-
-	if (vin->continuous)
-		/* Continuous Frame Capture Mode */
-		rvin_write(vin, VNFC_C_FRAME, VNFC_REG);
-	else
-		/* Single Frame Capture Mode */
-		rvin_write(vin, VNFC_S_FRAME, VNFC_REG);
-}
-
-static int rvin_capture_start(struct rvin_dev *vin)
-{
-	struct rvin_buffer *buf, *node;
-	int bufs, ret;
-
-	/* Count number of free buffers */
-	bufs = 0;
-	list_for_each_entry_safe(buf, node, &vin->buf_list, list)
-		bufs++;
-
-	/* Continuous capture requires more buffers then there are HW slots */
-	vin->continuous = bufs > HW_BUFFER_NUM;
-
-	if (!rvin_fill_hw(vin)) {
-		vin_err(vin, "HW not ready to start, not enough buffers available\n");
-		return -EINVAL;
-	}
-
-	rvin_crop_scale_comp(vin);
-
-	ret = rvin_setup(vin);
-	if (ret)
-		return ret;
-
-	rvin_capture_on(vin);
-
-	vin->state = RUNNING;
-
-	return 0;
-}
-
-static void rvin_capture_stop(struct rvin_dev *vin)
-{
-	/* Set continuous & single transfer off */
-	rvin_write(vin, 0, VNFC_REG);
-
-	/* Disable module */
-	rvin_write(vin, rvin_read(vin, VNMC_REG) & ~VNMC_ME, VNMC_REG);
-}
-
 /* -----------------------------------------------------------------------------
  * Crop and Scaling Gen2
  */
@@ -757,139 +458,442 @@ static const struct vin_coeff vin_coeff_set[] = {
 			  0x0370e83b, 0x0310d439, 0x03a0f83d,
 			  0x0370e83c, 0x0300d438, 0x03b0fc3c },
 	}
-};
+};
+
+static void rvin_set_coeff(struct rvin_dev *vin, unsigned short xs)
+{
+	int i;
+	const struct vin_coeff *p_prev_set = NULL;
+	const struct vin_coeff *p_set = NULL;
+
+	/* Look for suitable coefficient values */
+	for (i = 0; i < ARRAY_SIZE(vin_coeff_set); i++) {
+		p_prev_set = p_set;
+		p_set = &vin_coeff_set[i];
+
+		if (xs < p_set->xs_value)
+			break;
+	}
+
+	/* Use previous value if its XS value is closer */
+	if (p_prev_set && p_set &&
+	    xs - p_prev_set->xs_value < p_set->xs_value - xs)
+		p_set = p_prev_set;
+
+	/* Set coefficient registers */
+	rvin_write(vin, p_set->coeff_set[0], VNC1A_REG);
+	rvin_write(vin, p_set->coeff_set[1], VNC1B_REG);
+	rvin_write(vin, p_set->coeff_set[2], VNC1C_REG);
+
+	rvin_write(vin, p_set->coeff_set[3], VNC2A_REG);
+	rvin_write(vin, p_set->coeff_set[4], VNC2B_REG);
+	rvin_write(vin, p_set->coeff_set[5], VNC2C_REG);
+
+	rvin_write(vin, p_set->coeff_set[6], VNC3A_REG);
+	rvin_write(vin, p_set->coeff_set[7], VNC3B_REG);
+	rvin_write(vin, p_set->coeff_set[8], VNC3C_REG);
+
+	rvin_write(vin, p_set->coeff_set[9], VNC4A_REG);
+	rvin_write(vin, p_set->coeff_set[10], VNC4B_REG);
+	rvin_write(vin, p_set->coeff_set[11], VNC4C_REG);
+
+	rvin_write(vin, p_set->coeff_set[12], VNC5A_REG);
+	rvin_write(vin, p_set->coeff_set[13], VNC5B_REG);
+	rvin_write(vin, p_set->coeff_set[14], VNC5C_REG);
+
+	rvin_write(vin, p_set->coeff_set[15], VNC6A_REG);
+	rvin_write(vin, p_set->coeff_set[16], VNC6B_REG);
+	rvin_write(vin, p_set->coeff_set[17], VNC6C_REG);
+
+	rvin_write(vin, p_set->coeff_set[18], VNC7A_REG);
+	rvin_write(vin, p_set->coeff_set[19], VNC7B_REG);
+	rvin_write(vin, p_set->coeff_set[20], VNC7C_REG);
+
+	rvin_write(vin, p_set->coeff_set[21], VNC8A_REG);
+	rvin_write(vin, p_set->coeff_set[22], VNC8B_REG);
+	rvin_write(vin, p_set->coeff_set[23], VNC8C_REG);
+}
+
+void rvin_crop_scale_comp(struct rvin_dev *vin)
+{
+	u32 xs, ys;
+
+	/* Set Start/End Pixel/Line Pre-Clip */
+	rvin_write(vin, vin->crop.left, VNSPPRC_REG);
+	rvin_write(vin, vin->crop.left + vin->crop.width - 1, VNEPPRC_REG);
+	switch (vin->format.field) {
+	case V4L2_FIELD_INTERLACED:
+	case V4L2_FIELD_INTERLACED_TB:
+	case V4L2_FIELD_INTERLACED_BT:
+		rvin_write(vin, vin->crop.top / 2, VNSLPRC_REG);
+		rvin_write(vin, (vin->crop.top + vin->crop.height) / 2 - 1,
+			   VNELPRC_REG);
+		break;
+	default:
+		rvin_write(vin, vin->crop.top, VNSLPRC_REG);
+		rvin_write(vin, vin->crop.top + vin->crop.height - 1,
+			   VNELPRC_REG);
+		break;
+	}
+
+	/* Set scaling coefficient */
+	ys = 0;
+	if (vin->crop.height != vin->compose.height)
+		ys = (4096 * vin->crop.height) / vin->compose.height;
+	rvin_write(vin, ys, VNYS_REG);
+
+	xs = 0;
+	if (vin->crop.width != vin->compose.width)
+		xs = (4096 * vin->crop.width) / vin->compose.width;
+
+	/* Horizontal upscaling is up to double size */
+	if (xs > 0 && xs < 2048)
+		xs = 2048;
+
+	rvin_write(vin, xs, VNXS_REG);
+
+	/* Horizontal upscaling is done out by scaling down from double size */
+	if (xs < 4096)
+		xs *= 2;
+
+	rvin_set_coeff(vin, xs);
+
+	/* Set Start/End Pixel/Line Post-Clip */
+	rvin_write(vin, 0, VNSPPOC_REG);
+	rvin_write(vin, 0, VNSLPOC_REG);
+	rvin_write(vin, vin->format.width - 1, VNEPPOC_REG);
+	switch (vin->format.field) {
+	case V4L2_FIELD_INTERLACED:
+	case V4L2_FIELD_INTERLACED_TB:
+	case V4L2_FIELD_INTERLACED_BT:
+		rvin_write(vin, vin->format.height / 2 - 1, VNELPOC_REG);
+		break;
+	default:
+		rvin_write(vin, vin->format.height - 1, VNELPOC_REG);
+		break;
+	}
+
+	if (vin->format.pixelformat == V4L2_PIX_FMT_NV16)
+		rvin_write(vin, ALIGN(vin->format.width, 0x20), VNIS_REG);
+	else
+		rvin_write(vin, ALIGN(vin->format.width, 0x10), VNIS_REG);
+
+	vin_dbg(vin,
+		"Pre-Clip: %ux%u@%u:%u YS: %d XS: %d Post-Clip: %ux%u@%u:%u\n",
+		vin->crop.width, vin->crop.height, vin->crop.left,
+		vin->crop.top, ys, xs, vin->format.width, vin->format.height,
+		0, 0);
+}
+
+void rvin_scale_try(struct rvin_dev *vin, struct v4l2_pix_format *pix,
+		    u32 width, u32 height)
+{
+	/* All VIN channels on Gen2 have scalers */
+	pix->width = width;
+	pix->height = height;
+}
+
+/* -----------------------------------------------------------------------------
+ * Hardware setup
+ */
+
+static int rvin_setup(struct rvin_dev *vin)
+{
+	u32 vnmc, dmr, dmr2, interrupts;
+	v4l2_std_id std;
+	bool progressive = false, output_is_yuv = false, input_is_yuv = false;
+
+	switch (vin->format.field) {
+	case V4L2_FIELD_TOP:
+		vnmc = VNMC_IM_ODD;
+		break;
+	case V4L2_FIELD_BOTTOM:
+		vnmc = VNMC_IM_EVEN;
+		break;
+	case V4L2_FIELD_INTERLACED:
+		/* Default to TB */
+		vnmc = VNMC_IM_FULL;
+		/* Use BT if video standard can be read and is 60 Hz format */
+		if (!v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
+			if (std & V4L2_STD_525_60)
+				vnmc = VNMC_IM_FULL | VNMC_FOC;
+		}
+		break;
+	case V4L2_FIELD_INTERLACED_TB:
+		vnmc = VNMC_IM_FULL;
+		break;
+	case V4L2_FIELD_INTERLACED_BT:
+		vnmc = VNMC_IM_FULL | VNMC_FOC;
+		break;
+	case V4L2_FIELD_ALTERNATE:
+	case V4L2_FIELD_NONE:
+		if (vin->continuous) {
+			vnmc = VNMC_IM_ODD_EVEN;
+			progressive = true;
+		} else {
+			vnmc = VNMC_IM_ODD;
+		}
+		break;
+	default:
+		vnmc = VNMC_IM_ODD;
+		break;
+	}
+
+	/*
+	 * Input interface
+	 */
+	switch (vin->digital.code) {
+	case MEDIA_BUS_FMT_YUYV8_1X16:
+		/* BT.601/BT.1358 16bit YCbCr422 */
+		vnmc |= VNMC_INF_YUV16;
+		input_is_yuv = true;
+		break;
+	case MEDIA_BUS_FMT_UYVY8_2X8:
+		/* BT.656 8bit YCbCr422 or BT.601 8bit YCbCr422 */
+		vnmc |= vin->digital.mbus_cfg.type == V4L2_MBUS_BT656 ?
+			VNMC_INF_YUV8_BT656 : VNMC_INF_YUV8_BT601;
+		input_is_yuv = true;
+		break;
+	case MEDIA_BUS_FMT_RGB888_1X24:
+		vnmc |= VNMC_INF_RGB888;
+		break;
+	case MEDIA_BUS_FMT_UYVY10_2X10:
+		/* BT.656 10bit YCbCr422 or BT.601 10bit YCbCr422 */
+		vnmc |= vin->digital.mbus_cfg.type == V4L2_MBUS_BT656 ?
+			VNMC_INF_YUV10_BT656 : VNMC_INF_YUV10_BT601;
+		input_is_yuv = true;
+		break;
+	default:
+		break;
+	}
+
+	/* Enable VSYNC Field Toogle mode after one VSYNC input */
+	dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
+
+	/* Hsync Signal Polarity Select */
+	if (!(vin->digital.mbus_cfg.flags & V4L2_MBUS_HSYNC_ACTIVE_LOW))
+		dmr2 |= VNDMR2_HPS;
+
+	/* Vsync Signal Polarity Select */
+	if (!(vin->digital.mbus_cfg.flags & V4L2_MBUS_VSYNC_ACTIVE_LOW))
+		dmr2 |= VNDMR2_VPS;
+
+	/*
+	 * Output format
+	 */
+	switch (vin->format.pixelformat) {
+	case V4L2_PIX_FMT_NV16:
+		rvin_write(vin,
+			   ALIGN(vin->format.width * vin->format.height, 0x80),
+			   VNUVAOF_REG);
+		dmr = VNDMR_DTMD_YCSEP;
+		output_is_yuv = true;
+		break;
+	case V4L2_PIX_FMT_YUYV:
+		dmr = VNDMR_BPSM;
+		output_is_yuv = true;
+		break;
+	case V4L2_PIX_FMT_UYVY:
+		dmr = 0;
+		output_is_yuv = true;
+		break;
+	case V4L2_PIX_FMT_XRGB555:
+		dmr = VNDMR_DTMD_ARGB1555;
+		break;
+	case V4L2_PIX_FMT_RGB565:
+		dmr = 0;
+		break;
+	case V4L2_PIX_FMT_XBGR32:
+		/* Note: not supported on M1 */
+		dmr = VNDMR_EXRGB;
+		break;
+	default:
+		vin_err(vin, "Invalid pixelformat (0x%x)\n",
+			vin->format.pixelformat);
+		return -EINVAL;
+	}
 
-static void rvin_set_coeff(struct rvin_dev *vin, unsigned short xs)
+	/* Always update on field change */
+	vnmc |= VNMC_VUP;
+
+	/* If input and output use the same colorspace, use bypass mode */
+	if (input_is_yuv == output_is_yuv)
+		vnmc |= VNMC_BPS;
+
+	/* Progressive or interlaced mode */
+	interrupts = progressive ? VNIE_FIE : VNIE_EFE;
+
+	/* Ack interrupts */
+	rvin_write(vin, interrupts, VNINTS_REG);
+	/* Enable interrupts */
+	rvin_write(vin, interrupts, VNIE_REG);
+	/* Start capturing */
+	rvin_write(vin, dmr, VNDMR_REG);
+	rvin_write(vin, dmr2, VNDMR2_REG);
+
+	/* Enable module */
+	rvin_write(vin, vnmc | VNMC_ME, VNMC_REG);
+
+	return 0;
+}
+
+static void rvin_disable_interrupts(struct rvin_dev *vin)
 {
-	int i;
-	const struct vin_coeff *p_prev_set = NULL;
-	const struct vin_coeff *p_set = NULL;
+	rvin_write(vin, 0, VNIE_REG);
+}
 
-	/* Look for suitable coefficient values */
-	for (i = 0; i < ARRAY_SIZE(vin_coeff_set); i++) {
-		p_prev_set = p_set;
-		p_set = &vin_coeff_set[i];
+static u32 rvin_get_interrupt_status(struct rvin_dev *vin)
+{
+	return rvin_read(vin, VNINTS_REG);
+}
 
-		if (xs < p_set->xs_value)
-			break;
+static void rvin_ack_interrupt(struct rvin_dev *vin)
+{
+	rvin_write(vin, rvin_read(vin, VNINTS_REG), VNINTS_REG);
+}
+
+static bool rvin_capture_active(struct rvin_dev *vin)
+{
+	return rvin_read(vin, VNMS_REG) & VNMS_CA;
+}
+
+static int rvin_get_active_slot(struct rvin_dev *vin, u32 vnms)
+{
+	if (vin->continuous)
+		return (vnms & VNMS_FBS_MASK) >> VNMS_FBS_SHIFT;
+
+	return 0;
+}
+
+static enum v4l2_field rvin_get_active_field(struct rvin_dev *vin, u32 vnms)
+{
+	if (vin->format.field == V4L2_FIELD_ALTERNATE) {
+		/* If FS is set it's a Even field */
+		if (vnms & VNMS_FS)
+			return V4L2_FIELD_BOTTOM;
+		return V4L2_FIELD_TOP;
 	}
 
-	/* Use previous value if its XS value is closer */
-	if (p_prev_set && p_set &&
-	    xs - p_prev_set->xs_value < p_set->xs_value - xs)
-		p_set = p_prev_set;
+	return vin->format.field;
+}
 
-	/* Set coefficient registers */
-	rvin_write(vin, p_set->coeff_set[0], VNC1A_REG);
-	rvin_write(vin, p_set->coeff_set[1], VNC1B_REG);
-	rvin_write(vin, p_set->coeff_set[2], VNC1C_REG);
+static void rvin_set_slot_addr(struct rvin_dev *vin, int slot, dma_addr_t addr)
+{
+	const struct rvin_video_format *fmt;
+	int offsetx, offsety;
+	dma_addr_t offset;
 
-	rvin_write(vin, p_set->coeff_set[3], VNC2A_REG);
-	rvin_write(vin, p_set->coeff_set[4], VNC2B_REG);
-	rvin_write(vin, p_set->coeff_set[5], VNC2C_REG);
+	fmt = rvin_format_from_pixel(vin->format.pixelformat);
 
-	rvin_write(vin, p_set->coeff_set[6], VNC3A_REG);
-	rvin_write(vin, p_set->coeff_set[7], VNC3B_REG);
-	rvin_write(vin, p_set->coeff_set[8], VNC3C_REG);
+	/*
+	 * There is no HW support for composition do the beast we can
+	 * by modifying the buffer offset
+	 */
+	offsetx = vin->compose.left * fmt->bpp;
+	offsety = vin->compose.top * vin->format.bytesperline;
+	offset = addr + offsetx + offsety;
 
-	rvin_write(vin, p_set->coeff_set[9], VNC4A_REG);
-	rvin_write(vin, p_set->coeff_set[10], VNC4B_REG);
-	rvin_write(vin, p_set->coeff_set[11], VNC4C_REG);
+	/*
+	 * The address needs to be 128 bytes aligned. Driver should never accept
+	 * settings that do not satisfy this in the first place...
+	 */
+	if (WARN_ON((offsetx | offsety | offset) & HW_BUFFER_MASK))
+		return;
 
-	rvin_write(vin, p_set->coeff_set[12], VNC5A_REG);
-	rvin_write(vin, p_set->coeff_set[13], VNC5B_REG);
-	rvin_write(vin, p_set->coeff_set[14], VNC5C_REG);
+	rvin_write(vin, offset, VNMB_REG(slot));
+}
 
-	rvin_write(vin, p_set->coeff_set[15], VNC6A_REG);
-	rvin_write(vin, p_set->coeff_set[16], VNC6B_REG);
-	rvin_write(vin, p_set->coeff_set[17], VNC6C_REG);
+/* Moves a buffer from the queue to the HW slots */
+static bool rvin_fill_hw_slot(struct rvin_dev *vin, int slot)
+{
+	struct rvin_buffer *buf;
+	struct vb2_v4l2_buffer *vbuf;
+	dma_addr_t phys_addr_top;
 
-	rvin_write(vin, p_set->coeff_set[18], VNC7A_REG);
-	rvin_write(vin, p_set->coeff_set[19], VNC7B_REG);
-	rvin_write(vin, p_set->coeff_set[20], VNC7C_REG);
+	if (vin->queue_buf[slot] != NULL)
+		return true;
 
-	rvin_write(vin, p_set->coeff_set[21], VNC8A_REG);
-	rvin_write(vin, p_set->coeff_set[22], VNC8B_REG);
-	rvin_write(vin, p_set->coeff_set[23], VNC8C_REG);
+	if (list_empty(&vin->buf_list))
+		return false;
+
+	vin_dbg(vin, "Filling HW slot: %d\n", slot);
+
+	/* Keep track of buffer we give to HW */
+	buf = list_entry(vin->buf_list.next, struct rvin_buffer, list);
+	vbuf = &buf->vb;
+	list_del_init(to_buf_list(vbuf));
+	vin->queue_buf[slot] = vbuf;
+
+	/* Setup DMA */
+	phys_addr_top = vb2_dma_contig_plane_dma_addr(&vbuf->vb2_buf, 0);
+	rvin_set_slot_addr(vin, slot, phys_addr_top);
+
+	return true;
 }
 
-void rvin_crop_scale_comp(struct rvin_dev *vin)
+static bool rvin_fill_hw(struct rvin_dev *vin)
 {
-	u32 xs, ys;
+	int slot, limit;
 
-	/* Set Start/End Pixel/Line Pre-Clip */
-	rvin_write(vin, vin->crop.left, VNSPPRC_REG);
-	rvin_write(vin, vin->crop.left + vin->crop.width - 1, VNEPPRC_REG);
-	switch (vin->format.field) {
-	case V4L2_FIELD_INTERLACED:
-	case V4L2_FIELD_INTERLACED_TB:
-	case V4L2_FIELD_INTERLACED_BT:
-		rvin_write(vin, vin->crop.top / 2, VNSLPRC_REG);
-		rvin_write(vin, (vin->crop.top + vin->crop.height) / 2 - 1,
-			   VNELPRC_REG);
-		break;
-	default:
-		rvin_write(vin, vin->crop.top, VNSLPRC_REG);
-		rvin_write(vin, vin->crop.top + vin->crop.height - 1,
-			   VNELPRC_REG);
-		break;
-	}
+	limit = vin->continuous ? HW_BUFFER_NUM : 1;
 
-	/* Set scaling coefficient */
-	ys = 0;
-	if (vin->crop.height != vin->compose.height)
-		ys = (4096 * vin->crop.height) / vin->compose.height;
-	rvin_write(vin, ys, VNYS_REG);
+	for (slot = 0; slot < limit; slot++)
+		if (!rvin_fill_hw_slot(vin, slot))
+			return false;
+	return true;
+}
 
-	xs = 0;
-	if (vin->crop.width != vin->compose.width)
-		xs = (4096 * vin->crop.width) / vin->compose.width;
+static void rvin_capture_on(struct rvin_dev *vin)
+{
+	vin_dbg(vin, "Capture on in %s mode\n",
+		vin->continuous ? "continuous" : "single");
 
-	/* Horizontal upscaling is up to double size */
-	if (xs > 0 && xs < 2048)
-		xs = 2048;
+	if (vin->continuous)
+		/* Continuous Frame Capture Mode */
+		rvin_write(vin, VNFC_C_FRAME, VNFC_REG);
+	else
+		/* Single Frame Capture Mode */
+		rvin_write(vin, VNFC_S_FRAME, VNFC_REG);
+}
 
-	rvin_write(vin, xs, VNXS_REG);
+static int rvin_capture_start(struct rvin_dev *vin)
+{
+	struct rvin_buffer *buf, *node;
+	int bufs, ret;
 
-	/* Horizontal upscaling is done out by scaling down from double size */
-	if (xs < 4096)
-		xs *= 2;
+	/* Count number of free buffers */
+	bufs = 0;
+	list_for_each_entry_safe(buf, node, &vin->buf_list, list)
+		bufs++;
 
-	rvin_set_coeff(vin, xs);
+	/* Continuous capture requires more buffers then there are HW slots */
+	vin->continuous = bufs > HW_BUFFER_NUM;
 
-	/* Set Start/End Pixel/Line Post-Clip */
-	rvin_write(vin, 0, VNSPPOC_REG);
-	rvin_write(vin, 0, VNSLPOC_REG);
-	rvin_write(vin, vin->format.width - 1, VNEPPOC_REG);
-	switch (vin->format.field) {
-	case V4L2_FIELD_INTERLACED:
-	case V4L2_FIELD_INTERLACED_TB:
-	case V4L2_FIELD_INTERLACED_BT:
-		rvin_write(vin, vin->format.height / 2 - 1, VNELPOC_REG);
-		break;
-	default:
-		rvin_write(vin, vin->format.height - 1, VNELPOC_REG);
-		break;
+	if (!rvin_fill_hw(vin)) {
+		vin_err(vin, "HW not ready to start, not enough buffers available\n");
+		return -EINVAL;
 	}
 
-	if (vin->format.pixelformat == V4L2_PIX_FMT_NV16)
-		rvin_write(vin, ALIGN(vin->format.width, 0x20), VNIS_REG);
-	else
-		rvin_write(vin, ALIGN(vin->format.width, 0x10), VNIS_REG);
+	rvin_crop_scale_comp(vin);
 
-	vin_dbg(vin,
-		"Pre-Clip: %ux%u@%u:%u YS: %d XS: %d Post-Clip: %ux%u@%u:%u\n",
-		vin->crop.width, vin->crop.height, vin->crop.left,
-		vin->crop.top, ys, xs, vin->format.width, vin->format.height,
-		0, 0);
+	ret = rvin_setup(vin);
+	if (ret)
+		return ret;
+
+	rvin_capture_on(vin);
+
+	vin->state = RUNNING;
+
+	return 0;
 }
 
-void rvin_scale_try(struct rvin_dev *vin, struct v4l2_pix_format *pix,
-		    u32 width, u32 height)
+static void rvin_capture_stop(struct rvin_dev *vin)
 {
-	/* All VIN channels on Gen2 have scalers */
-	pix->width = width;
-	pix->height = height;
+	/* Set continuous & single transfer off */
+	rvin_write(vin, 0, VNFC_REG);
+
+	/* Disable module */
+	rvin_write(vin, rvin_read(vin, VNMC_REG) & ~VNMC_ME, VNMC_REG);
 }
 
 /* -----------------------------------------------------------------------------
-- 
2.14.0

[PATCH v6 07/25] rcar-vin: all Gen2 boards can scale simplify logic

[Niklas Söderlund]

The logic to preserve the requested format width and height are too
complex and come from a premature optimization for Gen3. All Gen2 SoC
can scale and the Gen3 implementation will not use these functions at
all so simply preserve the width and hight when interacting with the
subdevice much like the field is preserved simplifies the logic quiet a
bit.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/rcar-vin/rcar-dma.c  |  8 --------
 drivers/media/platform/rcar-vin/rcar-v4l2.c | 22 ++++++++++------------
 drivers/media/platform/rcar-vin/rcar-vin.h  |  2 --
 3 files changed, 10 insertions(+), 22 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index 03a79de197d19e43..5f9674dc898305ba 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -585,14 +585,6 @@ void rvin_crop_scale_comp(struct rvin_dev *vin)
 		0, 0);
 }
 
-void rvin_scale_try(struct rvin_dev *vin, struct v4l2_pix_format *pix,
-		    u32 width, u32 height)
-{
-	/* All VIN channels on Gen2 have scalers */
-	pix->width = width;
-	pix->height = height;
-}
-
 /* -----------------------------------------------------------------------------
  * Hardware setup
  */
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index ba88774bd5379a98..affdc128a75e502e 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -166,6 +166,7 @@ static int __rvin_try_format_source(struct rvin_dev *vin,
 		.which = which,
 	};
 	enum v4l2_field field;
+	u32 width, height;
 	int ret;
 
 	sd = vin_to_source(vin);
@@ -178,7 +179,10 @@ static int __rvin_try_format_source(struct rvin_dev *vin,
 
 	format.pad = vin->digital.source_pad;
 
+	/* Allow the video device to override field and to scale */
 	field = pix->field;
+	width = pix->width;
+	height = pix->height;
 
 	ret = v4l2_subdev_call(sd, pad, set_fmt, pad_cfg, &format);
 	if (ret < 0 && ret != -ENOIOCTLCMD)
@@ -191,6 +195,9 @@ static int __rvin_try_format_source(struct rvin_dev *vin,
 	source->width = pix->width;
 	source->height = pix->height;
 
+	pix->width = width;
+	pix->height = height;
+
 	vin_dbg(vin, "Source resolution: %ux%u\n", source->width,
 		source->height);
 
@@ -204,13 +211,9 @@ static int __rvin_try_format(struct rvin_dev *vin,
 			     struct v4l2_pix_format *pix,
 			     struct rvin_source_fmt *source)
 {
-	u32 rwidth, rheight, walign;
+	u32 walign;
 	int ret;
 
-	/* Requested */
-	rwidth = pix->width;
-	rheight = pix->height;
-
 	/* Keep current field if no specific one is asked for */
 	if (pix->field == V4L2_FIELD_ANY)
 		pix->field = vin->format.field;
@@ -248,10 +251,6 @@ static int __rvin_try_format(struct rvin_dev *vin,
 		break;
 	}
 
-	/* If source can't match format try if VIN can scale */
-	if (source->width != rwidth || source->height != rheight)
-		rvin_scale_try(vin, pix, rwidth, rheight);
-
 	/* HW limit width to a multiple of 32 (2^5) for NV16 else 2 (2^1) */
 	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
 
@@ -270,9 +269,8 @@ static int __rvin_try_format(struct rvin_dev *vin,
 		return -EINVAL;
 	}
 
-	vin_dbg(vin, "Requested %ux%u Got %ux%u bpl: %d size: %d\n",
-		rwidth, rheight, pix->width, pix->height,
-		pix->bytesperline, pix->sizeimage);
+	vin_dbg(vin, "Format %ux%u bpl: %d size: %d\n",
+		pix->width, pix->height, pix->bytesperline, pix->sizeimage);
 
 	return 0;
 }
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 2d8b362012ea46a3..b2bac06c0a3cfcb7 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -177,8 +177,6 @@ int rvin_reset_format(struct rvin_dev *vin);
 const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
 
 /* Cropping, composing and scaling */
-void rvin_scale_try(struct rvin_dev *vin, struct v4l2_pix_format *pix,
-		    u32 width, u32 height);
 void rvin_crop_scale_comp(struct rvin_dev *vin);
 
 #endif
-- 
2.14.0

[PATCH v6 08/25] rcar-vin: do not reset crop and compose when setting format

[Niklas Söderlund]

It was a bad idea to reset the crop and compose settings when a new
format is set. This would overwrite any crop/compose set by s_select and
cause unexpected behaviors, remove it. Also fold the reset helper in to
the only remaining caller.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/rcar-vin/rcar-v4l2.c | 21 +++++++--------------
 1 file changed, 7 insertions(+), 14 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index affdc128a75e502e..421820caf275b066 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -90,17 +90,6 @@ static u32 rvin_format_sizeimage(struct v4l2_pix_format *pix)
  * V4L2
  */
 
-static void rvin_reset_crop_compose(struct rvin_dev *vin)
-{
-	vin->crop.top = vin->crop.left = 0;
-	vin->crop.width = vin->source.width;
-	vin->crop.height = vin->source.height;
-
-	vin->compose.top = vin->compose.left = 0;
-	vin->compose.width = vin->format.width;
-	vin->compose.height = vin->format.height;
-}
-
 int rvin_reset_format(struct rvin_dev *vin)
 {
 	struct v4l2_subdev_format fmt = {
@@ -147,7 +136,13 @@ int rvin_reset_format(struct rvin_dev *vin)
 		break;
 	}
 
-	rvin_reset_crop_compose(vin);
+	vin->crop.top = vin->crop.left = 0;
+	vin->crop.width = mf->width;
+	vin->crop.height = mf->height;
+
+	vin->compose.top = vin->compose.left = 0;
+	vin->compose.width = mf->width;
+	vin->compose.height = mf->height;
 
 	vin->format.bytesperline = rvin_format_bytesperline(&vin->format);
 	vin->format.sizeimage = rvin_format_sizeimage(&vin->format);
@@ -317,8 +312,6 @@ static int rvin_s_fmt_vid_cap(struct file *file, void *priv,
 
 	vin->format = f->fmt.pix;
 
-	rvin_reset_crop_compose(vin);
-
 	return 0;
 }
 
-- 
2.14.0

[PATCH v6 09/25] rcar-vin: do not allow changing scaling and composing while streaming

[Niklas Söderlund]

It is possible on Gen2 to change the registers controlling composing and
scaling while the stream is running. Is however not a good idea to do so
and could result in trouble. There are also no good reason to allow
this, remove immediate reflection in hardware registers from
vidioc_s_selection and only configure scaling and composing when the
stream starts.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/rcar-vin/rcar-dma.c  | 2 +-
 drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ---
 drivers/media/platform/rcar-vin/rcar-vin.h  | 3 ---
 3 files changed, 1 insertion(+), 7 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index 5f9674dc898305ba..6cc880e5ef7e0718 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -514,7 +514,7 @@ static void rvin_set_coeff(struct rvin_dev *vin, unsigned short xs)
 	rvin_write(vin, p_set->coeff_set[23], VNC8C_REG);
 }
 
-void rvin_crop_scale_comp(struct rvin_dev *vin)
+static void rvin_crop_scale_comp(struct rvin_dev *vin)
 {
 	u32 xs, ys;
 
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index 421820caf275b066..305a74d033b2d9c5 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -436,9 +436,6 @@ static int rvin_s_selection(struct file *file, void *fh,
 		return -EINVAL;
 	}
 
-	/* HW supports modifying configuration while running */
-	rvin_crop_scale_comp(vin);
-
 	return 0;
 }
 
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index b2bac06c0a3cfcb7..fc70ded462ed3244 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -176,7 +176,4 @@ int rvin_reset_format(struct rvin_dev *vin);
 
 const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
 
-/* Cropping, composing and scaling */
-void rvin_crop_scale_comp(struct rvin_dev *vin);
-
 #endif
-- 
2.14.0

[PATCH v6 10/25] rcar-vin: read subdevice format for crop only when needed

[Niklas Söderlund]

Instead of caching the subdevice format each time the video device
format is set read it directly when its needed. As it turns out the
format is only needed when figuring out the max rectangle for cropping.

This simplify the code and makes it clearer what the source format is
used for.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/rcar-vin/rcar-v4l2.c | 88 ++++++++++++++---------------
 drivers/media/platform/rcar-vin/rcar-vin.h  | 12 ----
 2 files changed, 42 insertions(+), 58 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index 305a74d033b2d9c5..c8c764188b85a926 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -90,24 +90,30 @@ static u32 rvin_format_sizeimage(struct v4l2_pix_format *pix)
  * V4L2
  */
 
-int rvin_reset_format(struct rvin_dev *vin)
+static int rvin_get_sd_format(struct rvin_dev *vin, struct v4l2_pix_format *pix)
 {
 	struct v4l2_subdev_format fmt = {
 		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
+		.pad = vin->digital.source_pad,
 	};
-	struct v4l2_mbus_framefmt *mf = &fmt.format;
 	int ret;
 
-	fmt.pad = vin->digital.source_pad;
-
 	ret = v4l2_subdev_call(vin_to_source(vin), pad, get_fmt, NULL, &fmt);
 	if (ret)
 		return ret;
 
-	vin->format.width	= mf->width;
-	vin->format.height	= mf->height;
-	vin->format.colorspace	= mf->colorspace;
-	vin->format.field	= mf->field;
+	v4l2_fill_pix_format(pix, &fmt.format);
+
+	return 0;
+}
+
+int rvin_reset_format(struct rvin_dev *vin)
+{
+	int ret;
+
+	ret = rvin_get_sd_format(vin, &vin->format);
+	if (ret)
+		return ret;
 
 	/*
 	 * If the subdevice uses ALTERNATE field mode and G_STD is
@@ -137,12 +143,12 @@ int rvin_reset_format(struct rvin_dev *vin)
 	}
 
 	vin->crop.top = vin->crop.left = 0;
-	vin->crop.width = mf->width;
-	vin->crop.height = mf->height;
+	vin->crop.width = vin->format.width;
+	vin->crop.height = vin->format.height;
 
 	vin->compose.top = vin->compose.left = 0;
-	vin->compose.width = mf->width;
-	vin->compose.height = mf->height;
+	vin->compose.width = vin->format.width;
+	vin->compose.height = vin->format.height;
 
 	vin->format.bytesperline = rvin_format_bytesperline(&vin->format);
 	vin->format.sizeimage = rvin_format_sizeimage(&vin->format);
@@ -151,9 +157,7 @@ int rvin_reset_format(struct rvin_dev *vin)
 }
 
 static int __rvin_try_format_source(struct rvin_dev *vin,
-				    u32 which,
-				    struct v4l2_pix_format *pix,
-				    struct rvin_source_fmt *source)
+				    u32 which, struct v4l2_pix_format *pix)
 {
 	struct v4l2_subdev *sd;
 	struct v4l2_subdev_pad_config *pad_cfg;
@@ -186,25 +190,15 @@ static int __rvin_try_format_source(struct rvin_dev *vin,
 	v4l2_fill_pix_format(pix, &format.format);
 
 	pix->field = field;
-
-	source->width = pix->width;
-	source->height = pix->height;
-
 	pix->width = width;
 	pix->height = height;
-
-	vin_dbg(vin, "Source resolution: %ux%u\n", source->width,
-		source->height);
-
 done:
 	v4l2_subdev_free_pad_config(pad_cfg);
 	return ret;
 }
 
 static int __rvin_try_format(struct rvin_dev *vin,
-			     u32 which,
-			     struct v4l2_pix_format *pix,
-			     struct rvin_source_fmt *source)
+			     u32 which, struct v4l2_pix_format *pix)
 {
 	u32 walign;
 	int ret;
@@ -225,7 +219,7 @@ static int __rvin_try_format(struct rvin_dev *vin,
 	pix->sizeimage = 0;
 
 	/* Limit to source capabilities */
-	ret = __rvin_try_format_source(vin, which, pix, source);
+	ret = __rvin_try_format_source(vin, which, pix);
 	if (ret)
 		return ret;
 
@@ -234,7 +228,6 @@ static int __rvin_try_format(struct rvin_dev *vin,
 	case V4L2_FIELD_BOTTOM:
 	case V4L2_FIELD_ALTERNATE:
 		pix->height /= 2;
-		source->height /= 2;
 		break;
 	case V4L2_FIELD_NONE:
 	case V4L2_FIELD_INTERLACED_TB:
@@ -286,30 +279,23 @@ static int rvin_try_fmt_vid_cap(struct file *file, void *priv,
 				struct v4l2_format *f)
 {
 	struct rvin_dev *vin = video_drvdata(file);
-	struct rvin_source_fmt source;
 
-	return __rvin_try_format(vin, V4L2_SUBDEV_FORMAT_TRY, &f->fmt.pix,
-				 &source);
+	return __rvin_try_format(vin, V4L2_SUBDEV_FORMAT_TRY, &f->fmt.pix);
 }
 
 static int rvin_s_fmt_vid_cap(struct file *file, void *priv,
 			      struct v4l2_format *f)
 {
 	struct rvin_dev *vin = video_drvdata(file);
-	struct rvin_source_fmt source;
 	int ret;
 
 	if (vb2_is_busy(&vin->queue))
 		return -EBUSY;
 
-	ret = __rvin_try_format(vin, V4L2_SUBDEV_FORMAT_ACTIVE, &f->fmt.pix,
-				&source);
+	ret = __rvin_try_format(vin, V4L2_SUBDEV_FORMAT_ACTIVE, &f->fmt.pix);
 	if (ret)
 		return ret;
 
-	vin->source.width = source.width;
-	vin->source.height = source.height;
-
 	vin->format = f->fmt.pix;
 
 	return 0;
@@ -340,6 +326,8 @@ static int rvin_g_selection(struct file *file, void *fh,
 			    struct v4l2_selection *s)
 {
 	struct rvin_dev *vin = video_drvdata(file);
+	struct v4l2_pix_format pix;
+	int ret;
 
 	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
 		return -EINVAL;
@@ -347,9 +335,12 @@ static int rvin_g_selection(struct file *file, void *fh,
 	switch (s->target) {
 	case V4L2_SEL_TGT_CROP_BOUNDS:
 	case V4L2_SEL_TGT_CROP_DEFAULT:
+		ret = rvin_get_sd_format(vin, &pix);
+		if (ret)
+			return ret;
 		s->r.left = s->r.top = 0;
-		s->r.width = vin->source.width;
-		s->r.height = vin->source.height;
+		s->r.width = pix.width;
+		s->r.height = pix.height;
 		break;
 	case V4L2_SEL_TGT_CROP:
 		s->r = vin->crop;
@@ -375,12 +366,14 @@ static int rvin_s_selection(struct file *file, void *fh,
 {
 	struct rvin_dev *vin = video_drvdata(file);
 	const struct rvin_video_format *fmt;
+	struct v4l2_pix_format pix;
 	struct v4l2_rect r = s->r;
 	struct v4l2_rect max_rect;
 	struct v4l2_rect min_rect = {
 		.width = 6,
 		.height = 2,
 	};
+	int ret;
 
 	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
 		return -EINVAL;
@@ -390,22 +383,25 @@ static int rvin_s_selection(struct file *file, void *fh,
 	switch (s->target) {
 	case V4L2_SEL_TGT_CROP:
 		/* Can't crop outside of source input */
+		ret = rvin_get_sd_format(vin, &pix);
+		if (ret)
+			return ret;
 		max_rect.top = max_rect.left = 0;
-		max_rect.width = vin->source.width;
-		max_rect.height = vin->source.height;
+		max_rect.width = pix.width;
+		max_rect.height = pix.height;
 		v4l2_rect_map_inside(&r, &max_rect);
 
-		v4l_bound_align_image(&r.width, 2, vin->source.width, 1,
-				      &r.height, 4, vin->source.height, 2, 0);
+		v4l_bound_align_image(&r.width, 2, pix.width, 1,
+				      &r.height, 4, pix.height, 2, 0);
 
-		r.top  = clamp_t(s32, r.top, 0, vin->source.height - r.height);
-		r.left = clamp_t(s32, r.left, 0, vin->source.width - r.width);
+		r.top  = clamp_t(s32, r.top, 0, pix.height - r.height);
+		r.left = clamp_t(s32, r.left, 0, pix.width - r.width);
 
 		vin->crop = s->r = r;
 
 		vin_dbg(vin, "Cropped %dx%d@%d:%d of %dx%d\n",
 			r.width, r.height, r.left, r.top,
-			vin->source.width, vin->source.height);
+			pix.width, pix.height);
 		break;
 	case V4L2_SEL_TGT_COMPOSE:
 		/* Make sure compose rect fits inside output format */
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index fc70ded462ed3244..cd8d9a96f78f3267 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -48,16 +48,6 @@ enum rvin_dma_state {
 	STOPPING,
 };
 
-/**
- * struct rvin_source_fmt - Source information
- * @width:	Width from source
- * @height:	Height from source
- */
-struct rvin_source_fmt {
-	u32 width;
-	u32 height;
-};
-
 /**
  * struct rvin_video_format - Data format stored in memory
  * @fourcc:	Pixelformat
@@ -125,7 +115,6 @@ struct rvin_info {
  * @sequence:		V4L2 buffers sequence number
  * @state:		keeps track of operation state
  *
- * @source:		active format from the video source
  * @format:		active V4L2 pixel format
  *
  * @crop:		active cropping
@@ -152,7 +141,6 @@ struct rvin_dev {
 	unsigned int sequence;
 	enum rvin_dma_state state;
 
-	struct rvin_source_fmt source;
 	struct v4l2_pix_format format;
 
 	struct v4l2_rect crop;
-- 
2.14.0

[PATCH v6 11/25] rcar-vin: fix handling of single field frames (top, bottom and alternate fields)

[Niklas Söderlund]

It was never proper support in the VIN driver to deliver ALTERNATING
field format to user-space, remove this field option. For sources using
this field format instead use the VIN hardware feature of combining the
fields to an interlaced format. This mode of operation was previously
the default behavior and ALTERNATING was only delivered to user-space if
explicitly requested. Allowing this to be explicitly requested was a
mistake and was never properly tested and never worked due to the
constrains put on the field format when it comes to sequence numbers and
timestamps etc.

The height should not be cut in half for the format for TOP or BOTTOM
fields settings. This was a mistake and it was made visible by the
scaling refactoring. Correct behavior is that the user should request a
frame size that fits the half height frame reflected in the field
setting. If not the VIN will do it's best to scale the top or bottom to
the requested format and cropping and scaling do not work as expected.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/rcar-vin/rcar-dma.c  | 15 +--------
 drivers/media/platform/rcar-vin/rcar-v4l2.c | 48 +++++++++++------------------
 2 files changed, 19 insertions(+), 44 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index 6cc880e5ef7e0718..f22bec062db31772 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -617,7 +617,6 @@ static int rvin_setup(struct rvin_dev *vin)
 	case V4L2_FIELD_INTERLACED_BT:
 		vnmc = VNMC_IM_FULL | VNMC_FOC;
 		break;
-	case V4L2_FIELD_ALTERNATE:
 	case V4L2_FIELD_NONE:
 		if (vin->continuous) {
 			vnmc = VNMC_IM_ODD_EVEN;
@@ -757,18 +756,6 @@ static int rvin_get_active_slot(struct rvin_dev *vin, u32 vnms)
 	return 0;
 }
 
-static enum v4l2_field rvin_get_active_field(struct rvin_dev *vin, u32 vnms)
-{
-	if (vin->format.field == V4L2_FIELD_ALTERNATE) {
-		/* If FS is set it's a Even field */
-		if (vnms & VNMS_FS)
-			return V4L2_FIELD_BOTTOM;
-		return V4L2_FIELD_TOP;
-	}
-
-	return vin->format.field;
-}
-
 static void rvin_set_slot_addr(struct rvin_dev *vin, int slot, dma_addr_t addr)
 {
 	const struct rvin_video_format *fmt;
@@ -941,7 +928,7 @@ static irqreturn_t rvin_irq(int irq, void *data)
 		goto done;
 
 	/* Capture frame */
-	vin->queue_buf[slot]->field = rvin_get_active_field(vin, vnms);
+	vin->queue_buf[slot]->field = vin->format.field;
 	vin->queue_buf[slot]->sequence = sequence;
 	vin->queue_buf[slot]->vb2_buf.timestamp = ktime_get_ns();
 	vb2_buffer_done(&vin->queue_buf[slot]->vb2_buf, VB2_BUF_STATE_DONE);
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index c8c764188b85a926..9f0aac9c3398d613 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -102,6 +102,24 @@ static int rvin_get_sd_format(struct rvin_dev *vin, struct v4l2_pix_format *pix)
 	if (ret)
 		return ret;
 
+	switch (fmt.format.field) {
+	case V4L2_FIELD_TOP:
+	case V4L2_FIELD_BOTTOM:
+	case V4L2_FIELD_NONE:
+	case V4L2_FIELD_INTERLACED_TB:
+	case V4L2_FIELD_INTERLACED_BT:
+	case V4L2_FIELD_INTERLACED:
+		break;
+	case V4L2_FIELD_ALTERNATE:
+		/* Use VIN hardware to combine the two fields */
+		fmt.format.field = V4L2_FIELD_INTERLACED;
+		fmt.format.height *= 2;
+		break;
+	default:
+		vin->format.field = V4L2_FIELD_NONE;
+		break;
+	}
+
 	v4l2_fill_pix_format(pix, &fmt.format);
 
 	return 0;
@@ -115,33 +133,6 @@ int rvin_reset_format(struct rvin_dev *vin)
 	if (ret)
 		return ret;
 
-	/*
-	 * If the subdevice uses ALTERNATE field mode and G_STD is
-	 * implemented use the VIN HW to combine the two fields to
-	 * one INTERLACED frame. The ALTERNATE field mode can still
-	 * be requested in S_FMT and be respected, this is just the
-	 * default which is applied at probing or when S_STD is called.
-	 */
-	if (vin->format.field == V4L2_FIELD_ALTERNATE &&
-	    v4l2_subdev_has_op(vin_to_source(vin), video, g_std))
-		vin->format.field = V4L2_FIELD_INTERLACED;
-
-	switch (vin->format.field) {
-	case V4L2_FIELD_TOP:
-	case V4L2_FIELD_BOTTOM:
-	case V4L2_FIELD_ALTERNATE:
-		vin->format.height /= 2;
-		break;
-	case V4L2_FIELD_NONE:
-	case V4L2_FIELD_INTERLACED_TB:
-	case V4L2_FIELD_INTERLACED_BT:
-	case V4L2_FIELD_INTERLACED:
-		break;
-	default:
-		vin->format.field = V4L2_FIELD_NONE;
-		break;
-	}
-
 	vin->crop.top = vin->crop.left = 0;
 	vin->crop.width = vin->format.width;
 	vin->crop.height = vin->format.height;
@@ -226,9 +217,6 @@ static int __rvin_try_format(struct rvin_dev *vin,
 	switch (pix->field) {
 	case V4L2_FIELD_TOP:
 	case V4L2_FIELD_BOTTOM:
-	case V4L2_FIELD_ALTERNATE:
-		pix->height /= 2;
-		break;
 	case V4L2_FIELD_NONE:
 	case V4L2_FIELD_INTERLACED_TB:
 	case V4L2_FIELD_INTERLACED_BT:
-- 
2.14.0

[PATCH v6 12/25] rcar-vin: move media bus configuration to struct rvin_info

[Niklas Söderlund]

Bus configuration will once the driver is extended to to support Gen3
contain information not specific to only the directly connected parallel
subdevice. Move it to struct rvin_info to show it's not always coupled
to the parallel subdevice.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/rcar-vin/rcar-core.c | 14 +++++++-------
 drivers/media/platform/rcar-vin/rcar-dma.c  | 11 ++++++-----
 drivers/media/platform/rcar-vin/rcar-v4l2.c |  2 +-
 drivers/media/platform/rcar-vin/rcar-vin.h  |  9 ++++-----
 4 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index 4dc148e7835439ab..65f01b6781c0aefd 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -45,15 +45,15 @@ static int rvin_find_pad(struct v4l2_subdev *sd, int direction)
 	return -EINVAL;
 }
 
-static bool rvin_mbus_supported(struct rvin_graph_entity *entity)
+static bool rvin_mbus_supported(struct rvin_dev *vin)
 {
-	struct v4l2_subdev *sd = entity->subdev;
+	struct v4l2_subdev *sd = vin->digital.subdev;
 	struct v4l2_subdev_mbus_code_enum code = {
 		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
 	};
 
 	code.index = 0;
-	code.pad = entity->source_pad;
+	code.pad = vin->digital.source_pad;
 	while (!v4l2_subdev_call(sd, pad, enum_mbus_code, NULL, &code)) {
 		code.index++;
 		switch (code.code) {
@@ -61,7 +61,7 @@ static bool rvin_mbus_supported(struct rvin_graph_entity *entity)
 		case MEDIA_BUS_FMT_UYVY8_2X8:
 		case MEDIA_BUS_FMT_UYVY10_2X10:
 		case MEDIA_BUS_FMT_RGB888_1X24:
-			entity->code = code.code;
+			vin->code = code.code;
 			return true;
 		default:
 			break;
@@ -78,14 +78,14 @@ static int rvin_digital_notify_complete(struct v4l2_async_notifier *notifier)
 	int ret;
 
 	/* Verify subdevices mbus format */
-	if (!rvin_mbus_supported(&vin->digital)) {
+	if (!rvin_mbus_supported(vin)) {
 		vin_err(vin, "Unsupported media bus format for %s\n",
 			vin->digital.subdev->name);
 		return -EINVAL;
 	}
 
 	vin_dbg(vin, "Found media bus format for %s: %d\n",
-		vin->digital.subdev->name, vin->digital.code);
+		vin->digital.subdev->name, vin->code);
 
 	ret = v4l2_device_register_subdev_nodes(&vin->v4l2_dev);
 	if (ret < 0) {
@@ -219,7 +219,7 @@ static int rvin_digital_graph_parse(struct rvin_dev *vin)
 	}
 	of_node_put(np);
 
-	ret = rvin_digitial_parse_v4l2(vin, ep, &vin->digital.mbus_cfg);
+	ret = rvin_digitial_parse_v4l2(vin, ep, &vin->mbus_cfg);
 	of_node_put(ep);
 	if (ret)
 		return ret;
diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index f22bec062db31772..9362e7dba5e3ba95 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -633,7 +633,7 @@ static int rvin_setup(struct rvin_dev *vin)
 	/*
 	 * Input interface
 	 */
-	switch (vin->digital.code) {
+	switch (vin->code) {
 	case MEDIA_BUS_FMT_YUYV8_1X16:
 		/* BT.601/BT.1358 16bit YCbCr422 */
 		vnmc |= VNMC_INF_YUV16;
@@ -641,7 +641,7 @@ static int rvin_setup(struct rvin_dev *vin)
 		break;
 	case MEDIA_BUS_FMT_UYVY8_2X8:
 		/* BT.656 8bit YCbCr422 or BT.601 8bit YCbCr422 */
-		vnmc |= vin->digital.mbus_cfg.type == V4L2_MBUS_BT656 ?
+		vnmc |= vin->mbus_cfg.type == V4L2_MBUS_BT656 ?
 			VNMC_INF_YUV8_BT656 : VNMC_INF_YUV8_BT601;
 		input_is_yuv = true;
 		break;
@@ -650,7 +650,7 @@ static int rvin_setup(struct rvin_dev *vin)
 		break;
 	case MEDIA_BUS_FMT_UYVY10_2X10:
 		/* BT.656 10bit YCbCr422 or BT.601 10bit YCbCr422 */
-		vnmc |= vin->digital.mbus_cfg.type == V4L2_MBUS_BT656 ?
+		vnmc |= vin->mbus_cfg.type == V4L2_MBUS_BT656 ?
 			VNMC_INF_YUV10_BT656 : VNMC_INF_YUV10_BT601;
 		input_is_yuv = true;
 		break;
@@ -662,11 +662,11 @@ static int rvin_setup(struct rvin_dev *vin)
 	dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
 
 	/* Hsync Signal Polarity Select */
-	if (!(vin->digital.mbus_cfg.flags & V4L2_MBUS_HSYNC_ACTIVE_LOW))
+	if (!(vin->mbus_cfg.flags & V4L2_MBUS_HSYNC_ACTIVE_LOW))
 		dmr2 |= VNDMR2_HPS;
 
 	/* Vsync Signal Polarity Select */
-	if (!(vin->digital.mbus_cfg.flags & V4L2_MBUS_VSYNC_ACTIVE_LOW))
+	if (!(vin->mbus_cfg.flags & V4L2_MBUS_VSYNC_ACTIVE_LOW))
 		dmr2 |= VNDMR2_VPS;
 
 	/*
@@ -875,6 +875,7 @@ static void rvin_capture_stop(struct rvin_dev *vin)
 	rvin_write(vin, rvin_read(vin, VNMC_REG) & ~VNMC_ME, VNMC_REG);
 }
 
+
 /* -----------------------------------------------------------------------------
  * DMA Functions
  */
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index 9f0aac9c3398d613..fb9f802e553e9b80 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -161,7 +161,7 @@ static int __rvin_try_format_source(struct rvin_dev *vin,
 
 	sd = vin_to_source(vin);
 
-	v4l2_fill_mbus_format(&format.format, pix, vin->digital.code);
+	v4l2_fill_mbus_format(&format.format, pix, vin->code);
 
 	pad_cfg = v4l2_subdev_alloc_pad_config(sd);
 	if (pad_cfg == NULL)
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index cd8d9a96f78f3267..82f074c601ea4efe 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -62,8 +62,6 @@ struct rvin_video_format {
  * struct rvin_graph_entity - Video endpoint from async framework
  * @asd:	sub-device descriptor for async framework
  * @subdev:	subdevice matched using async framework
- * @code:	Media bus format from source
- * @mbus_cfg:	Media bus format from DT
  * @source_pad:	source pad of remote subdevice
  * @sink_pad:	sink pad of remote subdevice
  */
@@ -71,9 +69,6 @@ struct rvin_graph_entity {
 	struct v4l2_async_subdev asd;
 	struct v4l2_subdev *subdev;
 
-	u32 code;
-	struct v4l2_mbus_config mbus_cfg;
-
 	unsigned int source_pad;
 	unsigned int sink_pad;
 };
@@ -115,6 +110,8 @@ struct rvin_info {
  * @sequence:		V4L2 buffers sequence number
  * @state:		keeps track of operation state
  *
+ * @mbus_cfg:		media bus format from DT
+ * @code:		media bus coide from subdevice
  * @format:		active V4L2 pixel format
  *
  * @crop:		active cropping
@@ -141,6 +138,8 @@ struct rvin_dev {
 	unsigned int sequence;
 	enum rvin_dma_state state;
 
+	struct v4l2_mbus_config mbus_cfg;
+	u32 code;
 	struct v4l2_pix_format format;
 
 	struct v4l2_rect crop;
-- 
2.14.0

[PATCH v6 13/25] rcar-vin: enable Gen3 hardware configuration

[Niklas Söderlund]

Add the register needed to work with Gen3 hardware. This patch adds
the logic for how to work with the Gen3 hardware. More work is required
to enable the subdevice structure needed to configure capturing.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/rcar-vin/rcar-dma.c | 94 ++++++++++++++++++++----------
 drivers/media/platform/rcar-vin/rcar-vin.h |  1 +
 2 files changed, 64 insertions(+), 31 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index 9362e7dba5e3ba95..c4f8e81e88c99e28 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -33,21 +33,23 @@
 #define VNELPRC_REG	0x10	/* Video n End Line Pre-Clip Register */
 #define VNSPPRC_REG	0x14	/* Video n Start Pixel Pre-Clip Register */
 #define VNEPPRC_REG	0x18	/* Video n End Pixel Pre-Clip Register */
-#define VNSLPOC_REG	0x1C	/* Video n Start Line Post-Clip Register */
-#define VNELPOC_REG	0x20	/* Video n End Line Post-Clip Register */
-#define VNSPPOC_REG	0x24	/* Video n Start Pixel Post-Clip Register */
-#define VNEPPOC_REG	0x28	/* Video n End Pixel Post-Clip Register */
 #define VNIS_REG	0x2C	/* Video n Image Stride Register */
 #define VNMB_REG(m)	(0x30 + ((m) << 2)) /* Video n Memory Base m Register */
 #define VNIE_REG	0x40	/* Video n Interrupt Enable Register */
 #define VNINTS_REG	0x44	/* Video n Interrupt Status Register */
 #define VNSI_REG	0x48	/* Video n Scanline Interrupt Register */
 #define VNMTC_REG	0x4C	/* Video n Memory Transfer Control Register */
-#define VNYS_REG	0x50	/* Video n Y Scale Register */
-#define VNXS_REG	0x54	/* Video n X Scale Register */
 #define VNDMR_REG	0x58	/* Video n Data Mode Register */
 #define VNDMR2_REG	0x5C	/* Video n Data Mode Register 2 */
 #define VNUVAOF_REG	0x60	/* Video n UV Address Offset Register */
+
+/* Register offsets specific for Gen2 */
+#define VNSLPOC_REG	0x1C	/* Video n Start Line Post-Clip Register */
+#define VNELPOC_REG	0x20	/* Video n End Line Post-Clip Register */
+#define VNSPPOC_REG	0x24	/* Video n Start Pixel Post-Clip Register */
+#define VNEPPOC_REG	0x28	/* Video n End Pixel Post-Clip Register */
+#define VNYS_REG	0x50	/* Video n Y Scale Register */
+#define VNXS_REG	0x54	/* Video n X Scale Register */
 #define VNC1A_REG	0x80	/* Video n Coefficient Set C1A Register */
 #define VNC1B_REG	0x84	/* Video n Coefficient Set C1B Register */
 #define VNC1C_REG	0x88	/* Video n Coefficient Set C1C Register */
@@ -73,9 +75,13 @@
 #define VNC8B_REG	0xF4	/* Video n Coefficient Set C8B Register */
 #define VNC8C_REG	0xF8	/* Video n Coefficient Set C8C Register */
 
+/* Register offsets specific for Gen3 */
+#define VNCSI_IFMD_REG		0x20 /* Video n CSI2 Interface Mode Register */
 
 /* Register bit fields for R-Car VIN */
 /* Video n Main Control Register bits */
+#define VNMC_DPINE		(1 << 27) /* Gen3 specific */
+#define VNMC_SCLE		(1 << 26) /* Gen3 specific */
 #define VNMC_FOC		(1 << 21)
 #define VNMC_YCAL		(1 << 19)
 #define VNMC_INF_YUV8_BT656	(0 << 16)
@@ -119,6 +125,13 @@
 #define VNDMR2_FTEV		(1 << 17)
 #define VNDMR2_VLV(n)		((n & 0xf) << 12)
 
+/* Video n CSI2 Interface Mode Register (Gen3) */
+#define VNCSI_IFMD_DES2		(1 << 27)
+#define VNCSI_IFMD_DES1		(1 << 26)
+#define VNCSI_IFMD_DES0		(1 << 25)
+#define VNCSI_IFMD_CSI_CHSEL(n) ((n & 0xf) << 0)
+#define VNCSI_IFMD_CSI_CHSEL_MASK 0xf
+
 struct rvin_buffer {
 	struct vb2_v4l2_buffer vb;
 	struct list_head list;
@@ -514,28 +527,10 @@ static void rvin_set_coeff(struct rvin_dev *vin, unsigned short xs)
 	rvin_write(vin, p_set->coeff_set[23], VNC8C_REG);
 }
 
-static void rvin_crop_scale_comp(struct rvin_dev *vin)
+static void rvin_crop_scale_comp_gen2(struct rvin_dev *vin)
 {
 	u32 xs, ys;
 
-	/* Set Start/End Pixel/Line Pre-Clip */
-	rvin_write(vin, vin->crop.left, VNSPPRC_REG);
-	rvin_write(vin, vin->crop.left + vin->crop.width - 1, VNEPPRC_REG);
-	switch (vin->format.field) {
-	case V4L2_FIELD_INTERLACED:
-	case V4L2_FIELD_INTERLACED_TB:
-	case V4L2_FIELD_INTERLACED_BT:
-		rvin_write(vin, vin->crop.top / 2, VNSLPRC_REG);
-		rvin_write(vin, (vin->crop.top + vin->crop.height) / 2 - 1,
-			   VNELPRC_REG);
-		break;
-	default:
-		rvin_write(vin, vin->crop.top, VNSLPRC_REG);
-		rvin_write(vin, vin->crop.top + vin->crop.height - 1,
-			   VNELPRC_REG);
-		break;
-	}
-
 	/* Set scaling coefficient */
 	ys = 0;
 	if (vin->crop.height != vin->compose.height)
@@ -573,11 +568,6 @@ static void rvin_crop_scale_comp(struct rvin_dev *vin)
 		break;
 	}
 
-	if (vin->format.pixelformat == V4L2_PIX_FMT_NV16)
-		rvin_write(vin, ALIGN(vin->format.width, 0x20), VNIS_REG);
-	else
-		rvin_write(vin, ALIGN(vin->format.width, 0x10), VNIS_REG);
-
 	vin_dbg(vin,
 		"Pre-Clip: %ux%u@%u:%u YS: %d XS: %d Post-Clip: %ux%u@%u:%u\n",
 		vin->crop.width, vin->crop.height, vin->crop.left,
@@ -585,6 +575,37 @@ static void rvin_crop_scale_comp(struct rvin_dev *vin)
 		0, 0);
 }
 
+static void rvin_crop_scale_comp(struct rvin_dev *vin)
+{
+	/* Set Start/End Pixel/Line Pre-Clip */
+	rvin_write(vin, vin->crop.left, VNSPPRC_REG);
+	rvin_write(vin, vin->crop.left + vin->crop.width - 1, VNEPPRC_REG);
+
+	switch (vin->format.field) {
+	case V4L2_FIELD_INTERLACED:
+	case V4L2_FIELD_INTERLACED_TB:
+	case V4L2_FIELD_INTERLACED_BT:
+		rvin_write(vin, vin->crop.top / 2, VNSLPRC_REG);
+		rvin_write(vin, (vin->crop.top + vin->crop.height) / 2 - 1,
+			   VNELPRC_REG);
+		break;
+	default:
+		rvin_write(vin, vin->crop.top, VNSLPRC_REG);
+		rvin_write(vin, vin->crop.top + vin->crop.height - 1,
+			   VNELPRC_REG);
+		break;
+	}
+
+	/* TODO: Add support for the UDS scaler. */
+	if (vin->info->chip != RCAR_GEN3)
+		rvin_crop_scale_comp_gen2(vin);
+
+	if (vin->format.pixelformat == V4L2_PIX_FMT_NV16)
+		rvin_write(vin, ALIGN(vin->format.width, 0x20), VNIS_REG);
+	else
+		rvin_write(vin, ALIGN(vin->format.width, 0x10), VNIS_REG);
+}
+
 /* -----------------------------------------------------------------------------
  * Hardware setup
  */
@@ -659,7 +680,10 @@ static int rvin_setup(struct rvin_dev *vin)
 	}
 
 	/* Enable VSYNC Field Toogle mode after one VSYNC input */
-	dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
+	if (vin->info->chip == RCAR_GEN3)
+		dmr2 = VNDMR2_FTEV;
+	else
+		dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
 
 	/* Hsync Signal Polarity Select */
 	if (!(vin->mbus_cfg.flags & V4L2_MBUS_HSYNC_ACTIVE_LOW))
@@ -711,6 +735,14 @@ static int rvin_setup(struct rvin_dev *vin)
 	if (input_is_yuv == output_is_yuv)
 		vnmc |= VNMC_BPS;
 
+	if (vin->info->chip == RCAR_GEN3) {
+		/* Select between CSI-2 and Digital input */
+		if (vin->mbus_cfg.type == V4L2_MBUS_CSI2)
+			vnmc &= ~VNMC_DPINE;
+		else
+			vnmc |= VNMC_DPINE;
+	}
+
 	/* Progressive or interlaced mode */
 	interrupts = progressive ? VNIE_FIE : VNIE_EFE;
 
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 82f074c601ea4efe..c4608686666d2d81 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -33,6 +33,7 @@ enum chip_id {
 	RCAR_H1,
 	RCAR_M1,
 	RCAR_GEN2,
+	RCAR_GEN3,
 };
 
 /**
-- 
2.14.0

[PATCH v6 14/25] rcar-vin: add functions to manipulate Gen3 CHSEL value

[Niklas Söderlund]

On Gen3 the CSI-2 routing is controlled by the VnCSI_IFMD register. One
feature of this register is that it's only present in the VIN0 and VIN4
instances. The register in VIN0 controls the routing for VIN0-3 and the
register in VIN4 controls routing for VIN4-7.

To be able to control routing from a media device these functions need
to control runtime PM for the subgroup master (VIN0 and VIN4). The
subgroup master must be switched on before the register is manipulated,
once the operation is complete it's safe to switch the master off and
the new routing will still be in effect.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/rcar-vin/rcar-dma.c | 41 ++++++++++++++++++++++++++++++
 drivers/media/platform/rcar-vin/rcar-vin.h |  3 +++
 2 files changed, 44 insertions(+)

diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index c4f8e81e88c99e28..6206fab7b6cdc55a 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -16,6 +16,7 @@
 
 #include <linux/delay.h>
 #include <linux/interrupt.h>
+#include <linux/pm_runtime.h>
 
 #include <media/videobuf2-dma-contig.h>
 
@@ -1228,3 +1229,43 @@ int rvin_dma_probe(struct rvin_dev *vin, int irq)
 
 	return ret;
 }
+
+/* -----------------------------------------------------------------------------
+ * Gen3 CHSEL manipulation
+ */
+
+void rvin_set_chsel(struct rvin_dev *vin, u8 chsel)
+{
+	u32 ifmd;
+
+	pm_runtime_get_sync(vin->dev);
+
+	/*
+	 * Undocumented feature: Writing to VNCSI_IFMD_REG will go
+	 * through and on read back look correct but won't have
+	 * any effect if VNMC_REG is not first set to 0.
+	 */
+	rvin_write(vin, 0, VNMC_REG);
+
+	ifmd = VNCSI_IFMD_DES2 | VNCSI_IFMD_DES1 | VNCSI_IFMD_DES0 |
+		VNCSI_IFMD_CSI_CHSEL(chsel);
+
+	rvin_write(vin, ifmd, VNCSI_IFMD_REG);
+
+	vin_dbg(vin, "Set IFMD 0x%x\n", ifmd);
+
+	pm_runtime_put(vin->dev);
+}
+
+int rvin_get_chsel(struct rvin_dev *vin)
+{
+	int chsel;
+
+	pm_runtime_get_sync(vin->dev);
+
+	chsel = rvin_read(vin, VNCSI_IFMD_REG) & VNCSI_IFMD_CSI_CHSEL_MASK;
+
+	pm_runtime_put(vin->dev);
+
+	return chsel;
+}
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index c4608686666d2d81..94c606f2b8f2f246 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -164,4 +164,7 @@ int rvin_reset_format(struct rvin_dev *vin);
 
 const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
 
+void rvin_set_chsel(struct rvin_dev *vin, u8 chsel);
+int rvin_get_chsel(struct rvin_dev *vin);
+
 #endif
-- 
2.14.0

[PATCH v6 15/25] rcar-vin: add flag to switch to media controller mode

[Niklas Söderlund]

On Gen3 a media controller API needs to be used to allow userspace to
configure the subdevices in the pipeline instead of directly controlling
a single source subdevice, which is and will continue to be the mode of
operation on Gen2.

Prepare for these two modes of operation by adding a flag to struct
rvin_graph_entity which will control which mode to use.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/rcar-vin/rcar-core.c | 6 +++++-
 drivers/media/platform/rcar-vin/rcar-vin.h  | 2 ++
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index 65f01b6781c0aefd..fbbb22924cf3a045 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -279,18 +279,21 @@ static int rvin_digital_graph_init(struct rvin_dev *vin)
 
 static const struct rvin_info rcar_info_h1 = {
 	.chip = RCAR_H1,
+	.use_mc = false,
 	.max_width = 2048,
 	.max_height = 2048,
 };
 
 static const struct rvin_info rcar_info_m1 = {
 	.chip = RCAR_M1,
+	.use_mc = false,
 	.max_width = 2048,
 	.max_height = 2048,
 };
 
 static const struct rvin_info rcar_info_gen2 = {
 	.chip = RCAR_GEN2,
+	.use_mc = false,
 	.max_width = 2048,
 	.max_height = 2048,
 };
@@ -387,7 +390,8 @@ static int rcar_vin_remove(struct platform_device *pdev)
 	v4l2_async_notifier_unregister(&vin->notifier);
 
 	/* Checks internaly if handlers have been init or not */
-	v4l2_ctrl_handler_free(&vin->ctrl_handler);
+	if (!vin->info->use_mc)
+		v4l2_ctrl_handler_free(&vin->ctrl_handler);
 
 	rvin_v4l2_remove(vin);
 
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 94c606f2b8f2f246..819d9c04ed8ffb36 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -77,12 +77,14 @@ struct rvin_graph_entity {
 /**
  * struct rvin_info - Information about the particular VIN implementation
  * @chip:		type of VIN chip
+ * @use_mc:		use media controller instead of controlling subdevice
  *
  * max_width:		max input width the VIN supports
  * max_height:		max input height the VIN supports
  */
 struct rvin_info {
 	enum chip_id chip;
+	bool use_mc;
 
 	unsigned int max_width;
 	unsigned int max_height;
-- 
2.14.0

[PATCH v6 16/25] rcar-vin: break out format alignment and checking

[Niklas Söderlund]

Part of the format aliment and checking can be shared with the Gen3
format handling. Break that part out to it's own function. While doing
this clean up the checking and add more checks.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/rcar-vin/rcar-v4l2.c | 98 +++++++++++++++--------------
 1 file changed, 51 insertions(+), 47 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index fb9f802e553e9b80..f71dea8d5d40323a 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -86,6 +86,56 @@ static u32 rvin_format_sizeimage(struct v4l2_pix_format *pix)
 	return pix->bytesperline * pix->height;
 }
 
+static int rvin_format_align(struct rvin_dev *vin, struct v4l2_pix_format *pix)
+{
+	u32 walign;
+
+	/* If requested format is not supported fallback to the default */
+	if (!rvin_format_from_pixel(pix->pixelformat)) {
+		vin_dbg(vin, "Format 0x%x not found, using default 0x%x\n",
+			pix->pixelformat, RVIN_DEFAULT_FORMAT);
+		pix->pixelformat = RVIN_DEFAULT_FORMAT;
+	}
+
+	switch (pix->field) {
+	case V4L2_FIELD_TOP:
+	case V4L2_FIELD_BOTTOM:
+	case V4L2_FIELD_NONE:
+	case V4L2_FIELD_INTERLACED_TB:
+	case V4L2_FIELD_INTERLACED_BT:
+	case V4L2_FIELD_INTERLACED:
+		break;
+	default:
+		pix->field = V4L2_FIELD_NONE;
+		break;
+	}
+
+	/* Check that colorspace is resonable, if not keep current */
+	if (!pix->colorspace || pix->colorspace >= 0xff)
+		pix->colorspace = vin->format.colorspace;
+
+	/* HW limit width to a multiple of 32 (2^5) for NV16 else 2 (2^1) */
+	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
+
+	/* Limit to VIN capabilities */
+	v4l_bound_align_image(&pix->width, 2, vin->info->max_width, walign,
+			      &pix->height, 4, vin->info->max_height, 2, 0);
+
+	pix->bytesperline = rvin_format_bytesperline(pix);
+	pix->sizeimage = rvin_format_sizeimage(pix);
+
+	if (vin->info->chip == RCAR_M1 &&
+	    pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
+		vin_err(vin, "pixel format XBGR32 not supported on M1\n");
+		return -EINVAL;
+	}
+
+	vin_dbg(vin, "Format %ux%u bpl: %d size: %d\n",
+		pix->width, pix->height, pix->bytesperline, pix->sizeimage);
+
+	return 0;
+}
+
 /* -----------------------------------------------------------------------------
  * V4L2
  */
@@ -191,64 +241,18 @@ static int __rvin_try_format_source(struct rvin_dev *vin,
 static int __rvin_try_format(struct rvin_dev *vin,
 			     u32 which, struct v4l2_pix_format *pix)
 {
-	u32 walign;
 	int ret;
 
 	/* Keep current field if no specific one is asked for */
 	if (pix->field == V4L2_FIELD_ANY)
 		pix->field = vin->format.field;
 
-	/* If requested format is not supported fallback to the default */
-	if (!rvin_format_from_pixel(pix->pixelformat)) {
-		vin_dbg(vin, "Format 0x%x not found, using default 0x%x\n",
-			pix->pixelformat, RVIN_DEFAULT_FORMAT);
-		pix->pixelformat = RVIN_DEFAULT_FORMAT;
-	}
-
-	/* Always recalculate */
-	pix->bytesperline = 0;
-	pix->sizeimage = 0;
-
 	/* Limit to source capabilities */
 	ret = __rvin_try_format_source(vin, which, pix);
 	if (ret)
 		return ret;
 
-	switch (pix->field) {
-	case V4L2_FIELD_TOP:
-	case V4L2_FIELD_BOTTOM:
-	case V4L2_FIELD_NONE:
-	case V4L2_FIELD_INTERLACED_TB:
-	case V4L2_FIELD_INTERLACED_BT:
-	case V4L2_FIELD_INTERLACED:
-		break;
-	default:
-		pix->field = V4L2_FIELD_NONE;
-		break;
-	}
-
-	/* HW limit width to a multiple of 32 (2^5) for NV16 else 2 (2^1) */
-	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
-
-	/* Limit to VIN capabilities */
-	v4l_bound_align_image(&pix->width, 2, vin->info->max_width, walign,
-			      &pix->height, 4, vin->info->max_height, 2, 0);
-
-	pix->bytesperline = max_t(u32, pix->bytesperline,
-				  rvin_format_bytesperline(pix));
-	pix->sizeimage = max_t(u32, pix->sizeimage,
-			       rvin_format_sizeimage(pix));
-
-	if (vin->info->chip == RCAR_M1 &&
-	    pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
-		vin_err(vin, "pixel format XBGR32 not supported on M1\n");
-		return -EINVAL;
-	}
-
-	vin_dbg(vin, "Format %ux%u bpl: %d size: %d\n",
-		pix->width, pix->height, pix->bytesperline, pix->sizeimage);
-
-	return 0;
+	return rvin_format_align(vin, pix);
 }
 
 static int rvin_querycap(struct file *file, void *priv,
-- 
2.14.0

[PATCH v6 17/25] rcar-vin: use different v4l2 operations in media controller mode

[Niklas Söderlund]

When the driver runs in media controller mode it should not directly
control the subdevice instead userspace will be responsible for
configuring the pipeline. To be able to run in this mode a different set
of v4l2 operations needs to be used.

Add a new set of v4l2 operations to support the running without directly
interacting with the source subdevice.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/rcar-vin/rcar-dma.c  |   3 +-
 drivers/media/platform/rcar-vin/rcar-v4l2.c | 155 +++++++++++++++++++++++++++-
 drivers/media/platform/rcar-vin/rcar-vin.h  |   1 +
 3 files changed, 155 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index 6206fab7b6cdc55a..de1486ee2786b499 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -628,7 +628,8 @@ static int rvin_setup(struct rvin_dev *vin)
 		/* Default to TB */
 		vnmc = VNMC_IM_FULL;
 		/* Use BT if video standard can be read and is 60 Hz format */
-		if (!v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
+		if (!vin->info->use_mc &&
+		    !v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
 			if (std & V4L2_STD_525_60)
 				vnmc = VNMC_IM_FULL | VNMC_FOC;
 		}
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index f71dea8d5d40323a..9d540644bbe446f6 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -23,6 +23,9 @@
 #include "rcar-vin.h"
 
 #define RVIN_DEFAULT_FORMAT	V4L2_PIX_FMT_YUYV
+#define RVIN_DEFAULT_WIDTH	800
+#define RVIN_DEFAULT_HEIGHT	600
+#define RVIN_DEFAULT_COLORSPACE	V4L2_COLORSPACE_SRGB
 
 /* -----------------------------------------------------------------------------
  * Format Conversions
@@ -671,6 +674,84 @@ static const struct v4l2_ioctl_ops rvin_ioctl_ops = {
 	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
 };
 
+/* -----------------------------------------------------------------------------
+ * V4L2 Media Controller
+ */
+
+static int __rvin_mc_try_format(struct rvin_dev *vin,
+				struct v4l2_pix_format *pix)
+{
+	/* Keep current field if no specific one is asked for */
+	if (pix->field == V4L2_FIELD_ANY)
+		pix->field = vin->format.field;
+
+	return rvin_format_align(vin, pix);
+}
+
+static int rvin_mc_try_fmt_vid_cap(struct file *file, void *priv,
+				   struct v4l2_format *f)
+{
+	struct rvin_dev *vin = video_drvdata(file);
+
+	return __rvin_mc_try_format(vin, &f->fmt.pix);
+}
+
+static int rvin_mc_s_fmt_vid_cap(struct file *file, void *priv,
+				 struct v4l2_format *f)
+{
+	struct rvin_dev *vin = video_drvdata(file);
+	int ret;
+
+	if (vb2_is_busy(&vin->queue))
+		return -EBUSY;
+
+	ret = __rvin_mc_try_format(vin, &f->fmt.pix);
+	if (ret)
+		return ret;
+
+	vin->format = f->fmt.pix;
+
+	return 0;
+}
+
+static int rvin_mc_enum_input(struct file *file, void *priv,
+			      struct v4l2_input *i)
+{
+	if (i->index != 0)
+		return -EINVAL;
+
+	i->type = V4L2_INPUT_TYPE_CAMERA;
+	strlcpy(i->name, "Camera", sizeof(i->name));
+
+	return 0;
+}
+
+static const struct v4l2_ioctl_ops rvin_mc_ioctl_ops = {
+	.vidioc_querycap		= rvin_querycap,
+	.vidioc_try_fmt_vid_cap		= rvin_mc_try_fmt_vid_cap,
+	.vidioc_g_fmt_vid_cap		= rvin_g_fmt_vid_cap,
+	.vidioc_s_fmt_vid_cap		= rvin_mc_s_fmt_vid_cap,
+	.vidioc_enum_fmt_vid_cap	= rvin_enum_fmt_vid_cap,
+
+	.vidioc_enum_input		= rvin_mc_enum_input,
+	.vidioc_g_input			= rvin_g_input,
+	.vidioc_s_input			= rvin_s_input,
+
+	.vidioc_reqbufs			= vb2_ioctl_reqbufs,
+	.vidioc_create_bufs		= vb2_ioctl_create_bufs,
+	.vidioc_querybuf		= vb2_ioctl_querybuf,
+	.vidioc_qbuf			= vb2_ioctl_qbuf,
+	.vidioc_dqbuf			= vb2_ioctl_dqbuf,
+	.vidioc_expbuf			= vb2_ioctl_expbuf,
+	.vidioc_prepare_buf		= vb2_ioctl_prepare_buf,
+	.vidioc_streamon		= vb2_ioctl_streamon,
+	.vidioc_streamoff		= vb2_ioctl_streamoff,
+
+	.vidioc_log_status		= v4l2_ctrl_log_status,
+	.vidioc_subscribe_event		= rvin_subscribe_event,
+	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
+};
+
 /* -----------------------------------------------------------------------------
  * File Operations
  */
@@ -819,6 +900,60 @@ static const struct v4l2_file_operations rvin_fops = {
 	.read		= vb2_fop_read,
 };
 
+/* -----------------------------------------------------------------------------
+ * Media controller file Operations
+ */
+
+static int rvin_mc_open(struct file *file)
+{
+	struct rvin_dev *vin = video_drvdata(file);
+	int ret;
+
+	mutex_lock(&vin->lock);
+
+	file->private_data = vin;
+
+	ret = v4l2_fh_open(file);
+	if (ret)
+		goto unlock;
+
+	pm_runtime_get_sync(vin->dev);
+	v4l2_pipeline_pm_use(&vin->vdev.entity, 1);
+
+unlock:
+	mutex_unlock(&vin->lock);
+
+	return ret;
+}
+
+static int rvin_mc_release(struct file *file)
+{
+	struct rvin_dev *vin = video_drvdata(file);
+	int ret;
+
+	mutex_lock(&vin->lock);
+
+	/* the release helper will cleanup any on-going streaming */
+	ret = _vb2_fop_release(file, NULL);
+
+	v4l2_pipeline_pm_use(&vin->vdev.entity, 0);
+	pm_runtime_put(vin->dev);
+
+	mutex_unlock(&vin->lock);
+
+	return ret;
+}
+
+static const struct v4l2_file_operations rvin_mc_fops = {
+	.owner		= THIS_MODULE,
+	.unlocked_ioctl	= video_ioctl2,
+	.open		= rvin_mc_open,
+	.release	= rvin_mc_release,
+	.poll		= vb2_fop_poll,
+	.mmap		= vb2_fop_mmap,
+	.read		= vb2_fop_read,
+};
+
 void rvin_v4l2_remove(struct rvin_dev *vin)
 {
 	v4l2_info(&vin->v4l2_dev, "Removing %s\n",
@@ -851,19 +986,33 @@ int rvin_v4l2_probe(struct rvin_dev *vin)
 	vin->v4l2_dev.notify = rvin_notify;
 
 	/* video node */
-	vdev->fops = &rvin_fops;
 	vdev->v4l2_dev = &vin->v4l2_dev;
 	vdev->queue = &vin->queue;
 	snprintf(vdev->name, sizeof(vdev->name), "%s %s", KBUILD_MODNAME,
 		 dev_name(vin->dev));
 	vdev->release = video_device_release_empty;
-	vdev->ioctl_ops = &rvin_ioctl_ops;
 	vdev->lock = &vin->lock;
-	vdev->ctrl_handler = &vin->ctrl_handler;
 	vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING |
 		V4L2_CAP_READWRITE;
 
+	/* Set some form of default format */
 	vin->format.pixelformat	= RVIN_DEFAULT_FORMAT;
+	vin->format.width = RVIN_DEFAULT_WIDTH;
+	vin->format.height = RVIN_DEFAULT_HEIGHT;
+	vin->format.colorspace = RVIN_DEFAULT_COLORSPACE;
+
+	ret = rvin_format_align(vin, &vin->format);
+	if (ret)
+		return ret;
+
+	if (vin->info->use_mc) {
+		vdev->fops = &rvin_mc_fops;
+		vdev->ioctl_ops = &rvin_mc_ioctl_ops;
+	} else {
+		vdev->fops = &rvin_fops;
+		vdev->ioctl_ops = &rvin_ioctl_ops;
+		vdev->ctrl_handler = &vin->ctrl_handler;
+	}
 
 	ret = video_register_device(&vin->vdev, VFL_TYPE_GRABBER, -1);
 	if (ret) {
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 819d9c04ed8ffb36..12daff804bb6f77f 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -21,6 +21,7 @@
 #include <media/v4l2-ctrls.h>
 #include <media/v4l2-dev.h>
 #include <media/v4l2-device.h>
+#include <media/v4l2-mc.h>
 #include <media/videobuf2-v4l2.h>
 
 /* Number of HW buffers */
-- 
2.14.0

[PATCH v6 18/25] rcar-vin: prepare for media controller mode initialization

[Niklas Söderlund]

When running in media controller mode a media pad is needed, register
one. Also set the media bus format to CSI-2.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/rcar-vin/rcar-core.c | 23 ++++++++++++++++++++++-
 drivers/media/platform/rcar-vin/rcar-vin.h  |  4 ++++
 2 files changed, 26 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index fbbb22924cf3a045..dd0525f2ba336bc2 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -45,6 +45,10 @@ static int rvin_find_pad(struct v4l2_subdev *sd, int direction)
 	return -EINVAL;
 }
 
+/* -----------------------------------------------------------------------------
+ * Digital async notifier
+ */
+
 static bool rvin_mbus_supported(struct rvin_dev *vin)
 {
 	struct v4l2_subdev *sd = vin->digital.subdev;
@@ -273,6 +277,20 @@ static int rvin_digital_graph_init(struct rvin_dev *vin)
 	return 0;
 }
 
+/* -----------------------------------------------------------------------------
+ * Group async notifier
+ */
+
+static int rvin_group_init(struct rvin_dev *vin)
+{
+	/* All our sources are CSI-2 */
+	vin->mbus_cfg.type = V4L2_MBUS_CSI2;
+	vin->mbus_cfg.flags = 0;
+
+	vin->pad.flags = MEDIA_PAD_FL_SINK;
+	return media_entity_pads_init(&vin->vdev.entity, 1, &vin->pad);
+}
+
 /* -----------------------------------------------------------------------------
  * Platform Device Driver
  */
@@ -365,7 +383,10 @@ static int rcar_vin_probe(struct platform_device *pdev)
 	if (ret)
 		return ret;
 
-	ret = rvin_digital_graph_init(vin);
+	if (vin->info->use_mc)
+		ret = rvin_group_init(vin);
+	else
+		ret = rvin_digital_graph_init(vin);
 	if (ret < 0)
 		goto error;
 
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 12daff804bb6f77f..9c47669669c0469c 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -103,6 +103,8 @@ struct rvin_info {
  * @notifier:		V4L2 asynchronous subdevs notifier
  * @digital:		entity in the DT for local digital subdevice
  *
+ * @pad:		pad for media controller
+ *
  * @lock:		protects @queue
  * @queue:		vb2 buffers queue
  *
@@ -132,6 +134,8 @@ struct rvin_dev {
 	struct v4l2_async_notifier notifier;
 	struct rvin_graph_entity digital;
 
+	struct media_pad pad;
+
 	struct mutex lock;
 	struct vb2_queue queue;
 
-- 
2.14.0

[PATCH v6 19/25] rcar-vin: add group allocator functions

[Niklas Söderlund]

In media controller mode all VIN instances needs to be part of the same
media graph. There is also a need to each VIN instance to know and in
some cases be able to communicate with other VIN instances.

Add a allocator framework where the first VIN instance to be probed
creates a shared data structure and creates a media device.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/rcar-vin/rcar-core.c | 179 +++++++++++++++++++++++++++-
 drivers/media/platform/rcar-vin/rcar-vin.h  |  38 ++++++
 2 files changed, 216 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index dd0525f2ba336bc2..4218a73eb6885486 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -20,11 +20,170 @@
 #include <linux/of_graph.h>
 #include <linux/platform_device.h>
 #include <linux/pm_runtime.h>
+#include <linux/slab.h>
 
 #include <media/v4l2-fwnode.h>
 
 #include "rcar-vin.h"
 
+/* -----------------------------------------------------------------------------
+ * Gen3 CSI2 Group Allocator
+ */
+
+static int rvin_group_read_id(struct rvin_dev *vin, struct device_node *np)
+{
+	u32 val;
+	int ret;
+
+	ret = of_property_read_u32(np, "renesas,id", &val);
+	if (ret) {
+		vin_err(vin, "%s: No renesas,id property found\n",
+			of_node_full_name(np));
+		return -EINVAL;
+	}
+
+	if (val >= RCAR_VIN_NUM) {
+		vin_err(vin, "%s: Invalid renesas,id '%u'\n",
+			of_node_full_name(np), val);
+		return -EINVAL;
+	}
+
+	return val;
+}
+
+static DEFINE_MUTEX(rvin_group_lock);
+static struct rvin_group *rvin_group_data;
+
+static void rvin_group_release(struct kref *kref)
+{
+	struct rvin_group *group =
+		container_of(kref, struct rvin_group, refcount);
+
+	mutex_lock(&rvin_group_lock);
+
+	media_device_unregister(&group->mdev);
+	media_device_cleanup(&group->mdev);
+
+	rvin_group_data = NULL;
+
+	mutex_unlock(&rvin_group_lock);
+
+	kfree(group);
+}
+
+static struct rvin_group *__rvin_group_allocate(struct rvin_dev *vin)
+{
+	struct rvin_group *group;
+
+	if (rvin_group_data) {
+		group = rvin_group_data;
+		kref_get(&group->refcount);
+		vin_dbg(vin, "%s: get group=%p\n", __func__, group);
+		return group;
+	}
+
+	group = kzalloc(sizeof(*group), GFP_KERNEL);
+	if (!group)
+		return NULL;
+
+	kref_init(&group->refcount);
+	rvin_group_data = group;
+
+	vin_dbg(vin, "%s: alloc group=%p\n", __func__, group);
+	return group;
+}
+
+static int rvin_group_add_vin(struct rvin_dev *vin)
+{
+	int ret;
+
+	ret = rvin_group_read_id(vin, vin->dev->of_node);
+	if (ret < 0)
+		return ret;
+
+	mutex_lock(&vin->group->lock);
+
+	if (vin->group->vin[ret]) {
+		mutex_unlock(&vin->group->lock);
+		vin_err(vin, "VIN number %d already occupied\n", ret);
+		return -EINVAL;
+	}
+
+	vin->group->vin[ret] = vin;
+
+	mutex_unlock(&vin->group->lock);
+
+	vin_dbg(vin, "I'm VIN number %d", ret);
+
+	return 0;
+}
+
+static int rvin_group_allocate(struct rvin_dev *vin)
+{
+	struct rvin_group *group;
+	struct media_device *mdev;
+	int ret;
+
+	mutex_lock(&rvin_group_lock);
+
+	group = __rvin_group_allocate(vin);
+	if (!group) {
+		mutex_unlock(&rvin_group_lock);
+		return -ENOMEM;
+	}
+
+	/* Init group data if its not already initialized */
+	mdev = &group->mdev;
+	if (!mdev->dev) {
+		mutex_init(&group->lock);
+		mdev->dev = vin->dev;
+
+		strlcpy(mdev->driver_name, "Renesas VIN",
+			sizeof(mdev->driver_name));
+		strlcpy(mdev->model, vin->dev->of_node->name,
+			sizeof(mdev->model));
+		strlcpy(mdev->bus_info, of_node_full_name(vin->dev->of_node),
+			sizeof(mdev->bus_info));
+		media_device_init(mdev);
+
+		ret = media_device_register(mdev);
+		if (ret) {
+			vin_err(vin, "Failed to register media device\n");
+			kref_put(&group->refcount, rvin_group_release);
+			mutex_unlock(&rvin_group_lock);
+			return ret;
+		}
+	}
+
+	vin->group = group;
+	vin->v4l2_dev.mdev = mdev;
+
+	ret = rvin_group_add_vin(vin);
+	if (ret) {
+		kref_put(&group->refcount, rvin_group_release);
+		mutex_unlock(&rvin_group_lock);
+		return ret;
+	}
+
+	mutex_unlock(&rvin_group_lock);
+
+	return 0;
+}
+
+static void rvin_group_delete(struct rvin_dev *vin)
+{
+	unsigned int i;
+
+	mutex_lock(&vin->group->lock);
+	for (i = 0; i < RCAR_VIN_NUM; i++)
+		if (vin->group->vin[i] == vin)
+			vin->group->vin[i] = NULL;
+	mutex_unlock(&vin->group->lock);
+
+	vin_dbg(vin, "%s: group=%p\n", __func__, &vin->group);
+	kref_put(&vin->group->refcount, rvin_group_release);
+}
+
 /* -----------------------------------------------------------------------------
  * Async notifier
  */
@@ -283,12 +442,27 @@ static int rvin_digital_graph_init(struct rvin_dev *vin)
 
 static int rvin_group_init(struct rvin_dev *vin)
 {
+	int ret;
+
+	ret = rvin_group_allocate(vin);
+	if (ret)
+		return ret;
+
 	/* All our sources are CSI-2 */
 	vin->mbus_cfg.type = V4L2_MBUS_CSI2;
 	vin->mbus_cfg.flags = 0;
 
 	vin->pad.flags = MEDIA_PAD_FL_SINK;
-	return media_entity_pads_init(&vin->vdev.entity, 1, &vin->pad);
+	ret = media_entity_pads_init(&vin->vdev.entity, 1, &vin->pad);
+	if (ret)
+		goto error_group;
+
+	return 0;
+
+error_group:
+	rvin_group_delete(vin);
+
+	return ret;
 }
 
 /* -----------------------------------------------------------------------------
@@ -416,6 +590,9 @@ static int rcar_vin_remove(struct platform_device *pdev)
 
 	rvin_v4l2_remove(vin);
 
+	if (vin->info->use_mc)
+		rvin_group_delete(vin);
+
 	rvin_dma_remove(vin);
 
 	return 0;
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 9c47669669c0469c..88683aaee3b6acd5 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -17,6 +17,8 @@
 #ifndef __RCAR_VIN__
 #define __RCAR_VIN__
 
+#include <linux/kref.h>
+
 #include <media/v4l2-async.h>
 #include <media/v4l2-ctrls.h>
 #include <media/v4l2-dev.h>
@@ -30,6 +32,9 @@
 /* Address alignment mask for HW buffers */
 #define HW_BUFFER_MASK 0x7f
 
+/* Max number on VIN instances that can be in a system */
+#define RCAR_VIN_NUM 8
+
 enum chip_id {
 	RCAR_H1,
 	RCAR_M1,
@@ -37,6 +42,15 @@ enum chip_id {
 	RCAR_GEN3,
 };
 
+enum rvin_csi_id {
+	RVIN_CSI20,
+	RVIN_CSI21,
+	RVIN_CSI40,
+	RVIN_CSI41,
+	RVIN_CSI_MAX,
+	RVIN_NC, /* Not Connected */
+};
+
 /**
  * STOPPED  - No operation in progress
  * RUNNING  - Operation in progress have buffers
@@ -75,6 +89,8 @@ struct rvin_graph_entity {
 	unsigned int sink_pad;
 };
 
+struct rvin_group;
+
 /**
  * struct rvin_info - Information about the particular VIN implementation
  * @chip:		type of VIN chip
@@ -103,6 +119,7 @@ struct rvin_info {
  * @notifier:		V4L2 asynchronous subdevs notifier
  * @digital:		entity in the DT for local digital subdevice
  *
+ * @group:		Gen3 CSI group
  * @pad:		pad for media controller
  *
  * @lock:		protects @queue
@@ -134,6 +151,7 @@ struct rvin_dev {
 	struct v4l2_async_notifier notifier;
 	struct rvin_graph_entity digital;
 
+	struct rvin_group *group;
 	struct media_pad pad;
 
 	struct mutex lock;
@@ -162,6 +180,26 @@ struct rvin_dev {
 #define vin_warn(d, fmt, arg...)	dev_warn(d->dev, fmt, ##arg)
 #define vin_err(d, fmt, arg...)		dev_err(d->dev, fmt, ##arg)
 
+/**
+ * struct rvin_group - VIN CSI2 group information
+ * @refcount:		number of VIN instances using the group
+ *
+ * @mdev:		media device which represents the group
+ *
+ * @lock:		protects the vin and csi members
+ * @vin:		VIN instances which are part of the group
+ * @csi:		CSI-2 entities that are part of the group
+ */
+struct rvin_group {
+	struct kref refcount;
+
+	struct media_device mdev;
+
+	struct mutex lock;
+	struct rvin_dev *vin[RCAR_VIN_NUM];
+	struct rvin_graph_entity csi[RVIN_CSI_MAX];
+};
+
 int rvin_dma_probe(struct rvin_dev *vin, int irq);
 void rvin_dma_remove(struct rvin_dev *vin);
 
-- 
2.14.0

[PATCH v6 20/25] rcar-vin: add chsel information to rvin_info

[Niklas Söderlund]

Each Gen3 SoC has a limited set of predefined routing possibilities for
which CSI-2 device and virtual channel can be routed to which VIN
instance. Prepare to store this information in the struct rvin_info.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/rcar-vin/rcar-vin.h | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 88683aaee3b6acd5..617f254b52fe106d 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -35,6 +35,9 @@
 /* Max number on VIN instances that can be in a system */
 #define RCAR_VIN_NUM 8
 
+/* Max number of CHSEL values for any Gen3 SoC */
+#define RCAR_CHSEL_MAX 6
+
 enum chip_id {
 	RCAR_H1,
 	RCAR_M1,
@@ -91,6 +94,19 @@ struct rvin_graph_entity {
 
 struct rvin_group;
 
+
+/** struct rvin_group_chsel - Map a CSI2 device and channel for a CHSEL value
+ * @csi:		VIN internal number for CSI2 device
+ * @chan:		CSI-2 channel number on remote. Note that channel
+ *			is not the same as VC. The CSI-2 hardware have 4
+ *			channels it can output on but which VC is outputted
+ *			on which channel is configurable inside the CSI-2.
+ */
+struct rvin_group_chsel {
+	enum rvin_csi_id csi;
+	unsigned int chan;
+};
+
 /**
  * struct rvin_info - Information about the particular VIN implementation
  * @chip:		type of VIN chip
@@ -98,6 +114,9 @@ struct rvin_group;
  *
  * max_width:		max input width the VIN supports
  * max_height:		max input height the VIN supports
+ *
+ * num_chsels:		number of possible chsel values for this VIN
+ * chsels:		routing table VIN <-> CSI-2 for the chsel values
  */
 struct rvin_info {
 	enum chip_id chip;
@@ -105,6 +124,9 @@ struct rvin_info {
 
 	unsigned int max_width;
 	unsigned int max_height;
+
+	unsigned int num_chsels;
+	struct rvin_group_chsel chsels[RCAR_VIN_NUM][RCAR_CHSEL_MAX];
 };
 
 /**
-- 
2.14.0

[PATCH v6 21/25] rcar-vin: parse Gen3 OF and setup media graph

[Niklas Söderlund]

Parse the VIN Gen3 OF graph and register all CSI-2 devices in the VIN
group common media device. Once all CSI-2 subdevice is added to the
common media device create links between them.

The parsing and registering CSI-2 subdevices with the v4l2 async
framework is a collaborative effort shared between the VIN instances
which are part of the group. The fist rcar-vin instance parses OF and
finds all other VIN and CSI-2 nodes which are part of the graph. It
stores a bit mask of all VIN instances found and handles to all CSI-2
nodes.

The bit mask is used to figure out when all VIN instances have been
probed. Once the last VIN instance is probed this is detected and this
instance registers all CSI-2 subdevices in its private async notifier.
Once the .complete() callback of this notifier is called it creates the
media controller links between all entities in the graph.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/rcar-vin/rcar-core.c | 296 +++++++++++++++++++++++++++-
 drivers/media/platform/rcar-vin/rcar-vin.h  |   7 +-
 2 files changed, 301 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index 4218a73eb6885486..2aba442a0750e91a 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -440,10 +440,268 @@ static int rvin_digital_graph_init(struct rvin_dev *vin)
  * Group async notifier
  */
 
-static int rvin_group_init(struct rvin_dev *vin)
+/* group lock should be held when calling this function */
+static int rvin_group_add_link(struct rvin_dev *vin,
+			       struct media_entity *source,
+			       unsigned int source_idx,
+			       struct media_entity *sink,
+			       unsigned int sink_idx,
+			       u32 flags)
+{
+	struct media_pad *source_pad, *sink_pad;
+	int ret = 0;
+
+	source_pad = &source->pads[source_idx];
+	sink_pad = &sink->pads[sink_idx];
+
+	if (!media_entity_find_link(source_pad, sink_pad))
+		ret = media_create_pad_link(source, source_idx,
+					    sink, sink_idx, flags);
+
+	if (ret)
+		vin_err(vin, "Error adding link from %s to %s\n",
+			source->name, sink->name);
+
+	return ret;
+}
+
+static int rvin_group_update_links(struct rvin_dev *vin)
 {
+	struct media_entity *source, *sink;
+	struct rvin_dev *master;
+	unsigned int i, n, idx, chsel, csi;
+	u32 flags;
 	int ret;
 
+	mutex_lock(&vin->group->lock);
+
+	for (n = 0; n < RCAR_VIN_NUM; n++) {
+
+		/* Check that VIN is part of the group */
+		if (!vin->group->vin[n])
+			continue;
+
+		/* Check that subgroup master is part of the group */
+		master = vin->group->vin[n < 4 ? 0 : 4];
+		if (!master)
+			continue;
+
+		chsel = rvin_get_chsel(master);
+
+		for (i = 0; i < vin->info->num_chsels; i++) {
+			csi = vin->info->chsels[n][i].csi;
+
+			/* If the CSI-2 is out of bounds it's a noop, skip */
+			if (csi >= RVIN_CSI_MAX)
+				continue;
+
+			/* Check that CSI-2 are part of the group */
+			if (!vin->group->csi[csi].subdev)
+				continue;
+
+			source = &vin->group->csi[csi].subdev->entity;
+			sink = &vin->group->vin[n]->vdev.entity;
+			idx = vin->info->chsels[n][i].chan + 1;
+			flags = i == chsel ? MEDIA_LNK_FL_ENABLED : 0;
+
+			ret = rvin_group_add_link(vin, source, idx, sink, 0,
+						  flags);
+			if (ret)
+				goto out;
+		}
+	}
+out:
+	mutex_unlock(&vin->group->lock);
+
+	return ret;
+}
+
+static int rvin_group_notify_complete(struct v4l2_async_notifier *notifier)
+{
+	struct rvin_dev *vin = notifier_to_vin(notifier);
+	int ret;
+
+	ret = v4l2_device_register_subdev_nodes(&vin->v4l2_dev);
+	if (ret) {
+		vin_err(vin, "Failed to register subdev nodes\n");
+		return ret;
+	}
+
+	return rvin_group_update_links(vin);
+}
+
+static void rvin_group_notify_unbind(struct v4l2_async_notifier *notifier,
+				     struct v4l2_subdev *subdev,
+				     struct v4l2_async_subdev *asd)
+{
+	struct rvin_dev *vin = notifier_to_vin(notifier);
+	struct rvin_graph_entity *csi = to_rvin_graph_entity(asd);
+
+	mutex_lock(&vin->group->lock);
+	csi->subdev = NULL;
+	mutex_unlock(&vin->group->lock);
+}
+
+static int rvin_group_notify_bound(struct v4l2_async_notifier *notifier,
+				   struct v4l2_subdev *subdev,
+				   struct v4l2_async_subdev *asd)
+{
+	struct rvin_dev *vin = notifier_to_vin(notifier);
+	struct rvin_graph_entity *csi = to_rvin_graph_entity(asd);
+
+	v4l2_set_subdev_hostdata(subdev, vin);
+
+	mutex_lock(&vin->group->lock);
+	vin_dbg(vin, "Bound CSI-2 %s\n", subdev->name);
+	csi->subdev = subdev;
+	mutex_unlock(&vin->group->lock);
+
+	return 0;
+}
+
+static struct device_node *rvin_group_get_remote(struct rvin_dev *vin,
+						 struct device_node *node)
+{
+	struct device_node *np;
+
+	np = of_graph_get_remote_port_parent(node);
+	if (!np) {
+		vin_err(vin, "Remote not found %s\n", of_node_full_name(node));
+		return NULL;
+	}
+
+	/* Not all remotes are available, this is OK */
+	if (!of_device_is_available(np)) {
+		vin_dbg(vin, "Remote %s is not available\n",
+			of_node_full_name(np));
+		of_node_put(np);
+		return NULL;
+	}
+
+	return np;
+}
+
+/* group lock should be held when calling this function */
+static int rvin_group_graph_parse(struct rvin_dev *vin, struct device_node *np)
+{
+	int i, id, ret;
+
+	/* Read VIN id from DT */
+	id = rvin_group_read_id(vin, np);
+	if (id < 0)
+		return id;
+
+	/* Check if VIN is already handled */
+	if (vin->group->mask & BIT(id))
+		return 0;
+
+	vin->group->mask |= BIT(id);
+
+	vin_dbg(vin, "Handling VIN%d\n", id);
+
+	/* Parse all enpoints for CSI-2 and VIN nodes */
+	for (i = 0; i < RVIN_CSI_MAX; i++) {
+		struct device_node *ep, *csi, *remote;
+
+		/* Check if instance is connected to the CSI-2 */
+		ep = of_graph_get_endpoint_by_regs(np, 1, i);
+		if (!ep) {
+			vin_dbg(vin, "VIN%d: ep %d not connected\n", id, i);
+			continue;
+		}
+
+		if (vin->group->csi[i].asd.match.fwnode.fwnode) {
+			of_node_put(ep);
+			vin_dbg(vin, "VIN%d: ep %d already handled\n", id, i);
+			continue;
+		}
+
+		csi = rvin_group_get_remote(vin, ep);
+		of_node_put(ep);
+		if (!csi)
+			continue;
+
+		vin->group->csi[i].asd.match.fwnode.fwnode =
+			of_fwnode_handle(csi);
+		vin->group->csi[i].asd.match_type = V4L2_ASYNC_MATCH_FWNODE;
+
+		vin_dbg(vin, "VIN%d ep: %d handled CSI-2 %s\n", id, i,
+			of_node_full_name(csi));
+
+		/* Parse the CSI-2 for all VIN nodes connected to it */
+		ep = NULL;
+		while (1) {
+			ep = of_graph_get_next_endpoint(csi, ep);
+			if (!ep)
+				break;
+
+			remote = rvin_group_get_remote(vin, ep);
+			if (!remote)
+				continue;
+
+			if (of_match_node(vin->dev->driver->of_match_table,
+					  remote)) {
+				ret = rvin_group_graph_parse(vin, remote);
+				if (ret)
+					return ret;
+
+			}
+		}
+	}
+
+	return 0;
+}
+
+static int rvin_group_graph_register(struct rvin_dev *vin)
+{
+	struct v4l2_async_subdev **subdevs = NULL;
+	int i, n, ret, count = 0;
+
+	mutex_lock(&vin->group->lock);
+
+	/* Count how many CSI-2 nodes found */
+	for (i = 0; i < RVIN_CSI_MAX; i++)
+		if (vin->group->csi[i].asd.match.fwnode.fwnode)
+			count++;
+
+	if (!count) {
+		mutex_unlock(&vin->group->lock);
+		return 0;
+	}
+
+	/* Allocate and setup list of subdevices for the notifier */
+	subdevs = devm_kzalloc(vin->dev, sizeof(*subdevs) * count, GFP_KERNEL);
+	if (subdevs == NULL) {
+		mutex_unlock(&vin->group->lock);
+		return -ENOMEM;
+	}
+
+	n = 0;
+	for (i = 0; i < RVIN_CSI_MAX; i++)
+		if (vin->group->csi[i].asd.match.fwnode.fwnode)
+			subdevs[n++] = &vin->group->csi[i].asd;
+
+	vin_dbg(vin, "Claimed %d subdevices for group\n", count);
+
+	vin->notifier.num_subdevs = count;
+	vin->notifier.subdevs = subdevs;
+	vin->notifier.bound = rvin_group_notify_bound;
+	vin->notifier.unbind = rvin_group_notify_unbind;
+	vin->notifier.complete = rvin_group_notify_complete;
+
+	mutex_unlock(&vin->group->lock);
+
+	ret = v4l2_async_notifier_register(&vin->v4l2_dev, &vin->notifier);
+	if (ret < 0)
+		vin_err(vin, "Notifier registration failed\n");
+
+	return ret;
+}
+
+static int rvin_group_init(struct rvin_dev *vin)
+{
+	int i, ret, count_mask, count_vin = 0;
+
 	ret = rvin_group_allocate(vin);
 	if (ret)
 		return ret;
@@ -457,8 +715,44 @@ static int rvin_group_init(struct rvin_dev *vin)
 	if (ret)
 		goto error_group;
 
+	/*
+	 * Check number of registered VIN in group against the group mask.
+	 * If the mask is empty DT have not yet been parsed and if the
+	 * count match all VINs are registered and it's safe to register
+	 * the async notifier
+	 */
+	mutex_lock(&vin->group->lock);
+
+	if (!vin->group->mask) {
+		ret = rvin_group_graph_parse(vin, vin->dev->of_node);
+		if (ret) {
+			mutex_unlock(&vin->group->lock);
+			goto error_group;
+		}
+	}
+
+	for (i = 0; i < RCAR_VIN_NUM; i++)
+		if (vin->group->vin[i])
+			count_vin++;
+
+	count_mask = hweight_long(vin->group->mask);
+
+	mutex_unlock(&vin->group->lock);
+
+	ret = rvin_v4l2_probe(vin);
+	if (ret)
+		goto error_group;
+
+	if (count_vin == count_mask) {
+		ret = rvin_group_graph_register(vin);
+		if (ret)
+			goto error_vdev;
+	}
+
 	return 0;
 
+error_vdev:
+	rvin_v4l2_remove(vin);
 error_group:
 	rvin_group_delete(vin);
 
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 617f254b52fe106d..a41301833221c750 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -92,6 +92,9 @@ struct rvin_graph_entity {
 	unsigned int sink_pad;
 };
 
+#define to_rvin_graph_entity(asd) \
+	container_of(asd, struct rvin_graph_entity, asd)
+
 struct rvin_group;
 
 
@@ -208,7 +211,8 @@ struct rvin_dev {
  *
  * @mdev:		media device which represents the group
  *
- * @lock:		protects the vin and csi members
+ * @lock:		protects the mask, vin and csi members
+ * @mask:		Mask of VIN instances found in DT
  * @vin:		VIN instances which are part of the group
  * @csi:		CSI-2 entities that are part of the group
  */
@@ -218,6 +222,7 @@ struct rvin_group {
 	struct media_device mdev;
 
 	struct mutex lock;
+	unsigned long mask;
 	struct rvin_dev *vin[RCAR_VIN_NUM];
 	struct rvin_graph_entity csi[RVIN_CSI_MAX];
 };
-- 
2.14.0

[PATCH v6 22/25] rcar-vin: add link notify for Gen3

[Niklas Söderlund]

Add the ability to process media device link change request. Link
enablement are a bit complicated on Gen3, if it's possible to enable a
link depends on what other links already are enabled. On Gen3 the 8 VIN
are split into two subgroups (VIN0-3 and VIN4-7) and from a routing
perspective these two groups are independent of each other. Each
subgroups routing is controlled by the subgroup VIN master instance
(VIN0 and VIN4).

There are a limited number of possible route setups available for each
subgroup and the configuration of each setup is dictated by the
hardware. On H3 for example there are 6 possible route setups for each
subgroup to choose from.

This leads to the media device link notification code being rather large
since it will find the best routing configuration to try and accommodate
as many links as possible. When it's not possible to enable a new link
due to hardware constrains the link_notifier callback will return
-EMLINK.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/rcar-vin/rcar-core.c | 203 ++++++++++++++++++++++++++++
 1 file changed, 203 insertions(+)

diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index 2aba442a0750e91a..dec91e2f3ccdbd93 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -26,6 +26,207 @@
 
 #include "rcar-vin.h"
 
+/* -----------------------------------------------------------------------------
+ * Media Controller link notification
+ */
+
+static unsigned int rvin_group_csi_pad_to_chan(unsigned int pad)
+{
+	/*
+	 * The CSI2 driver is rcar-csi2 and we know it's pad layout are
+	 * 0: Source 1-4: Sinks so if we remove one from the pad we
+	 * get the rcar-vin internal CSI2 channel number
+	 */
+	return pad - 1;
+}
+
+/* group lock should be held when calling this function */
+static int rvin_group_entity_to_vin_num(struct rvin_group *group,
+					struct media_entity *entity)
+{
+	struct video_device *vdev;
+	int i;
+
+	if (!is_media_entity_v4l2_video_device(entity))
+		return -ENODEV;
+
+	vdev = media_entity_to_video_device(entity);
+
+	for (i = 0; i < RCAR_VIN_NUM; i++) {
+		if (!group->vin[i])
+			continue;
+
+		if (&group->vin[i]->vdev == vdev)
+			return i;
+	}
+
+	return -ENODEV;
+}
+
+/* group lock should be held when calling this function */
+static int rvin_group_entity_to_csi_num(struct rvin_group *group,
+					struct media_entity *entity)
+{
+	struct v4l2_subdev *sd;
+	int i;
+
+	if (!is_media_entity_v4l2_subdev(entity))
+		return -ENODEV;
+
+	sd = media_entity_to_v4l2_subdev(entity);
+
+	for (i = 0; i < RVIN_CSI_MAX; i++)
+		if (group->csi[i].subdev == sd)
+			return i;
+
+	return -ENODEV;
+}
+
+/* group lock should be held when calling this function */
+static void __rvin_group_build_link_list(struct rvin_group *group,
+					 struct rvin_group_chsel *map,
+					 int start, int len)
+{
+	struct media_pad *vin_pad, *remote_pad;
+	unsigned int n;
+
+	for (n = 0; n < len; n++) {
+		map[n].csi = -1;
+		map[n].chan = -1;
+
+		if (!group->vin[start + n])
+			continue;
+
+		vin_pad = &group->vin[start + n]->vdev.entity.pads[0];
+
+		remote_pad = media_entity_remote_pad(vin_pad);
+		if (!remote_pad)
+			continue;
+
+		map[n].csi =
+			rvin_group_entity_to_csi_num(group, remote_pad->entity);
+		map[n].chan = rvin_group_csi_pad_to_chan(remote_pad->index);
+	}
+}
+
+/* group lock should be held when calling this function */
+static int __rvin_group_try_get_chsel(struct rvin_group *group,
+				      struct rvin_group_chsel *map,
+				      int start, int len)
+{
+	const struct rvin_group_chsel *sel;
+	unsigned int i, n;
+	int chsel;
+
+	for (i = 0; i < group->vin[start]->info->num_chsels; i++) {
+		chsel = i;
+		for (n = 0; n < len; n++) {
+
+			/* If the link is not active it's OK */
+			if (map[n].csi == -1)
+				continue;
+
+			/* Check if chsel match requested link */
+			sel = &group->vin[start]->info->chsels[start + n][i];
+			if (map[n].csi != sel->csi ||
+			    map[n].chan != sel->chan) {
+				chsel = -1;
+				break;
+			}
+		}
+
+		/* A chsel which satisfy the links have been found */
+		if (chsel != -1)
+			return chsel;
+	}
+
+	/* No chsel can satisfy the requested links */
+	return -1;
+}
+
+/* group lock should be held when calling this function */
+static bool rvin_group_in_use(struct rvin_group *group)
+{
+	struct media_entity *entity;
+
+	media_device_for_each_entity(entity, &group->mdev)
+		if (entity->use_count)
+			return true;
+
+	return false;
+}
+
+static int rvin_group_link_notify(struct media_link *link, u32 flags,
+				  unsigned int notification)
+{
+	struct rvin_group *group = container_of(link->graph_obj.mdev,
+						struct rvin_group, mdev);
+	struct rvin_group_chsel chsel_map[4];
+	int vin_num, vin_master, csi_num, csi_chan;
+	unsigned int chsel;
+
+	mutex_lock(&group->lock);
+
+	vin_num = rvin_group_entity_to_vin_num(group, link->sink->entity);
+	csi_num = rvin_group_entity_to_csi_num(group, link->source->entity);
+	csi_chan = rvin_group_csi_pad_to_chan(link->source->index);
+
+	/*
+	 * Figure out which VIN node is the subgroup master.
+	 *
+	 * VIN0-3 are controlled by VIN0
+	 * VIN4-7 are controlled by VIN4
+	 */
+	vin_master = vin_num < 4 ? 0 : 4;
+
+	/* If not all devices exists something is horribly wrong */
+	if (vin_num < 0 || csi_num < 0 || !group->vin[vin_master])
+		goto error;
+
+	/* Special checking only needed for links which are to be enabled */
+	if (notification != MEDIA_DEV_NOTIFY_PRE_LINK_CH ||
+	    !(flags & MEDIA_LNK_FL_ENABLED))
+		goto out;
+
+	/* If any link in the group are in use, no new link can be enabled */
+	if (rvin_group_in_use(group))
+		goto error;
+
+	/* If the VIN already have a active link it's busy */
+	if (media_entity_remote_pad(&link->sink->entity->pads[0]))
+		goto error;
+
+	/* Build list of active links */
+	__rvin_group_build_link_list(group, chsel_map, vin_master, 4);
+
+	/* Add the new proposed link */
+	chsel_map[vin_num - vin_master].csi = csi_num;
+	chsel_map[vin_num - vin_master].chan = csi_chan;
+
+	/* See if there is a chsel value which match our link selection */
+	chsel = __rvin_group_try_get_chsel(group, chsel_map, vin_master, 4);
+
+	/* No chsel can provide the request links */
+	if (chsel == -1)
+		goto error;
+
+	/* Update chsel value at group master */
+	rvin_set_chsel(group->vin[vin_master], chsel);
+
+out:
+	mutex_unlock(&group->lock);
+
+	return v4l2_pipeline_link_notify(link, flags, notification);
+error:
+	mutex_unlock(&group->lock);
+
+	return -EMLINK;
+}
+
+static const struct media_device_ops rvin_media_ops = {
+	.link_notify = rvin_group_link_notify,
+};
+
 /* -----------------------------------------------------------------------------
  * Gen3 CSI2 Group Allocator
  */
@@ -146,6 +347,8 @@ static int rvin_group_allocate(struct rvin_dev *vin)
 			sizeof(mdev->bus_info));
 		media_device_init(mdev);
 
+		mdev->ops = &rvin_media_ops;
+
 		ret = media_device_register(mdev);
 		if (ret) {
 			vin_err(vin, "Failed to register media device\n");
-- 
2.14.0

[PATCH v6 23/25] rcar-vin: extend {start,stop}_streaming to work with media controller

[Niklas Söderlund]

The procedure to start or stop streaming using the none MC single
subdevice and the MC graph and multiple subdevices are quiet different.
Create a new function to abstract which method is used based on which
mode the driver is running in and add logic to start the MC graph.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/rcar-vin/rcar-dma.c | 112 +++++++++++++++++++++++++++--
 1 file changed, 105 insertions(+), 7 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index de1486ee2786b499..499253f94390f43e 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -1087,15 +1087,115 @@ static void rvin_buffer_queue(struct vb2_buffer *vb)
 	spin_unlock_irqrestore(&vin->qlock, flags);
 }
 
+static int rvin_set_stream(struct rvin_dev *vin, int on)
+{
+	struct v4l2_subdev_format fmt = {
+		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
+	};
+	struct media_pipeline *pipe;
+	struct  v4l2_subdev *sd;
+	struct media_pad *pad;
+	int ret;
+
+	/* Not media controller used, simply pass operation to subdevice */
+	if (!vin->info->use_mc) {
+		ret = v4l2_subdev_call(vin->digital.subdev, video, s_stream,
+				       on);
+
+		return ret == -ENOIOCTLCMD ? 0 : ret;
+	}
+
+	pad = media_entity_remote_pad(&vin->pad);
+	if (!pad)
+		return -EPIPE;
+
+	sd = media_entity_to_v4l2_subdev(pad->entity);
+	if (!sd)
+		return -EPIPE;
+
+	if (on) {
+		fmt.pad = pad->index;
+		if (v4l2_subdev_call(sd, pad, get_fmt, NULL, &fmt))
+			return -EPIPE;
+
+		switch (fmt.format.code) {
+		case MEDIA_BUS_FMT_YUYV8_1X16:
+		case MEDIA_BUS_FMT_UYVY8_2X8:
+		case MEDIA_BUS_FMT_UYVY10_2X10:
+		case MEDIA_BUS_FMT_RGB888_1X24:
+			vin->code = fmt.format.code;
+			break;
+		default:
+			return -EPIPE;
+		}
+
+		switch (fmt.format.field) {
+		case V4L2_FIELD_TOP:
+		case V4L2_FIELD_BOTTOM:
+		case V4L2_FIELD_NONE:
+		case V4L2_FIELD_INTERLACED_TB:
+		case V4L2_FIELD_INTERLACED_BT:
+		case V4L2_FIELD_INTERLACED:
+		case V4L2_FIELD_SEQ_TB:
+		case V4L2_FIELD_SEQ_BT:
+			/* Supported nativly */
+			break;
+		case V4L2_FIELD_ALTERNATE:
+			switch (vin->format.field) {
+			case V4L2_FIELD_TOP:
+			case V4L2_FIELD_BOTTOM:
+			case V4L2_FIELD_NONE:
+				break;
+			case V4L2_FIELD_INTERLACED_TB:
+			case V4L2_FIELD_INTERLACED_BT:
+			case V4L2_FIELD_INTERLACED:
+			case V4L2_FIELD_SEQ_TB:
+			case V4L2_FIELD_SEQ_BT:
+				/* Use VIN hardware to combine the two fields */
+				fmt.format.height *= 2;
+				break;
+			default:
+				return -EPIPE;
+			}
+			break;
+		default:
+			return -EPIPE;
+		}
+
+		if (fmt.format.width != vin->format.width ||
+		    fmt.format.height != vin->format.height)
+			return -EPIPE;
+
+		pipe = sd->entity.pipe ? sd->entity.pipe : &vin->vdev.pipe;
+		if (media_pipeline_start(&vin->vdev.entity, pipe))
+			return -EPIPE;
+
+		ret = v4l2_subdev_call(sd, video, s_stream, 1);
+		if (ret == -ENOIOCTLCMD)
+			ret = 0;
+		if (ret)
+			media_pipeline_stop(&vin->vdev.entity);
+	} else {
+		media_pipeline_stop(&vin->vdev.entity);
+		ret = v4l2_subdev_call(sd, video, s_stream, 0);
+	}
+
+	return ret;
+}
+
 static int rvin_start_streaming(struct vb2_queue *vq, unsigned int count)
 {
 	struct rvin_dev *vin = vb2_get_drv_priv(vq);
-	struct v4l2_subdev *sd;
 	unsigned long flags;
 	int ret;
 
-	sd = vin_to_source(vin);
-	v4l2_subdev_call(sd, video, s_stream, 1);
+	ret = rvin_set_stream(vin, 1);
+	if (ret) {
+		spin_lock_irqsave(&vin->qlock, flags);
+		return_all_buffers(vin, VB2_BUF_STATE_QUEUED);
+		spin_unlock_irqrestore(&vin->qlock, flags);
+		return ret;
+	}
 
 	spin_lock_irqsave(&vin->qlock, flags);
 
@@ -1104,7 +1204,7 @@ static int rvin_start_streaming(struct vb2_queue *vq, unsigned int count)
 	ret = rvin_capture_start(vin);
 	if (ret) {
 		return_all_buffers(vin, VB2_BUF_STATE_QUEUED);
-		v4l2_subdev_call(sd, video, s_stream, 0);
+		rvin_set_stream(vin, 0);
 	}
 
 	spin_unlock_irqrestore(&vin->qlock, flags);
@@ -1115,7 +1215,6 @@ static int rvin_start_streaming(struct vb2_queue *vq, unsigned int count)
 static void rvin_stop_streaming(struct vb2_queue *vq)
 {
 	struct rvin_dev *vin = vb2_get_drv_priv(vq);
-	struct v4l2_subdev *sd;
 	unsigned long flags;
 	int retries = 0;
 
@@ -1154,8 +1253,7 @@ static void rvin_stop_streaming(struct vb2_queue *vq)
 
 	spin_unlock_irqrestore(&vin->qlock, flags);
 
-	sd = vin_to_source(vin);
-	v4l2_subdev_call(sd, video, s_stream, 0);
+	rvin_set_stream(vin, 0);
 
 	/* disable interrupts */
 	rvin_disable_interrupts(vin);
-- 
2.14.0

[PATCH v6 24/25] rcar-vin: enable support for r8a7795

[Niklas Söderlund]

Add the SoC specific information for Renesas r8a7795.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/rcar-vin/Kconfig     |   2 +-
 drivers/media/platform/rcar-vin/rcar-core.c | 145 ++++++++++++++++++++++++++++
 2 files changed, 146 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/rcar-vin/Kconfig b/drivers/media/platform/rcar-vin/Kconfig
index af4c98b44d2e22cb..8fa7ee468c63afb9 100644
--- a/drivers/media/platform/rcar-vin/Kconfig
+++ b/drivers/media/platform/rcar-vin/Kconfig
@@ -6,7 +6,7 @@ config VIDEO_RCAR_VIN
 	select V4L2_FWNODE
 	---help---
 	  Support for Renesas R-Car Video Input (VIN) driver.
-	  Supports R-Car Gen2 SoCs.
+	  Supports R-Car Gen2 and Gen3 SoCs.
 
 	  To compile this driver as a module, choose M here: the
 	  module will be called rcar-vin.
diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index dec91e2f3ccdbd93..58d903ab9fb83faf 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -21,6 +21,7 @@
 #include <linux/platform_device.h>
 #include <linux/pm_runtime.h>
 #include <linux/slab.h>
+#include <linux/sys_soc.h>
 
 #include <media/v4l2-fwnode.h>
 
@@ -987,7 +988,139 @@ static const struct rvin_info rcar_info_gen2 = {
 	.max_height = 2048,
 };
 
+static const struct rvin_info rcar_info_r8a7795 = {
+	.chip = RCAR_GEN3,
+	.use_mc = true,
+	.max_width = 4096,
+	.max_height = 4096,
+
+	.num_chsels = 5,
+	.chsels = {
+		{
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+		}, {
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+		}, {
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 2 },
+			{ .csi = RVIN_CSI20, .chan = 2 },
+		}, {
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_CSI40, .chan = 3 },
+			{ .csi = RVIN_CSI20, .chan = 3 },
+		}, {
+			{ .csi = RVIN_CSI41, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI41, .chan = 1 },
+			{ .csi = RVIN_CSI41, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+		}, {
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI41, .chan = 1 },
+			{ .csi = RVIN_CSI41, .chan = 0 },
+			{ .csi = RVIN_CSI41, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+		}, {
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_CSI41, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI41, .chan = 2 },
+			{ .csi = RVIN_CSI20, .chan = 2 },
+		}, {
+			{ .csi = RVIN_CSI41, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_CSI41, .chan = 3 },
+			{ .csi = RVIN_CSI20, .chan = 3 },
+		},
+	},
+};
+
+static const struct rvin_info rcar_info_r8a7795es1 = {
+	.chip = RCAR_GEN3,
+	.use_mc = true,
+	.max_width = 4096,
+	.max_height = 4096,
+
+	.num_chsels = 6,
+	.chsels = {
+		{
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI21, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI21, .chan = 0 },
+		}, {
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI21, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_CSI21, .chan = 1 },
+		}, {
+			{ .csi = RVIN_CSI21, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 2 },
+			{ .csi = RVIN_CSI20, .chan = 2 },
+			{ .csi = RVIN_CSI21, .chan = 2 },
+		}, {
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_CSI21, .chan = 1 },
+			{ .csi = RVIN_CSI40, .chan = 3 },
+			{ .csi = RVIN_CSI20, .chan = 3 },
+			{ .csi = RVIN_CSI21, .chan = 3 },
+		}, {
+			{ .csi = RVIN_CSI41, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI21, .chan = 0 },
+			{ .csi = RVIN_CSI41, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI21, .chan = 0 },
+		}, {
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI21, .chan = 0 },
+			{ .csi = RVIN_CSI41, .chan = 0 },
+			{ .csi = RVIN_CSI41, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_CSI21, .chan = 1 },
+		}, {
+			{ .csi = RVIN_CSI21, .chan = 0 },
+			{ .csi = RVIN_CSI41, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI41, .chan = 2 },
+			{ .csi = RVIN_CSI20, .chan = 2 },
+			{ .csi = RVIN_CSI21, .chan = 2 },
+		}, {
+			{ .csi = RVIN_CSI41, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_CSI21, .chan = 1 },
+			{ .csi = RVIN_CSI41, .chan = 3 },
+			{ .csi = RVIN_CSI20, .chan = 3 },
+			{ .csi = RVIN_CSI21, .chan = 3 },
+		},
+	},
+};
+
 static const struct of_device_id rvin_of_id_table[] = {
+	{
+		.compatible = "renesas,vin-r8a7795",
+		.data = &rcar_info_r8a7795,
+	},
 	{
 		.compatible = "renesas,vin-r8a7794",
 		.data = &rcar_info_gen2,
@@ -1020,6 +1153,11 @@ static const struct of_device_id rvin_of_id_table[] = {
 };
 MODULE_DEVICE_TABLE(of, rvin_of_id_table);
 
+static const struct soc_device_attribute r8a7795es1[] = {
+	{ .soc_id = "r8a7795", .revision = "ES1.*" },
+	{ /* sentinel */ }
+};
+
 static int rcar_vin_probe(struct platform_device *pdev)
 {
 	const struct of_device_id *match;
@@ -1038,6 +1176,13 @@ static int rcar_vin_probe(struct platform_device *pdev)
 	vin->dev = &pdev->dev;
 	vin->info = match->data;
 
+	/*
+	 * Special care is needed on r8a7795 ES1.x since it
+	 * uses different routing then r8a7795 ES2.0.
+	 */
+	if (vin->info == &rcar_info_r8a7795 && soc_device_match(r8a7795es1))
+		vin->info = &rcar_info_r8a7795es1;
+
 	mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
 	if (mem == NULL)
 		return -EINVAL;
-- 
2.14.0

[PATCH v6 25/25] rcar-vin: enable support for r8a7796

[Niklas Söderlund]

Add the SoC specific information for Renesas r8a7796.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 .../devicetree/bindings/media/rcar_vin.txt         |  1 +
 drivers/media/platform/rcar-vin/rcar-core.c        | 64 ++++++++++++++++++++++
 2 files changed, 65 insertions(+)

diff --git a/Documentation/devicetree/bindings/media/rcar_vin.txt b/Documentation/devicetree/bindings/media/rcar_vin.txt
index be38ad89d71ad05d..767358f39512aa17 100644
--- a/Documentation/devicetree/bindings/media/rcar_vin.txt
+++ b/Documentation/devicetree/bindings/media/rcar_vin.txt
@@ -10,6 +10,7 @@ always slaves and support multiple input channels which can be either RGB,
 YUVU, BT656 or CSI-2.
 
  - compatible: Must be one or more of the following
+   - "renesas,vin-r8a7796" for the R8A7796 device
    - "renesas,vin-r8a7795" for the R8A7795 device
    - "renesas,vin-r8a7794" for the R8A7794 device
    - "renesas,vin-r8a7793" for the R8A7793 device
diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index 58d903ab9fb83faf..e01edd5f5925d26c 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -1116,7 +1116,71 @@ static const struct rvin_info rcar_info_r8a7795es1 = {
 	},
 };
 
+static const struct rvin_info rcar_info_r8a7796 = {
+	.chip = RCAR_GEN3,
+	.use_mc = true,
+	.max_width = 4096,
+	.max_height = 4096,
+
+	.num_chsels = 5,
+	.chsels = {
+		{
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_NC, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+		}, {
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_NC, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+		}, {
+			{ .csi = RVIN_NC, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 2 },
+			{ .csi = RVIN_CSI20, .chan = 2 },
+		}, {
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_NC, .chan = 1 },
+			{ .csi = RVIN_CSI40, .chan = 3 },
+			{ .csi = RVIN_CSI20, .chan = 3 },
+		}, {
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_NC, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+		}, {
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_NC, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+		}, {
+			{ .csi = RVIN_NC, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 2 },
+			{ .csi = RVIN_CSI20, .chan = 2 },
+		}, {
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_NC, .chan = 1 },
+			{ .csi = RVIN_CSI40, .chan = 3 },
+			{ .csi = RVIN_CSI20, .chan = 3 },
+		},
+	},
+};
+
 static const struct of_device_id rvin_of_id_table[] = {
+	{
+		.compatible = "renesas,vin-r8a7796",
+		.data = &rcar_info_r8a7796,
+	},
 	{
 		.compatible = "renesas,vin-r8a7795",
 		.data = &rcar_info_r8a7795,
-- 
2.14.0

Re: [PATCH v6 01/25] rcar-vin: add Gen3 devicetree bindings documentation

[Laurent Pinchart]

Hi Niklas,

On Wednesday, 23 August 2017 02:26:16 EEST Niklas Söderlund wrote:
> Document the devicetree bindings for the CSI-2 inputs available on Gen3.
> 
> There is a need to add a custom property 'renesas,id' and to define
> which CSI-2 input is described in which endpoint under the port@1 node.
> This information is needed since there are a set of predefined routes
> between each VIN and CSI-2 block. This routing table will be kept
> inside the driver but in order for it to act on it it must know which
> VIN and CSI-2 is which.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> ---
>  .../devicetree/bindings/media/rcar_vin.txt         | 106 ++++++++++++++++--
>  1 file changed, 96 insertions(+), 10 deletions(-)
> 
> diff --git a/Documentation/devicetree/bindings/media/rcar_vin.txt
> b/Documentation/devicetree/bindings/media/rcar_vin.txt index
> 6e4ef8caf759e5d3..be38ad89d71ad05d 100644
> --- a/Documentation/devicetree/bindings/media/rcar_vin.txt
> +++ b/Documentation/devicetree/bindings/media/rcar_vin.txt
> @@ -2,8 +2,12 @@ Renesas R-Car Video Input driver (rcar_vin)
>  -------------------------------------------
> 
>  The rcar_vin device provides video input capabilities for the Renesas R-Car
> -family of devices. The current blocks are always slaves and suppot one
> input -channel which can be either RGB, YUYV or BT656.
> +family of devices.
> +
> +On Gen2 the current blocks are always slaves and support one input channel
> +which can be either RGB, YUYV or BT656.

What do you mean by "are always slaves" ?

> On Gen3 the current blocks are
> +always slaves and support multiple input channels which can be either RGB,
> +YUVU, BT656 or CSI-2.

Strictly speaking VIN on Gen3 doesn't handle CSI-2, the CSI-2 receiver 
deserializes the video stream and produces a parallel input.

You could word this as follows.

Each VIN instance has a single parallel input that supports RGB and YUV video, 
with both external synchronization and BT.656 synchronization for the latter. 
Depending on the instance the VIN input is connected to external SoC pins, or 
on Gen3 to a CSI-2 receiver.

>   - compatible: Must be one or more of the following
>     - "renesas,vin-r8a7795" for the R8A7795 device
> @@ -28,7 +32,7 @@ channel which can be either RGB, YUYV or BT656.
>  Additionally, an alias named vinX will need to be created to specify
>  which video input device this is.
> 
> -The per-board settings:
> +The per-board settings Gen2:
>   - port sub-node describing a single endpoint connected to the vin
>     as described in video-interfaces.txt[1]. Only the first one will
>     be considered as each vin interface has one input port.
> @@ -36,13 +40,21 @@ The per-board settings:
>     These settings are used to work out video input format and widths
>     into the system.

Not related to this patch, but I don't understand how that sentence is related 
to the previous one.

> +The per-board settings Gen3:
> +- renesas,id - ID number of the VIN

You should define what the ID is.

> +- Port 0 - Digital video source (same as port node on Gen2)
> +- Port 1 - CSI-2 video sources
> +        - Endpoint 0 - sub-node describing the endpoint which is CSI20
> +        - Endpoint 1 - sub-node describing the endpoint which is CSI21
> +        - Endpoint 2 - sub-node describing the endpoint which is CSI40
> +        - Endpoint 3 - sub-node describing the endpoint which is CSI41

Given that the parallel input and CSI-2 input are mutually exclusive, 
shouldn't the VIN have a single port ?

I think nodes and endpoints need slightly more detailed documentation.

> -Device node example
> --------------------
> +Device node example Gen2
> +------------------------
> 
> -	aliases {
> -	       vin0 = &vin0;
> -	};
> +        aliases {
> +                vin0 = &vin0;
> +        };

Do we need the aliases ?

>          vin0: vin@0xe6ef0000 {
>                  compatible = "renesas,vin-r8a7790",
> "renesas,rcar-gen2-vin"; @@ -52,8 +64,8 @@ Device node example
>                  status = "disabled";
>          };
> 
> -Board setup example (vin1 composite video input)
> -------------------------------------------------
> +Board setup example Gen2 (vin1 composite video input)
> +-----------------------------------------------------
> 
>  &i2c2   {
>          status = "ok";
> @@ -92,6 +104,80 @@ Board setup example (vin1 composite video input)
>          };
>  };
> 
> +Device node example Gen3
> +------------------------
> +
> +        vin0: video@e6ef0000 {
> +                compatible = "renesas,vin-r8a7795";
> +                reg = <0 0xe6ef0000 0 0x1000>;
> +                interrupts = <GIC_SPI 188 IRQ_TYPE_LEVEL_HIGH>;
> +                clocks = <&cpg CPG_MOD 811>;
> +                power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
> +                status = "disabled";
> +
> +                renesas,id = <0>;
> +
> +                ports {
> +                        #address-cells = <1>;
> +                        #size-cells = <0>;
> +
> +                        port@1 {
> +                                #address-cells = <1>;
> +                                #size-cells = <0>;
> +
> +                                reg = <1>;
> +
> +                                vin0csi20: endpoint@0 {
> +                                        reg = <0>;
> +                                        remote-endpoint= <&csi20vin0>;
> +                                };
> +                                vin0csi21: endpoint@1 {
> +                                        reg = <1>;
> +                                        remote-endpoint= <&csi21vin0>;
> +                                };
> +                                vin0csi40: endpoint@2 {
> +                                        reg = <2>;
> +                                        remote-endpoint= <&csi40vin0>;
> +                                };
> +                        };
> +                };
> +        };
> +
> +        csi20: csi2@fea80000 {
> +                compatible = "renesas,r8a7795-csi2",
> "renesas,rcar-gen3-csi2"; +                reg = <0 0xfea80000 0 0x10000>;
> +                interrupts = <GIC_SPI 184 IRQ_TYPE_LEVEL_HIGH>;
> +                clocks = <&cpg CPG_MOD 714>;
> +                power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
> +                status = "disabled";
> +
> +                ports {
> +                        #address-cells = <1>;
> +                        #size-cells = <0>;
> +
> +                        port@0 {
> +                        #address-cells = <1>;
> +                        #size-cells = <0>;

Wrong indentation.

> +
> +                                reg = <0>;
> +                                csi20_in: endpoint@0 {

Do you need to number the endpoint ? If not you could omit the #address-cells 
and #size-cells properties. Otherwise you need a reg property here.

> +                                        clock-lanes = <0>;
> +                                        data-lanes = <1>;
> +                                        remote-endpoint = <&adv7482_txb>;
> +                                };
> +                        };
> 
> +                        port@1 {
> +                                #address-cells = <1>;
> +                                #size-cells = <0>;
> +
> +                                reg = <1>;
> +
> +                                csi20vin0: endpoint@0 {

I assume this one needs to be numbered as will have multiple endpoints (one 
per remote VIN).

> +                                        remote-endpoint = <&vin0csi20>;
> +                                };
> +                        };
> +                };
> +        };
> 
>  [1] video-interfaces.txt common video media interface


-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 02/25] rcar-vin: register the video device at probe time

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Wednesday, 23 August 2017 02:26:17 EEST Niklas Söderlund wrote:
> The driver registers the video device from the async complete callback
> and unregistered in the async unbind callback. This creates problems if
> if the subdevice is bound, unbound and later rebound. The second time
> video_register_device() is called it fails:
> 
>    kobject (eb3be918): tried to init an initialized object, something is
> seriously wrong.
> 
> To prevent this register the video device at prob time and don't allow

s/prob/probe/

> user-space to open the video device if the subdevice have not yet been
> bound.

s/have not yet been bound/is not bound yet/

> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> ---
>  drivers/media/platform/rcar-vin/rcar-core.c | 42 ++++++++++++++++++++++++--
>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 42 ++++----------------------
>  drivers/media/platform/rcar-vin/rcar-vin.h  |  1 +
>  3 files changed, 47 insertions(+), 38 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
> b/drivers/media/platform/rcar-vin/rcar-core.c index
> 77dff047c41c803e..aefbe8e3ccddb3e4 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -74,6 +74,7 @@ static bool rvin_mbus_supported(struct rvin_graph_entity
> *entity) static int rvin_digital_notify_complete(struct v4l2_async_notifier
> *notifier) {
>  	struct rvin_dev *vin = notifier_to_vin(notifier);
> +	struct v4l2_subdev *sd = vin_to_source(vin);
>  	int ret;
> 
>  	/* Verify subdevices mbus format */
> @@ -92,7 +93,35 @@ static int rvin_digital_notify_complete(struct
> v4l2_async_notifier *notifier) return ret;
>  	}
> 
> -	return rvin_v4l2_probe(vin);
> +	/* Add the controls */
> +	/*
> +	 * Currently the subdev with the largest number of controls (13) is
> +	 * ov6550. So let's pick 16 as a hint for the control handler. Note
> +	 * that this is a hint only: too large and you waste some memory, too
> +	 * small and there is a (very) small performance hit when looking up
> +	 * controls in the internal hash.

No need to copy the help text from the v4l2_ctrl_handler_init() documentation 
:-)

> +	 */
> +	ret = v4l2_ctrl_handler_init(&vin->ctrl_handler, 16);
> +	if (ret < 0)
> +		return ret;

This is racy. You set vdev->ctrl_handler at probe time, but only initialize 
the control handler later. I think it would be better to leave vdev-
>ctrl_handler to NULL and only set it here after initializing the handler. You 
also need proper locking.

> +	ret = v4l2_ctrl_add_handler(&vin->ctrl_handler, sd->ctrl_handler, NULL);
> +	if (ret < 0)
> +		return ret;
> +
> +	ret = v4l2_subdev_call(sd, video, g_tvnorms, &vin->vdev.tvnorms);
> +	if (ret < 0 && ret != -ENOIOCTLCMD && ret != -ENODEV)
> +		return ret;
> +
> +	if (vin->vdev.tvnorms == 0) {
> +		/* Disable the STD API if there are no tvnorms defined */
> +		v4l2_disable_ioctl(&vin->vdev, VIDIOC_G_STD);
> +		v4l2_disable_ioctl(&vin->vdev, VIDIOC_S_STD);
> +		v4l2_disable_ioctl(&vin->vdev, VIDIOC_QUERYSTD);
> +		v4l2_disable_ioctl(&vin->vdev, VIDIOC_ENUMSTD);
> +	}
> +
> +	return rvin_reset_format(vin);
>  }
> 
>  static void rvin_digital_notify_unbind(struct v4l2_async_notifier
> *notifier, @@ -102,7 +131,7 @@ static void
> rvin_digital_notify_unbind(struct v4l2_async_notifier *notifier, struct
> rvin_dev *vin = notifier_to_vin(notifier);
> 
>  	vin_dbg(vin, "unbind digital subdev %s\n", subdev->name);
> -	rvin_v4l2_remove(vin);
> +	v4l2_ctrl_handler_free(&vin->ctrl_handler);
>  	vin->digital.subdev = NULL;

You need locking here too. Nothing prevents an open file handle from issuing a 
control ioctl after the handler is freed by the subdev unbind.

>  }
> 
> @@ -231,6 +260,10 @@ static int rvin_digital_graph_init(struct rvin_dev
> *vin) vin->notifier.unbind = rvin_digital_notify_unbind;
>  	vin->notifier.complete = rvin_digital_notify_complete;
> 
> +	ret = rvin_v4l2_probe(vin);
> +	if (ret)
> +		return ret;

Registering the V4L2 devnodes in the rvin_digital_graph_init() function sounds 
weird. Maybe you should rename the function ? And while at it, I'd rename 
rvin_v4l2_probe() to rvin_v4l2_register().

>  	ret = v4l2_async_notifier_register(&vin->v4l2_dev, &vin->notifier);
>  	if (ret < 0) {
>  		vin_err(vin, "Notifier registration failed\n");
> @@ -314,6 +347,11 @@ static int rcar_vin_remove(struct platform_device
> *pdev)
> 
>  	v4l2_async_notifier_unregister(&vin->notifier);
> 
> +	/* Checks internaly if handlers have been init or not */
> +	v4l2_ctrl_handler_free(&vin->ctrl_handler);
> +
> +	rvin_v4l2_remove(vin);
> +
>  	rvin_dma_remove(vin);
> 
>  	return 0;
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> dd37ea8116804df3..81ff59c3b4744075 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -103,7 +103,7 @@ static void rvin_reset_crop_compose(struct rvin_dev
> *vin) vin->compose.height = vin->format.height;
>  }
> 
> -static int rvin_reset_format(struct rvin_dev *vin)
> +int rvin_reset_format(struct rvin_dev *vin)
>  {
>  	struct v4l2_subdev_format fmt = {
>  		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
> @@ -781,6 +781,11 @@ static int rvin_open(struct file *file)
> 
>  	mutex_lock(&vin->lock);
> 
> +	if (!vin->digital.subdev) {
> +		ret = -ENODEV;
> +		goto unlock;
> +	}

This is racy, vin->digital.subdev is set in the bound notifier, while you 
initialize the control handler in the complete notifier. I would perform 
initializations in the bound notifier instead of the complete notifier. Please 
make sure you use proper locking.

>  	file->private_data = vin;
> 
>  	ret = v4l2_fh_open(file);
> @@ -844,9 +849,6 @@ void rvin_v4l2_remove(struct rvin_dev *vin)
>  	v4l2_info(&vin->v4l2_dev, "Removing %s\n",
>  		  video_device_node_name(&vin->vdev));
> 
> -	/* Checks internaly if handlers have been init or not */
> -	v4l2_ctrl_handler_free(&vin->ctrl_handler);
> -
>  	/* Checks internaly if vdev have been init or not */
>  	video_unregister_device(&vin->vdev);
>  }
> @@ -869,41 +871,10 @@ static void rvin_notify(struct v4l2_subdev *sd,
>  int rvin_v4l2_probe(struct rvin_dev *vin)
>  {
>  	struct video_device *vdev = &vin->vdev;
> -	struct v4l2_subdev *sd = vin_to_source(vin);
>  	int ret;
> 
> -	v4l2_set_subdev_hostdata(sd, vin);
> -
>  	vin->v4l2_dev.notify = rvin_notify;
> 
> -	ret = v4l2_subdev_call(sd, video, g_tvnorms, &vin->vdev.tvnorms);
> -	if (ret < 0 && ret != -ENOIOCTLCMD && ret != -ENODEV)
> -		return ret;
> -
> -	if (vin->vdev.tvnorms == 0) {
> -		/* Disable the STD API if there are no tvnorms defined */
> -		v4l2_disable_ioctl(&vin->vdev, VIDIOC_G_STD);
> -		v4l2_disable_ioctl(&vin->vdev, VIDIOC_S_STD);
> -		v4l2_disable_ioctl(&vin->vdev, VIDIOC_QUERYSTD);
> -		v4l2_disable_ioctl(&vin->vdev, VIDIOC_ENUMSTD);
> -	}
> -
> -	/* Add the controls */
> -	/*
> -	 * Currently the subdev with the largest number of controls (13) is
> -	 * ov6550. So let's pick 16 as a hint for the control handler. Note
> -	 * that this is a hint only: too large and you waste some memory, too
> -	 * small and there is a (very) small performance hit when looking up
> -	 * controls in the internal hash.
> -	 */
> -	ret = v4l2_ctrl_handler_init(&vin->ctrl_handler, 16);
> -	if (ret < 0)
> -		return ret;
> -
> -	ret = v4l2_ctrl_add_handler(&vin->ctrl_handler, sd->ctrl_handler, NULL);
> -	if (ret < 0)
> -		return ret;
> -
>  	/* video node */
>  	vdev->fops = &rvin_fops;
>  	vdev->v4l2_dev = &vin->v4l2_dev;
> @@ -917,7 +888,6 @@ int rvin_v4l2_probe(struct rvin_dev *vin)
>  		V4L2_CAP_READWRITE;
> 
>  	vin->format.pixelformat	= RVIN_DEFAULT_FORMAT;
> -	rvin_reset_format(vin);
> 
>  	ret = video_register_device(&vin->vdev, VFL_TYPE_GRABBER, -1);
>  	if (ret) {


-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 03/25] rcar-vin: move chip information to own struct

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Wednesday, 23 August 2017 02:26:18 EEST Niklas Söderlund wrote:
> When Gen3 support is added to the driver more then chip id will be

s/then/than/
s/id/ID/

> different for the different Soc. To avoid a lot of if statements in the

s/Soc/SoCs/

> code create a struct chip_info to contain this information.

s/contain/store/

> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> ---
>  drivers/media/platform/rcar-vin/rcar-core.c | 49 ++++++++++++++++++++-----
>  drivers/media/platform/rcar-vin/rcar-v4l2.c |  3 +-
>  drivers/media/platform/rcar-vin/rcar-vin.h  | 12 +++++--
>  3 files changed, 53 insertions(+), 11 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
> b/drivers/media/platform/rcar-vin/rcar-core.c index
> aefbe8e3ccddb3e4..dae38de706b66b64 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -277,14 +277,47 @@ static int rvin_digital_graph_init(struct rvin_dev
> *vin) * Platform Device Driver
>   */
> 
> +static const struct rvin_info rcar_info_h1 = {
> +	.chip = RCAR_H1,
> +};
> +
> +static const struct rvin_info rcar_info_m1 = {
> +	.chip = RCAR_M1,
> +};
> +
> +static const struct rvin_info rcar_info_gen2 = {
> +	.chip = RCAR_GEN2,
> +};
> +
>  static const struct of_device_id rvin_of_id_table[] = {
> -	{ .compatible = "renesas,vin-r8a7794", .data = (void *)RCAR_GEN2 },
> -	{ .compatible = "renesas,vin-r8a7793", .data = (void *)RCAR_GEN2 },
> -	{ .compatible = "renesas,vin-r8a7791", .data = (void *)RCAR_GEN2 },
> -	{ .compatible = "renesas,vin-r8a7790", .data = (void *)RCAR_GEN2 },
> -	{ .compatible = "renesas,vin-r8a7779", .data = (void *)RCAR_H1 },
> -	{ .compatible = "renesas,vin-r8a7778", .data = (void *)RCAR_M1 },
> -	{ .compatible = "renesas,rcar-gen2-vin", .data = (void *)RCAR_GEN2 },
> +	{
> +		.compatible = "renesas,vin-r8a7794",
> +		.data = &rcar_info_gen2,
> +	},
> +	{
> +		.compatible = "renesas,vin-r8a7793",
> +		.data = &rcar_info_gen2,
> +	},
> +	{
> +		.compatible = "renesas,vin-r8a7791",
> +		.data = &rcar_info_gen2,
> +	},
> +	{
> +		.compatible = "renesas,vin-r8a7790",
> +		.data = &rcar_info_gen2,
> +	},
> +	{
> +		.compatible = "renesas,vin-r8a7779",
> +		.data = &rcar_info_h1,
> +	},
> +	{
> +		.compatible = "renesas,vin-r8a7778",
> +		.data = &rcar_info_m1,
> +	},

How about sorting the entries by compatible string ?

> +	{
> +		.compatible = "renesas,rcar-gen2-vin",
> +		.data = &rcar_info_gen2,
> +	},

Do we need the SoC-specific entries for the Gen2 SoCs if we have a generic one 
?

>  	{ },
>  };
>  MODULE_DEVICE_TABLE(of, rvin_of_id_table);
> @@ -305,7 +338,7 @@ static int rcar_vin_probe(struct platform_device *pdev)
>  		return -ENODEV;
> 
>  	vin->dev = &pdev->dev;
> -	vin->chip = (enum chip_id)match->data;
> +	vin->info = match->data;
> 
>  	mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
>  	if (mem == NULL)
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> 81ff59c3b4744075..02a08cf5acfce1ce 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -266,7 +266,8 @@ static int __rvin_try_format(struct rvin_dev *vin,
>  	pix->sizeimage = max_t(u32, pix->sizeimage,
>  			       rvin_format_sizeimage(pix));
> 
> -	if (vin->chip == RCAR_M1 && pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
> +	if (vin->info->chip == RCAR_M1 &&
> +	    pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
>  		vin_err(vin, "pixel format XBGR32 not supported on M1\n");
>  		return -EINVAL;
>  	}
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> b/drivers/media/platform/rcar-vin/rcar-vin.h index
> 9d0d4a5001b6ccd8..13466dfd72292fc0 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -88,11 +88,19 @@ struct rvin_graph_entity {
>  	unsigned int sink_pad;
>  };
> 
> +/**
> + * struct rvin_info - Information about the particular VIN implementation
> + * @chip:		type of VIN chip
> + */
> +struct rvin_info {
> +	enum chip_id chip;
> +};
> +
>  /**
>   * struct rvin_dev - Renesas VIN device structure
>   * @dev:		(OF) device
>   * @base:		device I/O register space remapped to virtual memory
> - * @chip:		type of VIN chip
> + * @info:		info about VIN instance
>   *
>   * @vdev:		V4L2 video device associated with VIN
>   * @v4l2_dev:		V4L2 device
> @@ -120,7 +128,7 @@ struct rvin_graph_entity {
>  struct rvin_dev {
>  	struct device *dev;
>  	void __iomem *base;
> -	enum chip_id chip;
> +	const struct rvin_info *info;
> 
>  	struct video_device vdev;
>  	struct v4l2_device v4l2_dev;


-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 03/25] rcar-vin: move chip information to own struct

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> When Gen3 support is added to the driver more then chip id will be

then -> than

> different for the different Soc. To avoid a lot of if statements in the
> code create a struct chip_info to contain this information.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>

Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Regards,

	Hans

> ---
>  drivers/media/platform/rcar-vin/rcar-core.c | 49 ++++++++++++++++++++++++-----
>  drivers/media/platform/rcar-vin/rcar-v4l2.c |  3 +-
>  drivers/media/platform/rcar-vin/rcar-vin.h  | 12 +++++--
>  3 files changed, 53 insertions(+), 11 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
> index aefbe8e3ccddb3e4..dae38de706b66b64 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -277,14 +277,47 @@ static int rvin_digital_graph_init(struct rvin_dev *vin)
>   * Platform Device Driver
>   */
>  
> +static const struct rvin_info rcar_info_h1 = {
> +	.chip = RCAR_H1,
> +};
> +
> +static const struct rvin_info rcar_info_m1 = {
> +	.chip = RCAR_M1,
> +};
> +
> +static const struct rvin_info rcar_info_gen2 = {
> +	.chip = RCAR_GEN2,
> +};
> +
>  static const struct of_device_id rvin_of_id_table[] = {
> -	{ .compatible = "renesas,vin-r8a7794", .data = (void *)RCAR_GEN2 },
> -	{ .compatible = "renesas,vin-r8a7793", .data = (void *)RCAR_GEN2 },
> -	{ .compatible = "renesas,vin-r8a7791", .data = (void *)RCAR_GEN2 },
> -	{ .compatible = "renesas,vin-r8a7790", .data = (void *)RCAR_GEN2 },
> -	{ .compatible = "renesas,vin-r8a7779", .data = (void *)RCAR_H1 },
> -	{ .compatible = "renesas,vin-r8a7778", .data = (void *)RCAR_M1 },
> -	{ .compatible = "renesas,rcar-gen2-vin", .data = (void *)RCAR_GEN2 },
> +	{
> +		.compatible = "renesas,vin-r8a7794",
> +		.data = &rcar_info_gen2,
> +	},
> +	{
> +		.compatible = "renesas,vin-r8a7793",
> +		.data = &rcar_info_gen2,
> +	},
> +	{
> +		.compatible = "renesas,vin-r8a7791",
> +		.data = &rcar_info_gen2,
> +	},
> +	{
> +		.compatible = "renesas,vin-r8a7790",
> +		.data = &rcar_info_gen2,
> +	},
> +	{
> +		.compatible = "renesas,vin-r8a7779",
> +		.data = &rcar_info_h1,
> +	},
> +	{
> +		.compatible = "renesas,vin-r8a7778",
> +		.data = &rcar_info_m1,
> +	},
> +	{
> +		.compatible = "renesas,rcar-gen2-vin",
> +		.data = &rcar_info_gen2,
> +	},
>  	{ },
>  };
>  MODULE_DEVICE_TABLE(of, rvin_of_id_table);
> @@ -305,7 +338,7 @@ static int rcar_vin_probe(struct platform_device *pdev)
>  		return -ENODEV;
>  
>  	vin->dev = &pdev->dev;
> -	vin->chip = (enum chip_id)match->data;
> +	vin->info = match->data;
>  
>  	mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
>  	if (mem == NULL)
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> index 81ff59c3b4744075..02a08cf5acfce1ce 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -266,7 +266,8 @@ static int __rvin_try_format(struct rvin_dev *vin,
>  	pix->sizeimage = max_t(u32, pix->sizeimage,
>  			       rvin_format_sizeimage(pix));
>  
> -	if (vin->chip == RCAR_M1 && pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
> +	if (vin->info->chip == RCAR_M1 &&
> +	    pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
>  		vin_err(vin, "pixel format XBGR32 not supported on M1\n");
>  		return -EINVAL;
>  	}
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
> index 9d0d4a5001b6ccd8..13466dfd72292fc0 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -88,11 +88,19 @@ struct rvin_graph_entity {
>  	unsigned int sink_pad;
>  };
>  
> +/**
> + * struct rvin_info - Information about the particular VIN implementation
> + * @chip:		type of VIN chip
> + */
> +struct rvin_info {
> +	enum chip_id chip;
> +};
> +
>  /**
>   * struct rvin_dev - Renesas VIN device structure
>   * @dev:		(OF) device
>   * @base:		device I/O register space remapped to virtual memory
> - * @chip:		type of VIN chip
> + * @info:		info about VIN instance
>   *
>   * @vdev:		V4L2 video device associated with VIN
>   * @v4l2_dev:		V4L2 device
> @@ -120,7 +128,7 @@ struct rvin_graph_entity {
>  struct rvin_dev {
>  	struct device *dev;
>  	void __iomem *base;
> -	enum chip_id chip;
> +	const struct rvin_info *info;
>  
>  	struct video_device vdev;
>  	struct v4l2_device v4l2_dev;
> 

Re: [PATCH v6 04/25] rcar-vin: move max width and height information to chip information

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> On Gen3 the max supported width and height will be different from Gen2.
> Move the limits to the struct rvin_info to prepare for Gen3 support.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>

Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Regards,

	Hans

> ---
>  drivers/media/platform/rcar-vin/rcar-core.c | 6 ++++++
>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 6 ++----
>  drivers/media/platform/rcar-vin/rcar-vin.h  | 6 ++++++
>  3 files changed, 14 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
> index dae38de706b66b64..4dc148e7835439ab 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -279,14 +279,20 @@ static int rvin_digital_graph_init(struct rvin_dev *vin)
>  
>  static const struct rvin_info rcar_info_h1 = {
>  	.chip = RCAR_H1,
> +	.max_width = 2048,
> +	.max_height = 2048,
>  };
>  
>  static const struct rvin_info rcar_info_m1 = {
>  	.chip = RCAR_M1,
> +	.max_width = 2048,
> +	.max_height = 2048,
>  };
>  
>  static const struct rvin_info rcar_info_gen2 = {
>  	.chip = RCAR_GEN2,
> +	.max_width = 2048,
> +	.max_height = 2048,
>  };
>  
>  static const struct of_device_id rvin_of_id_table[] = {
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> index 02a08cf5acfce1ce..3c4dd08261a0d3f5 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -23,8 +23,6 @@
>  #include "rcar-vin.h"
>  
>  #define RVIN_DEFAULT_FORMAT	V4L2_PIX_FMT_YUYV
> -#define RVIN_MAX_WIDTH		2048
> -#define RVIN_MAX_HEIGHT		2048
>  
>  /* -----------------------------------------------------------------------------
>   * Format Conversions
> @@ -258,8 +256,8 @@ static int __rvin_try_format(struct rvin_dev *vin,
>  	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
>  
>  	/* Limit to VIN capabilities */
> -	v4l_bound_align_image(&pix->width, 2, RVIN_MAX_WIDTH, walign,
> -			      &pix->height, 4, RVIN_MAX_HEIGHT, 2, 0);
> +	v4l_bound_align_image(&pix->width, 2, vin->info->max_width, walign,
> +			      &pix->height, 4, vin->info->max_height, 2, 0);
>  
>  	pix->bytesperline = max_t(u32, pix->bytesperline,
>  				  rvin_format_bytesperline(pix));
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
> index 13466dfd72292fc0..2d8b362012ea46a3 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -91,9 +91,15 @@ struct rvin_graph_entity {
>  /**
>   * struct rvin_info - Information about the particular VIN implementation
>   * @chip:		type of VIN chip
> + *
> + * max_width:		max input width the VIN supports
> + * max_height:		max input height the VIN supports
>   */
>  struct rvin_info {
>  	enum chip_id chip;
> +
> +	unsigned int max_width;
> +	unsigned int max_height;
>  };
>  
>  /**
> 

Re: [PATCH v6 05/25] rcar-vin: change name of video device

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> The rcar-vin driver needs to be part of a media controller to support
> Gen3. Give each VIN instance a unique name so it can be referenced from
> userspace.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>

Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Regards,

	Hans

> ---
>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> index 3c4dd08261a0d3f5..ba88774bd5379a98 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -878,7 +878,8 @@ int rvin_v4l2_probe(struct rvin_dev *vin)
>  	vdev->fops = &rvin_fops;
>  	vdev->v4l2_dev = &vin->v4l2_dev;
>  	vdev->queue = &vin->queue;
> -	strlcpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name));
> +	snprintf(vdev->name, sizeof(vdev->name), "%s %s", KBUILD_MODNAME,
> +		 dev_name(vin->dev));
>  	vdev->release = video_device_release_empty;
>  	vdev->ioctl_ops = &rvin_ioctl_ops;
>  	vdev->lock = &vin->lock;
> 

Re: [PATCH v6 06/25] rcar-vin: move functions regarding scaling

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> In preparation of refactoring the scaling code move the code regarding
> scaling to to the top of the file to avoid the need to add forward
> declarations. No code is changed in this commit only whole functions
> moved inside the same file.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Regards,

	Hans

> ---
>  drivers/media/platform/rcar-vin/rcar-dma.c | 806 +++++++++++++++--------------
>  1 file changed, 405 insertions(+), 401 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
> index b136844499f677cf..03a79de197d19e43 100644
> --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> @@ -138,305 +138,6 @@ static u32 rvin_read(struct rvin_dev *vin, u32 offset)
>  	return ioread32(vin->base + offset);
>  }
>  
> -static int rvin_setup(struct rvin_dev *vin)
> -{
> -	u32 vnmc, dmr, dmr2, interrupts;
> -	v4l2_std_id std;
> -	bool progressive = false, output_is_yuv = false, input_is_yuv = false;
> -
> -	switch (vin->format.field) {
> -	case V4L2_FIELD_TOP:
> -		vnmc = VNMC_IM_ODD;
> -		break;
> -	case V4L2_FIELD_BOTTOM:
> -		vnmc = VNMC_IM_EVEN;
> -		break;
> -	case V4L2_FIELD_INTERLACED:
> -		/* Default to TB */
> -		vnmc = VNMC_IM_FULL;
> -		/* Use BT if video standard can be read and is 60 Hz format */
> -		if (!v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
> -			if (std & V4L2_STD_525_60)
> -				vnmc = VNMC_IM_FULL | VNMC_FOC;
> -		}
> -		break;
> -	case V4L2_FIELD_INTERLACED_TB:
> -		vnmc = VNMC_IM_FULL;
> -		break;
> -	case V4L2_FIELD_INTERLACED_BT:
> -		vnmc = VNMC_IM_FULL | VNMC_FOC;
> -		break;
> -	case V4L2_FIELD_ALTERNATE:
> -	case V4L2_FIELD_NONE:
> -		if (vin->continuous) {
> -			vnmc = VNMC_IM_ODD_EVEN;
> -			progressive = true;
> -		} else {
> -			vnmc = VNMC_IM_ODD;
> -		}
> -		break;
> -	default:
> -		vnmc = VNMC_IM_ODD;
> -		break;
> -	}
> -
> -	/*
> -	 * Input interface
> -	 */
> -	switch (vin->digital.code) {
> -	case MEDIA_BUS_FMT_YUYV8_1X16:
> -		/* BT.601/BT.1358 16bit YCbCr422 */
> -		vnmc |= VNMC_INF_YUV16;
> -		input_is_yuv = true;
> -		break;
> -	case MEDIA_BUS_FMT_UYVY8_2X8:
> -		/* BT.656 8bit YCbCr422 or BT.601 8bit YCbCr422 */
> -		vnmc |= vin->digital.mbus_cfg.type == V4L2_MBUS_BT656 ?
> -			VNMC_INF_YUV8_BT656 : VNMC_INF_YUV8_BT601;
> -		input_is_yuv = true;
> -		break;
> -	case MEDIA_BUS_FMT_RGB888_1X24:
> -		vnmc |= VNMC_INF_RGB888;
> -		break;
> -	case MEDIA_BUS_FMT_UYVY10_2X10:
> -		/* BT.656 10bit YCbCr422 or BT.601 10bit YCbCr422 */
> -		vnmc |= vin->digital.mbus_cfg.type == V4L2_MBUS_BT656 ?
> -			VNMC_INF_YUV10_BT656 : VNMC_INF_YUV10_BT601;
> -		input_is_yuv = true;
> -		break;
> -	default:
> -		break;
> -	}
> -
> -	/* Enable VSYNC Field Toogle mode after one VSYNC input */
> -	dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
> -
> -	/* Hsync Signal Polarity Select */
> -	if (!(vin->digital.mbus_cfg.flags & V4L2_MBUS_HSYNC_ACTIVE_LOW))
> -		dmr2 |= VNDMR2_HPS;
> -
> -	/* Vsync Signal Polarity Select */
> -	if (!(vin->digital.mbus_cfg.flags & V4L2_MBUS_VSYNC_ACTIVE_LOW))
> -		dmr2 |= VNDMR2_VPS;
> -
> -	/*
> -	 * Output format
> -	 */
> -	switch (vin->format.pixelformat) {
> -	case V4L2_PIX_FMT_NV16:
> -		rvin_write(vin,
> -			   ALIGN(vin->format.width * vin->format.height, 0x80),
> -			   VNUVAOF_REG);
> -		dmr = VNDMR_DTMD_YCSEP;
> -		output_is_yuv = true;
> -		break;
> -	case V4L2_PIX_FMT_YUYV:
> -		dmr = VNDMR_BPSM;
> -		output_is_yuv = true;
> -		break;
> -	case V4L2_PIX_FMT_UYVY:
> -		dmr = 0;
> -		output_is_yuv = true;
> -		break;
> -	case V4L2_PIX_FMT_XRGB555:
> -		dmr = VNDMR_DTMD_ARGB1555;
> -		break;
> -	case V4L2_PIX_FMT_RGB565:
> -		dmr = 0;
> -		break;
> -	case V4L2_PIX_FMT_XBGR32:
> -		/* Note: not supported on M1 */
> -		dmr = VNDMR_EXRGB;
> -		break;
> -	default:
> -		vin_err(vin, "Invalid pixelformat (0x%x)\n",
> -			vin->format.pixelformat);
> -		return -EINVAL;
> -	}
> -
> -	/* Always update on field change */
> -	vnmc |= VNMC_VUP;
> -
> -	/* If input and output use the same colorspace, use bypass mode */
> -	if (input_is_yuv == output_is_yuv)
> -		vnmc |= VNMC_BPS;
> -
> -	/* Progressive or interlaced mode */
> -	interrupts = progressive ? VNIE_FIE : VNIE_EFE;
> -
> -	/* Ack interrupts */
> -	rvin_write(vin, interrupts, VNINTS_REG);
> -	/* Enable interrupts */
> -	rvin_write(vin, interrupts, VNIE_REG);
> -	/* Start capturing */
> -	rvin_write(vin, dmr, VNDMR_REG);
> -	rvin_write(vin, dmr2, VNDMR2_REG);
> -
> -	/* Enable module */
> -	rvin_write(vin, vnmc | VNMC_ME, VNMC_REG);
> -
> -	return 0;
> -}
> -
> -static void rvin_disable_interrupts(struct rvin_dev *vin)
> -{
> -	rvin_write(vin, 0, VNIE_REG);
> -}
> -
> -static u32 rvin_get_interrupt_status(struct rvin_dev *vin)
> -{
> -	return rvin_read(vin, VNINTS_REG);
> -}
> -
> -static void rvin_ack_interrupt(struct rvin_dev *vin)
> -{
> -	rvin_write(vin, rvin_read(vin, VNINTS_REG), VNINTS_REG);
> -}
> -
> -static bool rvin_capture_active(struct rvin_dev *vin)
> -{
> -	return rvin_read(vin, VNMS_REG) & VNMS_CA;
> -}
> -
> -static int rvin_get_active_slot(struct rvin_dev *vin, u32 vnms)
> -{
> -	if (vin->continuous)
> -		return (vnms & VNMS_FBS_MASK) >> VNMS_FBS_SHIFT;
> -
> -	return 0;
> -}
> -
> -static enum v4l2_field rvin_get_active_field(struct rvin_dev *vin, u32 vnms)
> -{
> -	if (vin->format.field == V4L2_FIELD_ALTERNATE) {
> -		/* If FS is set it's a Even field */
> -		if (vnms & VNMS_FS)
> -			return V4L2_FIELD_BOTTOM;
> -		return V4L2_FIELD_TOP;
> -	}
> -
> -	return vin->format.field;
> -}
> -
> -static void rvin_set_slot_addr(struct rvin_dev *vin, int slot, dma_addr_t addr)
> -{
> -	const struct rvin_video_format *fmt;
> -	int offsetx, offsety;
> -	dma_addr_t offset;
> -
> -	fmt = rvin_format_from_pixel(vin->format.pixelformat);
> -
> -	/*
> -	 * There is no HW support for composition do the beast we can
> -	 * by modifying the buffer offset
> -	 */
> -	offsetx = vin->compose.left * fmt->bpp;
> -	offsety = vin->compose.top * vin->format.bytesperline;
> -	offset = addr + offsetx + offsety;
> -
> -	/*
> -	 * The address needs to be 128 bytes aligned. Driver should never accept
> -	 * settings that do not satisfy this in the first place...
> -	 */
> -	if (WARN_ON((offsetx | offsety | offset) & HW_BUFFER_MASK))
> -		return;
> -
> -	rvin_write(vin, offset, VNMB_REG(slot));
> -}
> -
> -/* Moves a buffer from the queue to the HW slots */
> -static bool rvin_fill_hw_slot(struct rvin_dev *vin, int slot)
> -{
> -	struct rvin_buffer *buf;
> -	struct vb2_v4l2_buffer *vbuf;
> -	dma_addr_t phys_addr_top;
> -
> -	if (vin->queue_buf[slot] != NULL)
> -		return true;
> -
> -	if (list_empty(&vin->buf_list))
> -		return false;
> -
> -	vin_dbg(vin, "Filling HW slot: %d\n", slot);
> -
> -	/* Keep track of buffer we give to HW */
> -	buf = list_entry(vin->buf_list.next, struct rvin_buffer, list);
> -	vbuf = &buf->vb;
> -	list_del_init(to_buf_list(vbuf));
> -	vin->queue_buf[slot] = vbuf;
> -
> -	/* Setup DMA */
> -	phys_addr_top = vb2_dma_contig_plane_dma_addr(&vbuf->vb2_buf, 0);
> -	rvin_set_slot_addr(vin, slot, phys_addr_top);
> -
> -	return true;
> -}
> -
> -static bool rvin_fill_hw(struct rvin_dev *vin)
> -{
> -	int slot, limit;
> -
> -	limit = vin->continuous ? HW_BUFFER_NUM : 1;
> -
> -	for (slot = 0; slot < limit; slot++)
> -		if (!rvin_fill_hw_slot(vin, slot))
> -			return false;
> -	return true;
> -}
> -
> -static void rvin_capture_on(struct rvin_dev *vin)
> -{
> -	vin_dbg(vin, "Capture on in %s mode\n",
> -		vin->continuous ? "continuous" : "single");
> -
> -	if (vin->continuous)
> -		/* Continuous Frame Capture Mode */
> -		rvin_write(vin, VNFC_C_FRAME, VNFC_REG);
> -	else
> -		/* Single Frame Capture Mode */
> -		rvin_write(vin, VNFC_S_FRAME, VNFC_REG);
> -}
> -
> -static int rvin_capture_start(struct rvin_dev *vin)
> -{
> -	struct rvin_buffer *buf, *node;
> -	int bufs, ret;
> -
> -	/* Count number of free buffers */
> -	bufs = 0;
> -	list_for_each_entry_safe(buf, node, &vin->buf_list, list)
> -		bufs++;
> -
> -	/* Continuous capture requires more buffers then there are HW slots */
> -	vin->continuous = bufs > HW_BUFFER_NUM;
> -
> -	if (!rvin_fill_hw(vin)) {
> -		vin_err(vin, "HW not ready to start, not enough buffers available\n");
> -		return -EINVAL;
> -	}
> -
> -	rvin_crop_scale_comp(vin);
> -
> -	ret = rvin_setup(vin);
> -	if (ret)
> -		return ret;
> -
> -	rvin_capture_on(vin);
> -
> -	vin->state = RUNNING;
> -
> -	return 0;
> -}
> -
> -static void rvin_capture_stop(struct rvin_dev *vin)
> -{
> -	/* Set continuous & single transfer off */
> -	rvin_write(vin, 0, VNFC_REG);
> -
> -	/* Disable module */
> -	rvin_write(vin, rvin_read(vin, VNMC_REG) & ~VNMC_ME, VNMC_REG);
> -}
> -
>  /* -----------------------------------------------------------------------------
>   * Crop and Scaling Gen2
>   */
> @@ -757,139 +458,442 @@ static const struct vin_coeff vin_coeff_set[] = {
>  			  0x0370e83b, 0x0310d439, 0x03a0f83d,
>  			  0x0370e83c, 0x0300d438, 0x03b0fc3c },
>  	}
> -};
> +};
> +
> +static void rvin_set_coeff(struct rvin_dev *vin, unsigned short xs)
> +{
> +	int i;
> +	const struct vin_coeff *p_prev_set = NULL;
> +	const struct vin_coeff *p_set = NULL;
> +
> +	/* Look for suitable coefficient values */
> +	for (i = 0; i < ARRAY_SIZE(vin_coeff_set); i++) {
> +		p_prev_set = p_set;
> +		p_set = &vin_coeff_set[i];
> +
> +		if (xs < p_set->xs_value)
> +			break;
> +	}
> +
> +	/* Use previous value if its XS value is closer */
> +	if (p_prev_set && p_set &&
> +	    xs - p_prev_set->xs_value < p_set->xs_value - xs)
> +		p_set = p_prev_set;
> +
> +	/* Set coefficient registers */
> +	rvin_write(vin, p_set->coeff_set[0], VNC1A_REG);
> +	rvin_write(vin, p_set->coeff_set[1], VNC1B_REG);
> +	rvin_write(vin, p_set->coeff_set[2], VNC1C_REG);
> +
> +	rvin_write(vin, p_set->coeff_set[3], VNC2A_REG);
> +	rvin_write(vin, p_set->coeff_set[4], VNC2B_REG);
> +	rvin_write(vin, p_set->coeff_set[5], VNC2C_REG);
> +
> +	rvin_write(vin, p_set->coeff_set[6], VNC3A_REG);
> +	rvin_write(vin, p_set->coeff_set[7], VNC3B_REG);
> +	rvin_write(vin, p_set->coeff_set[8], VNC3C_REG);
> +
> +	rvin_write(vin, p_set->coeff_set[9], VNC4A_REG);
> +	rvin_write(vin, p_set->coeff_set[10], VNC4B_REG);
> +	rvin_write(vin, p_set->coeff_set[11], VNC4C_REG);
> +
> +	rvin_write(vin, p_set->coeff_set[12], VNC5A_REG);
> +	rvin_write(vin, p_set->coeff_set[13], VNC5B_REG);
> +	rvin_write(vin, p_set->coeff_set[14], VNC5C_REG);
> +
> +	rvin_write(vin, p_set->coeff_set[15], VNC6A_REG);
> +	rvin_write(vin, p_set->coeff_set[16], VNC6B_REG);
> +	rvin_write(vin, p_set->coeff_set[17], VNC6C_REG);
> +
> +	rvin_write(vin, p_set->coeff_set[18], VNC7A_REG);
> +	rvin_write(vin, p_set->coeff_set[19], VNC7B_REG);
> +	rvin_write(vin, p_set->coeff_set[20], VNC7C_REG);
> +
> +	rvin_write(vin, p_set->coeff_set[21], VNC8A_REG);
> +	rvin_write(vin, p_set->coeff_set[22], VNC8B_REG);
> +	rvin_write(vin, p_set->coeff_set[23], VNC8C_REG);
> +}
> +
> +void rvin_crop_scale_comp(struct rvin_dev *vin)
> +{
> +	u32 xs, ys;
> +
> +	/* Set Start/End Pixel/Line Pre-Clip */
> +	rvin_write(vin, vin->crop.left, VNSPPRC_REG);
> +	rvin_write(vin, vin->crop.left + vin->crop.width - 1, VNEPPRC_REG);
> +	switch (vin->format.field) {
> +	case V4L2_FIELD_INTERLACED:
> +	case V4L2_FIELD_INTERLACED_TB:
> +	case V4L2_FIELD_INTERLACED_BT:
> +		rvin_write(vin, vin->crop.top / 2, VNSLPRC_REG);
> +		rvin_write(vin, (vin->crop.top + vin->crop.height) / 2 - 1,
> +			   VNELPRC_REG);
> +		break;
> +	default:
> +		rvin_write(vin, vin->crop.top, VNSLPRC_REG);
> +		rvin_write(vin, vin->crop.top + vin->crop.height - 1,
> +			   VNELPRC_REG);
> +		break;
> +	}
> +
> +	/* Set scaling coefficient */
> +	ys = 0;
> +	if (vin->crop.height != vin->compose.height)
> +		ys = (4096 * vin->crop.height) / vin->compose.height;
> +	rvin_write(vin, ys, VNYS_REG);
> +
> +	xs = 0;
> +	if (vin->crop.width != vin->compose.width)
> +		xs = (4096 * vin->crop.width) / vin->compose.width;
> +
> +	/* Horizontal upscaling is up to double size */
> +	if (xs > 0 && xs < 2048)
> +		xs = 2048;
> +
> +	rvin_write(vin, xs, VNXS_REG);
> +
> +	/* Horizontal upscaling is done out by scaling down from double size */
> +	if (xs < 4096)
> +		xs *= 2;
> +
> +	rvin_set_coeff(vin, xs);
> +
> +	/* Set Start/End Pixel/Line Post-Clip */
> +	rvin_write(vin, 0, VNSPPOC_REG);
> +	rvin_write(vin, 0, VNSLPOC_REG);
> +	rvin_write(vin, vin->format.width - 1, VNEPPOC_REG);
> +	switch (vin->format.field) {
> +	case V4L2_FIELD_INTERLACED:
> +	case V4L2_FIELD_INTERLACED_TB:
> +	case V4L2_FIELD_INTERLACED_BT:
> +		rvin_write(vin, vin->format.height / 2 - 1, VNELPOC_REG);
> +		break;
> +	default:
> +		rvin_write(vin, vin->format.height - 1, VNELPOC_REG);
> +		break;
> +	}
> +
> +	if (vin->format.pixelformat == V4L2_PIX_FMT_NV16)
> +		rvin_write(vin, ALIGN(vin->format.width, 0x20), VNIS_REG);
> +	else
> +		rvin_write(vin, ALIGN(vin->format.width, 0x10), VNIS_REG);
> +
> +	vin_dbg(vin,
> +		"Pre-Clip: %ux%u@%u:%u YS: %d XS: %d Post-Clip: %ux%u@%u:%u\n",
> +		vin->crop.width, vin->crop.height, vin->crop.left,
> +		vin->crop.top, ys, xs, vin->format.width, vin->format.height,
> +		0, 0);
> +}
> +
> +void rvin_scale_try(struct rvin_dev *vin, struct v4l2_pix_format *pix,
> +		    u32 width, u32 height)
> +{
> +	/* All VIN channels on Gen2 have scalers */
> +	pix->width = width;
> +	pix->height = height;
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * Hardware setup
> + */
> +
> +static int rvin_setup(struct rvin_dev *vin)
> +{
> +	u32 vnmc, dmr, dmr2, interrupts;
> +	v4l2_std_id std;
> +	bool progressive = false, output_is_yuv = false, input_is_yuv = false;
> +
> +	switch (vin->format.field) {
> +	case V4L2_FIELD_TOP:
> +		vnmc = VNMC_IM_ODD;
> +		break;
> +	case V4L2_FIELD_BOTTOM:
> +		vnmc = VNMC_IM_EVEN;
> +		break;
> +	case V4L2_FIELD_INTERLACED:
> +		/* Default to TB */
> +		vnmc = VNMC_IM_FULL;
> +		/* Use BT if video standard can be read and is 60 Hz format */
> +		if (!v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
> +			if (std & V4L2_STD_525_60)
> +				vnmc = VNMC_IM_FULL | VNMC_FOC;
> +		}
> +		break;
> +	case V4L2_FIELD_INTERLACED_TB:
> +		vnmc = VNMC_IM_FULL;
> +		break;
> +	case V4L2_FIELD_INTERLACED_BT:
> +		vnmc = VNMC_IM_FULL | VNMC_FOC;
> +		break;
> +	case V4L2_FIELD_ALTERNATE:
> +	case V4L2_FIELD_NONE:
> +		if (vin->continuous) {
> +			vnmc = VNMC_IM_ODD_EVEN;
> +			progressive = true;
> +		} else {
> +			vnmc = VNMC_IM_ODD;
> +		}
> +		break;
> +	default:
> +		vnmc = VNMC_IM_ODD;
> +		break;
> +	}
> +
> +	/*
> +	 * Input interface
> +	 */
> +	switch (vin->digital.code) {
> +	case MEDIA_BUS_FMT_YUYV8_1X16:
> +		/* BT.601/BT.1358 16bit YCbCr422 */
> +		vnmc |= VNMC_INF_YUV16;
> +		input_is_yuv = true;
> +		break;
> +	case MEDIA_BUS_FMT_UYVY8_2X8:
> +		/* BT.656 8bit YCbCr422 or BT.601 8bit YCbCr422 */
> +		vnmc |= vin->digital.mbus_cfg.type == V4L2_MBUS_BT656 ?
> +			VNMC_INF_YUV8_BT656 : VNMC_INF_YUV8_BT601;
> +		input_is_yuv = true;
> +		break;
> +	case MEDIA_BUS_FMT_RGB888_1X24:
> +		vnmc |= VNMC_INF_RGB888;
> +		break;
> +	case MEDIA_BUS_FMT_UYVY10_2X10:
> +		/* BT.656 10bit YCbCr422 or BT.601 10bit YCbCr422 */
> +		vnmc |= vin->digital.mbus_cfg.type == V4L2_MBUS_BT656 ?
> +			VNMC_INF_YUV10_BT656 : VNMC_INF_YUV10_BT601;
> +		input_is_yuv = true;
> +		break;
> +	default:
> +		break;
> +	}
> +
> +	/* Enable VSYNC Field Toogle mode after one VSYNC input */
> +	dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
> +
> +	/* Hsync Signal Polarity Select */
> +	if (!(vin->digital.mbus_cfg.flags & V4L2_MBUS_HSYNC_ACTIVE_LOW))
> +		dmr2 |= VNDMR2_HPS;
> +
> +	/* Vsync Signal Polarity Select */
> +	if (!(vin->digital.mbus_cfg.flags & V4L2_MBUS_VSYNC_ACTIVE_LOW))
> +		dmr2 |= VNDMR2_VPS;
> +
> +	/*
> +	 * Output format
> +	 */
> +	switch (vin->format.pixelformat) {
> +	case V4L2_PIX_FMT_NV16:
> +		rvin_write(vin,
> +			   ALIGN(vin->format.width * vin->format.height, 0x80),
> +			   VNUVAOF_REG);
> +		dmr = VNDMR_DTMD_YCSEP;
> +		output_is_yuv = true;
> +		break;
> +	case V4L2_PIX_FMT_YUYV:
> +		dmr = VNDMR_BPSM;
> +		output_is_yuv = true;
> +		break;
> +	case V4L2_PIX_FMT_UYVY:
> +		dmr = 0;
> +		output_is_yuv = true;
> +		break;
> +	case V4L2_PIX_FMT_XRGB555:
> +		dmr = VNDMR_DTMD_ARGB1555;
> +		break;
> +	case V4L2_PIX_FMT_RGB565:
> +		dmr = 0;
> +		break;
> +	case V4L2_PIX_FMT_XBGR32:
> +		/* Note: not supported on M1 */
> +		dmr = VNDMR_EXRGB;
> +		break;
> +	default:
> +		vin_err(vin, "Invalid pixelformat (0x%x)\n",
> +			vin->format.pixelformat);
> +		return -EINVAL;
> +	}
>  
> -static void rvin_set_coeff(struct rvin_dev *vin, unsigned short xs)
> +	/* Always update on field change */
> +	vnmc |= VNMC_VUP;
> +
> +	/* If input and output use the same colorspace, use bypass mode */
> +	if (input_is_yuv == output_is_yuv)
> +		vnmc |= VNMC_BPS;
> +
> +	/* Progressive or interlaced mode */
> +	interrupts = progressive ? VNIE_FIE : VNIE_EFE;
> +
> +	/* Ack interrupts */
> +	rvin_write(vin, interrupts, VNINTS_REG);
> +	/* Enable interrupts */
> +	rvin_write(vin, interrupts, VNIE_REG);
> +	/* Start capturing */
> +	rvin_write(vin, dmr, VNDMR_REG);
> +	rvin_write(vin, dmr2, VNDMR2_REG);
> +
> +	/* Enable module */
> +	rvin_write(vin, vnmc | VNMC_ME, VNMC_REG);
> +
> +	return 0;
> +}
> +
> +static void rvin_disable_interrupts(struct rvin_dev *vin)
>  {
> -	int i;
> -	const struct vin_coeff *p_prev_set = NULL;
> -	const struct vin_coeff *p_set = NULL;
> +	rvin_write(vin, 0, VNIE_REG);
> +}
>  
> -	/* Look for suitable coefficient values */
> -	for (i = 0; i < ARRAY_SIZE(vin_coeff_set); i++) {
> -		p_prev_set = p_set;
> -		p_set = &vin_coeff_set[i];
> +static u32 rvin_get_interrupt_status(struct rvin_dev *vin)
> +{
> +	return rvin_read(vin, VNINTS_REG);
> +}
>  
> -		if (xs < p_set->xs_value)
> -			break;
> +static void rvin_ack_interrupt(struct rvin_dev *vin)
> +{
> +	rvin_write(vin, rvin_read(vin, VNINTS_REG), VNINTS_REG);
> +}
> +
> +static bool rvin_capture_active(struct rvin_dev *vin)
> +{
> +	return rvin_read(vin, VNMS_REG) & VNMS_CA;
> +}
> +
> +static int rvin_get_active_slot(struct rvin_dev *vin, u32 vnms)
> +{
> +	if (vin->continuous)
> +		return (vnms & VNMS_FBS_MASK) >> VNMS_FBS_SHIFT;
> +
> +	return 0;
> +}
> +
> +static enum v4l2_field rvin_get_active_field(struct rvin_dev *vin, u32 vnms)
> +{
> +	if (vin->format.field == V4L2_FIELD_ALTERNATE) {
> +		/* If FS is set it's a Even field */
> +		if (vnms & VNMS_FS)
> +			return V4L2_FIELD_BOTTOM;
> +		return V4L2_FIELD_TOP;
>  	}
>  
> -	/* Use previous value if its XS value is closer */
> -	if (p_prev_set && p_set &&
> -	    xs - p_prev_set->xs_value < p_set->xs_value - xs)
> -		p_set = p_prev_set;
> +	return vin->format.field;
> +}
>  
> -	/* Set coefficient registers */
> -	rvin_write(vin, p_set->coeff_set[0], VNC1A_REG);
> -	rvin_write(vin, p_set->coeff_set[1], VNC1B_REG);
> -	rvin_write(vin, p_set->coeff_set[2], VNC1C_REG);
> +static void rvin_set_slot_addr(struct rvin_dev *vin, int slot, dma_addr_t addr)
> +{
> +	const struct rvin_video_format *fmt;
> +	int offsetx, offsety;
> +	dma_addr_t offset;
>  
> -	rvin_write(vin, p_set->coeff_set[3], VNC2A_REG);
> -	rvin_write(vin, p_set->coeff_set[4], VNC2B_REG);
> -	rvin_write(vin, p_set->coeff_set[5], VNC2C_REG);
> +	fmt = rvin_format_from_pixel(vin->format.pixelformat);
>  
> -	rvin_write(vin, p_set->coeff_set[6], VNC3A_REG);
> -	rvin_write(vin, p_set->coeff_set[7], VNC3B_REG);
> -	rvin_write(vin, p_set->coeff_set[8], VNC3C_REG);
> +	/*
> +	 * There is no HW support for composition do the beast we can
> +	 * by modifying the buffer offset
> +	 */
> +	offsetx = vin->compose.left * fmt->bpp;
> +	offsety = vin->compose.top * vin->format.bytesperline;
> +	offset = addr + offsetx + offsety;
>  
> -	rvin_write(vin, p_set->coeff_set[9], VNC4A_REG);
> -	rvin_write(vin, p_set->coeff_set[10], VNC4B_REG);
> -	rvin_write(vin, p_set->coeff_set[11], VNC4C_REG);
> +	/*
> +	 * The address needs to be 128 bytes aligned. Driver should never accept
> +	 * settings that do not satisfy this in the first place...
> +	 */
> +	if (WARN_ON((offsetx | offsety | offset) & HW_BUFFER_MASK))
> +		return;
>  
> -	rvin_write(vin, p_set->coeff_set[12], VNC5A_REG);
> -	rvin_write(vin, p_set->coeff_set[13], VNC5B_REG);
> -	rvin_write(vin, p_set->coeff_set[14], VNC5C_REG);
> +	rvin_write(vin, offset, VNMB_REG(slot));
> +}
>  
> -	rvin_write(vin, p_set->coeff_set[15], VNC6A_REG);
> -	rvin_write(vin, p_set->coeff_set[16], VNC6B_REG);
> -	rvin_write(vin, p_set->coeff_set[17], VNC6C_REG);
> +/* Moves a buffer from the queue to the HW slots */
> +static bool rvin_fill_hw_slot(struct rvin_dev *vin, int slot)
> +{
> +	struct rvin_buffer *buf;
> +	struct vb2_v4l2_buffer *vbuf;
> +	dma_addr_t phys_addr_top;
>  
> -	rvin_write(vin, p_set->coeff_set[18], VNC7A_REG);
> -	rvin_write(vin, p_set->coeff_set[19], VNC7B_REG);
> -	rvin_write(vin, p_set->coeff_set[20], VNC7C_REG);
> +	if (vin->queue_buf[slot] != NULL)
> +		return true;
>  
> -	rvin_write(vin, p_set->coeff_set[21], VNC8A_REG);
> -	rvin_write(vin, p_set->coeff_set[22], VNC8B_REG);
> -	rvin_write(vin, p_set->coeff_set[23], VNC8C_REG);
> +	if (list_empty(&vin->buf_list))
> +		return false;
> +
> +	vin_dbg(vin, "Filling HW slot: %d\n", slot);
> +
> +	/* Keep track of buffer we give to HW */
> +	buf = list_entry(vin->buf_list.next, struct rvin_buffer, list);
> +	vbuf = &buf->vb;
> +	list_del_init(to_buf_list(vbuf));
> +	vin->queue_buf[slot] = vbuf;
> +
> +	/* Setup DMA */
> +	phys_addr_top = vb2_dma_contig_plane_dma_addr(&vbuf->vb2_buf, 0);
> +	rvin_set_slot_addr(vin, slot, phys_addr_top);
> +
> +	return true;
>  }
>  
> -void rvin_crop_scale_comp(struct rvin_dev *vin)
> +static bool rvin_fill_hw(struct rvin_dev *vin)
>  {
> -	u32 xs, ys;
> +	int slot, limit;
>  
> -	/* Set Start/End Pixel/Line Pre-Clip */
> -	rvin_write(vin, vin->crop.left, VNSPPRC_REG);
> -	rvin_write(vin, vin->crop.left + vin->crop.width - 1, VNEPPRC_REG);
> -	switch (vin->format.field) {
> -	case V4L2_FIELD_INTERLACED:
> -	case V4L2_FIELD_INTERLACED_TB:
> -	case V4L2_FIELD_INTERLACED_BT:
> -		rvin_write(vin, vin->crop.top / 2, VNSLPRC_REG);
> -		rvin_write(vin, (vin->crop.top + vin->crop.height) / 2 - 1,
> -			   VNELPRC_REG);
> -		break;
> -	default:
> -		rvin_write(vin, vin->crop.top, VNSLPRC_REG);
> -		rvin_write(vin, vin->crop.top + vin->crop.height - 1,
> -			   VNELPRC_REG);
> -		break;
> -	}
> +	limit = vin->continuous ? HW_BUFFER_NUM : 1;
>  
> -	/* Set scaling coefficient */
> -	ys = 0;
> -	if (vin->crop.height != vin->compose.height)
> -		ys = (4096 * vin->crop.height) / vin->compose.height;
> -	rvin_write(vin, ys, VNYS_REG);
> +	for (slot = 0; slot < limit; slot++)
> +		if (!rvin_fill_hw_slot(vin, slot))
> +			return false;
> +	return true;
> +}
>  
> -	xs = 0;
> -	if (vin->crop.width != vin->compose.width)
> -		xs = (4096 * vin->crop.width) / vin->compose.width;
> +static void rvin_capture_on(struct rvin_dev *vin)
> +{
> +	vin_dbg(vin, "Capture on in %s mode\n",
> +		vin->continuous ? "continuous" : "single");
>  
> -	/* Horizontal upscaling is up to double size */
> -	if (xs > 0 && xs < 2048)
> -		xs = 2048;
> +	if (vin->continuous)
> +		/* Continuous Frame Capture Mode */
> +		rvin_write(vin, VNFC_C_FRAME, VNFC_REG);
> +	else
> +		/* Single Frame Capture Mode */
> +		rvin_write(vin, VNFC_S_FRAME, VNFC_REG);
> +}
>  
> -	rvin_write(vin, xs, VNXS_REG);
> +static int rvin_capture_start(struct rvin_dev *vin)
> +{
> +	struct rvin_buffer *buf, *node;
> +	int bufs, ret;
>  
> -	/* Horizontal upscaling is done out by scaling down from double size */
> -	if (xs < 4096)
> -		xs *= 2;
> +	/* Count number of free buffers */
> +	bufs = 0;
> +	list_for_each_entry_safe(buf, node, &vin->buf_list, list)
> +		bufs++;
>  
> -	rvin_set_coeff(vin, xs);
> +	/* Continuous capture requires more buffers then there are HW slots */
> +	vin->continuous = bufs > HW_BUFFER_NUM;
>  
> -	/* Set Start/End Pixel/Line Post-Clip */
> -	rvin_write(vin, 0, VNSPPOC_REG);
> -	rvin_write(vin, 0, VNSLPOC_REG);
> -	rvin_write(vin, vin->format.width - 1, VNEPPOC_REG);
> -	switch (vin->format.field) {
> -	case V4L2_FIELD_INTERLACED:
> -	case V4L2_FIELD_INTERLACED_TB:
> -	case V4L2_FIELD_INTERLACED_BT:
> -		rvin_write(vin, vin->format.height / 2 - 1, VNELPOC_REG);
> -		break;
> -	default:
> -		rvin_write(vin, vin->format.height - 1, VNELPOC_REG);
> -		break;
> +	if (!rvin_fill_hw(vin)) {
> +		vin_err(vin, "HW not ready to start, not enough buffers available\n");
> +		return -EINVAL;
>  	}
>  
> -	if (vin->format.pixelformat == V4L2_PIX_FMT_NV16)
> -		rvin_write(vin, ALIGN(vin->format.width, 0x20), VNIS_REG);
> -	else
> -		rvin_write(vin, ALIGN(vin->format.width, 0x10), VNIS_REG);
> +	rvin_crop_scale_comp(vin);
>  
> -	vin_dbg(vin,
> -		"Pre-Clip: %ux%u@%u:%u YS: %d XS: %d Post-Clip: %ux%u@%u:%u\n",
> -		vin->crop.width, vin->crop.height, vin->crop.left,
> -		vin->crop.top, ys, xs, vin->format.width, vin->format.height,
> -		0, 0);
> +	ret = rvin_setup(vin);
> +	if (ret)
> +		return ret;
> +
> +	rvin_capture_on(vin);
> +
> +	vin->state = RUNNING;
> +
> +	return 0;
>  }
>  
> -void rvin_scale_try(struct rvin_dev *vin, struct v4l2_pix_format *pix,
> -		    u32 width, u32 height)
> +static void rvin_capture_stop(struct rvin_dev *vin)
>  {
> -	/* All VIN channels on Gen2 have scalers */
> -	pix->width = width;
> -	pix->height = height;
> +	/* Set continuous & single transfer off */
> +	rvin_write(vin, 0, VNFC_REG);
> +
> +	/* Disable module */
> +	rvin_write(vin, rvin_read(vin, VNMC_REG) & ~VNMC_ME, VNMC_REG);
>  }
>  
>  /* -----------------------------------------------------------------------------
> 

Re: [PATCH v6 07/25] rcar-vin: all Gen2 boards can scale simplify logic

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> The logic to preserve the requested format width and height are too
> complex and come from a premature optimization for Gen3. All Gen2 SoC
> can scale and the Gen3 implementation will not use these functions at
> all so simply preserve the width and hight when interacting with the

hight -> height

> subdevice much like the field is preserved simplifies the logic quiet a

quiet -> quite

> bit.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Regards,

	Hans


> ---
>  drivers/media/platform/rcar-vin/rcar-dma.c  |  8 --------
>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 22 ++++++++++------------
>  drivers/media/platform/rcar-vin/rcar-vin.h  |  2 --
>  3 files changed, 10 insertions(+), 22 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
> index 03a79de197d19e43..5f9674dc898305ba 100644
> --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> @@ -585,14 +585,6 @@ void rvin_crop_scale_comp(struct rvin_dev *vin)
>  		0, 0);
>  }
>  
> -void rvin_scale_try(struct rvin_dev *vin, struct v4l2_pix_format *pix,
> -		    u32 width, u32 height)
> -{
> -	/* All VIN channels on Gen2 have scalers */
> -	pix->width = width;
> -	pix->height = height;
> -}
> -
>  /* -----------------------------------------------------------------------------
>   * Hardware setup
>   */
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> index ba88774bd5379a98..affdc128a75e502e 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -166,6 +166,7 @@ static int __rvin_try_format_source(struct rvin_dev *vin,
>  		.which = which,
>  	};
>  	enum v4l2_field field;
> +	u32 width, height;
>  	int ret;
>  
>  	sd = vin_to_source(vin);
> @@ -178,7 +179,10 @@ static int __rvin_try_format_source(struct rvin_dev *vin,
>  
>  	format.pad = vin->digital.source_pad;
>  
> +	/* Allow the video device to override field and to scale */
>  	field = pix->field;
> +	width = pix->width;
> +	height = pix->height;
>  
>  	ret = v4l2_subdev_call(sd, pad, set_fmt, pad_cfg, &format);
>  	if (ret < 0 && ret != -ENOIOCTLCMD)
> @@ -191,6 +195,9 @@ static int __rvin_try_format_source(struct rvin_dev *vin,
>  	source->width = pix->width;
>  	source->height = pix->height;
>  
> +	pix->width = width;
> +	pix->height = height;
> +
>  	vin_dbg(vin, "Source resolution: %ux%u\n", source->width,
>  		source->height);
>  
> @@ -204,13 +211,9 @@ static int __rvin_try_format(struct rvin_dev *vin,
>  			     struct v4l2_pix_format *pix,
>  			     struct rvin_source_fmt *source)
>  {
> -	u32 rwidth, rheight, walign;
> +	u32 walign;
>  	int ret;
>  
> -	/* Requested */
> -	rwidth = pix->width;
> -	rheight = pix->height;
> -
>  	/* Keep current field if no specific one is asked for */
>  	if (pix->field == V4L2_FIELD_ANY)
>  		pix->field = vin->format.field;
> @@ -248,10 +251,6 @@ static int __rvin_try_format(struct rvin_dev *vin,
>  		break;
>  	}
>  
> -	/* If source can't match format try if VIN can scale */
> -	if (source->width != rwidth || source->height != rheight)
> -		rvin_scale_try(vin, pix, rwidth, rheight);
> -
>  	/* HW limit width to a multiple of 32 (2^5) for NV16 else 2 (2^1) */
>  	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
>  
> @@ -270,9 +269,8 @@ static int __rvin_try_format(struct rvin_dev *vin,
>  		return -EINVAL;
>  	}
>  
> -	vin_dbg(vin, "Requested %ux%u Got %ux%u bpl: %d size: %d\n",
> -		rwidth, rheight, pix->width, pix->height,
> -		pix->bytesperline, pix->sizeimage);
> +	vin_dbg(vin, "Format %ux%u bpl: %d size: %d\n",
> +		pix->width, pix->height, pix->bytesperline, pix->sizeimage);
>  
>  	return 0;
>  }
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
> index 2d8b362012ea46a3..b2bac06c0a3cfcb7 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -177,8 +177,6 @@ int rvin_reset_format(struct rvin_dev *vin);
>  const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
>  
>  /* Cropping, composing and scaling */
> -void rvin_scale_try(struct rvin_dev *vin, struct v4l2_pix_format *pix,
> -		    u32 width, u32 height);
>  void rvin_crop_scale_comp(struct rvin_dev *vin);
>  
>  #endif
> 

Re: [PATCH v6 08/25] rcar-vin: do not reset crop and compose when setting format

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> It was a bad idea to reset the crop and compose settings when a new
> format is set. This would overwrite any crop/compose set by s_select and
> cause unexpected behaviors, remove it. Also fold the reset helper in to
> the only remaining caller.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Regards,

	Hans

> ---
>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 21 +++++++--------------
>  1 file changed, 7 insertions(+), 14 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> index affdc128a75e502e..421820caf275b066 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -90,17 +90,6 @@ static u32 rvin_format_sizeimage(struct v4l2_pix_format *pix)
>   * V4L2
>   */
>  
> -static void rvin_reset_crop_compose(struct rvin_dev *vin)
> -{
> -	vin->crop.top = vin->crop.left = 0;
> -	vin->crop.width = vin->source.width;
> -	vin->crop.height = vin->source.height;
> -
> -	vin->compose.top = vin->compose.left = 0;
> -	vin->compose.width = vin->format.width;
> -	vin->compose.height = vin->format.height;
> -}
> -
>  int rvin_reset_format(struct rvin_dev *vin)
>  {
>  	struct v4l2_subdev_format fmt = {
> @@ -147,7 +136,13 @@ int rvin_reset_format(struct rvin_dev *vin)
>  		break;
>  	}
>  
> -	rvin_reset_crop_compose(vin);
> +	vin->crop.top = vin->crop.left = 0;
> +	vin->crop.width = mf->width;
> +	vin->crop.height = mf->height;
> +
> +	vin->compose.top = vin->compose.left = 0;
> +	vin->compose.width = mf->width;
> +	vin->compose.height = mf->height;
>  
>  	vin->format.bytesperline = rvin_format_bytesperline(&vin->format);
>  	vin->format.sizeimage = rvin_format_sizeimage(&vin->format);
> @@ -317,8 +312,6 @@ static int rvin_s_fmt_vid_cap(struct file *file, void *priv,
>  
>  	vin->format = f->fmt.pix;
>  
> -	rvin_reset_crop_compose(vin);
> -
>  	return 0;
>  }
>  
> 

Re: [PATCH v6 09/25] rcar-vin: do not allow changing scaling and composing while streaming

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> It is possible on Gen2 to change the registers controlling composing and
> scaling while the stream is running. Is however not a good idea to do so

Is -> It

> and could result in trouble. There are also no good reason to allow

reason -> reasons

> this, remove immediate reflection in hardware registers from
> vidioc_s_selection and only configure scaling and composing when the
> stream starts.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Regards,

	Hans

> ---
>  drivers/media/platform/rcar-vin/rcar-dma.c  | 2 +-
>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ---
>  drivers/media/platform/rcar-vin/rcar-vin.h  | 3 ---
>  3 files changed, 1 insertion(+), 7 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
> index 5f9674dc898305ba..6cc880e5ef7e0718 100644
> --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> @@ -514,7 +514,7 @@ static void rvin_set_coeff(struct rvin_dev *vin, unsigned short xs)
>  	rvin_write(vin, p_set->coeff_set[23], VNC8C_REG);
>  }
>  
> -void rvin_crop_scale_comp(struct rvin_dev *vin)
> +static void rvin_crop_scale_comp(struct rvin_dev *vin)
>  {
>  	u32 xs, ys;
>  
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> index 421820caf275b066..305a74d033b2d9c5 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -436,9 +436,6 @@ static int rvin_s_selection(struct file *file, void *fh,
>  		return -EINVAL;
>  	}
>  
> -	/* HW supports modifying configuration while running */
> -	rvin_crop_scale_comp(vin);
> -
>  	return 0;
>  }
>  
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
> index b2bac06c0a3cfcb7..fc70ded462ed3244 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -176,7 +176,4 @@ int rvin_reset_format(struct rvin_dev *vin);
>  
>  const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
>  
> -/* Cropping, composing and scaling */
> -void rvin_crop_scale_comp(struct rvin_dev *vin);
> -
>  #endif
> 

Re: [PATCH v6 10/25] rcar-vin: read subdevice format for crop only when needed

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> Instead of caching the subdevice format each time the video device
> format is set read it directly when its needed. As it turns out the

its -> it's

> format is only needed when figuring out the max rectangle for cropping.
> 
> This simplify the code and makes it clearer what the source format is

simplify -> simplifies

> used for.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Regards,

	Hans

> ---
>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 88 ++++++++++++++---------------
>  drivers/media/platform/rcar-vin/rcar-vin.h  | 12 ----
>  2 files changed, 42 insertions(+), 58 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> index 305a74d033b2d9c5..c8c764188b85a926 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -90,24 +90,30 @@ static u32 rvin_format_sizeimage(struct v4l2_pix_format *pix)
>   * V4L2
>   */
>  
> -int rvin_reset_format(struct rvin_dev *vin)
> +static int rvin_get_sd_format(struct rvin_dev *vin, struct v4l2_pix_format *pix)
>  {
>  	struct v4l2_subdev_format fmt = {
>  		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
> +		.pad = vin->digital.source_pad,
>  	};
> -	struct v4l2_mbus_framefmt *mf = &fmt.format;
>  	int ret;
>  
> -	fmt.pad = vin->digital.source_pad;
> -
>  	ret = v4l2_subdev_call(vin_to_source(vin), pad, get_fmt, NULL, &fmt);
>  	if (ret)
>  		return ret;
>  
> -	vin->format.width	= mf->width;
> -	vin->format.height	= mf->height;
> -	vin->format.colorspace	= mf->colorspace;
> -	vin->format.field	= mf->field;
> +	v4l2_fill_pix_format(pix, &fmt.format);
> +
> +	return 0;
> +}
> +
> +int rvin_reset_format(struct rvin_dev *vin)
> +{
> +	int ret;
> +
> +	ret = rvin_get_sd_format(vin, &vin->format);
> +	if (ret)
> +		return ret;
>  
>  	/*
>  	 * If the subdevice uses ALTERNATE field mode and G_STD is
> @@ -137,12 +143,12 @@ int rvin_reset_format(struct rvin_dev *vin)
>  	}
>  
>  	vin->crop.top = vin->crop.left = 0;
> -	vin->crop.width = mf->width;
> -	vin->crop.height = mf->height;
> +	vin->crop.width = vin->format.width;
> +	vin->crop.height = vin->format.height;
>  
>  	vin->compose.top = vin->compose.left = 0;
> -	vin->compose.width = mf->width;
> -	vin->compose.height = mf->height;
> +	vin->compose.width = vin->format.width;
> +	vin->compose.height = vin->format.height;
>  
>  	vin->format.bytesperline = rvin_format_bytesperline(&vin->format);
>  	vin->format.sizeimage = rvin_format_sizeimage(&vin->format);
> @@ -151,9 +157,7 @@ int rvin_reset_format(struct rvin_dev *vin)
>  }
>  
>  static int __rvin_try_format_source(struct rvin_dev *vin,
> -				    u32 which,
> -				    struct v4l2_pix_format *pix,
> -				    struct rvin_source_fmt *source)
> +				    u32 which, struct v4l2_pix_format *pix)
>  {
>  	struct v4l2_subdev *sd;
>  	struct v4l2_subdev_pad_config *pad_cfg;
> @@ -186,25 +190,15 @@ static int __rvin_try_format_source(struct rvin_dev *vin,
>  	v4l2_fill_pix_format(pix, &format.format);
>  
>  	pix->field = field;
> -
> -	source->width = pix->width;
> -	source->height = pix->height;
> -
>  	pix->width = width;
>  	pix->height = height;
> -
> -	vin_dbg(vin, "Source resolution: %ux%u\n", source->width,
> -		source->height);
> -
>  done:
>  	v4l2_subdev_free_pad_config(pad_cfg);
>  	return ret;
>  }
>  
>  static int __rvin_try_format(struct rvin_dev *vin,
> -			     u32 which,
> -			     struct v4l2_pix_format *pix,
> -			     struct rvin_source_fmt *source)
> +			     u32 which, struct v4l2_pix_format *pix)
>  {
>  	u32 walign;
>  	int ret;
> @@ -225,7 +219,7 @@ static int __rvin_try_format(struct rvin_dev *vin,
>  	pix->sizeimage = 0;
>  
>  	/* Limit to source capabilities */
> -	ret = __rvin_try_format_source(vin, which, pix, source);
> +	ret = __rvin_try_format_source(vin, which, pix);
>  	if (ret)
>  		return ret;
>  
> @@ -234,7 +228,6 @@ static int __rvin_try_format(struct rvin_dev *vin,
>  	case V4L2_FIELD_BOTTOM:
>  	case V4L2_FIELD_ALTERNATE:
>  		pix->height /= 2;
> -		source->height /= 2;
>  		break;
>  	case V4L2_FIELD_NONE:
>  	case V4L2_FIELD_INTERLACED_TB:
> @@ -286,30 +279,23 @@ static int rvin_try_fmt_vid_cap(struct file *file, void *priv,
>  				struct v4l2_format *f)
>  {
>  	struct rvin_dev *vin = video_drvdata(file);
> -	struct rvin_source_fmt source;
>  
> -	return __rvin_try_format(vin, V4L2_SUBDEV_FORMAT_TRY, &f->fmt.pix,
> -				 &source);
> +	return __rvin_try_format(vin, V4L2_SUBDEV_FORMAT_TRY, &f->fmt.pix);
>  }
>  
>  static int rvin_s_fmt_vid_cap(struct file *file, void *priv,
>  			      struct v4l2_format *f)
>  {
>  	struct rvin_dev *vin = video_drvdata(file);
> -	struct rvin_source_fmt source;
>  	int ret;
>  
>  	if (vb2_is_busy(&vin->queue))
>  		return -EBUSY;
>  
> -	ret = __rvin_try_format(vin, V4L2_SUBDEV_FORMAT_ACTIVE, &f->fmt.pix,
> -				&source);
> +	ret = __rvin_try_format(vin, V4L2_SUBDEV_FORMAT_ACTIVE, &f->fmt.pix);
>  	if (ret)
>  		return ret;
>  
> -	vin->source.width = source.width;
> -	vin->source.height = source.height;
> -
>  	vin->format = f->fmt.pix;
>  
>  	return 0;
> @@ -340,6 +326,8 @@ static int rvin_g_selection(struct file *file, void *fh,
>  			    struct v4l2_selection *s)
>  {
>  	struct rvin_dev *vin = video_drvdata(file);
> +	struct v4l2_pix_format pix;
> +	int ret;
>  
>  	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
>  		return -EINVAL;
> @@ -347,9 +335,12 @@ static int rvin_g_selection(struct file *file, void *fh,
>  	switch (s->target) {
>  	case V4L2_SEL_TGT_CROP_BOUNDS:
>  	case V4L2_SEL_TGT_CROP_DEFAULT:
> +		ret = rvin_get_sd_format(vin, &pix);
> +		if (ret)
> +			return ret;
>  		s->r.left = s->r.top = 0;
> -		s->r.width = vin->source.width;
> -		s->r.height = vin->source.height;
> +		s->r.width = pix.width;
> +		s->r.height = pix.height;
>  		break;
>  	case V4L2_SEL_TGT_CROP:
>  		s->r = vin->crop;
> @@ -375,12 +366,14 @@ static int rvin_s_selection(struct file *file, void *fh,
>  {
>  	struct rvin_dev *vin = video_drvdata(file);
>  	const struct rvin_video_format *fmt;
> +	struct v4l2_pix_format pix;
>  	struct v4l2_rect r = s->r;
>  	struct v4l2_rect max_rect;
>  	struct v4l2_rect min_rect = {
>  		.width = 6,
>  		.height = 2,
>  	};
> +	int ret;
>  
>  	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
>  		return -EINVAL;
> @@ -390,22 +383,25 @@ static int rvin_s_selection(struct file *file, void *fh,
>  	switch (s->target) {
>  	case V4L2_SEL_TGT_CROP:
>  		/* Can't crop outside of source input */
> +		ret = rvin_get_sd_format(vin, &pix);
> +		if (ret)
> +			return ret;
>  		max_rect.top = max_rect.left = 0;
> -		max_rect.width = vin->source.width;
> -		max_rect.height = vin->source.height;
> +		max_rect.width = pix.width;
> +		max_rect.height = pix.height;
>  		v4l2_rect_map_inside(&r, &max_rect);
>  
> -		v4l_bound_align_image(&r.width, 2, vin->source.width, 1,
> -				      &r.height, 4, vin->source.height, 2, 0);
> +		v4l_bound_align_image(&r.width, 2, pix.width, 1,
> +				      &r.height, 4, pix.height, 2, 0);
>  
> -		r.top  = clamp_t(s32, r.top, 0, vin->source.height - r.height);
> -		r.left = clamp_t(s32, r.left, 0, vin->source.width - r.width);
> +		r.top  = clamp_t(s32, r.top, 0, pix.height - r.height);
> +		r.left = clamp_t(s32, r.left, 0, pix.width - r.width);
>  
>  		vin->crop = s->r = r;
>  
>  		vin_dbg(vin, "Cropped %dx%d@%d:%d of %dx%d\n",
>  			r.width, r.height, r.left, r.top,
> -			vin->source.width, vin->source.height);
> +			pix.width, pix.height);
>  		break;
>  	case V4L2_SEL_TGT_COMPOSE:
>  		/* Make sure compose rect fits inside output format */
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
> index fc70ded462ed3244..cd8d9a96f78f3267 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -48,16 +48,6 @@ enum rvin_dma_state {
>  	STOPPING,
>  };
>  
> -/**
> - * struct rvin_source_fmt - Source information
> - * @width:	Width from source
> - * @height:	Height from source
> - */
> -struct rvin_source_fmt {
> -	u32 width;
> -	u32 height;
> -};
> -
>  /**
>   * struct rvin_video_format - Data format stored in memory
>   * @fourcc:	Pixelformat
> @@ -125,7 +115,6 @@ struct rvin_info {
>   * @sequence:		V4L2 buffers sequence number
>   * @state:		keeps track of operation state
>   *
> - * @source:		active format from the video source
>   * @format:		active V4L2 pixel format
>   *
>   * @crop:		active cropping
> @@ -152,7 +141,6 @@ struct rvin_dev {
>  	unsigned int sequence;
>  	enum rvin_dma_state state;
>  
> -	struct rvin_source_fmt source;
>  	struct v4l2_pix_format format;
>  
>  	struct v4l2_rect crop;
> 

Re: [PATCH v6 11/25] rcar-vin: fix handling of single field frames (top, bottom and alternate fields)

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> It was never proper support in the VIN driver to deliver ALTERNATING

It -> There

> field format to user-space, remove this field option. For sources using
> this field format instead use the VIN hardware feature of combining the
> fields to an interlaced format. This mode of operation was previously
> the default behavior and ALTERNATING was only delivered to user-space if
> explicitly requested. Allowing this to be explicitly requested was a
> mistake and was never properly tested and never worked due to the
> constrains put on the field format when it comes to sequence numbers and

contrains -> constraints

> timestamps etc.
> 
> The height should not be cut in half for the format for TOP or BOTTOM
> fields settings. This was a mistake and it was made visible by the
> scaling refactoring. Correct behavior is that the user should request a
> frame size that fits the half height frame reflected in the field
> setting. If not the VIN will do it's best to scale the top or bottom to

it's -> its

> the requested format and cropping and scaling do not work as expected.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Regards,

	Hans

> ---
>  drivers/media/platform/rcar-vin/rcar-dma.c  | 15 +--------
>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 48 +++++++++++------------------
>  2 files changed, 19 insertions(+), 44 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
> index 6cc880e5ef7e0718..f22bec062db31772 100644
> --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> @@ -617,7 +617,6 @@ static int rvin_setup(struct rvin_dev *vin)
>  	case V4L2_FIELD_INTERLACED_BT:
>  		vnmc = VNMC_IM_FULL | VNMC_FOC;
>  		break;
> -	case V4L2_FIELD_ALTERNATE:
>  	case V4L2_FIELD_NONE:
>  		if (vin->continuous) {
>  			vnmc = VNMC_IM_ODD_EVEN;
> @@ -757,18 +756,6 @@ static int rvin_get_active_slot(struct rvin_dev *vin, u32 vnms)
>  	return 0;
>  }
>  
> -static enum v4l2_field rvin_get_active_field(struct rvin_dev *vin, u32 vnms)
> -{
> -	if (vin->format.field == V4L2_FIELD_ALTERNATE) {
> -		/* If FS is set it's a Even field */
> -		if (vnms & VNMS_FS)
> -			return V4L2_FIELD_BOTTOM;
> -		return V4L2_FIELD_TOP;
> -	}
> -
> -	return vin->format.field;
> -}
> -
>  static void rvin_set_slot_addr(struct rvin_dev *vin, int slot, dma_addr_t addr)
>  {
>  	const struct rvin_video_format *fmt;
> @@ -941,7 +928,7 @@ static irqreturn_t rvin_irq(int irq, void *data)
>  		goto done;
>  
>  	/* Capture frame */
> -	vin->queue_buf[slot]->field = rvin_get_active_field(vin, vnms);
> +	vin->queue_buf[slot]->field = vin->format.field;
>  	vin->queue_buf[slot]->sequence = sequence;
>  	vin->queue_buf[slot]->vb2_buf.timestamp = ktime_get_ns();
>  	vb2_buffer_done(&vin->queue_buf[slot]->vb2_buf, VB2_BUF_STATE_DONE);
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> index c8c764188b85a926..9f0aac9c3398d613 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -102,6 +102,24 @@ static int rvin_get_sd_format(struct rvin_dev *vin, struct v4l2_pix_format *pix)
>  	if (ret)
>  		return ret;
>  
> +	switch (fmt.format.field) {
> +	case V4L2_FIELD_TOP:
> +	case V4L2_FIELD_BOTTOM:
> +	case V4L2_FIELD_NONE:
> +	case V4L2_FIELD_INTERLACED_TB:
> +	case V4L2_FIELD_INTERLACED_BT:
> +	case V4L2_FIELD_INTERLACED:
> +		break;
> +	case V4L2_FIELD_ALTERNATE:
> +		/* Use VIN hardware to combine the two fields */
> +		fmt.format.field = V4L2_FIELD_INTERLACED;
> +		fmt.format.height *= 2;
> +		break;
> +	default:
> +		vin->format.field = V4L2_FIELD_NONE;
> +		break;
> +	}
> +
>  	v4l2_fill_pix_format(pix, &fmt.format);
>  
>  	return 0;
> @@ -115,33 +133,6 @@ int rvin_reset_format(struct rvin_dev *vin)
>  	if (ret)
>  		return ret;
>  
> -	/*
> -	 * If the subdevice uses ALTERNATE field mode and G_STD is
> -	 * implemented use the VIN HW to combine the two fields to
> -	 * one INTERLACED frame. The ALTERNATE field mode can still
> -	 * be requested in S_FMT and be respected, this is just the
> -	 * default which is applied at probing or when S_STD is called.
> -	 */
> -	if (vin->format.field == V4L2_FIELD_ALTERNATE &&
> -	    v4l2_subdev_has_op(vin_to_source(vin), video, g_std))
> -		vin->format.field = V4L2_FIELD_INTERLACED;
> -
> -	switch (vin->format.field) {
> -	case V4L2_FIELD_TOP:
> -	case V4L2_FIELD_BOTTOM:
> -	case V4L2_FIELD_ALTERNATE:
> -		vin->format.height /= 2;
> -		break;
> -	case V4L2_FIELD_NONE:
> -	case V4L2_FIELD_INTERLACED_TB:
> -	case V4L2_FIELD_INTERLACED_BT:
> -	case V4L2_FIELD_INTERLACED:
> -		break;
> -	default:
> -		vin->format.field = V4L2_FIELD_NONE;
> -		break;
> -	}
> -
>  	vin->crop.top = vin->crop.left = 0;
>  	vin->crop.width = vin->format.width;
>  	vin->crop.height = vin->format.height;
> @@ -226,9 +217,6 @@ static int __rvin_try_format(struct rvin_dev *vin,
>  	switch (pix->field) {
>  	case V4L2_FIELD_TOP:
>  	case V4L2_FIELD_BOTTOM:
> -	case V4L2_FIELD_ALTERNATE:
> -		pix->height /= 2;
> -		break;
>  	case V4L2_FIELD_NONE:
>  	case V4L2_FIELD_INTERLACED_TB:
>  	case V4L2_FIELD_INTERLACED_BT:
> 

Re: [PATCH v6 12/25] rcar-vin: move media bus configuration to struct rvin_info

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> Bus configuration will once the driver is extended to to support Gen3

to to -> to

> contain information not specific to only the directly connected parallel
> subdevice. Move it to struct rvin_info to show it's not always coupled
> to the parallel subdevice.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Regards,

	Hans

> ---
>  drivers/media/platform/rcar-vin/rcar-core.c | 14 +++++++-------
>  drivers/media/platform/rcar-vin/rcar-dma.c  | 11 ++++++-----
>  drivers/media/platform/rcar-vin/rcar-v4l2.c |  2 +-
>  drivers/media/platform/rcar-vin/rcar-vin.h  |  9 ++++-----
>  4 files changed, 18 insertions(+), 18 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
> index 4dc148e7835439ab..65f01b6781c0aefd 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -45,15 +45,15 @@ static int rvin_find_pad(struct v4l2_subdev *sd, int direction)
>  	return -EINVAL;
>  }
>  
> -static bool rvin_mbus_supported(struct rvin_graph_entity *entity)
> +static bool rvin_mbus_supported(struct rvin_dev *vin)
>  {
> -	struct v4l2_subdev *sd = entity->subdev;
> +	struct v4l2_subdev *sd = vin->digital.subdev;
>  	struct v4l2_subdev_mbus_code_enum code = {
>  		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
>  	};
>  
>  	code.index = 0;
> -	code.pad = entity->source_pad;
> +	code.pad = vin->digital.source_pad;
>  	while (!v4l2_subdev_call(sd, pad, enum_mbus_code, NULL, &code)) {
>  		code.index++;
>  		switch (code.code) {
> @@ -61,7 +61,7 @@ static bool rvin_mbus_supported(struct rvin_graph_entity *entity)
>  		case MEDIA_BUS_FMT_UYVY8_2X8:
>  		case MEDIA_BUS_FMT_UYVY10_2X10:
>  		case MEDIA_BUS_FMT_RGB888_1X24:
> -			entity->code = code.code;
> +			vin->code = code.code;
>  			return true;
>  		default:
>  			break;
> @@ -78,14 +78,14 @@ static int rvin_digital_notify_complete(struct v4l2_async_notifier *notifier)
>  	int ret;
>  
>  	/* Verify subdevices mbus format */
> -	if (!rvin_mbus_supported(&vin->digital)) {
> +	if (!rvin_mbus_supported(vin)) {
>  		vin_err(vin, "Unsupported media bus format for %s\n",
>  			vin->digital.subdev->name);
>  		return -EINVAL;
>  	}
>  
>  	vin_dbg(vin, "Found media bus format for %s: %d\n",
> -		vin->digital.subdev->name, vin->digital.code);
> +		vin->digital.subdev->name, vin->code);
>  
>  	ret = v4l2_device_register_subdev_nodes(&vin->v4l2_dev);
>  	if (ret < 0) {
> @@ -219,7 +219,7 @@ static int rvin_digital_graph_parse(struct rvin_dev *vin)
>  	}
>  	of_node_put(np);
>  
> -	ret = rvin_digitial_parse_v4l2(vin, ep, &vin->digital.mbus_cfg);
> +	ret = rvin_digitial_parse_v4l2(vin, ep, &vin->mbus_cfg);
>  	of_node_put(ep);
>  	if (ret)
>  		return ret;
> diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
> index f22bec062db31772..9362e7dba5e3ba95 100644
> --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> @@ -633,7 +633,7 @@ static int rvin_setup(struct rvin_dev *vin)
>  	/*
>  	 * Input interface
>  	 */
> -	switch (vin->digital.code) {
> +	switch (vin->code) {
>  	case MEDIA_BUS_FMT_YUYV8_1X16:
>  		/* BT.601/BT.1358 16bit YCbCr422 */
>  		vnmc |= VNMC_INF_YUV16;
> @@ -641,7 +641,7 @@ static int rvin_setup(struct rvin_dev *vin)
>  		break;
>  	case MEDIA_BUS_FMT_UYVY8_2X8:
>  		/* BT.656 8bit YCbCr422 or BT.601 8bit YCbCr422 */
> -		vnmc |= vin->digital.mbus_cfg.type == V4L2_MBUS_BT656 ?
> +		vnmc |= vin->mbus_cfg.type == V4L2_MBUS_BT656 ?
>  			VNMC_INF_YUV8_BT656 : VNMC_INF_YUV8_BT601;
>  		input_is_yuv = true;
>  		break;
> @@ -650,7 +650,7 @@ static int rvin_setup(struct rvin_dev *vin)
>  		break;
>  	case MEDIA_BUS_FMT_UYVY10_2X10:
>  		/* BT.656 10bit YCbCr422 or BT.601 10bit YCbCr422 */
> -		vnmc |= vin->digital.mbus_cfg.type == V4L2_MBUS_BT656 ?
> +		vnmc |= vin->mbus_cfg.type == V4L2_MBUS_BT656 ?
>  			VNMC_INF_YUV10_BT656 : VNMC_INF_YUV10_BT601;
>  		input_is_yuv = true;
>  		break;
> @@ -662,11 +662,11 @@ static int rvin_setup(struct rvin_dev *vin)
>  	dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
>  
>  	/* Hsync Signal Polarity Select */
> -	if (!(vin->digital.mbus_cfg.flags & V4L2_MBUS_HSYNC_ACTIVE_LOW))
> +	if (!(vin->mbus_cfg.flags & V4L2_MBUS_HSYNC_ACTIVE_LOW))
>  		dmr2 |= VNDMR2_HPS;
>  
>  	/* Vsync Signal Polarity Select */
> -	if (!(vin->digital.mbus_cfg.flags & V4L2_MBUS_VSYNC_ACTIVE_LOW))
> +	if (!(vin->mbus_cfg.flags & V4L2_MBUS_VSYNC_ACTIVE_LOW))
>  		dmr2 |= VNDMR2_VPS;
>  
>  	/*
> @@ -875,6 +875,7 @@ static void rvin_capture_stop(struct rvin_dev *vin)
>  	rvin_write(vin, rvin_read(vin, VNMC_REG) & ~VNMC_ME, VNMC_REG);
>  }
>  
> +
>  /* -----------------------------------------------------------------------------
>   * DMA Functions
>   */
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> index 9f0aac9c3398d613..fb9f802e553e9b80 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -161,7 +161,7 @@ static int __rvin_try_format_source(struct rvin_dev *vin,
>  
>  	sd = vin_to_source(vin);
>  
> -	v4l2_fill_mbus_format(&format.format, pix, vin->digital.code);
> +	v4l2_fill_mbus_format(&format.format, pix, vin->code);
>  
>  	pad_cfg = v4l2_subdev_alloc_pad_config(sd);
>  	if (pad_cfg == NULL)
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
> index cd8d9a96f78f3267..82f074c601ea4efe 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -62,8 +62,6 @@ struct rvin_video_format {
>   * struct rvin_graph_entity - Video endpoint from async framework
>   * @asd:	sub-device descriptor for async framework
>   * @subdev:	subdevice matched using async framework
> - * @code:	Media bus format from source
> - * @mbus_cfg:	Media bus format from DT
>   * @source_pad:	source pad of remote subdevice
>   * @sink_pad:	sink pad of remote subdevice
>   */
> @@ -71,9 +69,6 @@ struct rvin_graph_entity {
>  	struct v4l2_async_subdev asd;
>  	struct v4l2_subdev *subdev;
>  
> -	u32 code;
> -	struct v4l2_mbus_config mbus_cfg;
> -
>  	unsigned int source_pad;
>  	unsigned int sink_pad;
>  };
> @@ -115,6 +110,8 @@ struct rvin_info {
>   * @sequence:		V4L2 buffers sequence number
>   * @state:		keeps track of operation state
>   *
> + * @mbus_cfg:		media bus format from DT
> + * @code:		media bus coide from subdevice
>   * @format:		active V4L2 pixel format
>   *
>   * @crop:		active cropping
> @@ -141,6 +138,8 @@ struct rvin_dev {
>  	unsigned int sequence;
>  	enum rvin_dma_state state;
>  
> +	struct v4l2_mbus_config mbus_cfg;
> +	u32 code;
>  	struct v4l2_pix_format format;
>  
>  	struct v4l2_rect crop;
> 

Re: [PATCH v6 13/25] rcar-vin: enable Gen3 hardware configuration

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> Add the register needed to work with Gen3 hardware. This patch adds
> the logic for how to work with the Gen3 hardware. More work is required
> to enable the subdevice structure needed to configure capturing.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Regards,

	Hans


> ---
>  drivers/media/platform/rcar-vin/rcar-dma.c | 94 ++++++++++++++++++++----------
>  drivers/media/platform/rcar-vin/rcar-vin.h |  1 +
>  2 files changed, 64 insertions(+), 31 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
> index 9362e7dba5e3ba95..c4f8e81e88c99e28 100644
> --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> @@ -33,21 +33,23 @@
>  #define VNELPRC_REG	0x10	/* Video n End Line Pre-Clip Register */
>  #define VNSPPRC_REG	0x14	/* Video n Start Pixel Pre-Clip Register */
>  #define VNEPPRC_REG	0x18	/* Video n End Pixel Pre-Clip Register */
> -#define VNSLPOC_REG	0x1C	/* Video n Start Line Post-Clip Register */
> -#define VNELPOC_REG	0x20	/* Video n End Line Post-Clip Register */
> -#define VNSPPOC_REG	0x24	/* Video n Start Pixel Post-Clip Register */
> -#define VNEPPOC_REG	0x28	/* Video n End Pixel Post-Clip Register */
>  #define VNIS_REG	0x2C	/* Video n Image Stride Register */
>  #define VNMB_REG(m)	(0x30 + ((m) << 2)) /* Video n Memory Base m Register */
>  #define VNIE_REG	0x40	/* Video n Interrupt Enable Register */
>  #define VNINTS_REG	0x44	/* Video n Interrupt Status Register */
>  #define VNSI_REG	0x48	/* Video n Scanline Interrupt Register */
>  #define VNMTC_REG	0x4C	/* Video n Memory Transfer Control Register */
> -#define VNYS_REG	0x50	/* Video n Y Scale Register */
> -#define VNXS_REG	0x54	/* Video n X Scale Register */
>  #define VNDMR_REG	0x58	/* Video n Data Mode Register */
>  #define VNDMR2_REG	0x5C	/* Video n Data Mode Register 2 */
>  #define VNUVAOF_REG	0x60	/* Video n UV Address Offset Register */
> +
> +/* Register offsets specific for Gen2 */
> +#define VNSLPOC_REG	0x1C	/* Video n Start Line Post-Clip Register */
> +#define VNELPOC_REG	0x20	/* Video n End Line Post-Clip Register */
> +#define VNSPPOC_REG	0x24	/* Video n Start Pixel Post-Clip Register */
> +#define VNEPPOC_REG	0x28	/* Video n End Pixel Post-Clip Register */
> +#define VNYS_REG	0x50	/* Video n Y Scale Register */
> +#define VNXS_REG	0x54	/* Video n X Scale Register */
>  #define VNC1A_REG	0x80	/* Video n Coefficient Set C1A Register */
>  #define VNC1B_REG	0x84	/* Video n Coefficient Set C1B Register */
>  #define VNC1C_REG	0x88	/* Video n Coefficient Set C1C Register */
> @@ -73,9 +75,13 @@
>  #define VNC8B_REG	0xF4	/* Video n Coefficient Set C8B Register */
>  #define VNC8C_REG	0xF8	/* Video n Coefficient Set C8C Register */
>  
> +/* Register offsets specific for Gen3 */
> +#define VNCSI_IFMD_REG		0x20 /* Video n CSI2 Interface Mode Register */
>  
>  /* Register bit fields for R-Car VIN */
>  /* Video n Main Control Register bits */
> +#define VNMC_DPINE		(1 << 27) /* Gen3 specific */
> +#define VNMC_SCLE		(1 << 26) /* Gen3 specific */
>  #define VNMC_FOC		(1 << 21)
>  #define VNMC_YCAL		(1 << 19)
>  #define VNMC_INF_YUV8_BT656	(0 << 16)
> @@ -119,6 +125,13 @@
>  #define VNDMR2_FTEV		(1 << 17)
>  #define VNDMR2_VLV(n)		((n & 0xf) << 12)
>  
> +/* Video n CSI2 Interface Mode Register (Gen3) */
> +#define VNCSI_IFMD_DES2		(1 << 27)
> +#define VNCSI_IFMD_DES1		(1 << 26)
> +#define VNCSI_IFMD_DES0		(1 << 25)
> +#define VNCSI_IFMD_CSI_CHSEL(n) ((n & 0xf) << 0)
> +#define VNCSI_IFMD_CSI_CHSEL_MASK 0xf
> +
>  struct rvin_buffer {
>  	struct vb2_v4l2_buffer vb;
>  	struct list_head list;
> @@ -514,28 +527,10 @@ static void rvin_set_coeff(struct rvin_dev *vin, unsigned short xs)
>  	rvin_write(vin, p_set->coeff_set[23], VNC8C_REG);
>  }
>  
> -static void rvin_crop_scale_comp(struct rvin_dev *vin)
> +static void rvin_crop_scale_comp_gen2(struct rvin_dev *vin)
>  {
>  	u32 xs, ys;
>  
> -	/* Set Start/End Pixel/Line Pre-Clip */
> -	rvin_write(vin, vin->crop.left, VNSPPRC_REG);
> -	rvin_write(vin, vin->crop.left + vin->crop.width - 1, VNEPPRC_REG);
> -	switch (vin->format.field) {
> -	case V4L2_FIELD_INTERLACED:
> -	case V4L2_FIELD_INTERLACED_TB:
> -	case V4L2_FIELD_INTERLACED_BT:
> -		rvin_write(vin, vin->crop.top / 2, VNSLPRC_REG);
> -		rvin_write(vin, (vin->crop.top + vin->crop.height) / 2 - 1,
> -			   VNELPRC_REG);
> -		break;
> -	default:
> -		rvin_write(vin, vin->crop.top, VNSLPRC_REG);
> -		rvin_write(vin, vin->crop.top + vin->crop.height - 1,
> -			   VNELPRC_REG);
> -		break;
> -	}
> -
>  	/* Set scaling coefficient */
>  	ys = 0;
>  	if (vin->crop.height != vin->compose.height)
> @@ -573,11 +568,6 @@ static void rvin_crop_scale_comp(struct rvin_dev *vin)
>  		break;
>  	}
>  
> -	if (vin->format.pixelformat == V4L2_PIX_FMT_NV16)
> -		rvin_write(vin, ALIGN(vin->format.width, 0x20), VNIS_REG);
> -	else
> -		rvin_write(vin, ALIGN(vin->format.width, 0x10), VNIS_REG);
> -
>  	vin_dbg(vin,
>  		"Pre-Clip: %ux%u@%u:%u YS: %d XS: %d Post-Clip: %ux%u@%u:%u\n",
>  		vin->crop.width, vin->crop.height, vin->crop.left,
> @@ -585,6 +575,37 @@ static void rvin_crop_scale_comp(struct rvin_dev *vin)
>  		0, 0);
>  }
>  
> +static void rvin_crop_scale_comp(struct rvin_dev *vin)
> +{
> +	/* Set Start/End Pixel/Line Pre-Clip */
> +	rvin_write(vin, vin->crop.left, VNSPPRC_REG);
> +	rvin_write(vin, vin->crop.left + vin->crop.width - 1, VNEPPRC_REG);
> +
> +	switch (vin->format.field) {
> +	case V4L2_FIELD_INTERLACED:
> +	case V4L2_FIELD_INTERLACED_TB:
> +	case V4L2_FIELD_INTERLACED_BT:
> +		rvin_write(vin, vin->crop.top / 2, VNSLPRC_REG);
> +		rvin_write(vin, (vin->crop.top + vin->crop.height) / 2 - 1,
> +			   VNELPRC_REG);
> +		break;
> +	default:
> +		rvin_write(vin, vin->crop.top, VNSLPRC_REG);
> +		rvin_write(vin, vin->crop.top + vin->crop.height - 1,
> +			   VNELPRC_REG);
> +		break;
> +	}
> +
> +	/* TODO: Add support for the UDS scaler. */
> +	if (vin->info->chip != RCAR_GEN3)
> +		rvin_crop_scale_comp_gen2(vin);
> +
> +	if (vin->format.pixelformat == V4L2_PIX_FMT_NV16)
> +		rvin_write(vin, ALIGN(vin->format.width, 0x20), VNIS_REG);
> +	else
> +		rvin_write(vin, ALIGN(vin->format.width, 0x10), VNIS_REG);
> +}
> +
>  /* -----------------------------------------------------------------------------
>   * Hardware setup
>   */
> @@ -659,7 +680,10 @@ static int rvin_setup(struct rvin_dev *vin)
>  	}
>  
>  	/* Enable VSYNC Field Toogle mode after one VSYNC input */
> -	dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
> +	if (vin->info->chip == RCAR_GEN3)
> +		dmr2 = VNDMR2_FTEV;
> +	else
> +		dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
>  
>  	/* Hsync Signal Polarity Select */
>  	if (!(vin->mbus_cfg.flags & V4L2_MBUS_HSYNC_ACTIVE_LOW))
> @@ -711,6 +735,14 @@ static int rvin_setup(struct rvin_dev *vin)
>  	if (input_is_yuv == output_is_yuv)
>  		vnmc |= VNMC_BPS;
>  
> +	if (vin->info->chip == RCAR_GEN3) {
> +		/* Select between CSI-2 and Digital input */
> +		if (vin->mbus_cfg.type == V4L2_MBUS_CSI2)
> +			vnmc &= ~VNMC_DPINE;
> +		else
> +			vnmc |= VNMC_DPINE;
> +	}
> +
>  	/* Progressive or interlaced mode */
>  	interrupts = progressive ? VNIE_FIE : VNIE_EFE;
>  
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
> index 82f074c601ea4efe..c4608686666d2d81 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -33,6 +33,7 @@ enum chip_id {
>  	RCAR_H1,
>  	RCAR_M1,
>  	RCAR_GEN2,
> +	RCAR_GEN3,
>  };
>  
>  /**
> 

Re: [PATCH v6 14/25] rcar-vin: add functions to manipulate Gen3 CHSEL value

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> On Gen3 the CSI-2 routing is controlled by the VnCSI_IFMD register. One
> feature of this register is that it's only present in the VIN0 and VIN4
> instances. The register in VIN0 controls the routing for VIN0-3 and the
> register in VIN4 controls routing for VIN4-7.
> 
> To be able to control routing from a media device these functions need
> to control runtime PM for the subgroup master (VIN0 and VIN4). The
> subgroup master must be switched on before the register is manipulated,
> once the operation is complete it's safe to switch the master off and
> the new routing will still be in effect.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Regards,

	Hans


> ---
>  drivers/media/platform/rcar-vin/rcar-dma.c | 41 ++++++++++++++++++++++++++++++
>  drivers/media/platform/rcar-vin/rcar-vin.h |  3 +++
>  2 files changed, 44 insertions(+)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
> index c4f8e81e88c99e28..6206fab7b6cdc55a 100644
> --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> @@ -16,6 +16,7 @@
>  
>  #include <linux/delay.h>
>  #include <linux/interrupt.h>
> +#include <linux/pm_runtime.h>
>  
>  #include <media/videobuf2-dma-contig.h>
>  
> @@ -1228,3 +1229,43 @@ int rvin_dma_probe(struct rvin_dev *vin, int irq)
>  
>  	return ret;
>  }
> +
> +/* -----------------------------------------------------------------------------
> + * Gen3 CHSEL manipulation
> + */
> +
> +void rvin_set_chsel(struct rvin_dev *vin, u8 chsel)
> +{
> +	u32 ifmd;
> +
> +	pm_runtime_get_sync(vin->dev);
> +
> +	/*
> +	 * Undocumented feature: Writing to VNCSI_IFMD_REG will go
> +	 * through and on read back look correct but won't have
> +	 * any effect if VNMC_REG is not first set to 0.
> +	 */
> +	rvin_write(vin, 0, VNMC_REG);
> +
> +	ifmd = VNCSI_IFMD_DES2 | VNCSI_IFMD_DES1 | VNCSI_IFMD_DES0 |
> +		VNCSI_IFMD_CSI_CHSEL(chsel);
> +
> +	rvin_write(vin, ifmd, VNCSI_IFMD_REG);
> +
> +	vin_dbg(vin, "Set IFMD 0x%x\n", ifmd);
> +
> +	pm_runtime_put(vin->dev);
> +}
> +
> +int rvin_get_chsel(struct rvin_dev *vin)
> +{
> +	int chsel;
> +
> +	pm_runtime_get_sync(vin->dev);
> +
> +	chsel = rvin_read(vin, VNCSI_IFMD_REG) & VNCSI_IFMD_CSI_CHSEL_MASK;
> +
> +	pm_runtime_put(vin->dev);
> +
> +	return chsel;
> +}
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
> index c4608686666d2d81..94c606f2b8f2f246 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -164,4 +164,7 @@ int rvin_reset_format(struct rvin_dev *vin);
>  
>  const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
>  
> +void rvin_set_chsel(struct rvin_dev *vin, u8 chsel);
> +int rvin_get_chsel(struct rvin_dev *vin);
> +
>  #endif
> 

Re: [PATCH v6 15/25] rcar-vin: add flag to switch to media controller mode

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> On Gen3 a media controller API needs to be used to allow userspace to
> configure the subdevices in the pipeline instead of directly controlling
> a single source subdevice, which is and will continue to be the mode of
> operation on Gen2.
> 
> Prepare for these two modes of operation by adding a flag to struct
> rvin_graph_entity which will control which mode to use.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Regards,

	Hans


> ---
>  drivers/media/platform/rcar-vin/rcar-core.c | 6 +++++-
>  drivers/media/platform/rcar-vin/rcar-vin.h  | 2 ++
>  2 files changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
> index 65f01b6781c0aefd..fbbb22924cf3a045 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -279,18 +279,21 @@ static int rvin_digital_graph_init(struct rvin_dev *vin)
>  
>  static const struct rvin_info rcar_info_h1 = {
>  	.chip = RCAR_H1,
> +	.use_mc = false,
>  	.max_width = 2048,
>  	.max_height = 2048,
>  };
>  
>  static const struct rvin_info rcar_info_m1 = {
>  	.chip = RCAR_M1,
> +	.use_mc = false,
>  	.max_width = 2048,
>  	.max_height = 2048,
>  };
>  
>  static const struct rvin_info rcar_info_gen2 = {
>  	.chip = RCAR_GEN2,
> +	.use_mc = false,
>  	.max_width = 2048,
>  	.max_height = 2048,
>  };
> @@ -387,7 +390,8 @@ static int rcar_vin_remove(struct platform_device *pdev)
>  	v4l2_async_notifier_unregister(&vin->notifier);
>  
>  	/* Checks internaly if handlers have been init or not */
> -	v4l2_ctrl_handler_free(&vin->ctrl_handler);
> +	if (!vin->info->use_mc)
> +		v4l2_ctrl_handler_free(&vin->ctrl_handler);
>  
>  	rvin_v4l2_remove(vin);
>  
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
> index 94c606f2b8f2f246..819d9c04ed8ffb36 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -77,12 +77,14 @@ struct rvin_graph_entity {
>  /**
>   * struct rvin_info - Information about the particular VIN implementation
>   * @chip:		type of VIN chip
> + * @use_mc:		use media controller instead of controlling subdevice
>   *
>   * max_width:		max input width the VIN supports
>   * max_height:		max input height the VIN supports
>   */
>  struct rvin_info {
>  	enum chip_id chip;
> +	bool use_mc;
>  
>  	unsigned int max_width;
>  	unsigned int max_height;
> 

Re: [PATCH v6 16/25] rcar-vin: break out format alignment and checking

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> Part of the format aliment and checking can be shared with the Gen3

aliment -> alignment

> format handling. Break that part out to it's own function. While doing

it's -> its

> this clean up the checking and add more checks.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Please note the small typo in a comment below.

Regards,

	Hans


> ---
>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 98 +++++++++++++++--------------
>  1 file changed, 51 insertions(+), 47 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> index fb9f802e553e9b80..f71dea8d5d40323a 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -86,6 +86,56 @@ static u32 rvin_format_sizeimage(struct v4l2_pix_format *pix)
>  	return pix->bytesperline * pix->height;
>  }
>  
> +static int rvin_format_align(struct rvin_dev *vin, struct v4l2_pix_format *pix)
> +{
> +	u32 walign;
> +
> +	/* If requested format is not supported fallback to the default */
> +	if (!rvin_format_from_pixel(pix->pixelformat)) {
> +		vin_dbg(vin, "Format 0x%x not found, using default 0x%x\n",
> +			pix->pixelformat, RVIN_DEFAULT_FORMAT);
> +		pix->pixelformat = RVIN_DEFAULT_FORMAT;
> +	}
> +
> +	switch (pix->field) {
> +	case V4L2_FIELD_TOP:
> +	case V4L2_FIELD_BOTTOM:
> +	case V4L2_FIELD_NONE:
> +	case V4L2_FIELD_INTERLACED_TB:
> +	case V4L2_FIELD_INTERLACED_BT:
> +	case V4L2_FIELD_INTERLACED:
> +		break;
> +	default:
> +		pix->field = V4L2_FIELD_NONE;
> +		break;
> +	}
> +
> +	/* Check that colorspace is resonable, if not keep current */

resonable -> reasonable

> +	if (!pix->colorspace || pix->colorspace >= 0xff)
> +		pix->colorspace = vin->format.colorspace;
> +
> +	/* HW limit width to a multiple of 32 (2^5) for NV16 else 2 (2^1) */
> +	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
> +
> +	/* Limit to VIN capabilities */
> +	v4l_bound_align_image(&pix->width, 2, vin->info->max_width, walign,
> +			      &pix->height, 4, vin->info->max_height, 2, 0);
> +
> +	pix->bytesperline = rvin_format_bytesperline(pix);
> +	pix->sizeimage = rvin_format_sizeimage(pix);
> +
> +	if (vin->info->chip == RCAR_M1 &&
> +	    pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
> +		vin_err(vin, "pixel format XBGR32 not supported on M1\n");
> +		return -EINVAL;
> +	}
> +
> +	vin_dbg(vin, "Format %ux%u bpl: %d size: %d\n",
> +		pix->width, pix->height, pix->bytesperline, pix->sizeimage);
> +
> +	return 0;
> +}
> +
>  /* -----------------------------------------------------------------------------
>   * V4L2
>   */
> @@ -191,64 +241,18 @@ static int __rvin_try_format_source(struct rvin_dev *vin,
>  static int __rvin_try_format(struct rvin_dev *vin,
>  			     u32 which, struct v4l2_pix_format *pix)
>  {
> -	u32 walign;
>  	int ret;
>  
>  	/* Keep current field if no specific one is asked for */
>  	if (pix->field == V4L2_FIELD_ANY)
>  		pix->field = vin->format.field;
>  
> -	/* If requested format is not supported fallback to the default */
> -	if (!rvin_format_from_pixel(pix->pixelformat)) {
> -		vin_dbg(vin, "Format 0x%x not found, using default 0x%x\n",
> -			pix->pixelformat, RVIN_DEFAULT_FORMAT);
> -		pix->pixelformat = RVIN_DEFAULT_FORMAT;
> -	}
> -
> -	/* Always recalculate */
> -	pix->bytesperline = 0;
> -	pix->sizeimage = 0;
> -
>  	/* Limit to source capabilities */
>  	ret = __rvin_try_format_source(vin, which, pix);
>  	if (ret)
>  		return ret;
>  
> -	switch (pix->field) {
> -	case V4L2_FIELD_TOP:
> -	case V4L2_FIELD_BOTTOM:
> -	case V4L2_FIELD_NONE:
> -	case V4L2_FIELD_INTERLACED_TB:
> -	case V4L2_FIELD_INTERLACED_BT:
> -	case V4L2_FIELD_INTERLACED:
> -		break;
> -	default:
> -		pix->field = V4L2_FIELD_NONE;
> -		break;
> -	}
> -
> -	/* HW limit width to a multiple of 32 (2^5) for NV16 else 2 (2^1) */
> -	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
> -
> -	/* Limit to VIN capabilities */
> -	v4l_bound_align_image(&pix->width, 2, vin->info->max_width, walign,
> -			      &pix->height, 4, vin->info->max_height, 2, 0);
> -
> -	pix->bytesperline = max_t(u32, pix->bytesperline,
> -				  rvin_format_bytesperline(pix));
> -	pix->sizeimage = max_t(u32, pix->sizeimage,
> -			       rvin_format_sizeimage(pix));
> -
> -	if (vin->info->chip == RCAR_M1 &&
> -	    pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
> -		vin_err(vin, "pixel format XBGR32 not supported on M1\n");
> -		return -EINVAL;
> -	}
> -
> -	vin_dbg(vin, "Format %ux%u bpl: %d size: %d\n",
> -		pix->width, pix->height, pix->bytesperline, pix->sizeimage);
> -
> -	return 0;
> +	return rvin_format_align(vin, pix);
>  }
>  
>  static int rvin_querycap(struct file *file, void *priv,
> 

Re: [PATCH v6 17/25] rcar-vin: use different v4l2 operations in media controller mode

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> When the driver runs in media controller mode it should not directly
> control the subdevice instead userspace will be responsible for
> configuring the pipeline. To be able to run in this mode a different set
> of v4l2 operations needs to be used.
> 
> Add a new set of v4l2 operations to support the running without directly
> interacting with the source subdevice.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Regards,

	Hans

> ---
>  drivers/media/platform/rcar-vin/rcar-dma.c  |   3 +-
>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 155 +++++++++++++++++++++++++++-
>  drivers/media/platform/rcar-vin/rcar-vin.h  |   1 +
>  3 files changed, 155 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
> index 6206fab7b6cdc55a..de1486ee2786b499 100644
> --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> @@ -628,7 +628,8 @@ static int rvin_setup(struct rvin_dev *vin)
>  		/* Default to TB */
>  		vnmc = VNMC_IM_FULL;
>  		/* Use BT if video standard can be read and is 60 Hz format */
> -		if (!v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
> +		if (!vin->info->use_mc &&
> +		    !v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
>  			if (std & V4L2_STD_525_60)
>  				vnmc = VNMC_IM_FULL | VNMC_FOC;
>  		}
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> index f71dea8d5d40323a..9d540644bbe446f6 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -23,6 +23,9 @@
>  #include "rcar-vin.h"
>  
>  #define RVIN_DEFAULT_FORMAT	V4L2_PIX_FMT_YUYV
> +#define RVIN_DEFAULT_WIDTH	800
> +#define RVIN_DEFAULT_HEIGHT	600
> +#define RVIN_DEFAULT_COLORSPACE	V4L2_COLORSPACE_SRGB
>  
>  /* -----------------------------------------------------------------------------
>   * Format Conversions
> @@ -671,6 +674,84 @@ static const struct v4l2_ioctl_ops rvin_ioctl_ops = {
>  	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
>  };
>  
> +/* -----------------------------------------------------------------------------
> + * V4L2 Media Controller
> + */
> +
> +static int __rvin_mc_try_format(struct rvin_dev *vin,
> +				struct v4l2_pix_format *pix)
> +{
> +	/* Keep current field if no specific one is asked for */
> +	if (pix->field == V4L2_FIELD_ANY)
> +		pix->field = vin->format.field;
> +
> +	return rvin_format_align(vin, pix);
> +}
> +
> +static int rvin_mc_try_fmt_vid_cap(struct file *file, void *priv,
> +				   struct v4l2_format *f)
> +{
> +	struct rvin_dev *vin = video_drvdata(file);
> +
> +	return __rvin_mc_try_format(vin, &f->fmt.pix);
> +}
> +
> +static int rvin_mc_s_fmt_vid_cap(struct file *file, void *priv,
> +				 struct v4l2_format *f)
> +{
> +	struct rvin_dev *vin = video_drvdata(file);
> +	int ret;
> +
> +	if (vb2_is_busy(&vin->queue))
> +		return -EBUSY;
> +
> +	ret = __rvin_mc_try_format(vin, &f->fmt.pix);
> +	if (ret)
> +		return ret;
> +
> +	vin->format = f->fmt.pix;
> +
> +	return 0;
> +}
> +
> +static int rvin_mc_enum_input(struct file *file, void *priv,
> +			      struct v4l2_input *i)
> +{
> +	if (i->index != 0)
> +		return -EINVAL;
> +
> +	i->type = V4L2_INPUT_TYPE_CAMERA;
> +	strlcpy(i->name, "Camera", sizeof(i->name));
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_ioctl_ops rvin_mc_ioctl_ops = {
> +	.vidioc_querycap		= rvin_querycap,
> +	.vidioc_try_fmt_vid_cap		= rvin_mc_try_fmt_vid_cap,
> +	.vidioc_g_fmt_vid_cap		= rvin_g_fmt_vid_cap,
> +	.vidioc_s_fmt_vid_cap		= rvin_mc_s_fmt_vid_cap,
> +	.vidioc_enum_fmt_vid_cap	= rvin_enum_fmt_vid_cap,
> +
> +	.vidioc_enum_input		= rvin_mc_enum_input,
> +	.vidioc_g_input			= rvin_g_input,
> +	.vidioc_s_input			= rvin_s_input,
> +
> +	.vidioc_reqbufs			= vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs		= vb2_ioctl_create_bufs,
> +	.vidioc_querybuf		= vb2_ioctl_querybuf,
> +	.vidioc_qbuf			= vb2_ioctl_qbuf,
> +	.vidioc_dqbuf			= vb2_ioctl_dqbuf,
> +	.vidioc_expbuf			= vb2_ioctl_expbuf,
> +	.vidioc_prepare_buf		= vb2_ioctl_prepare_buf,
> +	.vidioc_streamon		= vb2_ioctl_streamon,
> +	.vidioc_streamoff		= vb2_ioctl_streamoff,
> +
> +	.vidioc_log_status		= v4l2_ctrl_log_status,
> +	.vidioc_subscribe_event		= rvin_subscribe_event,
> +	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
> +};
> +
>  /* -----------------------------------------------------------------------------
>   * File Operations
>   */
> @@ -819,6 +900,60 @@ static const struct v4l2_file_operations rvin_fops = {
>  	.read		= vb2_fop_read,
>  };
>  
> +/* -----------------------------------------------------------------------------
> + * Media controller file Operations
> + */
> +
> +static int rvin_mc_open(struct file *file)
> +{
> +	struct rvin_dev *vin = video_drvdata(file);
> +	int ret;
> +
> +	mutex_lock(&vin->lock);
> +
> +	file->private_data = vin;
> +
> +	ret = v4l2_fh_open(file);
> +	if (ret)
> +		goto unlock;
> +
> +	pm_runtime_get_sync(vin->dev);
> +	v4l2_pipeline_pm_use(&vin->vdev.entity, 1);
> +
> +unlock:
> +	mutex_unlock(&vin->lock);
> +
> +	return ret;
> +}
> +
> +static int rvin_mc_release(struct file *file)
> +{
> +	struct rvin_dev *vin = video_drvdata(file);
> +	int ret;
> +
> +	mutex_lock(&vin->lock);
> +
> +	/* the release helper will cleanup any on-going streaming */
> +	ret = _vb2_fop_release(file, NULL);
> +
> +	v4l2_pipeline_pm_use(&vin->vdev.entity, 0);
> +	pm_runtime_put(vin->dev);
> +
> +	mutex_unlock(&vin->lock);
> +
> +	return ret;
> +}
> +
> +static const struct v4l2_file_operations rvin_mc_fops = {
> +	.owner		= THIS_MODULE,
> +	.unlocked_ioctl	= video_ioctl2,
> +	.open		= rvin_mc_open,
> +	.release	= rvin_mc_release,
> +	.poll		= vb2_fop_poll,
> +	.mmap		= vb2_fop_mmap,
> +	.read		= vb2_fop_read,
> +};
> +
>  void rvin_v4l2_remove(struct rvin_dev *vin)
>  {
>  	v4l2_info(&vin->v4l2_dev, "Removing %s\n",
> @@ -851,19 +986,33 @@ int rvin_v4l2_probe(struct rvin_dev *vin)
>  	vin->v4l2_dev.notify = rvin_notify;
>  
>  	/* video node */
> -	vdev->fops = &rvin_fops;
>  	vdev->v4l2_dev = &vin->v4l2_dev;
>  	vdev->queue = &vin->queue;
>  	snprintf(vdev->name, sizeof(vdev->name), "%s %s", KBUILD_MODNAME,
>  		 dev_name(vin->dev));
>  	vdev->release = video_device_release_empty;
> -	vdev->ioctl_ops = &rvin_ioctl_ops;
>  	vdev->lock = &vin->lock;
> -	vdev->ctrl_handler = &vin->ctrl_handler;
>  	vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING |
>  		V4L2_CAP_READWRITE;
>  
> +	/* Set some form of default format */
>  	vin->format.pixelformat	= RVIN_DEFAULT_FORMAT;
> +	vin->format.width = RVIN_DEFAULT_WIDTH;
> +	vin->format.height = RVIN_DEFAULT_HEIGHT;
> +	vin->format.colorspace = RVIN_DEFAULT_COLORSPACE;
> +
> +	ret = rvin_format_align(vin, &vin->format);
> +	if (ret)
> +		return ret;
> +
> +	if (vin->info->use_mc) {
> +		vdev->fops = &rvin_mc_fops;
> +		vdev->ioctl_ops = &rvin_mc_ioctl_ops;
> +	} else {
> +		vdev->fops = &rvin_fops;
> +		vdev->ioctl_ops = &rvin_ioctl_ops;
> +		vdev->ctrl_handler = &vin->ctrl_handler;
> +	}
>  
>  	ret = video_register_device(&vin->vdev, VFL_TYPE_GRABBER, -1);
>  	if (ret) {
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
> index 819d9c04ed8ffb36..12daff804bb6f77f 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -21,6 +21,7 @@
>  #include <media/v4l2-ctrls.h>
>  #include <media/v4l2-dev.h>
>  #include <media/v4l2-device.h>
> +#include <media/v4l2-mc.h>
>  #include <media/videobuf2-v4l2.h>
>  
>  /* Number of HW buffers */
> 

Re: [PATCH v6 18/25] rcar-vin: prepare for media controller mode initialization

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> When running in media controller mode a media pad is needed, register
> one. Also set the media bus format to CSI-2.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>


Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Regards,

	Hans


> ---
>  drivers/media/platform/rcar-vin/rcar-core.c | 23 ++++++++++++++++++++++-
>  drivers/media/platform/rcar-vin/rcar-vin.h  |  4 ++++
>  2 files changed, 26 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
> index fbbb22924cf3a045..dd0525f2ba336bc2 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -45,6 +45,10 @@ static int rvin_find_pad(struct v4l2_subdev *sd, int direction)
>  	return -EINVAL;
>  }
>  
> +/* -----------------------------------------------------------------------------
> + * Digital async notifier
> + */
> +
>  static bool rvin_mbus_supported(struct rvin_dev *vin)
>  {
>  	struct v4l2_subdev *sd = vin->digital.subdev;
> @@ -273,6 +277,20 @@ static int rvin_digital_graph_init(struct rvin_dev *vin)
>  	return 0;
>  }
>  
> +/* -----------------------------------------------------------------------------
> + * Group async notifier
> + */
> +
> +static int rvin_group_init(struct rvin_dev *vin)
> +{
> +	/* All our sources are CSI-2 */
> +	vin->mbus_cfg.type = V4L2_MBUS_CSI2;
> +	vin->mbus_cfg.flags = 0;
> +
> +	vin->pad.flags = MEDIA_PAD_FL_SINK;
> +	return media_entity_pads_init(&vin->vdev.entity, 1, &vin->pad);
> +}
> +
>  /* -----------------------------------------------------------------------------
>   * Platform Device Driver
>   */
> @@ -365,7 +383,10 @@ static int rcar_vin_probe(struct platform_device *pdev)
>  	if (ret)
>  		return ret;
>  
> -	ret = rvin_digital_graph_init(vin);
> +	if (vin->info->use_mc)
> +		ret = rvin_group_init(vin);
> +	else
> +		ret = rvin_digital_graph_init(vin);
>  	if (ret < 0)
>  		goto error;
>  
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
> index 12daff804bb6f77f..9c47669669c0469c 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -103,6 +103,8 @@ struct rvin_info {
>   * @notifier:		V4L2 asynchronous subdevs notifier
>   * @digital:		entity in the DT for local digital subdevice
>   *
> + * @pad:		pad for media controller
> + *
>   * @lock:		protects @queue
>   * @queue:		vb2 buffers queue
>   *
> @@ -132,6 +134,8 @@ struct rvin_dev {
>  	struct v4l2_async_notifier notifier;
>  	struct rvin_graph_entity digital;
>  
> +	struct media_pad pad;
> +
>  	struct mutex lock;
>  	struct vb2_queue queue;
>  
> 

Re: [PATCH v6 19/25] rcar-vin: add group allocator functions

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> In media controller mode all VIN instances needs to be part of the same
> media graph. There is also a need to each VIN instance to know and in
> some cases be able to communicate with other VIN instances.
> 
> Add a allocator framework where the first VIN instance to be probed

Add a -> Add an

> creates a shared data structure and creates a media device.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>


Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Note the small typo below.

Regards,

	Hans


> ---
>  drivers/media/platform/rcar-vin/rcar-core.c | 179 +++++++++++++++++++++++++++-
>  drivers/media/platform/rcar-vin/rcar-vin.h  |  38 ++++++
>  2 files changed, 216 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
> index dd0525f2ba336bc2..4218a73eb6885486 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -20,11 +20,170 @@
>  #include <linux/of_graph.h>
>  #include <linux/platform_device.h>
>  #include <linux/pm_runtime.h>
> +#include <linux/slab.h>
>  
>  #include <media/v4l2-fwnode.h>
>  
>  #include "rcar-vin.h"
>  
> +/* -----------------------------------------------------------------------------
> + * Gen3 CSI2 Group Allocator
> + */
> +
> +static int rvin_group_read_id(struct rvin_dev *vin, struct device_node *np)
> +{
> +	u32 val;
> +	int ret;
> +
> +	ret = of_property_read_u32(np, "renesas,id", &val);
> +	if (ret) {
> +		vin_err(vin, "%s: No renesas,id property found\n",
> +			of_node_full_name(np));
> +		return -EINVAL;
> +	}
> +
> +	if (val >= RCAR_VIN_NUM) {
> +		vin_err(vin, "%s: Invalid renesas,id '%u'\n",
> +			of_node_full_name(np), val);
> +		return -EINVAL;
> +	}
> +
> +	return val;
> +}
> +
> +static DEFINE_MUTEX(rvin_group_lock);
> +static struct rvin_group *rvin_group_data;
> +
> +static void rvin_group_release(struct kref *kref)
> +{
> +	struct rvin_group *group =
> +		container_of(kref, struct rvin_group, refcount);
> +
> +	mutex_lock(&rvin_group_lock);
> +
> +	media_device_unregister(&group->mdev);
> +	media_device_cleanup(&group->mdev);
> +
> +	rvin_group_data = NULL;
> +
> +	mutex_unlock(&rvin_group_lock);
> +
> +	kfree(group);
> +}
> +
> +static struct rvin_group *__rvin_group_allocate(struct rvin_dev *vin)
> +{
> +	struct rvin_group *group;
> +
> +	if (rvin_group_data) {
> +		group = rvin_group_data;
> +		kref_get(&group->refcount);
> +		vin_dbg(vin, "%s: get group=%p\n", __func__, group);
> +		return group;
> +	}
> +
> +	group = kzalloc(sizeof(*group), GFP_KERNEL);
> +	if (!group)
> +		return NULL;
> +
> +	kref_init(&group->refcount);
> +	rvin_group_data = group;
> +
> +	vin_dbg(vin, "%s: alloc group=%p\n", __func__, group);
> +	return group;
> +}
> +
> +static int rvin_group_add_vin(struct rvin_dev *vin)
> +{
> +	int ret;
> +
> +	ret = rvin_group_read_id(vin, vin->dev->of_node);
> +	if (ret < 0)
> +		return ret;
> +
> +	mutex_lock(&vin->group->lock);
> +
> +	if (vin->group->vin[ret]) {
> +		mutex_unlock(&vin->group->lock);
> +		vin_err(vin, "VIN number %d already occupied\n", ret);
> +		return -EINVAL;
> +	}
> +
> +	vin->group->vin[ret] = vin;
> +
> +	mutex_unlock(&vin->group->lock);
> +
> +	vin_dbg(vin, "I'm VIN number %d", ret);
> +
> +	return 0;
> +}
> +
> +static int rvin_group_allocate(struct rvin_dev *vin)
> +{
> +	struct rvin_group *group;
> +	struct media_device *mdev;
> +	int ret;
> +
> +	mutex_lock(&rvin_group_lock);
> +
> +	group = __rvin_group_allocate(vin);
> +	if (!group) {
> +		mutex_unlock(&rvin_group_lock);
> +		return -ENOMEM;
> +	}
> +
> +	/* Init group data if its not already initialized */

its -> it is

> +	mdev = &group->mdev;
> +	if (!mdev->dev) {
> +		mutex_init(&group->lock);
> +		mdev->dev = vin->dev;
> +
> +		strlcpy(mdev->driver_name, "Renesas VIN",
> +			sizeof(mdev->driver_name));
> +		strlcpy(mdev->model, vin->dev->of_node->name,
> +			sizeof(mdev->model));
> +		strlcpy(mdev->bus_info, of_node_full_name(vin->dev->of_node),
> +			sizeof(mdev->bus_info));
> +		media_device_init(mdev);
> +
> +		ret = media_device_register(mdev);
> +		if (ret) {
> +			vin_err(vin, "Failed to register media device\n");
> +			kref_put(&group->refcount, rvin_group_release);
> +			mutex_unlock(&rvin_group_lock);
> +			return ret;
> +		}
> +	}
> +
> +	vin->group = group;
> +	vin->v4l2_dev.mdev = mdev;
> +
> +	ret = rvin_group_add_vin(vin);
> +	if (ret) {
> +		kref_put(&group->refcount, rvin_group_release);
> +		mutex_unlock(&rvin_group_lock);
> +		return ret;
> +	}
> +
> +	mutex_unlock(&rvin_group_lock);
> +
> +	return 0;
> +}
> +
> +static void rvin_group_delete(struct rvin_dev *vin)
> +{
> +	unsigned int i;
> +
> +	mutex_lock(&vin->group->lock);
> +	for (i = 0; i < RCAR_VIN_NUM; i++)
> +		if (vin->group->vin[i] == vin)
> +			vin->group->vin[i] = NULL;
> +	mutex_unlock(&vin->group->lock);
> +
> +	vin_dbg(vin, "%s: group=%p\n", __func__, &vin->group);
> +	kref_put(&vin->group->refcount, rvin_group_release);
> +}
> +
>  /* -----------------------------------------------------------------------------
>   * Async notifier
>   */
> @@ -283,12 +442,27 @@ static int rvin_digital_graph_init(struct rvin_dev *vin)
>  
>  static int rvin_group_init(struct rvin_dev *vin)
>  {
> +	int ret;
> +
> +	ret = rvin_group_allocate(vin);
> +	if (ret)
> +		return ret;
> +
>  	/* All our sources are CSI-2 */
>  	vin->mbus_cfg.type = V4L2_MBUS_CSI2;
>  	vin->mbus_cfg.flags = 0;
>  
>  	vin->pad.flags = MEDIA_PAD_FL_SINK;
> -	return media_entity_pads_init(&vin->vdev.entity, 1, &vin->pad);
> +	ret = media_entity_pads_init(&vin->vdev.entity, 1, &vin->pad);
> +	if (ret)
> +		goto error_group;
> +
> +	return 0;
> +
> +error_group:
> +	rvin_group_delete(vin);
> +
> +	return ret;
>  }
>  
>  /* -----------------------------------------------------------------------------
> @@ -416,6 +590,9 @@ static int rcar_vin_remove(struct platform_device *pdev)
>  
>  	rvin_v4l2_remove(vin);
>  
> +	if (vin->info->use_mc)
> +		rvin_group_delete(vin);
> +
>  	rvin_dma_remove(vin);
>  
>  	return 0;
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
> index 9c47669669c0469c..88683aaee3b6acd5 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -17,6 +17,8 @@
>  #ifndef __RCAR_VIN__
>  #define __RCAR_VIN__
>  
> +#include <linux/kref.h>
> +
>  #include <media/v4l2-async.h>
>  #include <media/v4l2-ctrls.h>
>  #include <media/v4l2-dev.h>
> @@ -30,6 +32,9 @@
>  /* Address alignment mask for HW buffers */
>  #define HW_BUFFER_MASK 0x7f
>  
> +/* Max number on VIN instances that can be in a system */
> +#define RCAR_VIN_NUM 8
> +
>  enum chip_id {
>  	RCAR_H1,
>  	RCAR_M1,
> @@ -37,6 +42,15 @@ enum chip_id {
>  	RCAR_GEN3,
>  };
>  
> +enum rvin_csi_id {
> +	RVIN_CSI20,
> +	RVIN_CSI21,
> +	RVIN_CSI40,
> +	RVIN_CSI41,
> +	RVIN_CSI_MAX,
> +	RVIN_NC, /* Not Connected */
> +};
> +
>  /**
>   * STOPPED  - No operation in progress
>   * RUNNING  - Operation in progress have buffers
> @@ -75,6 +89,8 @@ struct rvin_graph_entity {
>  	unsigned int sink_pad;
>  };
>  
> +struct rvin_group;
> +
>  /**
>   * struct rvin_info - Information about the particular VIN implementation
>   * @chip:		type of VIN chip
> @@ -103,6 +119,7 @@ struct rvin_info {
>   * @notifier:		V4L2 asynchronous subdevs notifier
>   * @digital:		entity in the DT for local digital subdevice
>   *
> + * @group:		Gen3 CSI group
>   * @pad:		pad for media controller
>   *
>   * @lock:		protects @queue
> @@ -134,6 +151,7 @@ struct rvin_dev {
>  	struct v4l2_async_notifier notifier;
>  	struct rvin_graph_entity digital;
>  
> +	struct rvin_group *group;
>  	struct media_pad pad;
>  
>  	struct mutex lock;
> @@ -162,6 +180,26 @@ struct rvin_dev {
>  #define vin_warn(d, fmt, arg...)	dev_warn(d->dev, fmt, ##arg)
>  #define vin_err(d, fmt, arg...)		dev_err(d->dev, fmt, ##arg)
>  
> +/**
> + * struct rvin_group - VIN CSI2 group information
> + * @refcount:		number of VIN instances using the group
> + *
> + * @mdev:		media device which represents the group
> + *
> + * @lock:		protects the vin and csi members
> + * @vin:		VIN instances which are part of the group
> + * @csi:		CSI-2 entities that are part of the group
> + */
> +struct rvin_group {
> +	struct kref refcount;
> +
> +	struct media_device mdev;
> +
> +	struct mutex lock;
> +	struct rvin_dev *vin[RCAR_VIN_NUM];
> +	struct rvin_graph_entity csi[RVIN_CSI_MAX];
> +};
> +
>  int rvin_dma_probe(struct rvin_dev *vin, int irq);
>  void rvin_dma_remove(struct rvin_dev *vin);
>  
> 

Re: [PATCH v6 20/25] rcar-vin: add chsel information to rvin_info

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> Each Gen3 SoC has a limited set of predefined routing possibilities for
> which CSI-2 device and virtual channel can be routed to which VIN
> instance. Prepare to store this information in the struct rvin_info.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

I think I know what you are doing here, but the comments for struct
rvin_group_chsel need more work, they are confusing.

> ---
>  drivers/media/platform/rcar-vin/rcar-vin.h | 22 ++++++++++++++++++++++
>  1 file changed, 22 insertions(+)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
> index 88683aaee3b6acd5..617f254b52fe106d 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -35,6 +35,9 @@
>  /* Max number on VIN instances that can be in a system */
>  #define RCAR_VIN_NUM 8
>  
> +/* Max number of CHSEL values for any Gen3 SoC */
> +#define RCAR_CHSEL_MAX 6
> +
>  enum chip_id {
>  	RCAR_H1,
>  	RCAR_M1,
> @@ -91,6 +94,19 @@ struct rvin_graph_entity {
>  
>  struct rvin_group;
>  
> +
> +/** struct rvin_group_chsel - Map a CSI2 device and channel for a CHSEL value
> + * @csi:		VIN internal number for CSI2 device
> + * @chan:		CSI-2 channel number on remote. Note that channel

remote? I don't know what you mean here.

> + *			is not the same as VC. The CSI-2 hardware have 4

have -> has

> + *			channels it can output on but which VC is outputted

output to what?

> + *			on which channel is configurable inside the CSI-2.
> + */
> +struct rvin_group_chsel {
> +	enum rvin_csi_id csi;
> +	unsigned int chan;
> +};
> +
>  /**
>   * struct rvin_info - Information about the particular VIN implementation
>   * @chip:		type of VIN chip
> @@ -98,6 +114,9 @@ struct rvin_group;
>   *
>   * max_width:		max input width the VIN supports
>   * max_height:		max input height the VIN supports
> + *
> + * num_chsels:		number of possible chsel values for this VIN
> + * chsels:		routing table VIN <-> CSI-2 for the chsel values
>   */
>  struct rvin_info {
>  	enum chip_id chip;
> @@ -105,6 +124,9 @@ struct rvin_info {
>  
>  	unsigned int max_width;
>  	unsigned int max_height;
> +
> +	unsigned int num_chsels;
> +	struct rvin_group_chsel chsels[RCAR_VIN_NUM][RCAR_CHSEL_MAX];
>  };
>  
>  /**
> 

Regards,

	Hans

Re: [PATCH v6 21/25] rcar-vin: parse Gen3 OF and setup media graph

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> Parse the VIN Gen3 OF graph and register all CSI-2 devices in the VIN
> group common media device. Once all CSI-2 subdevice is added to the

subdevice is -> subdevices are

> common media device create links between them.
> 
> The parsing and registering CSI-2 subdevices with the v4l2 async
> framework is a collaborative effort shared between the VIN instances
> which are part of the group. The fist rcar-vin instance parses OF and

fist -> first

> finds all other VIN and CSI-2 nodes which are part of the graph. It
> stores a bit mask of all VIN instances found and handles to all CSI-2
> nodes.
> 
> The bit mask is used to figure out when all VIN instances have been
> probed. Once the last VIN instance is probed this is detected and this
> instance registers all CSI-2 subdevices in its private async notifier.
> Once the .complete() callback of this notifier is called it creates the
> media controller links between all entities in the graph.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

See comments below for several typo/naming issues.

Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Regards,

	Hans

> ---
>  drivers/media/platform/rcar-vin/rcar-core.c | 296 +++++++++++++++++++++++++++-
>  drivers/media/platform/rcar-vin/rcar-vin.h  |   7 +-
>  2 files changed, 301 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
> index 4218a73eb6885486..2aba442a0750e91a 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -440,10 +440,268 @@ static int rvin_digital_graph_init(struct rvin_dev *vin)
>   * Group async notifier
>   */
>  
> -static int rvin_group_init(struct rvin_dev *vin)
> +/* group lock should be held when calling this function */
> +static int rvin_group_add_link(struct rvin_dev *vin,
> +			       struct media_entity *source,
> +			       unsigned int source_idx,
> +			       struct media_entity *sink,
> +			       unsigned int sink_idx,
> +			       u32 flags)
> +{
> +	struct media_pad *source_pad, *sink_pad;
> +	int ret = 0;
> +
> +	source_pad = &source->pads[source_idx];
> +	sink_pad = &sink->pads[sink_idx];
> +
> +	if (!media_entity_find_link(source_pad, sink_pad))
> +		ret = media_create_pad_link(source, source_idx,
> +					    sink, sink_idx, flags);
> +
> +	if (ret)
> +		vin_err(vin, "Error adding link from %s to %s\n",
> +			source->name, sink->name);
> +
> +	return ret;
> +}
> +
> +static int rvin_group_update_links(struct rvin_dev *vin)
>  {
> +	struct media_entity *source, *sink;
> +	struct rvin_dev *master;
> +	unsigned int i, n, idx, chsel, csi;
> +	u32 flags;
>  	int ret;
>  
> +	mutex_lock(&vin->group->lock);
> +
> +	for (n = 0; n < RCAR_VIN_NUM; n++) {
> +
> +		/* Check that VIN is part of the group */
> +		if (!vin->group->vin[n])
> +			continue;
> +
> +		/* Check that subgroup master is part of the group */
> +		master = vin->group->vin[n < 4 ? 0 : 4];
> +		if (!master)
> +			continue;
> +
> +		chsel = rvin_get_chsel(master);
> +
> +		for (i = 0; i < vin->info->num_chsels; i++) {
> +			csi = vin->info->chsels[n][i].csi;
> +
> +			/* If the CSI-2 is out of bounds it's a noop, skip */
> +			if (csi >= RVIN_CSI_MAX)
> +				continue;
> +
> +			/* Check that CSI-2 are part of the group */
> +			if (!vin->group->csi[csi].subdev)
> +				continue;
> +
> +			source = &vin->group->csi[csi].subdev->entity;
> +			sink = &vin->group->vin[n]->vdev.entity;
> +			idx = vin->info->chsels[n][i].chan + 1;
> +			flags = i == chsel ? MEDIA_LNK_FL_ENABLED : 0;
> +
> +			ret = rvin_group_add_link(vin, source, idx, sink, 0,
> +						  flags);
> +			if (ret)
> +				goto out;
> +		}
> +	}
> +out:
> +	mutex_unlock(&vin->group->lock);
> +
> +	return ret;
> +}
> +
> +static int rvin_group_notify_complete(struct v4l2_async_notifier *notifier)
> +{
> +	struct rvin_dev *vin = notifier_to_vin(notifier);
> +	int ret;
> +
> +	ret = v4l2_device_register_subdev_nodes(&vin->v4l2_dev);
> +	if (ret) {
> +		vin_err(vin, "Failed to register subdev nodes\n");
> +		return ret;
> +	}
> +
> +	return rvin_group_update_links(vin);
> +}
> +
> +static void rvin_group_notify_unbind(struct v4l2_async_notifier *notifier,
> +				     struct v4l2_subdev *subdev,
> +				     struct v4l2_async_subdev *asd)
> +{
> +	struct rvin_dev *vin = notifier_to_vin(notifier);
> +	struct rvin_graph_entity *csi = to_rvin_graph_entity(asd);
> +
> +	mutex_lock(&vin->group->lock);
> +	csi->subdev = NULL;
> +	mutex_unlock(&vin->group->lock);
> +}
> +
> +static int rvin_group_notify_bound(struct v4l2_async_notifier *notifier,
> +				   struct v4l2_subdev *subdev,
> +				   struct v4l2_async_subdev *asd)
> +{
> +	struct rvin_dev *vin = notifier_to_vin(notifier);
> +	struct rvin_graph_entity *csi = to_rvin_graph_entity(asd);
> +
> +	v4l2_set_subdev_hostdata(subdev, vin);
> +
> +	mutex_lock(&vin->group->lock);
> +	vin_dbg(vin, "Bound CSI-2 %s\n", subdev->name);
> +	csi->subdev = subdev;
> +	mutex_unlock(&vin->group->lock);
> +
> +	return 0;
> +}
> +
> +static struct device_node *rvin_group_get_remote(struct rvin_dev *vin,
> +						 struct device_node *node)
> +{
> +	struct device_node *np;
> +
> +	np = of_graph_get_remote_port_parent(node);
> +	if (!np) {
> +		vin_err(vin, "Remote not found %s\n", of_node_full_name(node));

I prefer 'Remote port' instead of just 'Remote'.

> +		return NULL;
> +	}
> +
> +	/* Not all remotes are available, this is OK */

Ditto

> +	if (!of_device_is_available(np)) {
> +		vin_dbg(vin, "Remote %s is not available\n",

Ditto

I won't repeat this. Please check the patch series for similar occurrences.

> +			of_node_full_name(np));
> +		of_node_put(np);
> +		return NULL;
> +	}
> +
> +	return np;
> +}
> +
> +/* group lock should be held when calling this function */
> +static int rvin_group_graph_parse(struct rvin_dev *vin, struct device_node *np)
> +{
> +	int i, id, ret;
> +
> +	/* Read VIN id from DT */
> +	id = rvin_group_read_id(vin, np);
> +	if (id < 0)
> +		return id;
> +
> +	/* Check if VIN is already handled */
> +	if (vin->group->mask & BIT(id))
> +		return 0;
> +
> +	vin->group->mask |= BIT(id);
> +
> +	vin_dbg(vin, "Handling VIN%d\n", id);
> +
> +	/* Parse all enpoints for CSI-2 and VIN nodes */

enpoints -> endpoints

> +	for (i = 0; i < RVIN_CSI_MAX; i++) {
> +		struct device_node *ep, *csi, *remote;
> +
> +		/* Check if instance is connected to the CSI-2 */
> +		ep = of_graph_get_endpoint_by_regs(np, 1, i);
> +		if (!ep) {
> +			vin_dbg(vin, "VIN%d: ep %d not connected\n", id, i);
> +			continue;
> +		}
> +
> +		if (vin->group->csi[i].asd.match.fwnode.fwnode) {
> +			of_node_put(ep);
> +			vin_dbg(vin, "VIN%d: ep %d already handled\n", id, i);
> +			continue;
> +		}
> +
> +		csi = rvin_group_get_remote(vin, ep);
> +		of_node_put(ep);
> +		if (!csi)
> +			continue;
> +
> +		vin->group->csi[i].asd.match.fwnode.fwnode =
> +			of_fwnode_handle(csi);
> +		vin->group->csi[i].asd.match_type = V4L2_ASYNC_MATCH_FWNODE;
> +
> +		vin_dbg(vin, "VIN%d ep: %d handled CSI-2 %s\n", id, i,
> +			of_node_full_name(csi));
> +
> +		/* Parse the CSI-2 for all VIN nodes connected to it */
> +		ep = NULL;
> +		while (1) {
> +			ep = of_graph_get_next_endpoint(csi, ep);
> +			if (!ep)
> +				break;
> +
> +			remote = rvin_group_get_remote(vin, ep);
> +			if (!remote)
> +				continue;
> +
> +			if (of_match_node(vin->dev->driver->of_match_table,
> +					  remote)) {
> +				ret = rvin_group_graph_parse(vin, remote);
> +				if (ret)
> +					return ret;
> +
> +			}
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +static int rvin_group_graph_register(struct rvin_dev *vin)
> +{
> +	struct v4l2_async_subdev **subdevs = NULL;
> +	int i, n, ret, count = 0;
> +
> +	mutex_lock(&vin->group->lock);
> +
> +	/* Count how many CSI-2 nodes found */
> +	for (i = 0; i < RVIN_CSI_MAX; i++)
> +		if (vin->group->csi[i].asd.match.fwnode.fwnode)
> +			count++;
> +
> +	if (!count) {
> +		mutex_unlock(&vin->group->lock);
> +		return 0;
> +	}
> +
> +	/* Allocate and setup list of subdevices for the notifier */
> +	subdevs = devm_kzalloc(vin->dev, sizeof(*subdevs) * count, GFP_KERNEL);
> +	if (subdevs == NULL) {
> +		mutex_unlock(&vin->group->lock);
> +		return -ENOMEM;
> +	}
> +
> +	n = 0;
> +	for (i = 0; i < RVIN_CSI_MAX; i++)
> +		if (vin->group->csi[i].asd.match.fwnode.fwnode)
> +			subdevs[n++] = &vin->group->csi[i].asd;
> +
> +	vin_dbg(vin, "Claimed %d subdevices for group\n", count);
> +
> +	vin->notifier.num_subdevs = count;
> +	vin->notifier.subdevs = subdevs;
> +	vin->notifier.bound = rvin_group_notify_bound;
> +	vin->notifier.unbind = rvin_group_notify_unbind;
> +	vin->notifier.complete = rvin_group_notify_complete;
> +
> +	mutex_unlock(&vin->group->lock);
> +
> +	ret = v4l2_async_notifier_register(&vin->v4l2_dev, &vin->notifier);
> +	if (ret < 0)
> +		vin_err(vin, "Notifier registration failed\n");
> +
> +	return ret;
> +}
> +
> +static int rvin_group_init(struct rvin_dev *vin)
> +{
> +	int i, ret, count_mask, count_vin = 0;
> +
>  	ret = rvin_group_allocate(vin);
>  	if (ret)
>  		return ret;
> @@ -457,8 +715,44 @@ static int rvin_group_init(struct rvin_dev *vin)
>  	if (ret)
>  		goto error_group;
>  
> +	/*
> +	 * Check number of registered VIN in group against the group mask.

VIN -> VINs

> +	 * If the mask is empty DT have not yet been parsed and if the

have -> has

> +	 * count match all VINs are registered and it's safe to register

match -> matches

> +	 * the async notifier
> +	 */
> +	mutex_lock(&vin->group->lock);
> +
> +	if (!vin->group->mask) {
> +		ret = rvin_group_graph_parse(vin, vin->dev->of_node);
> +		if (ret) {
> +			mutex_unlock(&vin->group->lock);
> +			goto error_group;
> +		}
> +	}
> +
> +	for (i = 0; i < RCAR_VIN_NUM; i++)
> +		if (vin->group->vin[i])
> +			count_vin++;
> +
> +	count_mask = hweight_long(vin->group->mask);
> +
> +	mutex_unlock(&vin->group->lock);
> +
> +	ret = rvin_v4l2_probe(vin);
> +	if (ret)
> +		goto error_group;
> +
> +	if (count_vin == count_mask) {
> +		ret = rvin_group_graph_register(vin);
> +		if (ret)
> +			goto error_vdev;
> +	}
> +
>  	return 0;
>  
> +error_vdev:
> +	rvin_v4l2_remove(vin);
>  error_group:
>  	rvin_group_delete(vin);
>  
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
> index 617f254b52fe106d..a41301833221c750 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -92,6 +92,9 @@ struct rvin_graph_entity {
>  	unsigned int sink_pad;
>  };
>  
> +#define to_rvin_graph_entity(asd) \
> +	container_of(asd, struct rvin_graph_entity, asd)
> +
>  struct rvin_group;
>  
>  
> @@ -208,7 +211,8 @@ struct rvin_dev {
>   *
>   * @mdev:		media device which represents the group
>   *
> - * @lock:		protects the vin and csi members
> + * @lock:		protects the mask, vin and csi members
> + * @mask:		Mask of VIN instances found in DT
>   * @vin:		VIN instances which are part of the group
>   * @csi:		CSI-2 entities that are part of the group
>   */
> @@ -218,6 +222,7 @@ struct rvin_group {
>  	struct media_device mdev;
>  
>  	struct mutex lock;
> +	unsigned long mask;
>  	struct rvin_dev *vin[RCAR_VIN_NUM];
>  	struct rvin_graph_entity csi[RVIN_CSI_MAX];
>  };
> 

Re: [PATCH v6 22/25] rcar-vin: add link notify for Gen3

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> Add the ability to process media device link change request. Link
> enablement are a bit complicated on Gen3, if it's possible to enable a

enablement are -> enabling is

if -> whether or not

> link depends on what other links already are enabled. On Gen3 the 8 VIN

VIN -> VINs

> are split into two subgroups (VIN0-3 and VIN4-7) and from a routing
> perspective these two groups are independent of each other. Each
> subgroups routing is controlled by the subgroup VIN master instance

subgroups -> subgroup's

> (VIN0 and VIN4).
> 
> There are a limited number of possible route setups available for each
> subgroup and the configuration of each setup is dictated by the
> hardware. On H3 for example there are 6 possible route setups for each
> subgroup to choose from.
> 
> This leads to the media device link notification code being rather large
> since it will find the best routing configuration to try and accommodate
> as many links as possible. When it's not possible to enable a new link
> due to hardware constrains the link_notifier callback will return
> -EMLINK.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

See various comments below.

> ---
>  drivers/media/platform/rcar-vin/rcar-core.c | 203 ++++++++++++++++++++++++++++
>  1 file changed, 203 insertions(+)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
> index 2aba442a0750e91a..dec91e2f3ccdbd93 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -26,6 +26,207 @@
>  
>  #include "rcar-vin.h"
>  
> +/* -----------------------------------------------------------------------------
> + * Media Controller link notification
> + */
> +
> +static unsigned int rvin_group_csi_pad_to_chan(unsigned int pad)
> +{
> +	/*
> +	 * The CSI2 driver is rcar-csi2 and we know it's pad layout are

it's -> its

> +	 * 0: Source 1-4: Sinks so if we remove one from the pad we

I don't follow. '0: Source 1-4: Sinks' is too cryptic for me.

> +	 * get the rcar-vin internal CSI2 channel number
> +	 */
> +	return pad - 1;
> +}
> +
> +/* group lock should be held when calling this function */
> +static int rvin_group_entity_to_vin_num(struct rvin_group *group,
> +					struct media_entity *entity)
> +{
> +	struct video_device *vdev;
> +	int i;
> +
> +	if (!is_media_entity_v4l2_video_device(entity))
> +		return -ENODEV;
> +
> +	vdev = media_entity_to_video_device(entity);
> +
> +	for (i = 0; i < RCAR_VIN_NUM; i++) {
> +		if (!group->vin[i])
> +			continue;
> +
> +		if (&group->vin[i]->vdev == vdev)
> +			return i;
> +	}
> +
> +	return -ENODEV;
> +}
> +
> +/* group lock should be held when calling this function */
> +static int rvin_group_entity_to_csi_num(struct rvin_group *group,
> +					struct media_entity *entity)
> +{
> +	struct v4l2_subdev *sd;
> +	int i;
> +
> +	if (!is_media_entity_v4l2_subdev(entity))
> +		return -ENODEV;
> +
> +	sd = media_entity_to_v4l2_subdev(entity);
> +
> +	for (i = 0; i < RVIN_CSI_MAX; i++)
> +		if (group->csi[i].subdev == sd)
> +			return i;
> +
> +	return -ENODEV;
> +}
> +
> +/* group lock should be held when calling this function */
> +static void __rvin_group_build_link_list(struct rvin_group *group,
> +					 struct rvin_group_chsel *map,
> +					 int start, int len)
> +{
> +	struct media_pad *vin_pad, *remote_pad;
> +	unsigned int n;
> +
> +	for (n = 0; n < len; n++) {
> +		map[n].csi = -1;
> +		map[n].chan = -1;
> +
> +		if (!group->vin[start + n])
> +			continue;
> +
> +		vin_pad = &group->vin[start + n]->vdev.entity.pads[0];
> +
> +		remote_pad = media_entity_remote_pad(vin_pad);
> +		if (!remote_pad)
> +			continue;
> +
> +		map[n].csi =
> +			rvin_group_entity_to_csi_num(group, remote_pad->entity);
> +		map[n].chan = rvin_group_csi_pad_to_chan(remote_pad->index);
> +	}
> +}
> +
> +/* group lock should be held when calling this function */
> +static int __rvin_group_try_get_chsel(struct rvin_group *group,
> +				      struct rvin_group_chsel *map,
> +				      int start, int len)
> +{
> +	const struct rvin_group_chsel *sel;
> +	unsigned int i, n;
> +	int chsel;
> +
> +	for (i = 0; i < group->vin[start]->info->num_chsels; i++) {
> +		chsel = i;
> +		for (n = 0; n < len; n++) {
> +
> +			/* If the link is not active it's OK */
> +			if (map[n].csi == -1)
> +				continue;
> +
> +			/* Check if chsel match requested link */

match -> matches

> +			sel = &group->vin[start]->info->chsels[start + n][i];
> +			if (map[n].csi != sel->csi ||
> +			    map[n].chan != sel->chan) {
> +				chsel = -1;
> +				break;
> +			}
> +		}
> +
> +		/* A chsel which satisfy the links have been found */

satisfy -> satifies
have -> has

> +		if (chsel != -1)
> +			return chsel;
> +	}
> +
> +	/* No chsel can satisfy the requested links */
> +	return -1;
> +}
> +
> +/* group lock should be held when calling this function */
> +static bool rvin_group_in_use(struct rvin_group *group)
> +{
> +	struct media_entity *entity;
> +
> +	media_device_for_each_entity(entity, &group->mdev)
> +		if (entity->use_count)
> +			return true;
> +
> +	return false;
> +}
> +
> +static int rvin_group_link_notify(struct media_link *link, u32 flags,
> +				  unsigned int notification)
> +{
> +	struct rvin_group *group = container_of(link->graph_obj.mdev,
> +						struct rvin_group, mdev);
> +	struct rvin_group_chsel chsel_map[4];
> +	int vin_num, vin_master, csi_num, csi_chan;
> +	unsigned int chsel;
> +
> +	mutex_lock(&group->lock);
> +
> +	vin_num = rvin_group_entity_to_vin_num(group, link->sink->entity);
> +	csi_num = rvin_group_entity_to_csi_num(group, link->source->entity);
> +	csi_chan = rvin_group_csi_pad_to_chan(link->source->index);
> +
> +	/*
> +	 * Figure out which VIN node is the subgroup master.
> +	 *
> +	 * VIN0-3 are controlled by VIN0
> +	 * VIN4-7 are controlled by VIN4
> +	 */
> +	vin_master = vin_num < 4 ? 0 : 4;
> +
> +	/* If not all devices exists something is horribly wrong */

exists -> exist

> +	if (vin_num < 0 || csi_num < 0 || !group->vin[vin_master])
> +		goto error;
> +
> +	/* Special checking only needed for links which are to be enabled */
> +	if (notification != MEDIA_DEV_NOTIFY_PRE_LINK_CH ||
> +	    !(flags & MEDIA_LNK_FL_ENABLED))
> +		goto out;
> +
> +	/* If any link in the group are in use, no new link can be enabled */

are -> is

> +	if (rvin_group_in_use(group))
> +		goto error;
> +
> +	/* If the VIN already have a active link it's busy */

have -> has
a -> an

> +	if (media_entity_remote_pad(&link->sink->entity->pads[0]))
> +		goto error;
> +
> +	/* Build list of active links */
> +	__rvin_group_build_link_list(group, chsel_map, vin_master, 4);
> +
> +	/* Add the new proposed link */
> +	chsel_map[vin_num - vin_master].csi = csi_num;
> +	chsel_map[vin_num - vin_master].chan = csi_chan;
> +
> +	/* See if there is a chsel value which match our link selection */

match -> matches

> +	chsel = __rvin_group_try_get_chsel(group, chsel_map, vin_master, 4);
> +
> +	/* No chsel can provide the request links */

request -> requested

> +	if (chsel == -1)
> +		goto error;
> +
> +	/* Update chsel value at group master */
> +	rvin_set_chsel(group->vin[vin_master], chsel);
> +
> +out:
> +	mutex_unlock(&group->lock);
> +
> +	return v4l2_pipeline_link_notify(link, flags, notification);
> +error:
> +	mutex_unlock(&group->lock);
> +
> +	return -EMLINK;
> +}
> +
> +static const struct media_device_ops rvin_media_ops = {
> +	.link_notify = rvin_group_link_notify,
> +};
> +
>  /* -----------------------------------------------------------------------------
>   * Gen3 CSI2 Group Allocator
>   */
> @@ -146,6 +347,8 @@ static int rvin_group_allocate(struct rvin_dev *vin)
>  			sizeof(mdev->bus_info));
>  		media_device_init(mdev);
>  
> +		mdev->ops = &rvin_media_ops;
> +
>  		ret = media_device_register(mdev);
>  		if (ret) {
>  			vin_err(vin, "Failed to register media device\n");
> 

Regards,

	Hans

Re: [PATCH v6 23/25] rcar-vin: extend {start,stop}_streaming to work with media controller

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> The procedure to start or stop streaming using the none MC single

none MC -> non-MC

> subdevice and the MC graph and multiple subdevices are quiet different.

quiet -> quite

> Create a new function to abstract which method is used based on which
> mode the driver is running in and add logic to start the MC graph.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> ---
>  drivers/media/platform/rcar-vin/rcar-dma.c | 112 +++++++++++++++++++++++++++--
>  1 file changed, 105 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
> index de1486ee2786b499..499253f94390f43e 100644
> --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> @@ -1087,15 +1087,115 @@ static void rvin_buffer_queue(struct vb2_buffer *vb)
>  	spin_unlock_irqrestore(&vin->qlock, flags);
>  }
>  
> +static int rvin_set_stream(struct rvin_dev *vin, int on)
> +{
> +	struct v4l2_subdev_format fmt = {
> +		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
> +	};
> +	struct media_pipeline *pipe;
> +	struct  v4l2_subdev *sd;
> +	struct media_pad *pad;
> +	int ret;
> +
> +	/* Not media controller used, simply pass operation to subdevice */
> +	if (!vin->info->use_mc) {
> +		ret = v4l2_subdev_call(vin->digital.subdev, video, s_stream,
> +				       on);
> +
> +		return ret == -ENOIOCTLCMD ? 0 : ret;
> +	}
> +
> +	pad = media_entity_remote_pad(&vin->pad);
> +	if (!pad)
> +		return -EPIPE;
> +
> +	sd = media_entity_to_v4l2_subdev(pad->entity);
> +	if (!sd)
> +		return -EPIPE;
> +
> +	if (on) {

Handle !on first and return.

The remaining code then has one indent level less.

> +		fmt.pad = pad->index;
> +		if (v4l2_subdev_call(sd, pad, get_fmt, NULL, &fmt))
> +			return -EPIPE;
> +
> +		switch (fmt.format.code) {
> +		case MEDIA_BUS_FMT_YUYV8_1X16:
> +		case MEDIA_BUS_FMT_UYVY8_2X8:
> +		case MEDIA_BUS_FMT_UYVY10_2X10:
> +		case MEDIA_BUS_FMT_RGB888_1X24:
> +			vin->code = fmt.format.code;
> +			break;
> +		default:
> +			return -EPIPE;
> +		}
> +
> +		switch (fmt.format.field) {
> +		case V4L2_FIELD_TOP:
> +		case V4L2_FIELD_BOTTOM:
> +		case V4L2_FIELD_NONE:
> +		case V4L2_FIELD_INTERLACED_TB:
> +		case V4L2_FIELD_INTERLACED_BT:
> +		case V4L2_FIELD_INTERLACED:
> +		case V4L2_FIELD_SEQ_TB:
> +		case V4L2_FIELD_SEQ_BT:
> +			/* Supported nativly */
> +			break;
> +		case V4L2_FIELD_ALTERNATE:
> +			switch (vin->format.field) {
> +			case V4L2_FIELD_TOP:
> +			case V4L2_FIELD_BOTTOM:
> +			case V4L2_FIELD_NONE:
> +				break;
> +			case V4L2_FIELD_INTERLACED_TB:
> +			case V4L2_FIELD_INTERLACED_BT:
> +			case V4L2_FIELD_INTERLACED:
> +			case V4L2_FIELD_SEQ_TB:
> +			case V4L2_FIELD_SEQ_BT:
> +				/* Use VIN hardware to combine the two fields */
> +				fmt.format.height *= 2;
> +				break;
> +			default:
> +				return -EPIPE;
> +			}
> +			break;
> +		default:
> +			return -EPIPE;
> +		}
> +
> +		if (fmt.format.width != vin->format.width ||
> +		    fmt.format.height != vin->format.height)
> +			return -EPIPE;
> +
> +		pipe = sd->entity.pipe ? sd->entity.pipe : &vin->vdev.pipe;
> +		if (media_pipeline_start(&vin->vdev.entity, pipe))
> +			return -EPIPE;
> +
> +		ret = v4l2_subdev_call(sd, video, s_stream, 1);
> +		if (ret == -ENOIOCTLCMD)
> +			ret = 0;
> +		if (ret)
> +			media_pipeline_stop(&vin->vdev.entity);
> +	} else {
> +		media_pipeline_stop(&vin->vdev.entity);
> +		ret = v4l2_subdev_call(sd, video, s_stream, 0);
> +	}
> +
> +	return ret;
> +}
> +
>  static int rvin_start_streaming(struct vb2_queue *vq, unsigned int count)
>  {
>  	struct rvin_dev *vin = vb2_get_drv_priv(vq);
> -	struct v4l2_subdev *sd;
>  	unsigned long flags;
>  	int ret;
>  
> -	sd = vin_to_source(vin);
> -	v4l2_subdev_call(sd, video, s_stream, 1);
> +	ret = rvin_set_stream(vin, 1);
> +	if (ret) {
> +		spin_lock_irqsave(&vin->qlock, flags);
> +		return_all_buffers(vin, VB2_BUF_STATE_QUEUED);
> +		spin_unlock_irqrestore(&vin->qlock, flags);
> +		return ret;
> +	}
>  
>  	spin_lock_irqsave(&vin->qlock, flags);
>  
> @@ -1104,7 +1204,7 @@ static int rvin_start_streaming(struct vb2_queue *vq, unsigned int count)
>  	ret = rvin_capture_start(vin);
>  	if (ret) {
>  		return_all_buffers(vin, VB2_BUF_STATE_QUEUED);
> -		v4l2_subdev_call(sd, video, s_stream, 0);
> +		rvin_set_stream(vin, 0);
>  	}
>  
>  	spin_unlock_irqrestore(&vin->qlock, flags);
> @@ -1115,7 +1215,6 @@ static int rvin_start_streaming(struct vb2_queue *vq, unsigned int count)
>  static void rvin_stop_streaming(struct vb2_queue *vq)
>  {
>  	struct rvin_dev *vin = vb2_get_drv_priv(vq);
> -	struct v4l2_subdev *sd;
>  	unsigned long flags;
>  	int retries = 0;
>  
> @@ -1154,8 +1253,7 @@ static void rvin_stop_streaming(struct vb2_queue *vq)
>  
>  	spin_unlock_irqrestore(&vin->qlock, flags);
>  
> -	sd = vin_to_source(vin);
> -	v4l2_subdev_call(sd, video, s_stream, 0);
> +	rvin_set_stream(vin, 0);
>  
>  	/* disable interrupts */
>  	rvin_disable_interrupts(vin);
> 

Regards,

	Hans

Re: [PATCH v6 24/25] rcar-vin: enable support for r8a7795

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> Add the SoC specific information for Renesas r8a7795.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Regards,

	Hans

> ---
>  drivers/media/platform/rcar-vin/Kconfig     |   2 +-
>  drivers/media/platform/rcar-vin/rcar-core.c | 145 ++++++++++++++++++++++++++++
>  2 files changed, 146 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/Kconfig b/drivers/media/platform/rcar-vin/Kconfig
> index af4c98b44d2e22cb..8fa7ee468c63afb9 100644
> --- a/drivers/media/platform/rcar-vin/Kconfig
> +++ b/drivers/media/platform/rcar-vin/Kconfig
> @@ -6,7 +6,7 @@ config VIDEO_RCAR_VIN
>  	select V4L2_FWNODE
>  	---help---
>  	  Support for Renesas R-Car Video Input (VIN) driver.
> -	  Supports R-Car Gen2 SoCs.
> +	  Supports R-Car Gen2 and Gen3 SoCs.
>  
>  	  To compile this driver as a module, choose M here: the
>  	  module will be called rcar-vin.
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
> index dec91e2f3ccdbd93..58d903ab9fb83faf 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -21,6 +21,7 @@
>  #include <linux/platform_device.h>
>  #include <linux/pm_runtime.h>
>  #include <linux/slab.h>
> +#include <linux/sys_soc.h>
>  
>  #include <media/v4l2-fwnode.h>
>  
> @@ -987,7 +988,139 @@ static const struct rvin_info rcar_info_gen2 = {
>  	.max_height = 2048,
>  };
>  
> +static const struct rvin_info rcar_info_r8a7795 = {
> +	.chip = RCAR_GEN3,
> +	.use_mc = true,
> +	.max_width = 4096,
> +	.max_height = 4096,
> +
> +	.num_chsels = 5,
> +	.chsels = {
> +		{
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 1 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +		}, {
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 1 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +		}, {
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 2 },
> +			{ .csi = RVIN_CSI20, .chan = 2 },
> +		}, {
> +			{ .csi = RVIN_CSI40, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_CSI40, .chan = 3 },
> +			{ .csi = RVIN_CSI20, .chan = 3 },
> +		}, {
> +			{ .csi = RVIN_CSI41, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI41, .chan = 1 },
> +			{ .csi = RVIN_CSI41, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +		}, {
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI41, .chan = 1 },
> +			{ .csi = RVIN_CSI41, .chan = 0 },
> +			{ .csi = RVIN_CSI41, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +		}, {
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_CSI41, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI41, .chan = 2 },
> +			{ .csi = RVIN_CSI20, .chan = 2 },
> +		}, {
> +			{ .csi = RVIN_CSI41, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_CSI41, .chan = 3 },
> +			{ .csi = RVIN_CSI20, .chan = 3 },
> +		},
> +	},
> +};
> +
> +static const struct rvin_info rcar_info_r8a7795es1 = {
> +	.chip = RCAR_GEN3,
> +	.use_mc = true,
> +	.max_width = 4096,
> +	.max_height = 4096,
> +
> +	.num_chsels = 6,
> +	.chsels = {
> +		{
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI21, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI21, .chan = 0 },
> +		}, {
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI21, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_CSI21, .chan = 1 },
> +		}, {
> +			{ .csi = RVIN_CSI21, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 2 },
> +			{ .csi = RVIN_CSI20, .chan = 2 },
> +			{ .csi = RVIN_CSI21, .chan = 2 },
> +		}, {
> +			{ .csi = RVIN_CSI40, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_CSI21, .chan = 1 },
> +			{ .csi = RVIN_CSI40, .chan = 3 },
> +			{ .csi = RVIN_CSI20, .chan = 3 },
> +			{ .csi = RVIN_CSI21, .chan = 3 },
> +		}, {
> +			{ .csi = RVIN_CSI41, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI21, .chan = 0 },
> +			{ .csi = RVIN_CSI41, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI21, .chan = 0 },
> +		}, {
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI21, .chan = 0 },
> +			{ .csi = RVIN_CSI41, .chan = 0 },
> +			{ .csi = RVIN_CSI41, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_CSI21, .chan = 1 },
> +		}, {
> +			{ .csi = RVIN_CSI21, .chan = 0 },
> +			{ .csi = RVIN_CSI41, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI41, .chan = 2 },
> +			{ .csi = RVIN_CSI20, .chan = 2 },
> +			{ .csi = RVIN_CSI21, .chan = 2 },
> +		}, {
> +			{ .csi = RVIN_CSI41, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_CSI21, .chan = 1 },
> +			{ .csi = RVIN_CSI41, .chan = 3 },
> +			{ .csi = RVIN_CSI20, .chan = 3 },
> +			{ .csi = RVIN_CSI21, .chan = 3 },
> +		},
> +	},
> +};
> +
>  static const struct of_device_id rvin_of_id_table[] = {
> +	{
> +		.compatible = "renesas,vin-r8a7795",
> +		.data = &rcar_info_r8a7795,
> +	},
>  	{
>  		.compatible = "renesas,vin-r8a7794",
>  		.data = &rcar_info_gen2,
> @@ -1020,6 +1153,11 @@ static const struct of_device_id rvin_of_id_table[] = {
>  };
>  MODULE_DEVICE_TABLE(of, rvin_of_id_table);
>  
> +static const struct soc_device_attribute r8a7795es1[] = {
> +	{ .soc_id = "r8a7795", .revision = "ES1.*" },
> +	{ /* sentinel */ }
> +};
> +
>  static int rcar_vin_probe(struct platform_device *pdev)
>  {
>  	const struct of_device_id *match;
> @@ -1038,6 +1176,13 @@ static int rcar_vin_probe(struct platform_device *pdev)
>  	vin->dev = &pdev->dev;
>  	vin->info = match->data;
>  
> +	/*
> +	 * Special care is needed on r8a7795 ES1.x since it
> +	 * uses different routing then r8a7795 ES2.0.
> +	 */
> +	if (vin->info == &rcar_info_r8a7795 && soc_device_match(r8a7795es1))
> +		vin->info = &rcar_info_r8a7795es1;
> +
>  	mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
>  	if (mem == NULL)
>  		return -EINVAL;
> 

Re: [PATCH v6 25/25] rcar-vin: enable support for r8a7796

[Hans Verkuil]

On 23/08/17 01:26, Niklas Söderlund wrote:
> Add the SoC specific information for Renesas r8a7796.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Regards,

	Hans

> ---
>  .../devicetree/bindings/media/rcar_vin.txt         |  1 +
>  drivers/media/platform/rcar-vin/rcar-core.c        | 64 ++++++++++++++++++++++
>  2 files changed, 65 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/media/rcar_vin.txt b/Documentation/devicetree/bindings/media/rcar_vin.txt
> index be38ad89d71ad05d..767358f39512aa17 100644
> --- a/Documentation/devicetree/bindings/media/rcar_vin.txt
> +++ b/Documentation/devicetree/bindings/media/rcar_vin.txt
> @@ -10,6 +10,7 @@ always slaves and support multiple input channels which can be either RGB,
>  YUVU, BT656 or CSI-2.
>  
>   - compatible: Must be one or more of the following
> +   - "renesas,vin-r8a7796" for the R8A7796 device
>     - "renesas,vin-r8a7795" for the R8A7795 device
>     - "renesas,vin-r8a7794" for the R8A7794 device
>     - "renesas,vin-r8a7793" for the R8A7793 device
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
> index 58d903ab9fb83faf..e01edd5f5925d26c 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -1116,7 +1116,71 @@ static const struct rvin_info rcar_info_r8a7795es1 = {
>  	},
>  };
>  
> +static const struct rvin_info rcar_info_r8a7796 = {
> +	.chip = RCAR_GEN3,
> +	.use_mc = true,
> +	.max_width = 4096,
> +	.max_height = 4096,
> +
> +	.num_chsels = 5,
> +	.chsels = {
> +		{
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_NC, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +		}, {
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_NC, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +		}, {
> +			{ .csi = RVIN_NC, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 2 },
> +			{ .csi = RVIN_CSI20, .chan = 2 },
> +		}, {
> +			{ .csi = RVIN_CSI40, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_NC, .chan = 1 },
> +			{ .csi = RVIN_CSI40, .chan = 3 },
> +			{ .csi = RVIN_CSI20, .chan = 3 },
> +		}, {
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_NC, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +		}, {
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_NC, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +		}, {
> +			{ .csi = RVIN_NC, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 2 },
> +			{ .csi = RVIN_CSI20, .chan = 2 },
> +		}, {
> +			{ .csi = RVIN_CSI40, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_NC, .chan = 1 },
> +			{ .csi = RVIN_CSI40, .chan = 3 },
> +			{ .csi = RVIN_CSI20, .chan = 3 },
> +		},
> +	},
> +};
> +
>  static const struct of_device_id rvin_of_id_table[] = {
> +	{
> +		.compatible = "renesas,vin-r8a7796",
> +		.data = &rcar_info_r8a7796,
> +	},
>  	{
>  		.compatible = "renesas,vin-r8a7795",
>  		.data = &rcar_info_r8a7795,
> 

Re: [PATCH v6 24/25] rcar-vin: enable support for r8a7795

[Geert Uytterhoeven]

On Wed, Aug 23, 2017 at 1:26 AM, Niklas Söderlund
<niklas.soderlund+renesas@ragnatech.se> wrote:
> Add the SoC specific information for Renesas r8a7795.
>
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c

> @@ -1038,6 +1176,13 @@ static int rcar_vin_probe(struct platform_device *pdev)
>         vin->dev = &pdev->dev;
>         vin->info = match->data;
>
> +       /*
> +        * Special care is needed on r8a7795 ES1.x since it
> +        * uses different routing then r8a7795 ES2.0.

than

> +        */
> +       if (vin->info == &rcar_info_r8a7795 && soc_device_match(r8a7795es1))
> +               vin->info = &rcar_info_r8a7795es1;
> +
>         mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
>         if (mem == NULL)
>                 return -EINVAL;

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds