[PATCH v9 00/28] rcar-vin: Add Gen3 with media controller

[PATCH v9 00/28] rcar-vin: Add Gen3 with media controller

[Niklas Söderlund]

Hi,

This series adds Gen3 VIN support to rcar-vin driver for Renesas r8a7795,
r8a7796 and r8a77970. It is based on the media-tree and depends on some 
of Fabrizio Castro patches as they touches the order of the compatible 
strings in the documentation to reduce merge conflicts. The dependencies 
are:

[PATCH v2 1/4] dt-bindings: media: rcar_vin: Reverse SoC part number list
[PATCH v2 2/4] dt-bindings: media: rcar_vin: add device tree support for r8a774[35]

The driver is tested on Renesas H3 (r8a7795, ES2.0),
M3-W (r8a7796) together with the rcar-csi2 driver (posted separately and
not yet upstream) and the Salvator-X onboard ADV7482. It is also tested 
on the V3M (r8a77970) on the Eagle board together with its expansion 
board with a ADV7482.

It is possible to capture both CVBS and HDMI video streams,
v4l2-compliance passes with no errors and media-ctl can be used to
change the routing and formats for the different entities in the media
graph.

Gen2 compatibility is verified on Koelsch and no problems where found,
video can be captured just like before and v4l2-compliance passes
without errors or warnings just like before this series.

I have started on a very basic test suite for the VIN driver at:

  https://git.ragnatech.se/vin-tests

And as before the state of the driver and information about how to test
it can be found on the elinux wiki:

  http://elinux.org/R-Car/Tests:rcar-vin

* Changes since b8
- Fixed issue in rvin_group_init() where rvin_group_update_links() was 
  called, but after moving the video device registration to the 
  complete() callback this is now invalid.
- Fixed spelling in commit messages.
- Added review and acks from Hans, Kieran and Rob.

* Changes since v7
- Dropped '[PATCH v7 02/25] rcar-vin: register the video device at probe time'
- Add patch which renames four badly name functions. Some of the 
  renaming was in v7 part of the dropped patch 02/25. Make it a own 
  patch and rename all badly named functions in one patch.
- Add patch to replace part of the functionality of the dropped patch v7 
  02/25. The new patch keeps the subdevice (un)registration calls in the 
  async callbacks bind() and unbind() but moves the direct subdevice 
  initialization which only is used on Gen2 from the Gen2 and Gen3 
  shared rvin_v4l2_register().
- Add patch to enable Renesas V3M (r8a77970)
- Patch 'rcar-vin: parse Gen3 OF and setup media graph' have had code 
  additions since v7 since it now registers the video devices in the 
  async complete() callback instead of at probe time as an effect of 
  dropping v7 02/25.
  - The complete() callback now register all video devices.
  - The unbind() callback now unregister all video devices.
  - A new member '*notifier' is added to struct rvin_group which keeps 
    track of which rcar-vin instance have registered its notifier on the 
    groups behalf.
  For the reason above all Reviewed-by tags have been dropped for this 
  patch.
- Replaced all kernel messages which used of_node_full_name() as now 
  only returns the basename and not till full path, thanks Geert.

    printk("%s", of_node_full_name(ep)); -> printk("%pOF", ep);

- Added Reviewed-by tags from Hans, big thanks!

* Changes since v6
- Rebase ontop of latest media-tree which brings in the use of the
  fwnode async helpers for Gen2.
- Updated DT binding documentation, thanks Laurent for very helpful
  input!
- Removed help text which where copied in from v4l2_ctrl_handler_init()
  documentation when moving that code block, this was a residue from the
  soc_camera conversion and should have been removed at that time.
- Removed bad check of tvnorms which disables IOCTLs if it's not set,
  this was a residue from soc_camera conversion and have use in the
  current driver.
- Moved all subdevice initialization from complete to bound handler
  while improving the unbind handler. With this move all operations of
  the ctrl_handler from the subdevice is handled in either bound or
  unbind removing races pointed out by Laurent.
- Renamed rvin_v4l2_probe() -> rvin_v4l2_register() and
  rvin_v4l2_remove() -> rvin_v4l2_unregister().
- Fold rvin_mbus_supported() into its only caller.
- Sort compatible string entries in ascending order.
- Improved documentation for struct rvin_group_chsel.
- Clarify comment in rvin_group_csi_pad_to_chan().
- Make use of of_device_get_match_data() as suggested by Geert.
- Fixed spelling mistakes.
- Added review tags from Hans.

* Changes since v5
- Extract and make use of common format checking for both Gen2 and Gen3.
- Assign pad at declaration time in rvin_get_sd_format()
- Always call pm_runtime_{get_sync,put}() and v4l2_pipeline_pm_use()
  when opening/closing a video device, remove the check of
  v4l2_fh_is_singular_file().
- Make rvin_set_chsel() return void instead of int since it always
  return 0.
- Simplify the VIN group allocator functions.
- Make the group notifier callbacks and setup more robust.
- Moved the video device registration back to probe time.
- Add H3 ES2.0 support.
- Fix handling of single field formats (top, bottom, alternate) as this
  was obviously wrong before but hidden by the Gen2 scaler support.
- Added review tags from Kieran.

* Changes since v4 (Not posted to ML)
- Updated to the new fwnode functions.
- Moved the registration of the video devices to the async notification
  callback.

* Changes since v3
- Only add neighboring subdevices to the async notifier. Instead of
  parsing the whole OF graph depend on incremental async subnotifier to
  discover the whole pipeline. This is needed to support arbitrarily
  long graphs and support the new ADV7482 prototype driver which Kieran
  is working on.
- Fix warning from lockdep, reported by Kieran.
- Fix commit messages from feedback from Sergei, thanks.
- Fix chip info an OF device ids sorting order, thanks Geert.
- Use subdev->of_node instead of subdev->dev->of_node, thanks Kieran.

* Changes since v2
- Do not try to control the subdevices in the media graph from the rcar-vin
  driver. Have user-space configure to format in the pipeline instead.
- Add link validation before starting the stream.
- Rework on how the subdevices are and the video node behave by defining
  specific V4L2 operations for the MC mode of operation, this simplified
  the driver quit a bit, thanks Laurent!
- Add a new 'renesas,id' DT property which is needed to to be able to
  keep the VIN to CSI-2 routing table inside the driver. Previously this
  information was taken from the CSI-2 DT node which is obviously the
  wrong way to do things. Thanks Laurent for pointing this out.
- Fixed a memory leek in the group allocator function.
- Return -EMLINK instead of -EBUSY if a MC link is not possible given
  the current routing setup.
- Add comments to clarify that the 4 channels from the CSI-2 node is not
  directly related to CSI-2 virtual channels, the CSI-2 node can output
  any VC on any of its output channels.

* Changes since v1
- Remove unneeded casts as pointed out by Geert.
- Fix spelling and DT documentation as pointed out by Geert and Sergei,
  thanks!
- Refresh patch 2/32 with an updated version, thanks Sakari for pointing
  this out.
- Add Sakaris Ack to patch 1/32.
- Rebase on top of v4.9-rc1 instead of v4.9-rc3 to ease integration
  testing together with renesas-drivers tree.

Niklas Söderlund (28):
  rcar-vin: add Gen3 devicetree bindings documentation
  rcar-vin: rename poorly named initialize and cleanup functions
  rcar-vin: unregister video device on driver removal
  rcar-vin: move subdevice handling to async callbacks
  rcar-vin: move chip information to own struct
  rcar-vin: move max width and height information to chip information
  rcar-vin: change name of video device
  rcar-vin: move functions regarding scaling
  rcar-vin: all Gen2 boards can scale simplify logic
  rcar-vin: do not reset crop and compose when setting format
  rcar-vin: do not allow changing scaling and composing while streaming
  rcar-vin: read subdevice format for crop only when needed
  rcar-vin: fix handling of single field frames (top, bottom and
    alternate fields)
  rcar-vin: move media bus configuration to struct rvin_info
  rcar-vin: enable Gen3 hardware configuration
  rcar-vin: add function to manipulate Gen3 chsel value
  rcar-vin: add flag to switch to media controller mode
  rcar-vin: break out format alignment and checking
  rcar-vin: use different v4l2 operations in media controller mode
  rcar-vin: prepare for media controller mode initialization
  rcar-vin: add group allocator functions
  rcar-vin: add chsel information to rvin_info
  rcar-vin: parse Gen3 OF and setup media graph
  rcar-vin: add link notify for Gen3
  rcar-vin: extend {start,stop}_streaming to work with media controller
  rcar-vin: enable support for r8a7795
  rcar-vin: enable support for r8a7796
  rcar-vin: enable support for r8a77970

 .../devicetree/bindings/media/rcar_vin.txt         |  118 +-
 drivers/media/platform/rcar-vin/Kconfig            |    2 +-
 drivers/media/platform/rcar-vin/rcar-core.c        | 1159 ++++++++++++++++++--
 drivers/media/platform/rcar-vin/rcar-dma.c         |  990 ++++++++++-------
 drivers/media/platform/rcar-vin/rcar-v4l2.c        |  445 +++++---
 drivers/media/platform/rcar-vin/rcar-vin.h         |  131 ++-
 6 files changed, 2143 insertions(+), 702 deletions(-)

-- 
2.15.0

[PATCH v9 01/28] rcar-vin: add Gen3 devicetree bindings documentation

[Niklas Söderlund]

Document the devicetree bindings for the CSI-2 inputs available on Gen3.

There is a need to add a custom property 'renesas,id' and to define
which CSI-2 input is described in which endpoint under the port@1 node.
This information is needed since there are a set of predefined routes
between each VIN and CSI-2 block. This routing table will be kept
inside the driver but in order for it to act on it it must know which
VIN and CSI-2 is which.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Acked-by: Rob Herring <robh@kernel.org>
---
 .../devicetree/bindings/media/rcar_vin.txt         | 116 ++++++++++++++++++---
 1 file changed, 104 insertions(+), 12 deletions(-)

diff --git a/Documentation/devicetree/bindings/media/rcar_vin.txt b/Documentation/devicetree/bindings/media/rcar_vin.txt
index ff9697ed81396e64..5a95d9668d2c7dfd 100644
--- a/Documentation/devicetree/bindings/media/rcar_vin.txt
+++ b/Documentation/devicetree/bindings/media/rcar_vin.txt
@@ -2,8 +2,12 @@ Renesas R-Car Video Input driver (rcar_vin)
 -------------------------------------------
 
 The rcar_vin device provides video input capabilities for the Renesas R-Car
-family of devices. The current blocks are always slaves and suppot one input
-channel which can be either RGB, YUYV or BT656.
+family of devices.
+
+Each VIN instance has a single parallel input that supports RGB and YUV video,
+with both external synchronization and BT.656 synchronization for the latter.
+Depending on the instance the VIN input is connected to external SoC pins, or
+on Gen3 to a CSI-2 receiver.
 
  - compatible: Must be one or more of the following
    - "renesas,vin-r8a7743" for the R8A7743 device
@@ -31,21 +35,38 @@ channel which can be either RGB, YUYV or BT656.
 Additionally, an alias named vinX will need to be created to specify
 which video input device this is.
 
-The per-board settings:
+The per-board settings Gen2:
  - port sub-node describing a single endpoint connected to the vin
    as described in video-interfaces.txt[1]. Only the first one will
    be considered as each vin interface has one input port.
 
-   These settings are used to work out video input format and widths
-   into the system.
+The per-board settings Gen3:
+
+Gen3 can support both a single connected parallel input source from
+external SoC pins (port0) and/or multiple parallel input sources from
+local SoC CSI-2 receivers (port1) depending on SoC.
 
+- renesas,id - ID number of the VIN, VINx in the documentation.
+- ports
+    - port0 - sub-node describing a single endpoint connected to the VIN
+      from external SoC pins described in video-interfaces.txt[1]. Only
+      the first one will be considered as each VIN interface has at most
+      one set of SoC external input pins.
+    - port1 - sub-nodes describing one or more endpoints connected to
+      the VIN from local SoC CSI-2 receivers. The endpoint numbers must
+      use the following schema.
 
-Device node example
--------------------
+        - Endpoint 0 - sub-node describing the endpoint which is CSI20
+        - Endpoint 1 - sub-node describing the endpoint which is CSI21
+        - Endpoint 2 - sub-node describing the endpoint which is CSI40
+        - Endpoint 3 - sub-node describing the endpoint which is CSI41
 
-	aliases {
-	       vin0 = &vin0;
-	};
+Device node example Gen2
+------------------------
+
+        aliases {
+                vin0 = &vin0;
+        };
 
         vin0: vin@0xe6ef0000 {
                 compatible = "renesas,vin-r8a7790", "renesas,rcar-gen2-vin";
@@ -55,8 +76,8 @@ Device node example
                 status = "disabled";
         };
 
-Board setup example (vin1 composite video input)
-------------------------------------------------
+Board setup example Gen2 (vin1 composite video input)
+-----------------------------------------------------
 
 &i2c2   {
         status = "ok";
@@ -95,6 +116,77 @@ Board setup example (vin1 composite video input)
         };
 };
 
+Device node example Gen3
+------------------------
+
+        vin0: video@e6ef0000 {
+                compatible = "renesas,vin-r8a7795";
+                reg = <0 0xe6ef0000 0 0x1000>;
+                interrupts = <GIC_SPI 188 IRQ_TYPE_LEVEL_HIGH>;
+                clocks = <&cpg CPG_MOD 811>;
+                power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
+                resets = <&cpg 811>;
+                renesas,id = <0>;
+
+                ports {
+                        #address-cells = <1>;
+                        #size-cells = <0>;
+
+                        port@1 {
+                                #address-cells = <1>;
+                                #size-cells = <0>;
+
+                                reg = <1>;
+
+                                vin0csi20: endpoint@0 {
+                                        reg = <0>;
+                                        remote-endpoint= <&csi20vin0>;
+                                };
+                                vin0csi21: endpoint@1 {
+                                        reg = <1>;
+                                        remote-endpoint= <&csi21vin0>;
+                                };
+                                vin0csi40: endpoint@2 {
+                                        reg = <2>;
+                                        remote-endpoint= <&csi40vin0>;
+                                };
+                        };
+                };
+        };
+
+        csi20: csi2@fea80000 {
+                compatible = "renesas,r8a7795-csi2";
+                reg = <0 0xfea80000 0 0x10000>;
+                interrupts = <GIC_SPI 184 IRQ_TYPE_LEVEL_HIGH>;
+                clocks = <&cpg CPG_MOD 714>;
+                power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
+                resets = <&cpg 714>;
+
+                ports {
+                        #address-cells = <1>;
+                        #size-cells = <0>;
+
+                        port@0 {
+                                reg = <0>;
+                                csi20_in: endpoint {
+                                        clock-lanes = <0>;
+                                        data-lanes = <1>;
+                                        remote-endpoint = <&adv7482_txb>;
+                                };
+                        };
+
+                        port@1 {
+                                #address-cells = <1>;
+                                #size-cells = <0>;
 
+                                reg = <1>;
+
+                                csi20vin0: endpoint@0 {
+                                        reg = <0>;
+                                        remote-endpoint = <&vin0csi20>;
+                                };
+                        };
+                };
+        };
 
 [1] video-interfaces.txt common video media interface
-- 
2.15.0

[PATCH v9 02/28] rcar-vin: rename poorly named initialize and cleanup functions

[Niklas Söderlund]

The functions to initialize and cleanup the hardware and video device
where poorly named from the start. Rename them to better describe their
intended function.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-core.c | 10 +++++-----
 drivers/media/platform/rcar-vin/rcar-dma.c  |  6 +++---
 drivers/media/platform/rcar-vin/rcar-v4l2.c |  4 ++--
 drivers/media/platform/rcar-vin/rcar-vin.h  |  8 ++++----
 4 files changed, 14 insertions(+), 14 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index 108d776f32651b27..f7a4c21909da6923 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -93,7 +93,7 @@ static int rvin_digital_notify_complete(struct v4l2_async_notifier *notifier)
 		return ret;
 	}
 
-	return rvin_v4l2_probe(vin);
+	return rvin_v4l2_register(vin);
 }
 
 static void rvin_digital_notify_unbind(struct v4l2_async_notifier *notifier,
@@ -103,7 +103,7 @@ static void rvin_digital_notify_unbind(struct v4l2_async_notifier *notifier,
 	struct rvin_dev *vin = notifier_to_vin(notifier);
 
 	vin_dbg(vin, "unbind digital subdev %s\n", subdev->name);
-	rvin_v4l2_remove(vin);
+	rvin_v4l2_unregister(vin);
 	vin->digital->subdev = NULL;
 }
 
@@ -245,7 +245,7 @@ static int rcar_vin_probe(struct platform_device *pdev)
 	if (irq < 0)
 		return irq;
 
-	ret = rvin_dma_probe(vin, irq);
+	ret = rvin_dma_register(vin, irq);
 	if (ret)
 		return ret;
 
@@ -260,7 +260,7 @@ static int rcar_vin_probe(struct platform_device *pdev)
 
 	return 0;
 error:
-	rvin_dma_remove(vin);
+	rvin_dma_unregister(vin);
 	v4l2_async_notifier_cleanup(&vin->notifier);
 
 	return ret;
@@ -275,7 +275,7 @@ static int rcar_vin_remove(struct platform_device *pdev)
 	v4l2_async_notifier_unregister(&vin->notifier);
 	v4l2_async_notifier_cleanup(&vin->notifier);
 
-	rvin_dma_remove(vin);
+	rvin_dma_unregister(vin);
 
 	return 0;
 }
diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index 23fdff7a7370842e..d701b52d198243b5 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -1153,14 +1153,14 @@ static const struct vb2_ops rvin_qops = {
 	.wait_finish		= vb2_ops_wait_finish,
 };
 
-void rvin_dma_remove(struct rvin_dev *vin)
+void rvin_dma_unregister(struct rvin_dev *vin)
 {
 	mutex_destroy(&vin->lock);
 
 	v4l2_device_unregister(&vin->v4l2_dev);
 }
 
-int rvin_dma_probe(struct rvin_dev *vin, int irq)
+int rvin_dma_register(struct rvin_dev *vin, int irq)
 {
 	struct vb2_queue *q = &vin->queue;
 	int i, ret;
@@ -1208,7 +1208,7 @@ int rvin_dma_probe(struct rvin_dev *vin, int irq)
 
 	return 0;
 error:
-	rvin_dma_remove(vin);
+	rvin_dma_unregister(vin);
 
 	return ret;
 }
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index b479b882da12f62d..178aecc94962abe2 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -839,7 +839,7 @@ static const struct v4l2_file_operations rvin_fops = {
 	.read		= vb2_fop_read,
 };
 
-void rvin_v4l2_remove(struct rvin_dev *vin)
+void rvin_v4l2_unregister(struct rvin_dev *vin)
 {
 	v4l2_info(&vin->v4l2_dev, "Removing %s\n",
 		  video_device_node_name(&vin->vdev));
@@ -866,7 +866,7 @@ static void rvin_notify(struct v4l2_subdev *sd,
 	}
 }
 
-int rvin_v4l2_probe(struct rvin_dev *vin)
+int rvin_v4l2_register(struct rvin_dev *vin)
 {
 	struct video_device *vdev = &vin->vdev;
 	struct v4l2_subdev *sd = vin_to_source(vin);
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 5382078143fb3869..85cb7ec53d2b08b5 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -153,11 +153,11 @@ struct rvin_dev {
 #define vin_warn(d, fmt, arg...)	dev_warn(d->dev, fmt, ##arg)
 #define vin_err(d, fmt, arg...)		dev_err(d->dev, fmt, ##arg)
 
-int rvin_dma_probe(struct rvin_dev *vin, int irq);
-void rvin_dma_remove(struct rvin_dev *vin);
+int rvin_dma_register(struct rvin_dev *vin, int irq);
+void rvin_dma_unregister(struct rvin_dev *vin);
 
-int rvin_v4l2_probe(struct rvin_dev *vin);
-void rvin_v4l2_remove(struct rvin_dev *vin);
+int rvin_v4l2_register(struct rvin_dev *vin);
+void rvin_v4l2_unregister(struct rvin_dev *vin);
 
 const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
 
-- 
2.15.0

[PATCH v9 03/28] rcar-vin: unregister video device on driver removal

[Niklas Söderlund]

If the video device was registered by the complete() callback it should
be unregistered when the driver is removed. Protect from printing an
uninitialized video device node name by adding a check in
rvin_v4l2_unregister() to identify that the video device is registered.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-core.c | 2 ++
 drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 +++
 2 files changed, 5 insertions(+)

diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index f7a4c21909da6923..6d99542ec74b49a7 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -272,6 +272,8 @@ static int rcar_vin_remove(struct platform_device *pdev)
 
 	pm_runtime_disable(&pdev->dev);
 
+	rvin_v4l2_unregister(vin);
+
 	v4l2_async_notifier_unregister(&vin->notifier);
 	v4l2_async_notifier_cleanup(&vin->notifier);
 
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index 178aecc94962abe2..32a658214f48fa49 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -841,6 +841,9 @@ static const struct v4l2_file_operations rvin_fops = {
 
 void rvin_v4l2_unregister(struct rvin_dev *vin)
 {
+	if (!video_is_registered(&vin->vdev))
+		return;
+
 	v4l2_info(&vin->v4l2_dev, "Removing %s\n",
 		  video_device_node_name(&vin->vdev));
 
-- 
2.15.0

[PATCH v9 04/28] rcar-vin: move subdevice handling to async callbacks

[Niklas Söderlund]

In preparation for Gen3 support move the subdevice initialization and
clean up from rvin_v4l2_{register,unregister}() directly to the async
callbacks. This simplifies the addition of Gen3 support as the
rvin_v4l2_register() can be shared for both Gen2 and Gen3 while direct
subdevice control are only used on Gen2.

While moving this code drop a large comment which is copied from the
framework documentation and fold rvin_mbus_supported() into its only
caller.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-core.c | 105 ++++++++++++++++++----------
 drivers/media/platform/rcar-vin/rcar-v4l2.c |  35 ----------
 2 files changed, 67 insertions(+), 73 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index 6d99542ec74b49a7..6ab51acd676641ec 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -46,47 +46,11 @@ static int rvin_find_pad(struct v4l2_subdev *sd, int direction)
 	return -EINVAL;
 }
 
-static bool rvin_mbus_supported(struct rvin_graph_entity *entity)
-{
-	struct v4l2_subdev *sd = entity->subdev;
-	struct v4l2_subdev_mbus_code_enum code = {
-		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
-	};
-
-	code.index = 0;
-	code.pad = entity->source_pad;
-	while (!v4l2_subdev_call(sd, pad, enum_mbus_code, NULL, &code)) {
-		code.index++;
-		switch (code.code) {
-		case MEDIA_BUS_FMT_YUYV8_1X16:
-		case MEDIA_BUS_FMT_UYVY8_2X8:
-		case MEDIA_BUS_FMT_UYVY10_2X10:
-		case MEDIA_BUS_FMT_RGB888_1X24:
-			entity->code = code.code;
-			return true;
-		default:
-			break;
-		}
-	}
-
-	return false;
-}
-
 static int rvin_digital_notify_complete(struct v4l2_async_notifier *notifier)
 {
 	struct rvin_dev *vin = notifier_to_vin(notifier);
 	int ret;
 
-	/* Verify subdevices mbus format */
-	if (!rvin_mbus_supported(vin->digital)) {
-		vin_err(vin, "Unsupported media bus format for %s\n",
-			vin->digital->subdev->name);
-		return -EINVAL;
-	}
-
-	vin_dbg(vin, "Found media bus format for %s: %d\n",
-		vin->digital->subdev->name, vin->digital->code);
-
 	ret = v4l2_device_register_subdev_nodes(&vin->v4l2_dev);
 	if (ret < 0) {
 		vin_err(vin, "Failed to register subdev nodes\n");
@@ -103,8 +67,16 @@ static void rvin_digital_notify_unbind(struct v4l2_async_notifier *notifier,
 	struct rvin_dev *vin = notifier_to_vin(notifier);
 
 	vin_dbg(vin, "unbind digital subdev %s\n", subdev->name);
+
+	mutex_lock(&vin->lock);
+
 	rvin_v4l2_unregister(vin);
+	v4l2_ctrl_handler_free(&vin->ctrl_handler);
+
+	vin->vdev.ctrl_handler = NULL;
 	vin->digital->subdev = NULL;
+
+	mutex_unlock(&vin->lock);
 }
 
 static int rvin_digital_notify_bound(struct v4l2_async_notifier *notifier,
@@ -112,12 +84,14 @@ static int rvin_digital_notify_bound(struct v4l2_async_notifier *notifier,
 				     struct v4l2_async_subdev *asd)
 {
 	struct rvin_dev *vin = notifier_to_vin(notifier);
+	struct v4l2_subdev_mbus_code_enum code = {
+		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
+	};
 	int ret;
 
 	v4l2_set_subdev_hostdata(subdev, vin);
 
 	/* Find source and sink pad of remote subdevice */
-
 	ret = rvin_find_pad(subdev, MEDIA_PAD_FL_SOURCE);
 	if (ret < 0)
 		return ret;
@@ -126,21 +100,74 @@ static int rvin_digital_notify_bound(struct v4l2_async_notifier *notifier,
 	ret = rvin_find_pad(subdev, MEDIA_PAD_FL_SINK);
 	vin->digital->sink_pad = ret < 0 ? 0 : ret;
 
+	/* Find compatible subdevices mbus format */
+	vin->digital->code = 0;
+	code.index = 0;
+	code.pad = vin->digital->source_pad;
+	while (!vin->digital->code &&
+	       !v4l2_subdev_call(subdev, pad, enum_mbus_code, NULL, &code)) {
+		code.index++;
+		switch (code.code) {
+		case MEDIA_BUS_FMT_YUYV8_1X16:
+		case MEDIA_BUS_FMT_UYVY8_2X8:
+		case MEDIA_BUS_FMT_UYVY10_2X10:
+		case MEDIA_BUS_FMT_RGB888_1X24:
+			vin->digital->code = code.code;
+			vin_dbg(vin, "Found media bus format for %s: %d\n",
+				subdev->name, vin->digital->code);
+			break;
+		default:
+			break;
+		}
+	}
+
+	if (!vin->digital->code) {
+		vin_err(vin, "Unsupported media bus format for %s\n",
+			subdev->name);
+		return -EINVAL;
+	}
+
+	/* Read tvnorms */
+	ret = v4l2_subdev_call(subdev, video, g_tvnorms, &vin->vdev.tvnorms);
+	if (ret < 0 && ret != -ENOIOCTLCMD && ret != -ENODEV)
+		return ret;
+
+	mutex_lock(&vin->lock);
+
+	/* Add the controls */
+	ret = v4l2_ctrl_handler_init(&vin->ctrl_handler, 16);
+	if (ret < 0)
+		goto err;
+
+	ret = v4l2_ctrl_add_handler(&vin->ctrl_handler, subdev->ctrl_handler,
+				    NULL);
+	if (ret < 0)
+		goto err_ctrl;
+
+	vin->vdev.ctrl_handler = &vin->ctrl_handler;
+
 	vin->digital->subdev = subdev;
 
+	mutex_unlock(&vin->lock);
+
 	vin_dbg(vin, "bound subdev %s source pad: %u sink pad: %u\n",
 		subdev->name, vin->digital->source_pad,
 		vin->digital->sink_pad);
 
 	return 0;
+err_ctrl:
+	v4l2_ctrl_handler_free(&vin->ctrl_handler);
+err:
+	mutex_unlock(&vin->lock);
+	return ret;
 }
+
 static const struct v4l2_async_notifier_operations rvin_digital_notify_ops = {
 	.bound = rvin_digital_notify_bound,
 	.unbind = rvin_digital_notify_unbind,
 	.complete = rvin_digital_notify_complete,
 };
 
-
 static int rvin_digital_parse_v4l2(struct device *dev,
 				   struct v4l2_fwnode_endpoint *vep,
 				   struct v4l2_async_subdev *asd)
@@ -277,6 +304,8 @@ static int rcar_vin_remove(struct platform_device *pdev)
 	v4l2_async_notifier_unregister(&vin->notifier);
 	v4l2_async_notifier_cleanup(&vin->notifier);
 
+	v4l2_ctrl_handler_free(&vin->ctrl_handler);
+
 	rvin_dma_unregister(vin);
 
 	return 0;
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index 32a658214f48fa49..4a0610a6b4503501 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -847,9 +847,6 @@ void rvin_v4l2_unregister(struct rvin_dev *vin)
 	v4l2_info(&vin->v4l2_dev, "Removing %s\n",
 		  video_device_node_name(&vin->vdev));
 
-	/* Checks internaly if handlers have been init or not */
-	v4l2_ctrl_handler_free(&vin->ctrl_handler);
-
 	/* Checks internaly if vdev have been init or not */
 	video_unregister_device(&vin->vdev);
 }
@@ -872,41 +869,10 @@ static void rvin_notify(struct v4l2_subdev *sd,
 int rvin_v4l2_register(struct rvin_dev *vin)
 {
 	struct video_device *vdev = &vin->vdev;
-	struct v4l2_subdev *sd = vin_to_source(vin);
 	int ret;
 
-	v4l2_set_subdev_hostdata(sd, vin);
-
 	vin->v4l2_dev.notify = rvin_notify;
 
-	ret = v4l2_subdev_call(sd, video, g_tvnorms, &vin->vdev.tvnorms);
-	if (ret < 0 && ret != -ENOIOCTLCMD && ret != -ENODEV)
-		return ret;
-
-	if (vin->vdev.tvnorms == 0) {
-		/* Disable the STD API if there are no tvnorms defined */
-		v4l2_disable_ioctl(&vin->vdev, VIDIOC_G_STD);
-		v4l2_disable_ioctl(&vin->vdev, VIDIOC_S_STD);
-		v4l2_disable_ioctl(&vin->vdev, VIDIOC_QUERYSTD);
-		v4l2_disable_ioctl(&vin->vdev, VIDIOC_ENUMSTD);
-	}
-
-	/* Add the controls */
-	/*
-	 * Currently the subdev with the largest number of controls (13) is
-	 * ov6550. So let's pick 16 as a hint for the control handler. Note
-	 * that this is a hint only: too large and you waste some memory, too
-	 * small and there is a (very) small performance hit when looking up
-	 * controls in the internal hash.
-	 */
-	ret = v4l2_ctrl_handler_init(&vin->ctrl_handler, 16);
-	if (ret < 0)
-		return ret;
-
-	ret = v4l2_ctrl_add_handler(&vin->ctrl_handler, sd->ctrl_handler, NULL);
-	if (ret < 0)
-		return ret;
-
 	/* video node */
 	vdev->fops = &rvin_fops;
 	vdev->v4l2_dev = &vin->v4l2_dev;
@@ -915,7 +881,6 @@ int rvin_v4l2_register(struct rvin_dev *vin)
 	vdev->release = video_device_release_empty;
 	vdev->ioctl_ops = &rvin_ioctl_ops;
 	vdev->lock = &vin->lock;
-	vdev->ctrl_handler = &vin->ctrl_handler;
 	vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING |
 		V4L2_CAP_READWRITE;
 
-- 
2.15.0

[PATCH v9 05/28] rcar-vin: move chip information to own struct

[Niklas Söderlund]

When Gen3 support is added to the driver more than chip ID will be
different for the different SoCs. To avoid a lot of if statements in the
code create a struct chip_info to store this information.

And while we are at it sort the compatible string entries and make use
of of_device_get_match_data() which will always work as the driver is DT
only, so there's always a valid match.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-core.c | 54 ++++++++++++++++++++++-------
 drivers/media/platform/rcar-vin/rcar-v4l2.c |  3 +-
 drivers/media/platform/rcar-vin/rcar-vin.h  | 12 +++++--
 3 files changed, 53 insertions(+), 16 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index 6ab51acd676641ec..73c1700a409bfd35 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -230,21 +230,53 @@ static int rvin_digital_graph_init(struct rvin_dev *vin)
  * Platform Device Driver
  */
 
+static const struct rvin_info rcar_info_h1 = {
+	.chip = RCAR_H1,
+};
+
+static const struct rvin_info rcar_info_m1 = {
+	.chip = RCAR_M1,
+};
+
+static const struct rvin_info rcar_info_gen2 = {
+	.chip = RCAR_GEN2,
+};
+
 static const struct of_device_id rvin_of_id_table[] = {
-	{ .compatible = "renesas,vin-r8a7794", .data = (void *)RCAR_GEN2 },
-	{ .compatible = "renesas,vin-r8a7793", .data = (void *)RCAR_GEN2 },
-	{ .compatible = "renesas,vin-r8a7791", .data = (void *)RCAR_GEN2 },
-	{ .compatible = "renesas,vin-r8a7790", .data = (void *)RCAR_GEN2 },
-	{ .compatible = "renesas,vin-r8a7779", .data = (void *)RCAR_H1 },
-	{ .compatible = "renesas,vin-r8a7778", .data = (void *)RCAR_M1 },
-	{ .compatible = "renesas,rcar-gen2-vin", .data = (void *)RCAR_GEN2 },
+	{
+		.compatible = "renesas,vin-r8a7778",
+		.data = &rcar_info_m1,
+	},
+	{
+		.compatible = "renesas,vin-r8a7779",
+		.data = &rcar_info_h1,
+	},
+	{
+		.compatible = "renesas,vin-r8a7790",
+		.data = &rcar_info_gen2,
+	},
+	{
+		.compatible = "renesas,vin-r8a7791",
+		.data = &rcar_info_gen2,
+	},
+	{
+		.compatible = "renesas,vin-r8a7793",
+		.data = &rcar_info_gen2,
+	},
+	{
+		.compatible = "renesas,vin-r8a7794",
+		.data = &rcar_info_gen2,
+	},
+	{
+		.compatible = "renesas,rcar-gen2-vin",
+		.data = &rcar_info_gen2,
+	},
 	{ },
 };
 MODULE_DEVICE_TABLE(of, rvin_of_id_table);
 
 static int rcar_vin_probe(struct platform_device *pdev)
 {
-	const struct of_device_id *match;
 	struct rvin_dev *vin;
 	struct resource *mem;
 	int irq, ret;
@@ -253,12 +285,8 @@ static int rcar_vin_probe(struct platform_device *pdev)
 	if (!vin)
 		return -ENOMEM;
 
-	match = of_match_device(of_match_ptr(rvin_of_id_table), &pdev->dev);
-	if (!match)
-		return -ENODEV;
-
 	vin->dev = &pdev->dev;
-	vin->chip = (enum chip_id)match->data;
+	vin->info = of_device_get_match_data(&pdev->dev);
 
 	mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
 	if (mem == NULL)
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index 4a0610a6b4503501..b1caa04921aa23bb 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -266,7 +266,8 @@ static int __rvin_try_format(struct rvin_dev *vin,
 	pix->sizeimage = max_t(u32, pix->sizeimage,
 			       rvin_format_sizeimage(pix));
 
-	if (vin->chip == RCAR_M1 && pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
+	if (vin->info->chip == RCAR_M1 &&
+	    pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
 		vin_err(vin, "pixel format XBGR32 not supported on M1\n");
 		return -EINVAL;
 	}
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 85cb7ec53d2b08b5..0d3949c8c08c8f63 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -88,11 +88,19 @@ struct rvin_graph_entity {
 	unsigned int sink_pad;
 };
 
+/**
+ * struct rvin_info - Information about the particular VIN implementation
+ * @chip:		type of VIN chip
+ */
+struct rvin_info {
+	enum chip_id chip;
+};
+
 /**
  * struct rvin_dev - Renesas VIN device structure
  * @dev:		(OF) device
  * @base:		device I/O register space remapped to virtual memory
- * @chip:		type of VIN chip
+ * @info:		info about VIN instance
  *
  * @vdev:		V4L2 video device associated with VIN
  * @v4l2_dev:		V4L2 device
@@ -120,7 +128,7 @@ struct rvin_graph_entity {
 struct rvin_dev {
 	struct device *dev;
 	void __iomem *base;
-	enum chip_id chip;
+	const struct rvin_info *info;
 
 	struct video_device vdev;
 	struct v4l2_device v4l2_dev;
-- 
2.15.0

[PATCH v9 06/28] rcar-vin: move max width and height information to chip information

[Niklas Söderlund]

On Gen3 the max supported width and height will be different from Gen2.
Move the limits to the struct rvin_info to prepare for Gen3 support.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-core.c | 6 ++++++
 drivers/media/platform/rcar-vin/rcar-v4l2.c | 6 ++----
 drivers/media/platform/rcar-vin/rcar-vin.h  | 6 ++++++
 3 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index 73c1700a409bfd35..03d3cd63e38bee11 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -232,14 +232,20 @@ static int rvin_digital_graph_init(struct rvin_dev *vin)
 
 static const struct rvin_info rcar_info_h1 = {
 	.chip = RCAR_H1,
+	.max_width = 2048,
+	.max_height = 2048,
 };
 
 static const struct rvin_info rcar_info_m1 = {
 	.chip = RCAR_M1,
+	.max_width = 2048,
+	.max_height = 2048,
 };
 
 static const struct rvin_info rcar_info_gen2 = {
 	.chip = RCAR_GEN2,
+	.max_width = 2048,
+	.max_height = 2048,
 };
 
 static const struct of_device_id rvin_of_id_table[] = {
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index b1caa04921aa23bb..59ec6d3d119590aa 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -23,8 +23,6 @@
 #include "rcar-vin.h"
 
 #define RVIN_DEFAULT_FORMAT	V4L2_PIX_FMT_YUYV
-#define RVIN_MAX_WIDTH		2048
-#define RVIN_MAX_HEIGHT		2048
 
 /* -----------------------------------------------------------------------------
  * Format Conversions
@@ -258,8 +256,8 @@ static int __rvin_try_format(struct rvin_dev *vin,
 	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
 
 	/* Limit to VIN capabilities */
-	v4l_bound_align_image(&pix->width, 2, RVIN_MAX_WIDTH, walign,
-			      &pix->height, 4, RVIN_MAX_HEIGHT, 2, 0);
+	v4l_bound_align_image(&pix->width, 2, vin->info->max_width, walign,
+			      &pix->height, 4, vin->info->max_height, 2, 0);
 
 	pix->bytesperline = max_t(u32, pix->bytesperline,
 				  rvin_format_bytesperline(pix));
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 0d3949c8c08c8f63..646f897f5c05ec4e 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -91,9 +91,15 @@ struct rvin_graph_entity {
 /**
  * struct rvin_info - Information about the particular VIN implementation
  * @chip:		type of VIN chip
+ *
+ * max_width:		max input width the VIN supports
+ * max_height:		max input height the VIN supports
  */
 struct rvin_info {
 	enum chip_id chip;
+
+	unsigned int max_width;
+	unsigned int max_height;
 };
 
 /**
-- 
2.15.0

[PATCH v9 07/28] rcar-vin: change name of video device

[Niklas Söderlund]

The rcar-vin driver needs to be part of a media controller to support
Gen3. Give each VIN instance a unique name so it can be referenced from
userspace.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index 59ec6d3d119590aa..19de99133f048960 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -876,7 +876,8 @@ int rvin_v4l2_register(struct rvin_dev *vin)
 	vdev->fops = &rvin_fops;
 	vdev->v4l2_dev = &vin->v4l2_dev;
 	vdev->queue = &vin->queue;
-	strlcpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name));
+	snprintf(vdev->name, sizeof(vdev->name), "%s %s", KBUILD_MODNAME,
+		 dev_name(vin->dev));
 	vdev->release = video_device_release_empty;
 	vdev->ioctl_ops = &rvin_ioctl_ops;
 	vdev->lock = &vin->lock;
-- 
2.15.0

[PATCH v9 08/28] rcar-vin: move functions regarding scaling

[Niklas Söderlund]

In preparation of refactoring the scaling code move the code regarding
scaling to to the top of the file to avoid the need to add forward
declarations. No code is changed in this commit only whole functions
moved inside the same file.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-dma.c | 806 +++++++++++++++--------------
 1 file changed, 405 insertions(+), 401 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index d701b52d198243b5..a7cda3922cb74baa 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -138,305 +138,6 @@ static u32 rvin_read(struct rvin_dev *vin, u32 offset)
 	return ioread32(vin->base + offset);
 }
 
-static int rvin_setup(struct rvin_dev *vin)
-{
-	u32 vnmc, dmr, dmr2, interrupts;
-	v4l2_std_id std;
-	bool progressive = false, output_is_yuv = false, input_is_yuv = false;
-
-	switch (vin->format.field) {
-	case V4L2_FIELD_TOP:
-		vnmc = VNMC_IM_ODD;
-		break;
-	case V4L2_FIELD_BOTTOM:
-		vnmc = VNMC_IM_EVEN;
-		break;
-	case V4L2_FIELD_INTERLACED:
-		/* Default to TB */
-		vnmc = VNMC_IM_FULL;
-		/* Use BT if video standard can be read and is 60 Hz format */
-		if (!v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
-			if (std & V4L2_STD_525_60)
-				vnmc = VNMC_IM_FULL | VNMC_FOC;
-		}
-		break;
-	case V4L2_FIELD_INTERLACED_TB:
-		vnmc = VNMC_IM_FULL;
-		break;
-	case V4L2_FIELD_INTERLACED_BT:
-		vnmc = VNMC_IM_FULL | VNMC_FOC;
-		break;
-	case V4L2_FIELD_ALTERNATE:
-	case V4L2_FIELD_NONE:
-		if (vin->continuous) {
-			vnmc = VNMC_IM_ODD_EVEN;
-			progressive = true;
-		} else {
-			vnmc = VNMC_IM_ODD;
-		}
-		break;
-	default:
-		vnmc = VNMC_IM_ODD;
-		break;
-	}
-
-	/*
-	 * Input interface
-	 */
-	switch (vin->digital->code) {
-	case MEDIA_BUS_FMT_YUYV8_1X16:
-		/* BT.601/BT.1358 16bit YCbCr422 */
-		vnmc |= VNMC_INF_YUV16;
-		input_is_yuv = true;
-		break;
-	case MEDIA_BUS_FMT_UYVY8_2X8:
-		/* BT.656 8bit YCbCr422 or BT.601 8bit YCbCr422 */
-		vnmc |= vin->digital->mbus_cfg.type == V4L2_MBUS_BT656 ?
-			VNMC_INF_YUV8_BT656 : VNMC_INF_YUV8_BT601;
-		input_is_yuv = true;
-		break;
-	case MEDIA_BUS_FMT_RGB888_1X24:
-		vnmc |= VNMC_INF_RGB888;
-		break;
-	case MEDIA_BUS_FMT_UYVY10_2X10:
-		/* BT.656 10bit YCbCr422 or BT.601 10bit YCbCr422 */
-		vnmc |= vin->digital->mbus_cfg.type == V4L2_MBUS_BT656 ?
-			VNMC_INF_YUV10_BT656 : VNMC_INF_YUV10_BT601;
-		input_is_yuv = true;
-		break;
-	default:
-		break;
-	}
-
-	/* Enable VSYNC Field Toogle mode after one VSYNC input */
-	dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
-
-	/* Hsync Signal Polarity Select */
-	if (!(vin->digital->mbus_cfg.flags & V4L2_MBUS_HSYNC_ACTIVE_LOW))
-		dmr2 |= VNDMR2_HPS;
-
-	/* Vsync Signal Polarity Select */
-	if (!(vin->digital->mbus_cfg.flags & V4L2_MBUS_VSYNC_ACTIVE_LOW))
-		dmr2 |= VNDMR2_VPS;
-
-	/*
-	 * Output format
-	 */
-	switch (vin->format.pixelformat) {
-	case V4L2_PIX_FMT_NV16:
-		rvin_write(vin,
-			   ALIGN(vin->format.width * vin->format.height, 0x80),
-			   VNUVAOF_REG);
-		dmr = VNDMR_DTMD_YCSEP;
-		output_is_yuv = true;
-		break;
-	case V4L2_PIX_FMT_YUYV:
-		dmr = VNDMR_BPSM;
-		output_is_yuv = true;
-		break;
-	case V4L2_PIX_FMT_UYVY:
-		dmr = 0;
-		output_is_yuv = true;
-		break;
-	case V4L2_PIX_FMT_XRGB555:
-		dmr = VNDMR_DTMD_ARGB1555;
-		break;
-	case V4L2_PIX_FMT_RGB565:
-		dmr = 0;
-		break;
-	case V4L2_PIX_FMT_XBGR32:
-		/* Note: not supported on M1 */
-		dmr = VNDMR_EXRGB;
-		break;
-	default:
-		vin_err(vin, "Invalid pixelformat (0x%x)\n",
-			vin->format.pixelformat);
-		return -EINVAL;
-	}
-
-	/* Always update on field change */
-	vnmc |= VNMC_VUP;
-
-	/* If input and output use the same colorspace, use bypass mode */
-	if (input_is_yuv == output_is_yuv)
-		vnmc |= VNMC_BPS;
-
-	/* Progressive or interlaced mode */
-	interrupts = progressive ? VNIE_FIE : VNIE_EFE;
-
-	/* Ack interrupts */
-	rvin_write(vin, interrupts, VNINTS_REG);
-	/* Enable interrupts */
-	rvin_write(vin, interrupts, VNIE_REG);
-	/* Start capturing */
-	rvin_write(vin, dmr, VNDMR_REG);
-	rvin_write(vin, dmr2, VNDMR2_REG);
-
-	/* Enable module */
-	rvin_write(vin, vnmc | VNMC_ME, VNMC_REG);
-
-	return 0;
-}
-
-static void rvin_disable_interrupts(struct rvin_dev *vin)
-{
-	rvin_write(vin, 0, VNIE_REG);
-}
-
-static u32 rvin_get_interrupt_status(struct rvin_dev *vin)
-{
-	return rvin_read(vin, VNINTS_REG);
-}
-
-static void rvin_ack_interrupt(struct rvin_dev *vin)
-{
-	rvin_write(vin, rvin_read(vin, VNINTS_REG), VNINTS_REG);
-}
-
-static bool rvin_capture_active(struct rvin_dev *vin)
-{
-	return rvin_read(vin, VNMS_REG) & VNMS_CA;
-}
-
-static int rvin_get_active_slot(struct rvin_dev *vin, u32 vnms)
-{
-	if (vin->continuous)
-		return (vnms & VNMS_FBS_MASK) >> VNMS_FBS_SHIFT;
-
-	return 0;
-}
-
-static enum v4l2_field rvin_get_active_field(struct rvin_dev *vin, u32 vnms)
-{
-	if (vin->format.field == V4L2_FIELD_ALTERNATE) {
-		/* If FS is set it's a Even field */
-		if (vnms & VNMS_FS)
-			return V4L2_FIELD_BOTTOM;
-		return V4L2_FIELD_TOP;
-	}
-
-	return vin->format.field;
-}
-
-static void rvin_set_slot_addr(struct rvin_dev *vin, int slot, dma_addr_t addr)
-{
-	const struct rvin_video_format *fmt;
-	int offsetx, offsety;
-	dma_addr_t offset;
-
-	fmt = rvin_format_from_pixel(vin->format.pixelformat);
-
-	/*
-	 * There is no HW support for composition do the beast we can
-	 * by modifying the buffer offset
-	 */
-	offsetx = vin->compose.left * fmt->bpp;
-	offsety = vin->compose.top * vin->format.bytesperline;
-	offset = addr + offsetx + offsety;
-
-	/*
-	 * The address needs to be 128 bytes aligned. Driver should never accept
-	 * settings that do not satisfy this in the first place...
-	 */
-	if (WARN_ON((offsetx | offsety | offset) & HW_BUFFER_MASK))
-		return;
-
-	rvin_write(vin, offset, VNMB_REG(slot));
-}
-
-/* Moves a buffer from the queue to the HW slots */
-static bool rvin_fill_hw_slot(struct rvin_dev *vin, int slot)
-{
-	struct rvin_buffer *buf;
-	struct vb2_v4l2_buffer *vbuf;
-	dma_addr_t phys_addr_top;
-
-	if (vin->queue_buf[slot] != NULL)
-		return true;
-
-	if (list_empty(&vin->buf_list))
-		return false;
-
-	vin_dbg(vin, "Filling HW slot: %d\n", slot);
-
-	/* Keep track of buffer we give to HW */
-	buf = list_entry(vin->buf_list.next, struct rvin_buffer, list);
-	vbuf = &buf->vb;
-	list_del_init(to_buf_list(vbuf));
-	vin->queue_buf[slot] = vbuf;
-
-	/* Setup DMA */
-	phys_addr_top = vb2_dma_contig_plane_dma_addr(&vbuf->vb2_buf, 0);
-	rvin_set_slot_addr(vin, slot, phys_addr_top);
-
-	return true;
-}
-
-static bool rvin_fill_hw(struct rvin_dev *vin)
-{
-	int slot, limit;
-
-	limit = vin->continuous ? HW_BUFFER_NUM : 1;
-
-	for (slot = 0; slot < limit; slot++)
-		if (!rvin_fill_hw_slot(vin, slot))
-			return false;
-	return true;
-}
-
-static void rvin_capture_on(struct rvin_dev *vin)
-{
-	vin_dbg(vin, "Capture on in %s mode\n",
-		vin->continuous ? "continuous" : "single");
-
-	if (vin->continuous)
-		/* Continuous Frame Capture Mode */
-		rvin_write(vin, VNFC_C_FRAME, VNFC_REG);
-	else
-		/* Single Frame Capture Mode */
-		rvin_write(vin, VNFC_S_FRAME, VNFC_REG);
-}
-
-static int rvin_capture_start(struct rvin_dev *vin)
-{
-	struct rvin_buffer *buf, *node;
-	int bufs, ret;
-
-	/* Count number of free buffers */
-	bufs = 0;
-	list_for_each_entry_safe(buf, node, &vin->buf_list, list)
-		bufs++;
-
-	/* Continuous capture requires more buffers then there are HW slots */
-	vin->continuous = bufs > HW_BUFFER_NUM;
-
-	if (!rvin_fill_hw(vin)) {
-		vin_err(vin, "HW not ready to start, not enough buffers available\n");
-		return -EINVAL;
-	}
-
-	rvin_crop_scale_comp(vin);
-
-	ret = rvin_setup(vin);
-	if (ret)
-		return ret;
-
-	rvin_capture_on(vin);
-
-	vin->state = RUNNING;
-
-	return 0;
-}
-
-static void rvin_capture_stop(struct rvin_dev *vin)
-{
-	/* Set continuous & single transfer off */
-	rvin_write(vin, 0, VNFC_REG);
-
-	/* Disable module */
-	rvin_write(vin, rvin_read(vin, VNMC_REG) & ~VNMC_ME, VNMC_REG);
-}
-
 /* -----------------------------------------------------------------------------
  * Crop and Scaling Gen2
  */
@@ -757,139 +458,442 @@ static const struct vin_coeff vin_coeff_set[] = {
 			  0x0370e83b, 0x0310d439, 0x03a0f83d,
 			  0x0370e83c, 0x0300d438, 0x03b0fc3c },
 	}
-};
+};
+
+static void rvin_set_coeff(struct rvin_dev *vin, unsigned short xs)
+{
+	int i;
+	const struct vin_coeff *p_prev_set = NULL;
+	const struct vin_coeff *p_set = NULL;
+
+	/* Look for suitable coefficient values */
+	for (i = 0; i < ARRAY_SIZE(vin_coeff_set); i++) {
+		p_prev_set = p_set;
+		p_set = &vin_coeff_set[i];
+
+		if (xs < p_set->xs_value)
+			break;
+	}
+
+	/* Use previous value if its XS value is closer */
+	if (p_prev_set && p_set &&
+	    xs - p_prev_set->xs_value < p_set->xs_value - xs)
+		p_set = p_prev_set;
+
+	/* Set coefficient registers */
+	rvin_write(vin, p_set->coeff_set[0], VNC1A_REG);
+	rvin_write(vin, p_set->coeff_set[1], VNC1B_REG);
+	rvin_write(vin, p_set->coeff_set[2], VNC1C_REG);
+
+	rvin_write(vin, p_set->coeff_set[3], VNC2A_REG);
+	rvin_write(vin, p_set->coeff_set[4], VNC2B_REG);
+	rvin_write(vin, p_set->coeff_set[5], VNC2C_REG);
+
+	rvin_write(vin, p_set->coeff_set[6], VNC3A_REG);
+	rvin_write(vin, p_set->coeff_set[7], VNC3B_REG);
+	rvin_write(vin, p_set->coeff_set[8], VNC3C_REG);
+
+	rvin_write(vin, p_set->coeff_set[9], VNC4A_REG);
+	rvin_write(vin, p_set->coeff_set[10], VNC4B_REG);
+	rvin_write(vin, p_set->coeff_set[11], VNC4C_REG);
+
+	rvin_write(vin, p_set->coeff_set[12], VNC5A_REG);
+	rvin_write(vin, p_set->coeff_set[13], VNC5B_REG);
+	rvin_write(vin, p_set->coeff_set[14], VNC5C_REG);
+
+	rvin_write(vin, p_set->coeff_set[15], VNC6A_REG);
+	rvin_write(vin, p_set->coeff_set[16], VNC6B_REG);
+	rvin_write(vin, p_set->coeff_set[17], VNC6C_REG);
+
+	rvin_write(vin, p_set->coeff_set[18], VNC7A_REG);
+	rvin_write(vin, p_set->coeff_set[19], VNC7B_REG);
+	rvin_write(vin, p_set->coeff_set[20], VNC7C_REG);
+
+	rvin_write(vin, p_set->coeff_set[21], VNC8A_REG);
+	rvin_write(vin, p_set->coeff_set[22], VNC8B_REG);
+	rvin_write(vin, p_set->coeff_set[23], VNC8C_REG);
+}
+
+void rvin_crop_scale_comp(struct rvin_dev *vin)
+{
+	u32 xs, ys;
+
+	/* Set Start/End Pixel/Line Pre-Clip */
+	rvin_write(vin, vin->crop.left, VNSPPRC_REG);
+	rvin_write(vin, vin->crop.left + vin->crop.width - 1, VNEPPRC_REG);
+	switch (vin->format.field) {
+	case V4L2_FIELD_INTERLACED:
+	case V4L2_FIELD_INTERLACED_TB:
+	case V4L2_FIELD_INTERLACED_BT:
+		rvin_write(vin, vin->crop.top / 2, VNSLPRC_REG);
+		rvin_write(vin, (vin->crop.top + vin->crop.height) / 2 - 1,
+			   VNELPRC_REG);
+		break;
+	default:
+		rvin_write(vin, vin->crop.top, VNSLPRC_REG);
+		rvin_write(vin, vin->crop.top + vin->crop.height - 1,
+			   VNELPRC_REG);
+		break;
+	}
+
+	/* Set scaling coefficient */
+	ys = 0;
+	if (vin->crop.height != vin->compose.height)
+		ys = (4096 * vin->crop.height) / vin->compose.height;
+	rvin_write(vin, ys, VNYS_REG);
+
+	xs = 0;
+	if (vin->crop.width != vin->compose.width)
+		xs = (4096 * vin->crop.width) / vin->compose.width;
+
+	/* Horizontal upscaling is up to double size */
+	if (xs > 0 && xs < 2048)
+		xs = 2048;
+
+	rvin_write(vin, xs, VNXS_REG);
+
+	/* Horizontal upscaling is done out by scaling down from double size */
+	if (xs < 4096)
+		xs *= 2;
+
+	rvin_set_coeff(vin, xs);
+
+	/* Set Start/End Pixel/Line Post-Clip */
+	rvin_write(vin, 0, VNSPPOC_REG);
+	rvin_write(vin, 0, VNSLPOC_REG);
+	rvin_write(vin, vin->format.width - 1, VNEPPOC_REG);
+	switch (vin->format.field) {
+	case V4L2_FIELD_INTERLACED:
+	case V4L2_FIELD_INTERLACED_TB:
+	case V4L2_FIELD_INTERLACED_BT:
+		rvin_write(vin, vin->format.height / 2 - 1, VNELPOC_REG);
+		break;
+	default:
+		rvin_write(vin, vin->format.height - 1, VNELPOC_REG);
+		break;
+	}
+
+	if (vin->format.pixelformat == V4L2_PIX_FMT_NV16)
+		rvin_write(vin, ALIGN(vin->format.width, 0x20), VNIS_REG);
+	else
+		rvin_write(vin, ALIGN(vin->format.width, 0x10), VNIS_REG);
+
+	vin_dbg(vin,
+		"Pre-Clip: %ux%u@%u:%u YS: %d XS: %d Post-Clip: %ux%u@%u:%u\n",
+		vin->crop.width, vin->crop.height, vin->crop.left,
+		vin->crop.top, ys, xs, vin->format.width, vin->format.height,
+		0, 0);
+}
+
+void rvin_scale_try(struct rvin_dev *vin, struct v4l2_pix_format *pix,
+		    u32 width, u32 height)
+{
+	/* All VIN channels on Gen2 have scalers */
+	pix->width = width;
+	pix->height = height;
+}
+
+/* -----------------------------------------------------------------------------
+ * Hardware setup
+ */
+
+static int rvin_setup(struct rvin_dev *vin)
+{
+	u32 vnmc, dmr, dmr2, interrupts;
+	v4l2_std_id std;
+	bool progressive = false, output_is_yuv = false, input_is_yuv = false;
+
+	switch (vin->format.field) {
+	case V4L2_FIELD_TOP:
+		vnmc = VNMC_IM_ODD;
+		break;
+	case V4L2_FIELD_BOTTOM:
+		vnmc = VNMC_IM_EVEN;
+		break;
+	case V4L2_FIELD_INTERLACED:
+		/* Default to TB */
+		vnmc = VNMC_IM_FULL;
+		/* Use BT if video standard can be read and is 60 Hz format */
+		if (!v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
+			if (std & V4L2_STD_525_60)
+				vnmc = VNMC_IM_FULL | VNMC_FOC;
+		}
+		break;
+	case V4L2_FIELD_INTERLACED_TB:
+		vnmc = VNMC_IM_FULL;
+		break;
+	case V4L2_FIELD_INTERLACED_BT:
+		vnmc = VNMC_IM_FULL | VNMC_FOC;
+		break;
+	case V4L2_FIELD_ALTERNATE:
+	case V4L2_FIELD_NONE:
+		if (vin->continuous) {
+			vnmc = VNMC_IM_ODD_EVEN;
+			progressive = true;
+		} else {
+			vnmc = VNMC_IM_ODD;
+		}
+		break;
+	default:
+		vnmc = VNMC_IM_ODD;
+		break;
+	}
+
+	/*
+	 * Input interface
+	 */
+	switch (vin->digital->code) {
+	case MEDIA_BUS_FMT_YUYV8_1X16:
+		/* BT.601/BT.1358 16bit YCbCr422 */
+		vnmc |= VNMC_INF_YUV16;
+		input_is_yuv = true;
+		break;
+	case MEDIA_BUS_FMT_UYVY8_2X8:
+		/* BT.656 8bit YCbCr422 or BT.601 8bit YCbCr422 */
+		vnmc |= vin->digital->mbus_cfg.type == V4L2_MBUS_BT656 ?
+			VNMC_INF_YUV8_BT656 : VNMC_INF_YUV8_BT601;
+		input_is_yuv = true;
+		break;
+	case MEDIA_BUS_FMT_RGB888_1X24:
+		vnmc |= VNMC_INF_RGB888;
+		break;
+	case MEDIA_BUS_FMT_UYVY10_2X10:
+		/* BT.656 10bit YCbCr422 or BT.601 10bit YCbCr422 */
+		vnmc |= vin->digital->mbus_cfg.type == V4L2_MBUS_BT656 ?
+			VNMC_INF_YUV10_BT656 : VNMC_INF_YUV10_BT601;
+		input_is_yuv = true;
+		break;
+	default:
+		break;
+	}
+
+	/* Enable VSYNC Field Toogle mode after one VSYNC input */
+	dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
+
+	/* Hsync Signal Polarity Select */
+	if (!(vin->digital->mbus_cfg.flags & V4L2_MBUS_HSYNC_ACTIVE_LOW))
+		dmr2 |= VNDMR2_HPS;
+
+	/* Vsync Signal Polarity Select */
+	if (!(vin->digital->mbus_cfg.flags & V4L2_MBUS_VSYNC_ACTIVE_LOW))
+		dmr2 |= VNDMR2_VPS;
+
+	/*
+	 * Output format
+	 */
+	switch (vin->format.pixelformat) {
+	case V4L2_PIX_FMT_NV16:
+		rvin_write(vin,
+			   ALIGN(vin->format.width * vin->format.height, 0x80),
+			   VNUVAOF_REG);
+		dmr = VNDMR_DTMD_YCSEP;
+		output_is_yuv = true;
+		break;
+	case V4L2_PIX_FMT_YUYV:
+		dmr = VNDMR_BPSM;
+		output_is_yuv = true;
+		break;
+	case V4L2_PIX_FMT_UYVY:
+		dmr = 0;
+		output_is_yuv = true;
+		break;
+	case V4L2_PIX_FMT_XRGB555:
+		dmr = VNDMR_DTMD_ARGB1555;
+		break;
+	case V4L2_PIX_FMT_RGB565:
+		dmr = 0;
+		break;
+	case V4L2_PIX_FMT_XBGR32:
+		/* Note: not supported on M1 */
+		dmr = VNDMR_EXRGB;
+		break;
+	default:
+		vin_err(vin, "Invalid pixelformat (0x%x)\n",
+			vin->format.pixelformat);
+		return -EINVAL;
+	}
 
-static void rvin_set_coeff(struct rvin_dev *vin, unsigned short xs)
+	/* Always update on field change */
+	vnmc |= VNMC_VUP;
+
+	/* If input and output use the same colorspace, use bypass mode */
+	if (input_is_yuv == output_is_yuv)
+		vnmc |= VNMC_BPS;
+
+	/* Progressive or interlaced mode */
+	interrupts = progressive ? VNIE_FIE : VNIE_EFE;
+
+	/* Ack interrupts */
+	rvin_write(vin, interrupts, VNINTS_REG);
+	/* Enable interrupts */
+	rvin_write(vin, interrupts, VNIE_REG);
+	/* Start capturing */
+	rvin_write(vin, dmr, VNDMR_REG);
+	rvin_write(vin, dmr2, VNDMR2_REG);
+
+	/* Enable module */
+	rvin_write(vin, vnmc | VNMC_ME, VNMC_REG);
+
+	return 0;
+}
+
+static void rvin_disable_interrupts(struct rvin_dev *vin)
 {
-	int i;
-	const struct vin_coeff *p_prev_set = NULL;
-	const struct vin_coeff *p_set = NULL;
+	rvin_write(vin, 0, VNIE_REG);
+}
 
-	/* Look for suitable coefficient values */
-	for (i = 0; i < ARRAY_SIZE(vin_coeff_set); i++) {
-		p_prev_set = p_set;
-		p_set = &vin_coeff_set[i];
+static u32 rvin_get_interrupt_status(struct rvin_dev *vin)
+{
+	return rvin_read(vin, VNINTS_REG);
+}
 
-		if (xs < p_set->xs_value)
-			break;
+static void rvin_ack_interrupt(struct rvin_dev *vin)
+{
+	rvin_write(vin, rvin_read(vin, VNINTS_REG), VNINTS_REG);
+}
+
+static bool rvin_capture_active(struct rvin_dev *vin)
+{
+	return rvin_read(vin, VNMS_REG) & VNMS_CA;
+}
+
+static int rvin_get_active_slot(struct rvin_dev *vin, u32 vnms)
+{
+	if (vin->continuous)
+		return (vnms & VNMS_FBS_MASK) >> VNMS_FBS_SHIFT;
+
+	return 0;
+}
+
+static enum v4l2_field rvin_get_active_field(struct rvin_dev *vin, u32 vnms)
+{
+	if (vin->format.field == V4L2_FIELD_ALTERNATE) {
+		/* If FS is set it's a Even field */
+		if (vnms & VNMS_FS)
+			return V4L2_FIELD_BOTTOM;
+		return V4L2_FIELD_TOP;
 	}
 
-	/* Use previous value if its XS value is closer */
-	if (p_prev_set && p_set &&
-	    xs - p_prev_set->xs_value < p_set->xs_value - xs)
-		p_set = p_prev_set;
+	return vin->format.field;
+}
 
-	/* Set coefficient registers */
-	rvin_write(vin, p_set->coeff_set[0], VNC1A_REG);
-	rvin_write(vin, p_set->coeff_set[1], VNC1B_REG);
-	rvin_write(vin, p_set->coeff_set[2], VNC1C_REG);
+static void rvin_set_slot_addr(struct rvin_dev *vin, int slot, dma_addr_t addr)
+{
+	const struct rvin_video_format *fmt;
+	int offsetx, offsety;
+	dma_addr_t offset;
 
-	rvin_write(vin, p_set->coeff_set[3], VNC2A_REG);
-	rvin_write(vin, p_set->coeff_set[4], VNC2B_REG);
-	rvin_write(vin, p_set->coeff_set[5], VNC2C_REG);
+	fmt = rvin_format_from_pixel(vin->format.pixelformat);
 
-	rvin_write(vin, p_set->coeff_set[6], VNC3A_REG);
-	rvin_write(vin, p_set->coeff_set[7], VNC3B_REG);
-	rvin_write(vin, p_set->coeff_set[8], VNC3C_REG);
+	/*
+	 * There is no HW support for composition do the beast we can
+	 * by modifying the buffer offset
+	 */
+	offsetx = vin->compose.left * fmt->bpp;
+	offsety = vin->compose.top * vin->format.bytesperline;
+	offset = addr + offsetx + offsety;
 
-	rvin_write(vin, p_set->coeff_set[9], VNC4A_REG);
-	rvin_write(vin, p_set->coeff_set[10], VNC4B_REG);
-	rvin_write(vin, p_set->coeff_set[11], VNC4C_REG);
+	/*
+	 * The address needs to be 128 bytes aligned. Driver should never accept
+	 * settings that do not satisfy this in the first place...
+	 */
+	if (WARN_ON((offsetx | offsety | offset) & HW_BUFFER_MASK))
+		return;
 
-	rvin_write(vin, p_set->coeff_set[12], VNC5A_REG);
-	rvin_write(vin, p_set->coeff_set[13], VNC5B_REG);
-	rvin_write(vin, p_set->coeff_set[14], VNC5C_REG);
+	rvin_write(vin, offset, VNMB_REG(slot));
+}
 
-	rvin_write(vin, p_set->coeff_set[15], VNC6A_REG);
-	rvin_write(vin, p_set->coeff_set[16], VNC6B_REG);
-	rvin_write(vin, p_set->coeff_set[17], VNC6C_REG);
+/* Moves a buffer from the queue to the HW slots */
+static bool rvin_fill_hw_slot(struct rvin_dev *vin, int slot)
+{
+	struct rvin_buffer *buf;
+	struct vb2_v4l2_buffer *vbuf;
+	dma_addr_t phys_addr_top;
 
-	rvin_write(vin, p_set->coeff_set[18], VNC7A_REG);
-	rvin_write(vin, p_set->coeff_set[19], VNC7B_REG);
-	rvin_write(vin, p_set->coeff_set[20], VNC7C_REG);
+	if (vin->queue_buf[slot] != NULL)
+		return true;
 
-	rvin_write(vin, p_set->coeff_set[21], VNC8A_REG);
-	rvin_write(vin, p_set->coeff_set[22], VNC8B_REG);
-	rvin_write(vin, p_set->coeff_set[23], VNC8C_REG);
+	if (list_empty(&vin->buf_list))
+		return false;
+
+	vin_dbg(vin, "Filling HW slot: %d\n", slot);
+
+	/* Keep track of buffer we give to HW */
+	buf = list_entry(vin->buf_list.next, struct rvin_buffer, list);
+	vbuf = &buf->vb;
+	list_del_init(to_buf_list(vbuf));
+	vin->queue_buf[slot] = vbuf;
+
+	/* Setup DMA */
+	phys_addr_top = vb2_dma_contig_plane_dma_addr(&vbuf->vb2_buf, 0);
+	rvin_set_slot_addr(vin, slot, phys_addr_top);
+
+	return true;
 }
 
-void rvin_crop_scale_comp(struct rvin_dev *vin)
+static bool rvin_fill_hw(struct rvin_dev *vin)
 {
-	u32 xs, ys;
+	int slot, limit;
 
-	/* Set Start/End Pixel/Line Pre-Clip */
-	rvin_write(vin, vin->crop.left, VNSPPRC_REG);
-	rvin_write(vin, vin->crop.left + vin->crop.width - 1, VNEPPRC_REG);
-	switch (vin->format.field) {
-	case V4L2_FIELD_INTERLACED:
-	case V4L2_FIELD_INTERLACED_TB:
-	case V4L2_FIELD_INTERLACED_BT:
-		rvin_write(vin, vin->crop.top / 2, VNSLPRC_REG);
-		rvin_write(vin, (vin->crop.top + vin->crop.height) / 2 - 1,
-			   VNELPRC_REG);
-		break;
-	default:
-		rvin_write(vin, vin->crop.top, VNSLPRC_REG);
-		rvin_write(vin, vin->crop.top + vin->crop.height - 1,
-			   VNELPRC_REG);
-		break;
-	}
+	limit = vin->continuous ? HW_BUFFER_NUM : 1;
 
-	/* Set scaling coefficient */
-	ys = 0;
-	if (vin->crop.height != vin->compose.height)
-		ys = (4096 * vin->crop.height) / vin->compose.height;
-	rvin_write(vin, ys, VNYS_REG);
+	for (slot = 0; slot < limit; slot++)
+		if (!rvin_fill_hw_slot(vin, slot))
+			return false;
+	return true;
+}
 
-	xs = 0;
-	if (vin->crop.width != vin->compose.width)
-		xs = (4096 * vin->crop.width) / vin->compose.width;
+static void rvin_capture_on(struct rvin_dev *vin)
+{
+	vin_dbg(vin, "Capture on in %s mode\n",
+		vin->continuous ? "continuous" : "single");
 
-	/* Horizontal upscaling is up to double size */
-	if (xs > 0 && xs < 2048)
-		xs = 2048;
+	if (vin->continuous)
+		/* Continuous Frame Capture Mode */
+		rvin_write(vin, VNFC_C_FRAME, VNFC_REG);
+	else
+		/* Single Frame Capture Mode */
+		rvin_write(vin, VNFC_S_FRAME, VNFC_REG);
+}
 
-	rvin_write(vin, xs, VNXS_REG);
+static int rvin_capture_start(struct rvin_dev *vin)
+{
+	struct rvin_buffer *buf, *node;
+	int bufs, ret;
 
-	/* Horizontal upscaling is done out by scaling down from double size */
-	if (xs < 4096)
-		xs *= 2;
+	/* Count number of free buffers */
+	bufs = 0;
+	list_for_each_entry_safe(buf, node, &vin->buf_list, list)
+		bufs++;
 
-	rvin_set_coeff(vin, xs);
+	/* Continuous capture requires more buffers then there are HW slots */
+	vin->continuous = bufs > HW_BUFFER_NUM;
 
-	/* Set Start/End Pixel/Line Post-Clip */
-	rvin_write(vin, 0, VNSPPOC_REG);
-	rvin_write(vin, 0, VNSLPOC_REG);
-	rvin_write(vin, vin->format.width - 1, VNEPPOC_REG);
-	switch (vin->format.field) {
-	case V4L2_FIELD_INTERLACED:
-	case V4L2_FIELD_INTERLACED_TB:
-	case V4L2_FIELD_INTERLACED_BT:
-		rvin_write(vin, vin->format.height / 2 - 1, VNELPOC_REG);
-		break;
-	default:
-		rvin_write(vin, vin->format.height - 1, VNELPOC_REG);
-		break;
+	if (!rvin_fill_hw(vin)) {
+		vin_err(vin, "HW not ready to start, not enough buffers available\n");
+		return -EINVAL;
 	}
 
-	if (vin->format.pixelformat == V4L2_PIX_FMT_NV16)
-		rvin_write(vin, ALIGN(vin->format.width, 0x20), VNIS_REG);
-	else
-		rvin_write(vin, ALIGN(vin->format.width, 0x10), VNIS_REG);
+	rvin_crop_scale_comp(vin);
 
-	vin_dbg(vin,
-		"Pre-Clip: %ux%u@%u:%u YS: %d XS: %d Post-Clip: %ux%u@%u:%u\n",
-		vin->crop.width, vin->crop.height, vin->crop.left,
-		vin->crop.top, ys, xs, vin->format.width, vin->format.height,
-		0, 0);
+	ret = rvin_setup(vin);
+	if (ret)
+		return ret;
+
+	rvin_capture_on(vin);
+
+	vin->state = RUNNING;
+
+	return 0;
 }
 
-void rvin_scale_try(struct rvin_dev *vin, struct v4l2_pix_format *pix,
-		    u32 width, u32 height)
+static void rvin_capture_stop(struct rvin_dev *vin)
 {
-	/* All VIN channels on Gen2 have scalers */
-	pix->width = width;
-	pix->height = height;
+	/* Set continuous & single transfer off */
+	rvin_write(vin, 0, VNFC_REG);
+
+	/* Disable module */
+	rvin_write(vin, rvin_read(vin, VNMC_REG) & ~VNMC_ME, VNMC_REG);
 }
 
 /* -----------------------------------------------------------------------------
-- 
2.15.0

[PATCH v9 09/28] rcar-vin: all Gen2 boards can scale simplify logic

[Niklas Söderlund]

The logic to preserve the requested format width and height are too
complex and come from a premature optimization for Gen3. All Gen2 SoC
can scale and the Gen3 implementation will not use these functions at
all so simply preserve the width and height when interacting with the
subdevice much like the field is preserved simplifies the logic quite a
bit.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-dma.c  |  8 --------
 drivers/media/platform/rcar-vin/rcar-v4l2.c | 22 ++++++++++------------
 drivers/media/platform/rcar-vin/rcar-vin.h  |  2 --
 3 files changed, 10 insertions(+), 22 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index a7cda3922cb74baa..fd14be20a6604d7a 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -585,14 +585,6 @@ void rvin_crop_scale_comp(struct rvin_dev *vin)
 		0, 0);
 }
 
-void rvin_scale_try(struct rvin_dev *vin, struct v4l2_pix_format *pix,
-		    u32 width, u32 height)
-{
-	/* All VIN channels on Gen2 have scalers */
-	pix->width = width;
-	pix->height = height;
-}
-
 /* -----------------------------------------------------------------------------
  * Hardware setup
  */
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index 19de99133f048960..1c5e7f6d5b963740 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -166,6 +166,7 @@ static int __rvin_try_format_source(struct rvin_dev *vin,
 		.which = which,
 	};
 	enum v4l2_field field;
+	u32 width, height;
 	int ret;
 
 	sd = vin_to_source(vin);
@@ -178,7 +179,10 @@ static int __rvin_try_format_source(struct rvin_dev *vin,
 
 	format.pad = vin->digital->source_pad;
 
+	/* Allow the video device to override field and to scale */
 	field = pix->field;
+	width = pix->width;
+	height = pix->height;
 
 	ret = v4l2_subdev_call(sd, pad, set_fmt, pad_cfg, &format);
 	if (ret < 0 && ret != -ENOIOCTLCMD)
@@ -191,6 +195,9 @@ static int __rvin_try_format_source(struct rvin_dev *vin,
 	source->width = pix->width;
 	source->height = pix->height;
 
+	pix->width = width;
+	pix->height = height;
+
 	vin_dbg(vin, "Source resolution: %ux%u\n", source->width,
 		source->height);
 
@@ -204,13 +211,9 @@ static int __rvin_try_format(struct rvin_dev *vin,
 			     struct v4l2_pix_format *pix,
 			     struct rvin_source_fmt *source)
 {
-	u32 rwidth, rheight, walign;
+	u32 walign;
 	int ret;
 
-	/* Requested */
-	rwidth = pix->width;
-	rheight = pix->height;
-
 	/* Keep current field if no specific one is asked for */
 	if (pix->field == V4L2_FIELD_ANY)
 		pix->field = vin->format.field;
@@ -248,10 +251,6 @@ static int __rvin_try_format(struct rvin_dev *vin,
 		break;
 	}
 
-	/* If source can't match format try if VIN can scale */
-	if (source->width != rwidth || source->height != rheight)
-		rvin_scale_try(vin, pix, rwidth, rheight);
-
 	/* HW limit width to a multiple of 32 (2^5) for NV16 else 2 (2^1) */
 	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
 
@@ -270,9 +269,8 @@ static int __rvin_try_format(struct rvin_dev *vin,
 		return -EINVAL;
 	}
 
-	vin_dbg(vin, "Requested %ux%u Got %ux%u bpl: %d size: %d\n",
-		rwidth, rheight, pix->width, pix->height,
-		pix->bytesperline, pix->sizeimage);
+	vin_dbg(vin, "Format %ux%u bpl: %d size: %d\n",
+		pix->width, pix->height, pix->bytesperline, pix->sizeimage);
 
 	return 0;
 }
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 646f897f5c05ec4e..36d0f0cc4ce01a6e 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -176,8 +176,6 @@ void rvin_v4l2_unregister(struct rvin_dev *vin);
 const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
 
 /* Cropping, composing and scaling */
-void rvin_scale_try(struct rvin_dev *vin, struct v4l2_pix_format *pix,
-		    u32 width, u32 height);
 void rvin_crop_scale_comp(struct rvin_dev *vin);
 
 #endif
-- 
2.15.0

[PATCH v9 10/28] rcar-vin: do not reset crop and compose when setting format

[Niklas Söderlund]

It was a bad idea to reset the crop and compose settings when a new
format is set. This would overwrite any crop/compose set by s_select and
cause unexpected behaviors, remove it. Also fold the reset helper in to
the only remaining caller.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-v4l2.c | 21 +++++++--------------
 1 file changed, 7 insertions(+), 14 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index 1c5e7f6d5b963740..254fa1c8770275a5 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -90,17 +90,6 @@ static u32 rvin_format_sizeimage(struct v4l2_pix_format *pix)
  * V4L2
  */
 
-static void rvin_reset_crop_compose(struct rvin_dev *vin)
-{
-	vin->crop.top = vin->crop.left = 0;
-	vin->crop.width = vin->source.width;
-	vin->crop.height = vin->source.height;
-
-	vin->compose.top = vin->compose.left = 0;
-	vin->compose.width = vin->format.width;
-	vin->compose.height = vin->format.height;
-}
-
 static int rvin_reset_format(struct rvin_dev *vin)
 {
 	struct v4l2_subdev_format fmt = {
@@ -147,7 +136,13 @@ static int rvin_reset_format(struct rvin_dev *vin)
 		break;
 	}
 
-	rvin_reset_crop_compose(vin);
+	vin->crop.top = vin->crop.left = 0;
+	vin->crop.width = mf->width;
+	vin->crop.height = mf->height;
+
+	vin->compose.top = vin->compose.left = 0;
+	vin->compose.width = mf->width;
+	vin->compose.height = mf->height;
 
 	vin->format.bytesperline = rvin_format_bytesperline(&vin->format);
 	vin->format.sizeimage = rvin_format_sizeimage(&vin->format);
@@ -317,8 +312,6 @@ static int rvin_s_fmt_vid_cap(struct file *file, void *priv,
 
 	vin->format = f->fmt.pix;
 
-	rvin_reset_crop_compose(vin);
-
 	return 0;
 }
 
-- 
2.15.0

[PATCH v9 11/28] rcar-vin: do not allow changing scaling and composing while streaming

[Niklas Söderlund]

It is possible on Gen2 to change the registers controlling composing and
scaling while the stream is running. It is however not a good idea to do
so and could result in trouble. There are also no good reasons to allow
this, remove immediate reflection in hardware registers from
vidioc_s_selection and only configure scaling and composing when the
stream starts.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-dma.c  | 2 +-
 drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ---
 drivers/media/platform/rcar-vin/rcar-vin.h  | 3 ---
 3 files changed, 1 insertion(+), 7 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index fd14be20a6604d7a..7be5080f742825fb 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -514,7 +514,7 @@ static void rvin_set_coeff(struct rvin_dev *vin, unsigned short xs)
 	rvin_write(vin, p_set->coeff_set[23], VNC8C_REG);
 }
 
-void rvin_crop_scale_comp(struct rvin_dev *vin)
+static void rvin_crop_scale_comp(struct rvin_dev *vin)
 {
 	u32 xs, ys;
 
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index 254fa1c8770275a5..d6298c684ab2d731 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -436,9 +436,6 @@ static int rvin_s_selection(struct file *file, void *fh,
 		return -EINVAL;
 	}
 
-	/* HW supports modifying configuration while running */
-	rvin_crop_scale_comp(vin);
-
 	return 0;
 }
 
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 36d0f0cc4ce01a6e..67541b483ee43c52 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -175,7 +175,4 @@ void rvin_v4l2_unregister(struct rvin_dev *vin);
 
 const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
 
-/* Cropping, composing and scaling */
-void rvin_crop_scale_comp(struct rvin_dev *vin);
-
 #endif
-- 
2.15.0

[PATCH v9 12/28] rcar-vin: read subdevice format for crop only when needed

[Niklas Söderlund]

Instead of caching the subdevice format each time the video device
format is set read it directly when it's needed. As it turns out the
format is only needed when figuring out the max rectangle for cropping.

This simplifies the code and makes it clearer what the source format is
used for.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-v4l2.c | 88 ++++++++++++++---------------
 drivers/media/platform/rcar-vin/rcar-vin.h  | 12 ----
 2 files changed, 42 insertions(+), 58 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index d6298c684ab2d731..9cf9ff48ac1e2f4f 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -90,24 +90,30 @@ static u32 rvin_format_sizeimage(struct v4l2_pix_format *pix)
  * V4L2
  */
 
-static int rvin_reset_format(struct rvin_dev *vin)
+static int rvin_get_sd_format(struct rvin_dev *vin, struct v4l2_pix_format *pix)
 {
 	struct v4l2_subdev_format fmt = {
 		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
+		.pad = vin->digital->source_pad,
 	};
-	struct v4l2_mbus_framefmt *mf = &fmt.format;
 	int ret;
 
-	fmt.pad = vin->digital->source_pad;
-
 	ret = v4l2_subdev_call(vin_to_source(vin), pad, get_fmt, NULL, &fmt);
 	if (ret)
 		return ret;
 
-	vin->format.width	= mf->width;
-	vin->format.height	= mf->height;
-	vin->format.colorspace	= mf->colorspace;
-	vin->format.field	= mf->field;
+	v4l2_fill_pix_format(pix, &fmt.format);
+
+	return 0;
+}
+
+static int rvin_reset_format(struct rvin_dev *vin)
+{
+	int ret;
+
+	ret = rvin_get_sd_format(vin, &vin->format);
+	if (ret)
+		return ret;
 
 	/*
 	 * If the subdevice uses ALTERNATE field mode and G_STD is
@@ -137,12 +143,12 @@ static int rvin_reset_format(struct rvin_dev *vin)
 	}
 
 	vin->crop.top = vin->crop.left = 0;
-	vin->crop.width = mf->width;
-	vin->crop.height = mf->height;
+	vin->crop.width = vin->format.width;
+	vin->crop.height = vin->format.height;
 
 	vin->compose.top = vin->compose.left = 0;
-	vin->compose.width = mf->width;
-	vin->compose.height = mf->height;
+	vin->compose.width = vin->format.width;
+	vin->compose.height = vin->format.height;
 
 	vin->format.bytesperline = rvin_format_bytesperline(&vin->format);
 	vin->format.sizeimage = rvin_format_sizeimage(&vin->format);
@@ -151,9 +157,7 @@ static int rvin_reset_format(struct rvin_dev *vin)
 }
 
 static int __rvin_try_format_source(struct rvin_dev *vin,
-				    u32 which,
-				    struct v4l2_pix_format *pix,
-				    struct rvin_source_fmt *source)
+				    u32 which, struct v4l2_pix_format *pix)
 {
 	struct v4l2_subdev *sd;
 	struct v4l2_subdev_pad_config *pad_cfg;
@@ -186,25 +190,15 @@ static int __rvin_try_format_source(struct rvin_dev *vin,
 	v4l2_fill_pix_format(pix, &format.format);
 
 	pix->field = field;
-
-	source->width = pix->width;
-	source->height = pix->height;
-
 	pix->width = width;
 	pix->height = height;
-
-	vin_dbg(vin, "Source resolution: %ux%u\n", source->width,
-		source->height);
-
 done:
 	v4l2_subdev_free_pad_config(pad_cfg);
 	return ret;
 }
 
 static int __rvin_try_format(struct rvin_dev *vin,
-			     u32 which,
-			     struct v4l2_pix_format *pix,
-			     struct rvin_source_fmt *source)
+			     u32 which, struct v4l2_pix_format *pix)
 {
 	u32 walign;
 	int ret;
@@ -225,7 +219,7 @@ static int __rvin_try_format(struct rvin_dev *vin,
 	pix->sizeimage = 0;
 
 	/* Limit to source capabilities */
-	ret = __rvin_try_format_source(vin, which, pix, source);
+	ret = __rvin_try_format_source(vin, which, pix);
 	if (ret)
 		return ret;
 
@@ -234,7 +228,6 @@ static int __rvin_try_format(struct rvin_dev *vin,
 	case V4L2_FIELD_BOTTOM:
 	case V4L2_FIELD_ALTERNATE:
 		pix->height /= 2;
-		source->height /= 2;
 		break;
 	case V4L2_FIELD_NONE:
 	case V4L2_FIELD_INTERLACED_TB:
@@ -286,30 +279,23 @@ static int rvin_try_fmt_vid_cap(struct file *file, void *priv,
 				struct v4l2_format *f)
 {
 	struct rvin_dev *vin = video_drvdata(file);
-	struct rvin_source_fmt source;
 
-	return __rvin_try_format(vin, V4L2_SUBDEV_FORMAT_TRY, &f->fmt.pix,
-				 &source);
+	return __rvin_try_format(vin, V4L2_SUBDEV_FORMAT_TRY, &f->fmt.pix);
 }
 
 static int rvin_s_fmt_vid_cap(struct file *file, void *priv,
 			      struct v4l2_format *f)
 {
 	struct rvin_dev *vin = video_drvdata(file);
-	struct rvin_source_fmt source;
 	int ret;
 
 	if (vb2_is_busy(&vin->queue))
 		return -EBUSY;
 
-	ret = __rvin_try_format(vin, V4L2_SUBDEV_FORMAT_ACTIVE, &f->fmt.pix,
-				&source);
+	ret = __rvin_try_format(vin, V4L2_SUBDEV_FORMAT_ACTIVE, &f->fmt.pix);
 	if (ret)
 		return ret;
 
-	vin->source.width = source.width;
-	vin->source.height = source.height;
-
 	vin->format = f->fmt.pix;
 
 	return 0;
@@ -340,6 +326,8 @@ static int rvin_g_selection(struct file *file, void *fh,
 			    struct v4l2_selection *s)
 {
 	struct rvin_dev *vin = video_drvdata(file);
+	struct v4l2_pix_format pix;
+	int ret;
 
 	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
 		return -EINVAL;
@@ -347,9 +335,12 @@ static int rvin_g_selection(struct file *file, void *fh,
 	switch (s->target) {
 	case V4L2_SEL_TGT_CROP_BOUNDS:
 	case V4L2_SEL_TGT_CROP_DEFAULT:
+		ret = rvin_get_sd_format(vin, &pix);
+		if (ret)
+			return ret;
 		s->r.left = s->r.top = 0;
-		s->r.width = vin->source.width;
-		s->r.height = vin->source.height;
+		s->r.width = pix.width;
+		s->r.height = pix.height;
 		break;
 	case V4L2_SEL_TGT_CROP:
 		s->r = vin->crop;
@@ -375,12 +366,14 @@ static int rvin_s_selection(struct file *file, void *fh,
 {
 	struct rvin_dev *vin = video_drvdata(file);
 	const struct rvin_video_format *fmt;
+	struct v4l2_pix_format pix;
 	struct v4l2_rect r = s->r;
 	struct v4l2_rect max_rect;
 	struct v4l2_rect min_rect = {
 		.width = 6,
 		.height = 2,
 	};
+	int ret;
 
 	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
 		return -EINVAL;
@@ -390,22 +383,25 @@ static int rvin_s_selection(struct file *file, void *fh,
 	switch (s->target) {
 	case V4L2_SEL_TGT_CROP:
 		/* Can't crop outside of source input */
+		ret = rvin_get_sd_format(vin, &pix);
+		if (ret)
+			return ret;
 		max_rect.top = max_rect.left = 0;
-		max_rect.width = vin->source.width;
-		max_rect.height = vin->source.height;
+		max_rect.width = pix.width;
+		max_rect.height = pix.height;
 		v4l2_rect_map_inside(&r, &max_rect);
 
-		v4l_bound_align_image(&r.width, 2, vin->source.width, 1,
-				      &r.height, 4, vin->source.height, 2, 0);
+		v4l_bound_align_image(&r.width, 2, pix.width, 1,
+				      &r.height, 4, pix.height, 2, 0);
 
-		r.top  = clamp_t(s32, r.top, 0, vin->source.height - r.height);
-		r.left = clamp_t(s32, r.left, 0, vin->source.width - r.width);
+		r.top  = clamp_t(s32, r.top, 0, pix.height - r.height);
+		r.left = clamp_t(s32, r.left, 0, pix.width - r.width);
 
 		vin->crop = s->r = r;
 
 		vin_dbg(vin, "Cropped %dx%d@%d:%d of %dx%d\n",
 			r.width, r.height, r.left, r.top,
-			vin->source.width, vin->source.height);
+			pix.width, pix.height);
 		break;
 	case V4L2_SEL_TGT_COMPOSE:
 		/* Make sure compose rect fits inside output format */
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 67541b483ee43c52..f8e0e7cedeaa6c38 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -48,16 +48,6 @@ enum rvin_dma_state {
 	STOPPING,
 };
 
-/**
- * struct rvin_source_fmt - Source information
- * @width:	Width from source
- * @height:	Height from source
- */
-struct rvin_source_fmt {
-	u32 width;
-	u32 height;
-};
-
 /**
  * struct rvin_video_format - Data format stored in memory
  * @fourcc:	Pixelformat
@@ -125,7 +115,6 @@ struct rvin_info {
  * @sequence:		V4L2 buffers sequence number
  * @state:		keeps track of operation state
  *
- * @source:		active format from the video source
  * @format:		active V4L2 pixel format
  *
  * @crop:		active cropping
@@ -152,7 +141,6 @@ struct rvin_dev {
 	unsigned int sequence;
 	enum rvin_dma_state state;
 
-	struct rvin_source_fmt source;
 	struct v4l2_pix_format format;
 
 	struct v4l2_rect crop;
-- 
2.15.0

[PATCH v9 13/28] rcar-vin: fix handling of single field frames (top, bottom and alternate fields)

[Niklas Söderlund]

There was never proper support in the VIN driver to deliver ALTERNATING
field format to user-space, remove this field option. For sources using
this field format instead use the VIN hardware feature of combining the
fields to an interlaced format. This mode of operation was previously
the default behavior and ALTERNATING was only delivered to user-space if
explicitly requested. Allowing this to be explicitly requested was a
mistake and was never properly tested and never worked due to the
constraints put on the field format when it comes to sequence numbers and
timestamps etc.

The height should not be cut in half for the format for TOP or BOTTOM
fields settings. This was a mistake and it was made visible by the
scaling refactoring. Correct behavior is that the user should request a
frame size that fits the half height frame reflected in the field
setting. If not the VIN will do its best to scale the top or bottom to
the requested format and cropping and scaling do not work as expected.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-dma.c  | 15 +--------
 drivers/media/platform/rcar-vin/rcar-v4l2.c | 48 +++++++++++------------------
 2 files changed, 19 insertions(+), 44 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index 7be5080f742825fb..e6478088d9464221 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -617,7 +617,6 @@ static int rvin_setup(struct rvin_dev *vin)
 	case V4L2_FIELD_INTERLACED_BT:
 		vnmc = VNMC_IM_FULL | VNMC_FOC;
 		break;
-	case V4L2_FIELD_ALTERNATE:
 	case V4L2_FIELD_NONE:
 		if (vin->continuous) {
 			vnmc = VNMC_IM_ODD_EVEN;
@@ -757,18 +756,6 @@ static int rvin_get_active_slot(struct rvin_dev *vin, u32 vnms)
 	return 0;
 }
 
-static enum v4l2_field rvin_get_active_field(struct rvin_dev *vin, u32 vnms)
-{
-	if (vin->format.field == V4L2_FIELD_ALTERNATE) {
-		/* If FS is set it's a Even field */
-		if (vnms & VNMS_FS)
-			return V4L2_FIELD_BOTTOM;
-		return V4L2_FIELD_TOP;
-	}
-
-	return vin->format.field;
-}
-
 static void rvin_set_slot_addr(struct rvin_dev *vin, int slot, dma_addr_t addr)
 {
 	const struct rvin_video_format *fmt;
@@ -941,7 +928,7 @@ static irqreturn_t rvin_irq(int irq, void *data)
 		goto done;
 
 	/* Capture frame */
-	vin->queue_buf[slot]->field = rvin_get_active_field(vin, vnms);
+	vin->queue_buf[slot]->field = vin->format.field;
 	vin->queue_buf[slot]->sequence = sequence;
 	vin->queue_buf[slot]->vb2_buf.timestamp = ktime_get_ns();
 	vb2_buffer_done(&vin->queue_buf[slot]->vb2_buf, VB2_BUF_STATE_DONE);
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index 9cf9ff48ac1e2f4f..37fe1f6c646b0ea3 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -102,6 +102,24 @@ static int rvin_get_sd_format(struct rvin_dev *vin, struct v4l2_pix_format *pix)
 	if (ret)
 		return ret;
 
+	switch (fmt.format.field) {
+	case V4L2_FIELD_TOP:
+	case V4L2_FIELD_BOTTOM:
+	case V4L2_FIELD_NONE:
+	case V4L2_FIELD_INTERLACED_TB:
+	case V4L2_FIELD_INTERLACED_BT:
+	case V4L2_FIELD_INTERLACED:
+		break;
+	case V4L2_FIELD_ALTERNATE:
+		/* Use VIN hardware to combine the two fields */
+		fmt.format.field = V4L2_FIELD_INTERLACED;
+		fmt.format.height *= 2;
+		break;
+	default:
+		vin->format.field = V4L2_FIELD_NONE;
+		break;
+	}
+
 	v4l2_fill_pix_format(pix, &fmt.format);
 
 	return 0;
@@ -115,33 +133,6 @@ static int rvin_reset_format(struct rvin_dev *vin)
 	if (ret)
 		return ret;
 
-	/*
-	 * If the subdevice uses ALTERNATE field mode and G_STD is
-	 * implemented use the VIN HW to combine the two fields to
-	 * one INTERLACED frame. The ALTERNATE field mode can still
-	 * be requested in S_FMT and be respected, this is just the
-	 * default which is applied at probing or when S_STD is called.
-	 */
-	if (vin->format.field == V4L2_FIELD_ALTERNATE &&
-	    v4l2_subdev_has_op(vin_to_source(vin), video, g_std))
-		vin->format.field = V4L2_FIELD_INTERLACED;
-
-	switch (vin->format.field) {
-	case V4L2_FIELD_TOP:
-	case V4L2_FIELD_BOTTOM:
-	case V4L2_FIELD_ALTERNATE:
-		vin->format.height /= 2;
-		break;
-	case V4L2_FIELD_NONE:
-	case V4L2_FIELD_INTERLACED_TB:
-	case V4L2_FIELD_INTERLACED_BT:
-	case V4L2_FIELD_INTERLACED:
-		break;
-	default:
-		vin->format.field = V4L2_FIELD_NONE;
-		break;
-	}
-
 	vin->crop.top = vin->crop.left = 0;
 	vin->crop.width = vin->format.width;
 	vin->crop.height = vin->format.height;
@@ -226,9 +217,6 @@ static int __rvin_try_format(struct rvin_dev *vin,
 	switch (pix->field) {
 	case V4L2_FIELD_TOP:
 	case V4L2_FIELD_BOTTOM:
-	case V4L2_FIELD_ALTERNATE:
-		pix->height /= 2;
-		break;
 	case V4L2_FIELD_NONE:
 	case V4L2_FIELD_INTERLACED_TB:
 	case V4L2_FIELD_INTERLACED_BT:
-- 
2.15.0

[PATCH v9 14/28] rcar-vin: move media bus configuration to struct rvin_info

[Niklas Söderlund]

Bus configuration will once the driver is extended to support Gen3
contain information not specific to only the directly connected parallel
subdevice. Move it to struct rvin_info to show it's not always coupled
to the parallel subdevice.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-core.c | 18 +++++++++---------
 drivers/media/platform/rcar-vin/rcar-dma.c  | 11 ++++++-----
 drivers/media/platform/rcar-vin/rcar-v4l2.c |  2 +-
 drivers/media/platform/rcar-vin/rcar-vin.h  |  9 ++++-----
 4 files changed, 20 insertions(+), 20 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index 03d3cd63e38bee11..7d49904cab9cb2d9 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -101,10 +101,10 @@ static int rvin_digital_notify_bound(struct v4l2_async_notifier *notifier,
 	vin->digital->sink_pad = ret < 0 ? 0 : ret;
 
 	/* Find compatible subdevices mbus format */
-	vin->digital->code = 0;
+	vin->code = 0;
 	code.index = 0;
 	code.pad = vin->digital->source_pad;
-	while (!vin->digital->code &&
+	while (!vin->code &&
 	       !v4l2_subdev_call(subdev, pad, enum_mbus_code, NULL, &code)) {
 		code.index++;
 		switch (code.code) {
@@ -112,16 +112,16 @@ static int rvin_digital_notify_bound(struct v4l2_async_notifier *notifier,
 		case MEDIA_BUS_FMT_UYVY8_2X8:
 		case MEDIA_BUS_FMT_UYVY10_2X10:
 		case MEDIA_BUS_FMT_RGB888_1X24:
-			vin->digital->code = code.code;
+			vin->code = code.code;
 			vin_dbg(vin, "Found media bus format for %s: %d\n",
-				subdev->name, vin->digital->code);
+				subdev->name, vin->code);
 			break;
 		default:
 			break;
 		}
 	}
 
-	if (!vin->digital->code) {
+	if (!vin->code) {
 		vin_err(vin, "Unsupported media bus format for %s\n",
 			subdev->name);
 		return -EINVAL;
@@ -179,16 +179,16 @@ static int rvin_digital_parse_v4l2(struct device *dev,
 	if (vep->base.port || vep->base.id)
 		return -ENOTCONN;
 
-	rvge->mbus_cfg.type = vep->bus_type;
+	vin->mbus_cfg.type = vep->bus_type;
 
-	switch (rvge->mbus_cfg.type) {
+	switch (vin->mbus_cfg.type) {
 	case V4L2_MBUS_PARALLEL:
 		vin_dbg(vin, "Found PARALLEL media bus\n");
-		rvge->mbus_cfg.flags = vep->bus.parallel.flags;
+		vin->mbus_cfg.flags = vep->bus.parallel.flags;
 		break;
 	case V4L2_MBUS_BT656:
 		vin_dbg(vin, "Found BT656 media bus\n");
-		rvge->mbus_cfg.flags = 0;
+		vin->mbus_cfg.flags = 0;
 		break;
 	default:
 		vin_err(vin, "Unknown media bus type\n");
diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index e6478088d9464221..d7660f485a2df9e4 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -633,7 +633,7 @@ static int rvin_setup(struct rvin_dev *vin)
 	/*
 	 * Input interface
 	 */
-	switch (vin->digital->code) {
+	switch (vin->code) {
 	case MEDIA_BUS_FMT_YUYV8_1X16:
 		/* BT.601/BT.1358 16bit YCbCr422 */
 		vnmc |= VNMC_INF_YUV16;
@@ -641,7 +641,7 @@ static int rvin_setup(struct rvin_dev *vin)
 		break;
 	case MEDIA_BUS_FMT_UYVY8_2X8:
 		/* BT.656 8bit YCbCr422 or BT.601 8bit YCbCr422 */
-		vnmc |= vin->digital->mbus_cfg.type == V4L2_MBUS_BT656 ?
+		vnmc |= vin->mbus_cfg.type == V4L2_MBUS_BT656 ?
 			VNMC_INF_YUV8_BT656 : VNMC_INF_YUV8_BT601;
 		input_is_yuv = true;
 		break;
@@ -650,7 +650,7 @@ static int rvin_setup(struct rvin_dev *vin)
 		break;
 	case MEDIA_BUS_FMT_UYVY10_2X10:
 		/* BT.656 10bit YCbCr422 or BT.601 10bit YCbCr422 */
-		vnmc |= vin->digital->mbus_cfg.type == V4L2_MBUS_BT656 ?
+		vnmc |= vin->mbus_cfg.type == V4L2_MBUS_BT656 ?
 			VNMC_INF_YUV10_BT656 : VNMC_INF_YUV10_BT601;
 		input_is_yuv = true;
 		break;
@@ -662,11 +662,11 @@ static int rvin_setup(struct rvin_dev *vin)
 	dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
 
 	/* Hsync Signal Polarity Select */
-	if (!(vin->digital->mbus_cfg.flags & V4L2_MBUS_HSYNC_ACTIVE_LOW))
+	if (!(vin->mbus_cfg.flags & V4L2_MBUS_HSYNC_ACTIVE_LOW))
 		dmr2 |= VNDMR2_HPS;
 
 	/* Vsync Signal Polarity Select */
-	if (!(vin->digital->mbus_cfg.flags & V4L2_MBUS_VSYNC_ACTIVE_LOW))
+	if (!(vin->mbus_cfg.flags & V4L2_MBUS_VSYNC_ACTIVE_LOW))
 		dmr2 |= VNDMR2_VPS;
 
 	/*
@@ -875,6 +875,7 @@ static void rvin_capture_stop(struct rvin_dev *vin)
 	rvin_write(vin, rvin_read(vin, VNMC_REG) & ~VNMC_ME, VNMC_REG);
 }
 
+
 /* -----------------------------------------------------------------------------
  * DMA Functions
  */
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index 37fe1f6c646b0ea3..56c5183f55922e1d 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -161,7 +161,7 @@ static int __rvin_try_format_source(struct rvin_dev *vin,
 
 	sd = vin_to_source(vin);
 
-	v4l2_fill_mbus_format(&format.format, pix, vin->digital->code);
+	v4l2_fill_mbus_format(&format.format, pix, vin->code);
 
 	pad_cfg = v4l2_subdev_alloc_pad_config(sd);
 	if (pad_cfg == NULL)
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index f8e0e7cedeaa6c38..118f45b656920d39 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -62,8 +62,6 @@ struct rvin_video_format {
  * struct rvin_graph_entity - Video endpoint from async framework
  * @asd:	sub-device descriptor for async framework
  * @subdev:	subdevice matched using async framework
- * @code:	Media bus format from source
- * @mbus_cfg:	Media bus format from DT
  * @source_pad:	source pad of remote subdevice
  * @sink_pad:	sink pad of remote subdevice
  */
@@ -71,9 +69,6 @@ struct rvin_graph_entity {
 	struct v4l2_async_subdev asd;
 	struct v4l2_subdev *subdev;
 
-	u32 code;
-	struct v4l2_mbus_config mbus_cfg;
-
 	unsigned int source_pad;
 	unsigned int sink_pad;
 };
@@ -115,6 +110,8 @@ struct rvin_info {
  * @sequence:		V4L2 buffers sequence number
  * @state:		keeps track of operation state
  *
+ * @mbus_cfg:		media bus format from DT
+ * @code:		media bus coide from subdevice
  * @format:		active V4L2 pixel format
  *
  * @crop:		active cropping
@@ -141,6 +138,8 @@ struct rvin_dev {
 	unsigned int sequence;
 	enum rvin_dma_state state;
 
+	struct v4l2_mbus_config mbus_cfg;
+	u32 code;
 	struct v4l2_pix_format format;
 
 	struct v4l2_rect crop;
-- 
2.15.0

[PATCH v9 15/28] rcar-vin: enable Gen3 hardware configuration

[Niklas Söderlund]

Add the register needed to work with Gen3 hardware. This patch adds
the logic for how to work with the Gen3 hardware. More work is required
to enable the subdevice structure needed to configure capturing.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-dma.c | 94 ++++++++++++++++++++----------
 drivers/media/platform/rcar-vin/rcar-vin.h |  1 +
 2 files changed, 64 insertions(+), 31 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index d7660f485a2df9e4..ace95d5b543a17e3 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -33,21 +33,23 @@
 #define VNELPRC_REG	0x10	/* Video n End Line Pre-Clip Register */
 #define VNSPPRC_REG	0x14	/* Video n Start Pixel Pre-Clip Register */
 #define VNEPPRC_REG	0x18	/* Video n End Pixel Pre-Clip Register */
-#define VNSLPOC_REG	0x1C	/* Video n Start Line Post-Clip Register */
-#define VNELPOC_REG	0x20	/* Video n End Line Post-Clip Register */
-#define VNSPPOC_REG	0x24	/* Video n Start Pixel Post-Clip Register */
-#define VNEPPOC_REG	0x28	/* Video n End Pixel Post-Clip Register */
 #define VNIS_REG	0x2C	/* Video n Image Stride Register */
 #define VNMB_REG(m)	(0x30 + ((m) << 2)) /* Video n Memory Base m Register */
 #define VNIE_REG	0x40	/* Video n Interrupt Enable Register */
 #define VNINTS_REG	0x44	/* Video n Interrupt Status Register */
 #define VNSI_REG	0x48	/* Video n Scanline Interrupt Register */
 #define VNMTC_REG	0x4C	/* Video n Memory Transfer Control Register */
-#define VNYS_REG	0x50	/* Video n Y Scale Register */
-#define VNXS_REG	0x54	/* Video n X Scale Register */
 #define VNDMR_REG	0x58	/* Video n Data Mode Register */
 #define VNDMR2_REG	0x5C	/* Video n Data Mode Register 2 */
 #define VNUVAOF_REG	0x60	/* Video n UV Address Offset Register */
+
+/* Register offsets specific for Gen2 */
+#define VNSLPOC_REG	0x1C	/* Video n Start Line Post-Clip Register */
+#define VNELPOC_REG	0x20	/* Video n End Line Post-Clip Register */
+#define VNSPPOC_REG	0x24	/* Video n Start Pixel Post-Clip Register */
+#define VNEPPOC_REG	0x28	/* Video n End Pixel Post-Clip Register */
+#define VNYS_REG	0x50	/* Video n Y Scale Register */
+#define VNXS_REG	0x54	/* Video n X Scale Register */
 #define VNC1A_REG	0x80	/* Video n Coefficient Set C1A Register */
 #define VNC1B_REG	0x84	/* Video n Coefficient Set C1B Register */
 #define VNC1C_REG	0x88	/* Video n Coefficient Set C1C Register */
@@ -73,9 +75,13 @@
 #define VNC8B_REG	0xF4	/* Video n Coefficient Set C8B Register */
 #define VNC8C_REG	0xF8	/* Video n Coefficient Set C8C Register */
 
+/* Register offsets specific for Gen3 */
+#define VNCSI_IFMD_REG		0x20 /* Video n CSI2 Interface Mode Register */
 
 /* Register bit fields for R-Car VIN */
 /* Video n Main Control Register bits */
+#define VNMC_DPINE		(1 << 27) /* Gen3 specific */
+#define VNMC_SCLE		(1 << 26) /* Gen3 specific */
 #define VNMC_FOC		(1 << 21)
 #define VNMC_YCAL		(1 << 19)
 #define VNMC_INF_YUV8_BT656	(0 << 16)
@@ -119,6 +125,13 @@
 #define VNDMR2_FTEV		(1 << 17)
 #define VNDMR2_VLV(n)		((n & 0xf) << 12)
 
+/* Video n CSI2 Interface Mode Register (Gen3) */
+#define VNCSI_IFMD_DES2		(1 << 27)
+#define VNCSI_IFMD_DES1		(1 << 26)
+#define VNCSI_IFMD_DES0		(1 << 25)
+#define VNCSI_IFMD_CSI_CHSEL(n) ((n & 0xf) << 0)
+#define VNCSI_IFMD_CSI_CHSEL_MASK 0xf
+
 struct rvin_buffer {
 	struct vb2_v4l2_buffer vb;
 	struct list_head list;
@@ -514,28 +527,10 @@ static void rvin_set_coeff(struct rvin_dev *vin, unsigned short xs)
 	rvin_write(vin, p_set->coeff_set[23], VNC8C_REG);
 }
 
-static void rvin_crop_scale_comp(struct rvin_dev *vin)
+static void rvin_crop_scale_comp_gen2(struct rvin_dev *vin)
 {
 	u32 xs, ys;
 
-	/* Set Start/End Pixel/Line Pre-Clip */
-	rvin_write(vin, vin->crop.left, VNSPPRC_REG);
-	rvin_write(vin, vin->crop.left + vin->crop.width - 1, VNEPPRC_REG);
-	switch (vin->format.field) {
-	case V4L2_FIELD_INTERLACED:
-	case V4L2_FIELD_INTERLACED_TB:
-	case V4L2_FIELD_INTERLACED_BT:
-		rvin_write(vin, vin->crop.top / 2, VNSLPRC_REG);
-		rvin_write(vin, (vin->crop.top + vin->crop.height) / 2 - 1,
-			   VNELPRC_REG);
-		break;
-	default:
-		rvin_write(vin, vin->crop.top, VNSLPRC_REG);
-		rvin_write(vin, vin->crop.top + vin->crop.height - 1,
-			   VNELPRC_REG);
-		break;
-	}
-
 	/* Set scaling coefficient */
 	ys = 0;
 	if (vin->crop.height != vin->compose.height)
@@ -573,11 +568,6 @@ static void rvin_crop_scale_comp(struct rvin_dev *vin)
 		break;
 	}
 
-	if (vin->format.pixelformat == V4L2_PIX_FMT_NV16)
-		rvin_write(vin, ALIGN(vin->format.width, 0x20), VNIS_REG);
-	else
-		rvin_write(vin, ALIGN(vin->format.width, 0x10), VNIS_REG);
-
 	vin_dbg(vin,
 		"Pre-Clip: %ux%u@%u:%u YS: %d XS: %d Post-Clip: %ux%u@%u:%u\n",
 		vin->crop.width, vin->crop.height, vin->crop.left,
@@ -585,6 +575,37 @@ static void rvin_crop_scale_comp(struct rvin_dev *vin)
 		0, 0);
 }
 
+static void rvin_crop_scale_comp(struct rvin_dev *vin)
+{
+	/* Set Start/End Pixel/Line Pre-Clip */
+	rvin_write(vin, vin->crop.left, VNSPPRC_REG);
+	rvin_write(vin, vin->crop.left + vin->crop.width - 1, VNEPPRC_REG);
+
+	switch (vin->format.field) {
+	case V4L2_FIELD_INTERLACED:
+	case V4L2_FIELD_INTERLACED_TB:
+	case V4L2_FIELD_INTERLACED_BT:
+		rvin_write(vin, vin->crop.top / 2, VNSLPRC_REG);
+		rvin_write(vin, (vin->crop.top + vin->crop.height) / 2 - 1,
+			   VNELPRC_REG);
+		break;
+	default:
+		rvin_write(vin, vin->crop.top, VNSLPRC_REG);
+		rvin_write(vin, vin->crop.top + vin->crop.height - 1,
+			   VNELPRC_REG);
+		break;
+	}
+
+	/* TODO: Add support for the UDS scaler. */
+	if (vin->info->chip != RCAR_GEN3)
+		rvin_crop_scale_comp_gen2(vin);
+
+	if (vin->format.pixelformat == V4L2_PIX_FMT_NV16)
+		rvin_write(vin, ALIGN(vin->format.width, 0x20), VNIS_REG);
+	else
+		rvin_write(vin, ALIGN(vin->format.width, 0x10), VNIS_REG);
+}
+
 /* -----------------------------------------------------------------------------
  * Hardware setup
  */
@@ -659,7 +680,10 @@ static int rvin_setup(struct rvin_dev *vin)
 	}
 
 	/* Enable VSYNC Field Toogle mode after one VSYNC input */
-	dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
+	if (vin->info->chip == RCAR_GEN3)
+		dmr2 = VNDMR2_FTEV;
+	else
+		dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
 
 	/* Hsync Signal Polarity Select */
 	if (!(vin->mbus_cfg.flags & V4L2_MBUS_HSYNC_ACTIVE_LOW))
@@ -711,6 +735,14 @@ static int rvin_setup(struct rvin_dev *vin)
 	if (input_is_yuv == output_is_yuv)
 		vnmc |= VNMC_BPS;
 
+	if (vin->info->chip == RCAR_GEN3) {
+		/* Select between CSI-2 and Digital input */
+		if (vin->mbus_cfg.type == V4L2_MBUS_CSI2)
+			vnmc &= ~VNMC_DPINE;
+		else
+			vnmc |= VNMC_DPINE;
+	}
+
 	/* Progressive or interlaced mode */
 	interrupts = progressive ? VNIE_FIE : VNIE_EFE;
 
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 118f45b656920d39..a440effe4b86af31 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -33,6 +33,7 @@ enum chip_id {
 	RCAR_H1,
 	RCAR_M1,
 	RCAR_GEN2,
+	RCAR_GEN3,
 };
 
 /**
-- 
2.15.0

[PATCH v9 16/28] rcar-vin: add function to manipulate Gen3 chsel value

[Niklas Söderlund]

On Gen3 the CSI-2 routing is controlled by the VnCSI_IFMD register. One
feature of this register is that it's only present in the VIN0 and VIN4
instances. The register in VIN0 controls the routing for VIN0-3 and the
register in VIN4 controls routing for VIN4-7.

To be able to control routing from a media device this function is need
to control runtime PM for the subgroup master (VIN0 and VIN4). The
subgroup master must be switched on before the register is manipulated,
once the operation is complete it's safe to switch the master off and
the new routing will still be in effect.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-dma.c | 25 +++++++++++++++++++++++++
 drivers/media/platform/rcar-vin/rcar-vin.h |  2 ++
 2 files changed, 27 insertions(+)

diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index ace95d5b543a17e3..d2788d8bb9565aaa 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -16,6 +16,7 @@
 
 #include <linux/delay.h>
 #include <linux/interrupt.h>
+#include <linux/pm_runtime.h>
 
 #include <media/videobuf2-dma-contig.h>
 
@@ -1228,3 +1229,27 @@ int rvin_dma_register(struct rvin_dev *vin, int irq)
 
 	return ret;
 }
+
+/* -----------------------------------------------------------------------------
+ * Gen3 CHSEL manipulation
+ */
+
+void rvin_set_chsel(struct rvin_dev *vin, u8 chsel)
+{
+	u32 ifmd, vnmc;
+
+	pm_runtime_get_sync(vin->dev);
+
+	/* Make register writes take effect immediately */
+	vnmc = rvin_read(vin, VNMC_REG) & ~VNMC_VUP;
+	rvin_write(vin, vnmc, VNMC_REG);
+
+	ifmd = VNCSI_IFMD_DES2 | VNCSI_IFMD_DES1 | VNCSI_IFMD_DES0 |
+		VNCSI_IFMD_CSI_CHSEL(chsel);
+
+	rvin_write(vin, ifmd, VNCSI_IFMD_REG);
+
+	vin_dbg(vin, "Set IFMD 0x%x\n", ifmd);
+
+	pm_runtime_put(vin->dev);
+}
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index a440effe4b86af31..7819c760c2c13422 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -163,4 +163,6 @@ void rvin_v4l2_unregister(struct rvin_dev *vin);
 
 const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
 
+void rvin_set_chsel(struct rvin_dev *vin, u8 chsel);
+
 #endif
-- 
2.15.0

[PATCH v9 17/28] rcar-vin: add flag to switch to media controller mode

[Niklas Söderlund]

On Gen3 a media controller API needs to be used to allow userspace to
configure the subdevices in the pipeline instead of directly controlling
a single source subdevice, which is and will continue to be the mode of
operation on Gen2.

Prepare for these two modes of operation by adding a flag to struct
rvin_info which will control which mode to use.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-core.c | 6 +++++-
 drivers/media/platform/rcar-vin/rcar-vin.h  | 2 ++
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index 7d49904cab9cb2d9..61f48ecc1ab815ec 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -232,18 +232,21 @@ static int rvin_digital_graph_init(struct rvin_dev *vin)
 
 static const struct rvin_info rcar_info_h1 = {
 	.chip = RCAR_H1,
+	.use_mc = false,
 	.max_width = 2048,
 	.max_height = 2048,
 };
 
 static const struct rvin_info rcar_info_m1 = {
 	.chip = RCAR_M1,
+	.use_mc = false,
 	.max_width = 2048,
 	.max_height = 2048,
 };
 
 static const struct rvin_info rcar_info_gen2 = {
 	.chip = RCAR_GEN2,
+	.use_mc = false,
 	.max_width = 2048,
 	.max_height = 2048,
 };
@@ -338,7 +341,8 @@ static int rcar_vin_remove(struct platform_device *pdev)
 	v4l2_async_notifier_unregister(&vin->notifier);
 	v4l2_async_notifier_cleanup(&vin->notifier);
 
-	v4l2_ctrl_handler_free(&vin->ctrl_handler);
+	if (!vin->info->use_mc)
+		v4l2_ctrl_handler_free(&vin->ctrl_handler);
 
 	rvin_dma_unregister(vin);
 
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 7819c760c2c13422..0747873c2b9cb74c 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -77,12 +77,14 @@ struct rvin_graph_entity {
 /**
  * struct rvin_info - Information about the particular VIN implementation
  * @chip:		type of VIN chip
+ * @use_mc:		use media controller instead of controlling subdevice
  *
  * max_width:		max input width the VIN supports
  * max_height:		max input height the VIN supports
  */
 struct rvin_info {
 	enum chip_id chip;
+	bool use_mc;
 
 	unsigned int max_width;
 	unsigned int max_height;
-- 
2.15.0

[PATCH v9 18/28] rcar-vin: break out format alignment and checking

[Niklas Söderlund]

Part of the format alignment and checking can be shared with the Gen3
format handling. Break that part out to its own function. While doing
this clean up the checking and add more checks.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-v4l2.c | 98 +++++++++++++++--------------
 1 file changed, 51 insertions(+), 47 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index 56c5183f55922e1d..0ffbf0c16fb7b00e 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -86,6 +86,56 @@ static u32 rvin_format_sizeimage(struct v4l2_pix_format *pix)
 	return pix->bytesperline * pix->height;
 }
 
+static int rvin_format_align(struct rvin_dev *vin, struct v4l2_pix_format *pix)
+{
+	u32 walign;
+
+	/* If requested format is not supported fallback to the default */
+	if (!rvin_format_from_pixel(pix->pixelformat)) {
+		vin_dbg(vin, "Format 0x%x not found, using default 0x%x\n",
+			pix->pixelformat, RVIN_DEFAULT_FORMAT);
+		pix->pixelformat = RVIN_DEFAULT_FORMAT;
+	}
+
+	switch (pix->field) {
+	case V4L2_FIELD_TOP:
+	case V4L2_FIELD_BOTTOM:
+	case V4L2_FIELD_NONE:
+	case V4L2_FIELD_INTERLACED_TB:
+	case V4L2_FIELD_INTERLACED_BT:
+	case V4L2_FIELD_INTERLACED:
+		break;
+	default:
+		pix->field = V4L2_FIELD_NONE;
+		break;
+	}
+
+	/* Check that colorspace is reasonable, if not keep current */
+	if (!pix->colorspace || pix->colorspace >= 0xff)
+		pix->colorspace = vin->format.colorspace;
+
+	/* HW limit width to a multiple of 32 (2^5) for NV16 else 2 (2^1) */
+	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
+
+	/* Limit to VIN capabilities */
+	v4l_bound_align_image(&pix->width, 2, vin->info->max_width, walign,
+			      &pix->height, 4, vin->info->max_height, 2, 0);
+
+	pix->bytesperline = rvin_format_bytesperline(pix);
+	pix->sizeimage = rvin_format_sizeimage(pix);
+
+	if (vin->info->chip == RCAR_M1 &&
+	    pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
+		vin_err(vin, "pixel format XBGR32 not supported on M1\n");
+		return -EINVAL;
+	}
+
+	vin_dbg(vin, "Format %ux%u bpl: %d size: %d\n",
+		pix->width, pix->height, pix->bytesperline, pix->sizeimage);
+
+	return 0;
+}
+
 /* -----------------------------------------------------------------------------
  * V4L2
  */
@@ -191,64 +241,18 @@ static int __rvin_try_format_source(struct rvin_dev *vin,
 static int __rvin_try_format(struct rvin_dev *vin,
 			     u32 which, struct v4l2_pix_format *pix)
 {
-	u32 walign;
 	int ret;
 
 	/* Keep current field if no specific one is asked for */
 	if (pix->field == V4L2_FIELD_ANY)
 		pix->field = vin->format.field;
 
-	/* If requested format is not supported fallback to the default */
-	if (!rvin_format_from_pixel(pix->pixelformat)) {
-		vin_dbg(vin, "Format 0x%x not found, using default 0x%x\n",
-			pix->pixelformat, RVIN_DEFAULT_FORMAT);
-		pix->pixelformat = RVIN_DEFAULT_FORMAT;
-	}
-
-	/* Always recalculate */
-	pix->bytesperline = 0;
-	pix->sizeimage = 0;
-
 	/* Limit to source capabilities */
 	ret = __rvin_try_format_source(vin, which, pix);
 	if (ret)
 		return ret;
 
-	switch (pix->field) {
-	case V4L2_FIELD_TOP:
-	case V4L2_FIELD_BOTTOM:
-	case V4L2_FIELD_NONE:
-	case V4L2_FIELD_INTERLACED_TB:
-	case V4L2_FIELD_INTERLACED_BT:
-	case V4L2_FIELD_INTERLACED:
-		break;
-	default:
-		pix->field = V4L2_FIELD_NONE;
-		break;
-	}
-
-	/* HW limit width to a multiple of 32 (2^5) for NV16 else 2 (2^1) */
-	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
-
-	/* Limit to VIN capabilities */
-	v4l_bound_align_image(&pix->width, 2, vin->info->max_width, walign,
-			      &pix->height, 4, vin->info->max_height, 2, 0);
-
-	pix->bytesperline = max_t(u32, pix->bytesperline,
-				  rvin_format_bytesperline(pix));
-	pix->sizeimage = max_t(u32, pix->sizeimage,
-			       rvin_format_sizeimage(pix));
-
-	if (vin->info->chip == RCAR_M1 &&
-	    pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
-		vin_err(vin, "pixel format XBGR32 not supported on M1\n");
-		return -EINVAL;
-	}
-
-	vin_dbg(vin, "Format %ux%u bpl: %d size: %d\n",
-		pix->width, pix->height, pix->bytesperline, pix->sizeimage);
-
-	return 0;
+	return rvin_format_align(vin, pix);
 }
 
 static int rvin_querycap(struct file *file, void *priv,
-- 
2.15.0

[PATCH v9 19/28] rcar-vin: use different v4l2 operations in media controller mode

[Niklas Söderlund]

When the driver runs in media controller mode it should not directly
control the subdevice instead userspace will be responsible for
configuring the pipeline. To be able to run in this mode a different set
of v4l2 operations needs to be used.

Add a new set of v4l2 operations to support the running without directly
interacting with the source subdevice.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-dma.c  |   3 +-
 drivers/media/platform/rcar-vin/rcar-v4l2.c | 155 +++++++++++++++++++++++++++-
 drivers/media/platform/rcar-vin/rcar-vin.h  |   1 +
 3 files changed, 155 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index d2788d8bb9565aaa..6c5df13b30d6dd14 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -628,7 +628,8 @@ static int rvin_setup(struct rvin_dev *vin)
 		/* Default to TB */
 		vnmc = VNMC_IM_FULL;
 		/* Use BT if video standard can be read and is 60 Hz format */
-		if (!v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
+		if (!vin->info->use_mc &&
+		    !v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
 			if (std & V4L2_STD_525_60)
 				vnmc = VNMC_IM_FULL | VNMC_FOC;
 		}
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index 0ffbf0c16fb7b00e..5fea2856fd61030f 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -23,6 +23,9 @@
 #include "rcar-vin.h"
 
 #define RVIN_DEFAULT_FORMAT	V4L2_PIX_FMT_YUYV
+#define RVIN_DEFAULT_WIDTH	800
+#define RVIN_DEFAULT_HEIGHT	600
+#define RVIN_DEFAULT_COLORSPACE	V4L2_COLORSPACE_SRGB
 
 /* -----------------------------------------------------------------------------
  * Format Conversions
@@ -671,6 +674,84 @@ static const struct v4l2_ioctl_ops rvin_ioctl_ops = {
 	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
 };
 
+/* -----------------------------------------------------------------------------
+ * V4L2 Media Controller
+ */
+
+static int __rvin_mc_try_format(struct rvin_dev *vin,
+				struct v4l2_pix_format *pix)
+{
+	/* Keep current field if no specific one is asked for */
+	if (pix->field == V4L2_FIELD_ANY)
+		pix->field = vin->format.field;
+
+	return rvin_format_align(vin, pix);
+}
+
+static int rvin_mc_try_fmt_vid_cap(struct file *file, void *priv,
+				   struct v4l2_format *f)
+{
+	struct rvin_dev *vin = video_drvdata(file);
+
+	return __rvin_mc_try_format(vin, &f->fmt.pix);
+}
+
+static int rvin_mc_s_fmt_vid_cap(struct file *file, void *priv,
+				 struct v4l2_format *f)
+{
+	struct rvin_dev *vin = video_drvdata(file);
+	int ret;
+
+	if (vb2_is_busy(&vin->queue))
+		return -EBUSY;
+
+	ret = __rvin_mc_try_format(vin, &f->fmt.pix);
+	if (ret)
+		return ret;
+
+	vin->format = f->fmt.pix;
+
+	return 0;
+}
+
+static int rvin_mc_enum_input(struct file *file, void *priv,
+			      struct v4l2_input *i)
+{
+	if (i->index != 0)
+		return -EINVAL;
+
+	i->type = V4L2_INPUT_TYPE_CAMERA;
+	strlcpy(i->name, "Camera", sizeof(i->name));
+
+	return 0;
+}
+
+static const struct v4l2_ioctl_ops rvin_mc_ioctl_ops = {
+	.vidioc_querycap		= rvin_querycap,
+	.vidioc_try_fmt_vid_cap		= rvin_mc_try_fmt_vid_cap,
+	.vidioc_g_fmt_vid_cap		= rvin_g_fmt_vid_cap,
+	.vidioc_s_fmt_vid_cap		= rvin_mc_s_fmt_vid_cap,
+	.vidioc_enum_fmt_vid_cap	= rvin_enum_fmt_vid_cap,
+
+	.vidioc_enum_input		= rvin_mc_enum_input,
+	.vidioc_g_input			= rvin_g_input,
+	.vidioc_s_input			= rvin_s_input,
+
+	.vidioc_reqbufs			= vb2_ioctl_reqbufs,
+	.vidioc_create_bufs		= vb2_ioctl_create_bufs,
+	.vidioc_querybuf		= vb2_ioctl_querybuf,
+	.vidioc_qbuf			= vb2_ioctl_qbuf,
+	.vidioc_dqbuf			= vb2_ioctl_dqbuf,
+	.vidioc_expbuf			= vb2_ioctl_expbuf,
+	.vidioc_prepare_buf		= vb2_ioctl_prepare_buf,
+	.vidioc_streamon		= vb2_ioctl_streamon,
+	.vidioc_streamoff		= vb2_ioctl_streamoff,
+
+	.vidioc_log_status		= v4l2_ctrl_log_status,
+	.vidioc_subscribe_event		= rvin_subscribe_event,
+	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
+};
+
 /* -----------------------------------------------------------------------------
  * File Operations
  */
@@ -814,6 +895,60 @@ static const struct v4l2_file_operations rvin_fops = {
 	.read		= vb2_fop_read,
 };
 
+/* -----------------------------------------------------------------------------
+ * Media controller file Operations
+ */
+
+static int rvin_mc_open(struct file *file)
+{
+	struct rvin_dev *vin = video_drvdata(file);
+	int ret;
+
+	mutex_lock(&vin->lock);
+
+	file->private_data = vin;
+
+	ret = v4l2_fh_open(file);
+	if (ret)
+		goto unlock;
+
+	pm_runtime_get_sync(vin->dev);
+	v4l2_pipeline_pm_use(&vin->vdev.entity, 1);
+
+unlock:
+	mutex_unlock(&vin->lock);
+
+	return ret;
+}
+
+static int rvin_mc_release(struct file *file)
+{
+	struct rvin_dev *vin = video_drvdata(file);
+	int ret;
+
+	mutex_lock(&vin->lock);
+
+	/* the release helper will cleanup any on-going streaming */
+	ret = _vb2_fop_release(file, NULL);
+
+	v4l2_pipeline_pm_use(&vin->vdev.entity, 0);
+	pm_runtime_put(vin->dev);
+
+	mutex_unlock(&vin->lock);
+
+	return ret;
+}
+
+static const struct v4l2_file_operations rvin_mc_fops = {
+	.owner		= THIS_MODULE,
+	.unlocked_ioctl	= video_ioctl2,
+	.open		= rvin_mc_open,
+	.release	= rvin_mc_release,
+	.poll		= vb2_fop_poll,
+	.mmap		= vb2_fop_mmap,
+	.read		= vb2_fop_read,
+};
+
 void rvin_v4l2_unregister(struct rvin_dev *vin)
 {
 	if (!video_is_registered(&vin->vdev))
@@ -849,19 +984,33 @@ int rvin_v4l2_register(struct rvin_dev *vin)
 	vin->v4l2_dev.notify = rvin_notify;
 
 	/* video node */
-	vdev->fops = &rvin_fops;
 	vdev->v4l2_dev = &vin->v4l2_dev;
 	vdev->queue = &vin->queue;
 	snprintf(vdev->name, sizeof(vdev->name), "%s %s", KBUILD_MODNAME,
 		 dev_name(vin->dev));
 	vdev->release = video_device_release_empty;
-	vdev->ioctl_ops = &rvin_ioctl_ops;
 	vdev->lock = &vin->lock;
 	vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING |
 		V4L2_CAP_READWRITE;
 
+	/* Set some form of default format */
 	vin->format.pixelformat	= RVIN_DEFAULT_FORMAT;
-	rvin_reset_format(vin);
+	vin->format.width = RVIN_DEFAULT_WIDTH;
+	vin->format.height = RVIN_DEFAULT_HEIGHT;
+	vin->format.colorspace = RVIN_DEFAULT_COLORSPACE;
+
+	if (vin->info->use_mc) {
+		vdev->fops = &rvin_mc_fops;
+		vdev->ioctl_ops = &rvin_mc_ioctl_ops;
+	} else {
+		vdev->fops = &rvin_fops;
+		vdev->ioctl_ops = &rvin_ioctl_ops;
+		rvin_reset_format(vin);
+	}
+
+	ret = rvin_format_align(vin, &vin->format);
+	if (ret)
+		return ret;
 
 	ret = video_register_device(&vin->vdev, VFL_TYPE_GRABBER, -1);
 	if (ret) {
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 0747873c2b9cb74c..fd3cd781be0ab1cf 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -21,6 +21,7 @@
 #include <media/v4l2-ctrls.h>
 #include <media/v4l2-dev.h>
 #include <media/v4l2-device.h>
+#include <media/v4l2-mc.h>
 #include <media/videobuf2-v4l2.h>
 
 /* Number of HW buffers */
-- 
2.15.0

[PATCH v9 20/28] rcar-vin: prepare for media controller mode initialization

[Niklas Söderlund]

When running in media controller mode a media pad is needed, register
one. Also set the media bus format to CSI-2.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-core.c | 24 ++++++++++++++++++++++--
 drivers/media/platform/rcar-vin/rcar-vin.h  |  4 ++++
 2 files changed, 26 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index 61f48ecc1ab815ec..45de4079fd835759 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -46,6 +46,10 @@ static int rvin_find_pad(struct v4l2_subdev *sd, int direction)
 	return -EINVAL;
 }
 
+/* -----------------------------------------------------------------------------
+ * Digital async notifier
+ */
+
 static int rvin_digital_notify_complete(struct v4l2_async_notifier *notifier)
 {
 	struct rvin_dev *vin = notifier_to_vin(notifier);
@@ -226,6 +230,20 @@ static int rvin_digital_graph_init(struct rvin_dev *vin)
 	return 0;
 }
 
+/* -----------------------------------------------------------------------------
+ * Group async notifier
+ */
+
+static int rvin_group_init(struct rvin_dev *vin)
+{
+	/* All our sources are CSI-2 */
+	vin->mbus_cfg.type = V4L2_MBUS_CSI2;
+	vin->mbus_cfg.flags = 0;
+
+	vin->pad.flags = MEDIA_PAD_FL_SINK;
+	return media_entity_pads_init(&vin->vdev.entity, 1, &vin->pad);
+}
+
 /* -----------------------------------------------------------------------------
  * Platform Device Driver
  */
@@ -314,8 +332,10 @@ static int rcar_vin_probe(struct platform_device *pdev)
 		return ret;
 
 	platform_set_drvdata(pdev, vin);
-
-	ret = rvin_digital_graph_init(vin);
+	if (vin->info->use_mc)
+		ret = rvin_group_init(vin);
+	else
+		ret = rvin_digital_graph_init(vin);
 	if (ret < 0)
 		goto error;
 
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index fd3cd781be0ab1cf..07d270a976893cdb 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -103,6 +103,8 @@ struct rvin_info {
  * @notifier:		V4L2 asynchronous subdevs notifier
  * @digital:		entity in the DT for local digital subdevice
  *
+ * @pad:		pad for media controller
+ *
  * @lock:		protects @queue
  * @queue:		vb2 buffers queue
  *
@@ -132,6 +134,8 @@ struct rvin_dev {
 	struct v4l2_async_notifier notifier;
 	struct rvin_graph_entity *digital;
 
+	struct media_pad pad;
+
 	struct mutex lock;
 	struct vb2_queue queue;
 
-- 
2.15.0

[PATCH v9 21/28] rcar-vin: add group allocator functions

[Niklas Söderlund]

In media controller mode all VIN instances needs to be part of the same
media graph. There is also a need to each VIN instance to know and in
some cases be able to communicate with other VIN instances.

Add an allocator framework where the first VIN instance to be probed
creates a shared data structure and creates a media device.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-core.c | 179 +++++++++++++++++++++++++++-
 drivers/media/platform/rcar-vin/rcar-vin.h  |  38 ++++++
 2 files changed, 215 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index 45de4079fd835759..a6713fd61dd87a88 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -20,12 +20,170 @@
 #include <linux/of_graph.h>
 #include <linux/platform_device.h>
 #include <linux/pm_runtime.h>
+#include <linux/slab.h>
 
 #include <media/v4l2-async.h>
 #include <media/v4l2-fwnode.h>
 
 #include "rcar-vin.h"
 
+/* -----------------------------------------------------------------------------
+ * Gen3 CSI2 Group Allocator
+ */
+
+static int rvin_group_read_id(struct rvin_dev *vin, struct device_node *np)
+{
+	u32 val;
+	int ret;
+
+	ret = of_property_read_u32(np, "renesas,id", &val);
+	if (ret) {
+		vin_err(vin, "%pOF: No renesas,id property found\n", np);
+		return -EINVAL;
+	}
+
+	if (val >= RCAR_VIN_NUM) {
+		vin_err(vin, "%pOF: Invalid renesas,id '%u'\n", np, val);
+		return -EINVAL;
+	}
+
+	return val;
+}
+
+static DEFINE_MUTEX(rvin_group_lock);
+static struct rvin_group *rvin_group_data;
+
+static void rvin_group_release(struct kref *kref)
+{
+	struct rvin_group *group =
+		container_of(kref, struct rvin_group, refcount);
+
+	mutex_lock(&rvin_group_lock);
+
+	media_device_unregister(&group->mdev);
+	media_device_cleanup(&group->mdev);
+
+	rvin_group_data = NULL;
+
+	mutex_unlock(&rvin_group_lock);
+
+	kfree(group);
+}
+
+static struct rvin_group *__rvin_group_allocate(struct rvin_dev *vin)
+{
+	struct rvin_group *group;
+
+	if (rvin_group_data) {
+		group = rvin_group_data;
+		kref_get(&group->refcount);
+		vin_dbg(vin, "%s: get group=%p\n", __func__, group);
+		return group;
+	}
+
+	group = kzalloc(sizeof(*group), GFP_KERNEL);
+	if (!group)
+		return NULL;
+
+	kref_init(&group->refcount);
+	rvin_group_data = group;
+
+	vin_dbg(vin, "%s: alloc group=%p\n", __func__, group);
+	return group;
+}
+
+static int rvin_group_add_vin(struct rvin_dev *vin)
+{
+	int ret;
+
+	ret = rvin_group_read_id(vin, vin->dev->of_node);
+	if (ret < 0)
+		return ret;
+
+	mutex_lock(&vin->group->lock);
+
+	if (vin->group->vin[ret]) {
+		mutex_unlock(&vin->group->lock);
+		vin_err(vin, "VIN number %d already occupied\n", ret);
+		return -EINVAL;
+	}
+
+	vin->group->vin[ret] = vin;
+
+	mutex_unlock(&vin->group->lock);
+
+	vin_dbg(vin, "I'm VIN number %d", ret);
+
+	return 0;
+}
+
+static int rvin_group_allocate(struct rvin_dev *vin)
+{
+	struct rvin_group *group;
+	struct media_device *mdev;
+	int ret;
+
+	mutex_lock(&rvin_group_lock);
+
+	group = __rvin_group_allocate(vin);
+	if (!group) {
+		mutex_unlock(&rvin_group_lock);
+		return -ENOMEM;
+	}
+
+	/* Init group data if it is not already initialized */
+	mdev = &group->mdev;
+	if (!mdev->dev) {
+		mutex_init(&group->lock);
+		mdev->dev = vin->dev;
+
+		strlcpy(mdev->driver_name, "Renesas VIN",
+			sizeof(mdev->driver_name));
+		strlcpy(mdev->model, vin->dev->of_node->name,
+			sizeof(mdev->model));
+
+		snprintf(mdev->bus_info, sizeof(mdev->bus_info), "platform:%s",
+			 dev_name(mdev->dev));
+		media_device_init(mdev);
+
+		ret = media_device_register(mdev);
+		if (ret) {
+			vin_err(vin, "Failed to register media device\n");
+			kref_put(&group->refcount, rvin_group_release);
+			mutex_unlock(&rvin_group_lock);
+			return ret;
+		}
+	}
+
+	vin->group = group;
+	vin->v4l2_dev.mdev = mdev;
+
+	ret = rvin_group_add_vin(vin);
+	if (ret) {
+		kref_put(&group->refcount, rvin_group_release);
+		mutex_unlock(&rvin_group_lock);
+		return ret;
+	}
+
+	mutex_unlock(&rvin_group_lock);
+
+	return 0;
+}
+
+static void rvin_group_delete(struct rvin_dev *vin)
+{
+	unsigned int i;
+
+	mutex_lock(&vin->group->lock);
+	for (i = 0; i < RCAR_VIN_NUM; i++)
+		if (vin->group->vin[i] == vin)
+			vin->group->vin[i] = NULL;
+	mutex_unlock(&vin->group->lock);
+
+	vin_dbg(vin, "%s: group=%p\n", __func__, &vin->group);
+	kref_put(&vin->group->refcount, rvin_group_release);
+}
+
 /* -----------------------------------------------------------------------------
  * Async notifier
  */
@@ -236,12 +394,27 @@ static int rvin_digital_graph_init(struct rvin_dev *vin)
 
 static int rvin_group_init(struct rvin_dev *vin)
 {
+	int ret;
+
+	ret = rvin_group_allocate(vin);
+	if (ret)
+		return ret;
+
 	/* All our sources are CSI-2 */
 	vin->mbus_cfg.type = V4L2_MBUS_CSI2;
 	vin->mbus_cfg.flags = 0;
 
 	vin->pad.flags = MEDIA_PAD_FL_SINK;
-	return media_entity_pads_init(&vin->vdev.entity, 1, &vin->pad);
+	ret = media_entity_pads_init(&vin->vdev.entity, 1, &vin->pad);
+	if (ret)
+		goto error_group;
+
+	return 0;
+
+error_group:
+	rvin_group_delete(vin);
+
+	return ret;
 }
 
 /* -----------------------------------------------------------------------------
@@ -361,7 +534,9 @@ static int rcar_vin_remove(struct platform_device *pdev)
 	v4l2_async_notifier_unregister(&vin->notifier);
 	v4l2_async_notifier_cleanup(&vin->notifier);
 
-	if (!vin->info->use_mc)
+	if (vin->info->use_mc)
+		rvin_group_delete(vin);
+	else
 		v4l2_ctrl_handler_free(&vin->ctrl_handler);
 
 	rvin_dma_unregister(vin);
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 07d270a976893cdb..5f736a3500b6e10f 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -17,6 +17,8 @@
 #ifndef __RCAR_VIN__
 #define __RCAR_VIN__
 
+#include <linux/kref.h>
+
 #include <media/v4l2-async.h>
 #include <media/v4l2-ctrls.h>
 #include <media/v4l2-dev.h>
@@ -30,6 +32,9 @@
 /* Address alignment mask for HW buffers */
 #define HW_BUFFER_MASK 0x7f
 
+/* Max number on VIN instances that can be in a system */
+#define RCAR_VIN_NUM 8
+
 enum chip_id {
 	RCAR_H1,
 	RCAR_M1,
@@ -37,6 +42,15 @@ enum chip_id {
 	RCAR_GEN3,
 };
 
+enum rvin_csi_id {
+	RVIN_CSI20,
+	RVIN_CSI21,
+	RVIN_CSI40,
+	RVIN_CSI41,
+	RVIN_CSI_MAX,
+	RVIN_NC, /* Not Connected */
+};
+
 /**
  * STOPPED  - No operation in progress
  * RUNNING  - Operation in progress have buffers
@@ -75,6 +89,8 @@ struct rvin_graph_entity {
 	unsigned int sink_pad;
 };
 
+struct rvin_group;
+
 /**
  * struct rvin_info - Information about the particular VIN implementation
  * @chip:		type of VIN chip
@@ -103,6 +119,7 @@ struct rvin_info {
  * @notifier:		V4L2 asynchronous subdevs notifier
  * @digital:		entity in the DT for local digital subdevice
  *
+ * @group:		Gen3 CSI group
  * @pad:		pad for media controller
  *
  * @lock:		protects @queue
@@ -134,6 +151,7 @@ struct rvin_dev {
 	struct v4l2_async_notifier notifier;
 	struct rvin_graph_entity *digital;
 
+	struct rvin_group *group;
 	struct media_pad pad;
 
 	struct mutex lock;
@@ -162,6 +180,26 @@ struct rvin_dev {
 #define vin_warn(d, fmt, arg...)	dev_warn(d->dev, fmt, ##arg)
 #define vin_err(d, fmt, arg...)		dev_err(d->dev, fmt, ##arg)
 
+/**
+ * struct rvin_group - VIN CSI2 group information
+ * @refcount:		number of VIN instances using the group
+ *
+ * @mdev:		media device which represents the group
+ *
+ * @lock:		protects the vin and csi members
+ * @vin:		VIN instances which are part of the group
+ * @csi:		CSI-2 entities that are part of the group
+ */
+struct rvin_group {
+	struct kref refcount;
+
+	struct media_device mdev;
+
+	struct mutex lock;
+	struct rvin_dev *vin[RCAR_VIN_NUM];
+	struct rvin_graph_entity csi[RVIN_CSI_MAX];
+};
+
 int rvin_dma_register(struct rvin_dev *vin, int irq);
 void rvin_dma_unregister(struct rvin_dev *vin);
 
-- 
2.15.0

[PATCH v9 22/28] rcar-vin: add chsel information to rvin_info

[Niklas Söderlund]

Each Gen3 SoC has a limited set of predefined routing possibilities for
which CSI-2 device and virtual channel can be routed to which VIN
instance. Prepare to store this information in the struct rvin_info.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-vin.h | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 5f736a3500b6e10f..41bf24aa8a1a0aed 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -35,6 +35,9 @@
 /* Max number on VIN instances that can be in a system */
 #define RCAR_VIN_NUM 8
 
+/* Max number of CHSEL values for any Gen3 SoC */
+#define RCAR_CHSEL_MAX 6
+
 enum chip_id {
 	RCAR_H1,
 	RCAR_M1,
@@ -91,6 +94,22 @@ struct rvin_graph_entity {
 
 struct rvin_group;
 
+/** struct rvin_group_chsel - Map a CSI-2 receiver and channel to a CHSEL value
+ * @csi:		VIN internal number for CSI-2 device
+ * @chan:		Output channel of the CSI-2 receiver. Each R-Car CSI-2
+ *			receiver has four output channels facing the VIN
+ *			devices, each channel can carry one CSI-2 Virtual
+ *			Channel (VC) and there are no correlation between
+ *			output channel number and CSI-2 VC. It's up to the
+ *			CSI-2 receiver driver to configure which VC is
+ *			outputted on which channel, the VIN devices only
+ *			cares about output channels.
+ */
+struct rvin_group_chsel {
+	enum rvin_csi_id csi;
+	unsigned int chan;
+};
+
 /**
  * struct rvin_info - Information about the particular VIN implementation
  * @chip:		type of VIN chip
@@ -98,6 +117,9 @@ struct rvin_group;
  *
  * max_width:		max input width the VIN supports
  * max_height:		max input height the VIN supports
+ *
+ * num_chsels:		number of possible chsel values for this VIN
+ * chsels:		routing table VIN <-> CSI-2 for the chsel values
  */
 struct rvin_info {
 	enum chip_id chip;
@@ -105,6 +127,9 @@ struct rvin_info {
 
 	unsigned int max_width;
 	unsigned int max_height;
+
+	unsigned int num_chsels;
+	struct rvin_group_chsel chsels[RCAR_VIN_NUM][RCAR_CHSEL_MAX];
 };
 
 /**
-- 
2.15.0

[PATCH v9 23/28] rcar-vin: parse Gen3 OF and setup media graph

[Niklas Söderlund]

Parse the VIN Gen3 OF graph and register all CSI-2 devices in the VIN
group common media device. Once all CSI-2 subdevices are added to the
common media device create links between them.

The parsing and registering CSI-2 subdevices with the v4l2 async
framework is a collaborative effort shared between the VIN instances
which are part of the group. The first rcar-vin instance parses OF and
finds all other VIN and CSI-2 nodes which are part of the graph. It
stores a bit mask of all VIN instances found and handles to all CSI-2
nodes.

The bit mask is used to figure out when all VIN instances have been
probed. Once the last VIN instance is probed this is detected and this
instance registers all CSI-2 subdevices in its private async notifier.
Once the .complete() callback of this notifier is called it registers
all video devices and creates the media controller links between all
entities.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-core.c | 320 +++++++++++++++++++++++++++-
 drivers/media/platform/rcar-vin/rcar-vin.h  |  10 +-
 2 files changed, 326 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index a6713fd61dd87a88..4a147437fa69c364 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -86,6 +86,7 @@ static struct rvin_group *__rvin_group_allocate(struct rvin_dev *vin)
 		return NULL;
 
 	kref_init(&group->refcount);
+	group->notifier = NULL;
 	rvin_group_data = group;
 
 	vin_dbg(vin, "%s: alloc group=%p\n", __func__, group);
@@ -392,10 +393,281 @@ static int rvin_digital_graph_init(struct rvin_dev *vin)
  * Group async notifier
  */
 
-static int rvin_group_init(struct rvin_dev *vin)
+/* group lock should be held when calling this function */
+static int rvin_group_add_link(struct rvin_dev *vin,
+			       struct media_entity *source,
+			       unsigned int source_idx,
+			       struct media_entity *sink,
+			       unsigned int sink_idx,
+			       u32 flags)
+{
+	struct media_pad *source_pad, *sink_pad;
+	int ret = 0;
+
+	source_pad = &source->pads[source_idx];
+	sink_pad = &sink->pads[sink_idx];
+
+	if (!media_entity_find_link(source_pad, sink_pad))
+		ret = media_create_pad_link(source, source_idx,
+					    sink, sink_idx, flags);
+
+	if (ret)
+		vin_err(vin, "Error adding link from %s to %s\n",
+			source->name, sink->name);
+
+	return ret;
+}
+
+static int rvin_group_update_links(struct rvin_dev *vin)
+{
+	struct media_entity *source, *sink;
+	struct rvin_dev *master;
+	unsigned int i, n, idx, csi;
+	int ret = 0;
+
+	mutex_lock(&vin->group->lock);
+
+	for (n = 0; n < RCAR_VIN_NUM; n++) {
+
+		/* Check that VIN is part of the group */
+		if (!vin->group->vin[n])
+			continue;
+
+		/* Check that subgroup master is part of the group */
+		master = vin->group->vin[n < 4 ? 0 : 4];
+		if (!master)
+			continue;
+
+		for (i = 0; i < vin->info->num_chsels; i++) {
+			csi = vin->info->chsels[n][i].csi;
+
+			/* If the CSI-2 is out of bounds it's a noop, skip */
+			if (csi >= RVIN_CSI_MAX)
+				continue;
+
+			/* Check that CSI-2 are part of the group */
+			if (!vin->group->csi[csi].subdev)
+				continue;
+
+			source = &vin->group->csi[csi].subdev->entity;
+			sink = &vin->group->vin[n]->vdev.entity;
+			idx = vin->info->chsels[n][i].chan + 1;
+
+			ret = rvin_group_add_link(vin, source, idx, sink, 0,
+						  0);
+			if (ret)
+				goto out;
+		}
+	}
+out:
+	mutex_unlock(&vin->group->lock);
+
+	return ret;
+}
+
+static int rvin_group_notify_complete(struct v4l2_async_notifier *notifier)
 {
+	struct rvin_dev *vin = notifier_to_vin(notifier);
+	unsigned int i;
 	int ret;
 
+	ret = v4l2_device_register_subdev_nodes(&vin->v4l2_dev);
+	if (ret) {
+		vin_err(vin, "Failed to register subdev nodes\n");
+		return ret;
+	}
+
+	for (i = 0; i < RCAR_VIN_NUM; i++) {
+		if (vin->group->vin[i]) {
+			ret = rvin_v4l2_register(vin->group->vin[i]);
+			if (ret)
+				return ret;
+		}
+	}
+
+	return rvin_group_update_links(vin);
+}
+
+static void rvin_group_notify_unbind(struct v4l2_async_notifier *notifier,
+				     struct v4l2_subdev *subdev,
+				     struct v4l2_async_subdev *asd)
+{
+	struct rvin_dev *vin = notifier_to_vin(notifier);
+	struct rvin_graph_entity *csi = to_rvin_graph_entity(asd);
+	unsigned int i;
+
+	for (i = 0; i < RCAR_VIN_NUM; i++)
+		if (vin->group->vin[i])
+			rvin_v4l2_unregister(vin->group->vin[i]);
+
+	mutex_lock(&vin->group->lock);
+	csi->subdev = NULL;
+	mutex_unlock(&vin->group->lock);
+}
+
+static int rvin_group_notify_bound(struct v4l2_async_notifier *notifier,
+				   struct v4l2_subdev *subdev,
+				   struct v4l2_async_subdev *asd)
+{
+	struct rvin_dev *vin = notifier_to_vin(notifier);
+	struct rvin_graph_entity *csi = to_rvin_graph_entity(asd);
+
+	v4l2_set_subdev_hostdata(subdev, vin);
+
+	mutex_lock(&vin->group->lock);
+	vin_dbg(vin, "Bound CSI-2 %s\n", subdev->name);
+	csi->subdev = subdev;
+	mutex_unlock(&vin->group->lock);
+
+	return 0;
+}
+
+static const struct v4l2_async_notifier_operations rvin_group_notify_ops = {
+	.bound = rvin_group_notify_bound,
+	.unbind = rvin_group_notify_unbind,
+	.complete = rvin_group_notify_complete,
+};
+
+static struct device_node *rvin_group_get_remote(struct rvin_dev *vin,
+						 struct device_node *node)
+{
+	struct device_node *np;
+
+	np = of_graph_get_remote_port_parent(node);
+	if (!np) {
+		vin_err(vin, "Remote port not found %pOF\n", node);
+		return NULL;
+	}
+
+	/* Not all remote ports are available, this is OK */
+	if (!of_device_is_available(np)) {
+		vin_dbg(vin, "Remote port %pOF is not available\n", np);
+		of_node_put(np);
+		return NULL;
+	}
+
+	return np;
+}
+
+/* group lock should be held when calling this function */
+static int rvin_group_graph_parse(struct rvin_dev *vin, struct device_node *np)
+{
+	int i, id, ret;
+
+	/* Read VIN id from DT */
+	id = rvin_group_read_id(vin, np);
+	if (id < 0)
+		return id;
+
+	/* Check if VIN is already handled */
+	if (vin->group->mask & BIT(id))
+		return 0;
+
+	vin->group->mask |= BIT(id);
+
+	vin_dbg(vin, "Handling VIN%d\n", id);
+
+	/* Parse all endpoints for CSI-2 and VIN nodes */
+	for (i = 0; i < RVIN_CSI_MAX; i++) {
+		struct device_node *ep, *csi, *remote;
+
+		/* Check if instance is connected to the CSI-2 */
+		ep = of_graph_get_endpoint_by_regs(np, 1, i);
+		if (!ep) {
+			vin_dbg(vin, "VIN%d: ep %d not connected\n", id, i);
+			continue;
+		}
+
+		if (vin->group->csi[i].asd.match.fwnode.fwnode) {
+			of_node_put(ep);
+			vin_dbg(vin, "VIN%d: ep %d already handled\n", id, i);
+			continue;
+		}
+
+		csi = rvin_group_get_remote(vin, ep);
+		of_node_put(ep);
+		if (!csi)
+			continue;
+
+		vin->group->csi[i].asd.match.fwnode.fwnode =
+			of_fwnode_handle(csi);
+		vin->group->csi[i].asd.match_type = V4L2_ASYNC_MATCH_FWNODE;
+
+		vin_dbg(vin, "VIN%d ep: %d handled CSI-2 %pOF\n", id, i, csi);
+
+		/* Parse the CSI-2 for all VIN nodes connected to it */
+		ep = NULL;
+		while (1) {
+			ep = of_graph_get_next_endpoint(csi, ep);
+			if (!ep)
+				break;
+
+			remote = rvin_group_get_remote(vin, ep);
+			if (!remote)
+				continue;
+
+			if (of_match_node(vin->dev->driver->of_match_table,
+					  remote)) {
+				ret = rvin_group_graph_parse(vin, remote);
+				if (ret)
+					return ret;
+
+			}
+		}
+	}
+
+	return 0;
+}
+
+static int rvin_group_graph_register(struct rvin_dev *vin)
+{
+	struct v4l2_async_subdev **subdevs = NULL;
+	int i, n, ret, count = 0;
+
+	mutex_lock(&vin->group->lock);
+
+	/* Count how many CSI-2 nodes found */
+	for (i = 0; i < RVIN_CSI_MAX; i++)
+		if (vin->group->csi[i].asd.match.fwnode.fwnode)
+			count++;
+
+	if (!count) {
+		mutex_unlock(&vin->group->lock);
+		return 0;
+	}
+
+	/* Allocate and setup list of subdevices for the notifier */
+	subdevs = devm_kzalloc(vin->dev, sizeof(*subdevs) * count, GFP_KERNEL);
+	if (subdevs == NULL) {
+		mutex_unlock(&vin->group->lock);
+		return -ENOMEM;
+	}
+
+	n = 0;
+	for (i = 0; i < RVIN_CSI_MAX; i++)
+		if (vin->group->csi[i].asd.match.fwnode.fwnode)
+			subdevs[n++] = &vin->group->csi[i].asd;
+
+	vin_dbg(vin, "Claimed %d subdevices for group\n", count);
+
+	vin->notifier.num_subdevs = count;
+	vin->notifier.subdevs = subdevs;
+	vin->notifier.ops = &rvin_group_notify_ops;
+
+	mutex_unlock(&vin->group->lock);
+
+	ret = v4l2_async_notifier_register(&vin->v4l2_dev, &vin->notifier);
+	if (ret < 0)
+		vin_err(vin, "Notifier registration failed\n");
+
+	return ret;
+}
+
+static int rvin_group_init(struct rvin_dev *vin)
+{
+	int i, ret, count_mask, count_vin = 0;
+	bool reg_notifier = false;
+
 	ret = rvin_group_allocate(vin);
 	if (ret)
 		return ret;
@@ -409,8 +681,45 @@ static int rvin_group_init(struct rvin_dev *vin)
 	if (ret)
 		goto error_group;
 
+	/*
+	 * Check number of registered VINs in group against the group mask.
+	 * If the mask is empty DT has not yet been parsed and if the
+	 * count matches all VINs are registered and it's safe to register
+	 * the async notifier
+	 */
+	mutex_lock(&vin->group->lock);
+
+	if (!vin->group->mask) {
+		ret = rvin_group_graph_parse(vin, vin->dev->of_node);
+		if (ret) {
+			mutex_unlock(&vin->group->lock);
+			goto error_group;
+		}
+	}
+
+	for (i = 0; i < RCAR_VIN_NUM; i++)
+		if (vin->group->vin[i])
+			count_vin++;
+
+	count_mask = hweight_long(vin->group->mask);
+
+	if (count_vin == count_mask && !vin->group->notifier) {
+		vin->group->notifier = &vin->notifier;
+		reg_notifier = true;
+	}
+
+	mutex_unlock(&vin->group->lock);
+
+	if (reg_notifier) {
+		ret = rvin_group_graph_register(vin);
+		if (ret)
+			goto error_vdev;
+	}
+
 	return 0;
 
+error_vdev:
+	rvin_v4l2_unregister(vin);
 error_group:
 	rvin_group_delete(vin);
 
@@ -534,10 +843,15 @@ static int rcar_vin_remove(struct platform_device *pdev)
 	v4l2_async_notifier_unregister(&vin->notifier);
 	v4l2_async_notifier_cleanup(&vin->notifier);
 
-	if (vin->info->use_mc)
+	if (vin->info->use_mc) {
+		mutex_lock(&vin->group->lock);
+		if (vin->group->notifier == &vin->notifier)
+			vin->group->notifier = NULL;
+		mutex_unlock(&vin->group->lock);
 		rvin_group_delete(vin);
-	else
+	} else {
 		v4l2_ctrl_handler_free(&vin->ctrl_handler);
+	}
 
 	rvin_dma_unregister(vin);
 
diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h b/drivers/media/platform/rcar-vin/rcar-vin.h
index 41bf24aa8a1a0aed..2081ec6d3e7ac64f 100644
--- a/drivers/media/platform/rcar-vin/rcar-vin.h
+++ b/drivers/media/platform/rcar-vin/rcar-vin.h
@@ -92,6 +92,9 @@ struct rvin_graph_entity {
 	unsigned int sink_pad;
 };
 
+#define to_rvin_graph_entity(asd) \
+	container_of(asd, struct rvin_graph_entity, asd)
+
 struct rvin_group;
 
 /** struct rvin_group_chsel - Map a CSI-2 receiver and channel to a CHSEL value
@@ -211,7 +214,10 @@ struct rvin_dev {
  *
  * @mdev:		media device which represents the group
  *
- * @lock:		protects the vin and csi members
+ * @lock:		protects the mask, vin and csi members
+ * @mask:		Mask of VIN instances found in DT
+ * @notifier:		Pointer to the notifer of a VIN which handles the
+ *			groups async sub-devices.
  * @vin:		VIN instances which are part of the group
  * @csi:		CSI-2 entities that are part of the group
  */
@@ -221,6 +227,8 @@ struct rvin_group {
 	struct media_device mdev;
 
 	struct mutex lock;
+	unsigned long mask;
+	struct v4l2_async_notifier *notifier;
 	struct rvin_dev *vin[RCAR_VIN_NUM];
 	struct rvin_graph_entity csi[RVIN_CSI_MAX];
 };
-- 
2.15.0

[PATCH v9 24/28] rcar-vin: add link notify for Gen3

[Niklas Söderlund]

Add the ability to process media device link change request. Link
enabling is a bit complicated on Gen3, whether or not it's possible to
enable a link depends on what other links already are enabled. On Gen3
the 8 VINs are split into two subgroup's (VIN0-3 and VIN4-7) and from a
routing perspective these two groups are independent of each other.
Each subgroup's routing is controlled by the subgroup VIN master
instance (VIN0 and VIN4).

There are a limited number of possible route setups available for each
subgroup and the configuration of each setup is dictated by the
hardware. On H3 for example there are 6 possible route setups for each
subgroup to choose from.

This leads to the media device link notification code being rather large
since it will find the best routing configuration to try and accommodate
as many links as possible. When it's not possible to enable a new link
due to hardware constrains the link_notifier callback will return
-EMLINK.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-core.c | 205 ++++++++++++++++++++++++++++
 1 file changed, 205 insertions(+)

diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index 4a147437fa69c364..09ebeff1580556dc 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -27,6 +27,209 @@
 
 #include "rcar-vin.h"
 
+/* -----------------------------------------------------------------------------
+ * Media Controller link notification
+ */
+
+static unsigned int rvin_group_csi_pad_to_chan(unsigned int pad)
+{
+	/*
+	 * The companion CSI-2 receiver driver (rcar-csi2) is known
+	 * and we know it have one source pad (pad 0) and four sink
+	 * pads (pad 1-4). So to translate a pad on the remote
+	 * CSI-2 receiver to the VIN internal channel number simply
+	 * subtract one from the pad number.
+	 */
+	return pad - 1;
+}
+
+/* group lock should be held when calling this function */
+static int rvin_group_entity_to_vin_num(struct rvin_group *group,
+					struct media_entity *entity)
+{
+	struct video_device *vdev;
+	int i;
+
+	if (!is_media_entity_v4l2_video_device(entity))
+		return -ENODEV;
+
+	vdev = media_entity_to_video_device(entity);
+
+	for (i = 0; i < RCAR_VIN_NUM; i++) {
+		if (!group->vin[i])
+			continue;
+
+		if (&group->vin[i]->vdev == vdev)
+			return i;
+	}
+
+	return -ENODEV;
+}
+
+/* group lock should be held when calling this function */
+static int rvin_group_entity_to_csi_num(struct rvin_group *group,
+					struct media_entity *entity)
+{
+	struct v4l2_subdev *sd;
+	int i;
+
+	if (!is_media_entity_v4l2_subdev(entity))
+		return -ENODEV;
+
+	sd = media_entity_to_v4l2_subdev(entity);
+
+	for (i = 0; i < RVIN_CSI_MAX; i++)
+		if (group->csi[i].subdev == sd)
+			return i;
+
+	return -ENODEV;
+}
+
+/* group lock should be held when calling this function */
+static void __rvin_group_build_link_list(struct rvin_group *group,
+					 struct rvin_group_chsel *map,
+					 int start, int len)
+{
+	struct media_pad *vin_pad, *remote_pad;
+	unsigned int n;
+
+	for (n = 0; n < len; n++) {
+		map[n].csi = -1;
+		map[n].chan = -1;
+
+		if (!group->vin[start + n])
+			continue;
+
+		vin_pad = &group->vin[start + n]->vdev.entity.pads[0];
+
+		remote_pad = media_entity_remote_pad(vin_pad);
+		if (!remote_pad)
+			continue;
+
+		map[n].csi =
+			rvin_group_entity_to_csi_num(group, remote_pad->entity);
+		map[n].chan = rvin_group_csi_pad_to_chan(remote_pad->index);
+	}
+}
+
+/* group lock should be held when calling this function */
+static int __rvin_group_try_get_chsel(struct rvin_group *group,
+				      struct rvin_group_chsel *map,
+				      int start, int len)
+{
+	const struct rvin_group_chsel *sel;
+	unsigned int i, n;
+	int chsel;
+
+	for (i = 0; i < group->vin[start]->info->num_chsels; i++) {
+		chsel = i;
+		for (n = 0; n < len; n++) {
+
+			/* If the link is not active it's OK */
+			if (map[n].csi == -1)
+				continue;
+
+			/* Check if chsel matches requested link */
+			sel = &group->vin[start]->info->chsels[start + n][i];
+			if (map[n].csi != sel->csi ||
+			    map[n].chan != sel->chan) {
+				chsel = -1;
+				break;
+			}
+		}
+
+		/* A chsel which satisfies the links has been found */
+		if (chsel != -1)
+			return chsel;
+	}
+
+	/* No chsel can satisfy the requested links */
+	return -1;
+}
+
+/* group lock should be held when calling this function */
+static bool rvin_group_in_use(struct rvin_group *group)
+{
+	struct media_entity *entity;
+
+	media_device_for_each_entity(entity, &group->mdev)
+		if (entity->use_count)
+			return true;
+
+	return false;
+}
+
+static int rvin_group_link_notify(struct media_link *link, u32 flags,
+				  unsigned int notification)
+{
+	struct rvin_group *group = container_of(link->graph_obj.mdev,
+						struct rvin_group, mdev);
+	struct rvin_group_chsel chsel_map[4];
+	int vin_num, vin_master, csi_num, csi_chan;
+	unsigned int chsel;
+
+	mutex_lock(&group->lock);
+
+	vin_num = rvin_group_entity_to_vin_num(group, link->sink->entity);
+	csi_num = rvin_group_entity_to_csi_num(group, link->source->entity);
+	csi_chan = rvin_group_csi_pad_to_chan(link->source->index);
+
+	/*
+	 * Figure out which VIN node is the subgroup master.
+	 *
+	 * VIN0-3 are controlled by VIN0
+	 * VIN4-7 are controlled by VIN4
+	 */
+	vin_master = vin_num < 4 ? 0 : 4;
+
+	/* If not all devices exist something is horribly wrong */
+	if (vin_num < 0 || csi_num < 0 || !group->vin[vin_master])
+		goto error;
+
+	/* Special checking only needed for links which are to be enabled */
+	if (notification != MEDIA_DEV_NOTIFY_PRE_LINK_CH ||
+	    !(flags & MEDIA_LNK_FL_ENABLED))
+		goto out;
+
+	/* If any link in the group is in use, no new link can be enabled */
+	if (rvin_group_in_use(group))
+		goto error;
+
+	/* If the VIN already has an active link it's busy */
+	if (media_entity_remote_pad(&link->sink->entity->pads[0]))
+		goto error;
+
+	/* Build list of active links */
+	__rvin_group_build_link_list(group, chsel_map, vin_master, 4);
+
+	/* Add the new proposed link */
+	chsel_map[vin_num - vin_master].csi = csi_num;
+	chsel_map[vin_num - vin_master].chan = csi_chan;
+
+	/* See if there is a chsel value which matches our link selection */
+	chsel = __rvin_group_try_get_chsel(group, chsel_map, vin_master, 4);
+
+	/* No chsel can provide the requested links */
+	if (chsel == -1)
+		goto error;
+
+	/* Update chsel value at group master */
+	rvin_set_chsel(group->vin[vin_master], chsel);
+
+out:
+	mutex_unlock(&group->lock);
+
+	return v4l2_pipeline_link_notify(link, flags, notification);
+error:
+	mutex_unlock(&group->lock);
+
+	return -EMLINK;
+}
+
+static const struct media_device_ops rvin_media_ops = {
+	.link_notify = rvin_group_link_notify,
+};
+
 /* -----------------------------------------------------------------------------
  * Gen3 CSI2 Group Allocator
  */
@@ -147,6 +350,8 @@ static int rvin_group_allocate(struct rvin_dev *vin)
 			 dev_name(mdev->dev));
 		media_device_init(mdev);
 
+		mdev->ops = &rvin_media_ops;
+
 		ret = media_device_register(mdev);
 		if (ret) {
 			vin_err(vin, "Failed to register media device\n");
-- 
2.15.0

[PATCH v9 25/28] rcar-vin: extend {start,stop}_streaming to work with media controller

[Niklas Söderlund]

The procedure to start or stop streaming using the non-MC single
subdevice and the MC graph and multiple subdevices are quite different.
Create a new function to abstract which method is used based on which
mode the driver is running in and add logic to start the MC graph.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/rcar-dma.c | 112 +++++++++++++++++++++++++++--
 1 file changed, 105 insertions(+), 7 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index 6c5df13b30d6dd14..8a6674a891aab357 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -1087,15 +1087,115 @@ static void rvin_buffer_queue(struct vb2_buffer *vb)
 	spin_unlock_irqrestore(&vin->qlock, flags);
 }
 
+static int rvin_set_stream(struct rvin_dev *vin, int on)
+{
+	struct v4l2_subdev_format fmt = {
+		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
+	};
+	struct media_pipeline *pipe;
+	struct  v4l2_subdev *sd;
+	struct media_pad *pad;
+	int ret;
+
+	/* No media controller used, simply pass operation to subdevice */
+	if (!vin->info->use_mc) {
+		ret = v4l2_subdev_call(vin->digital->subdev, video, s_stream,
+				       on);
+
+		return ret == -ENOIOCTLCMD ? 0 : ret;
+	}
+
+	pad = media_entity_remote_pad(&vin->pad);
+	if (!pad)
+		return -EPIPE;
+
+	sd = media_entity_to_v4l2_subdev(pad->entity);
+	if (!sd)
+		return -EPIPE;
+
+	if (!on) {
+		media_pipeline_stop(&vin->vdev.entity);
+		return v4l2_subdev_call(sd, video, s_stream, 0);
+	}
+
+	fmt.pad = pad->index;
+	if (v4l2_subdev_call(sd, pad, get_fmt, NULL, &fmt))
+		return -EPIPE;
+
+	switch (fmt.format.code) {
+	case MEDIA_BUS_FMT_YUYV8_1X16:
+	case MEDIA_BUS_FMT_UYVY8_2X8:
+	case MEDIA_BUS_FMT_UYVY10_2X10:
+	case MEDIA_BUS_FMT_RGB888_1X24:
+		vin->code = fmt.format.code;
+		break;
+	default:
+		return -EPIPE;
+	}
+
+	switch (fmt.format.field) {
+	case V4L2_FIELD_TOP:
+	case V4L2_FIELD_BOTTOM:
+	case V4L2_FIELD_NONE:
+	case V4L2_FIELD_INTERLACED_TB:
+	case V4L2_FIELD_INTERLACED_BT:
+	case V4L2_FIELD_INTERLACED:
+	case V4L2_FIELD_SEQ_TB:
+	case V4L2_FIELD_SEQ_BT:
+		/* Supported natively */
+		break;
+	case V4L2_FIELD_ALTERNATE:
+		switch (vin->format.field) {
+		case V4L2_FIELD_TOP:
+		case V4L2_FIELD_BOTTOM:
+		case V4L2_FIELD_NONE:
+			break;
+		case V4L2_FIELD_INTERLACED_TB:
+		case V4L2_FIELD_INTERLACED_BT:
+		case V4L2_FIELD_INTERLACED:
+		case V4L2_FIELD_SEQ_TB:
+		case V4L2_FIELD_SEQ_BT:
+			/* Use VIN hardware to combine the two fields */
+			fmt.format.height *= 2;
+			break;
+		default:
+			return -EPIPE;
+		}
+		break;
+	default:
+		return -EPIPE;
+	}
+
+	if (fmt.format.width != vin->format.width ||
+	    fmt.format.height != vin->format.height)
+		return -EPIPE;
+
+	pipe = sd->entity.pipe ? sd->entity.pipe : &vin->vdev.pipe;
+	if (media_pipeline_start(&vin->vdev.entity, pipe))
+		return -EPIPE;
+
+	ret = v4l2_subdev_call(sd, video, s_stream, 1);
+	if (ret == -ENOIOCTLCMD)
+		ret = 0;
+	if (ret)
+		media_pipeline_stop(&vin->vdev.entity);
+
+	return ret;
+}
+
 static int rvin_start_streaming(struct vb2_queue *vq, unsigned int count)
 {
 	struct rvin_dev *vin = vb2_get_drv_priv(vq);
-	struct v4l2_subdev *sd;
 	unsigned long flags;
 	int ret;
 
-	sd = vin_to_source(vin);
-	v4l2_subdev_call(sd, video, s_stream, 1);
+	ret = rvin_set_stream(vin, 1);
+	if (ret) {
+		spin_lock_irqsave(&vin->qlock, flags);
+		return_all_buffers(vin, VB2_BUF_STATE_QUEUED);
+		spin_unlock_irqrestore(&vin->qlock, flags);
+		return ret;
+	}
 
 	spin_lock_irqsave(&vin->qlock, flags);
 
@@ -1104,7 +1204,7 @@ static int rvin_start_streaming(struct vb2_queue *vq, unsigned int count)
 	ret = rvin_capture_start(vin);
 	if (ret) {
 		return_all_buffers(vin, VB2_BUF_STATE_QUEUED);
-		v4l2_subdev_call(sd, video, s_stream, 0);
+		rvin_set_stream(vin, 0);
 	}
 
 	spin_unlock_irqrestore(&vin->qlock, flags);
@@ -1115,7 +1215,6 @@ static int rvin_start_streaming(struct vb2_queue *vq, unsigned int count)
 static void rvin_stop_streaming(struct vb2_queue *vq)
 {
 	struct rvin_dev *vin = vb2_get_drv_priv(vq);
-	struct v4l2_subdev *sd;
 	unsigned long flags;
 	int retries = 0;
 
@@ -1154,8 +1253,7 @@ static void rvin_stop_streaming(struct vb2_queue *vq)
 
 	spin_unlock_irqrestore(&vin->qlock, flags);
 
-	sd = vin_to_source(vin);
-	v4l2_subdev_call(sd, video, s_stream, 0);
+	rvin_set_stream(vin, 0);
 
 	/* disable interrupts */
 	rvin_disable_interrupts(vin);
-- 
2.15.0

[PATCH v9 26/28] rcar-vin: enable support for r8a7795

[Niklas Söderlund]

Add the SoC specific information for Renesas r8a7795 ES1.x and ES2.0.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/rcar-vin/Kconfig     |   2 +-
 drivers/media/platform/rcar-vin/rcar-core.c | 150 ++++++++++++++++++++++++++++
 2 files changed, 151 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/rcar-vin/Kconfig b/drivers/media/platform/rcar-vin/Kconfig
index af4c98b44d2e22cb..8fa7ee468c63afb9 100644
--- a/drivers/media/platform/rcar-vin/Kconfig
+++ b/drivers/media/platform/rcar-vin/Kconfig
@@ -6,7 +6,7 @@ config VIDEO_RCAR_VIN
 	select V4L2_FWNODE
 	---help---
 	  Support for Renesas R-Car Video Input (VIN) driver.
-	  Supports R-Car Gen2 SoCs.
+	  Supports R-Car Gen2 and Gen3 SoCs.
 
 	  To compile this driver as a module, choose M here: the
 	  module will be called rcar-vin.
diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index 09ebeff1580556dc..66a8144fbba437d3 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -21,6 +21,7 @@
 #include <linux/platform_device.h>
 #include <linux/pm_runtime.h>
 #include <linux/slab.h>
+#include <linux/sys_soc.h>
 
 #include <media/v4l2-async.h>
 #include <media/v4l2-fwnode.h>
@@ -956,6 +957,134 @@ static const struct rvin_info rcar_info_gen2 = {
 	.max_height = 2048,
 };
 
+static const struct rvin_info rcar_info_r8a7795 = {
+	.chip = RCAR_GEN3,
+	.use_mc = true,
+	.max_width = 4096,
+	.max_height = 4096,
+
+	.num_chsels = 5,
+	.chsels = {
+		{
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+		}, {
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+		}, {
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 2 },
+			{ .csi = RVIN_CSI20, .chan = 2 },
+		}, {
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_CSI40, .chan = 3 },
+			{ .csi = RVIN_CSI20, .chan = 3 },
+		}, {
+			{ .csi = RVIN_CSI41, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI41, .chan = 1 },
+			{ .csi = RVIN_CSI41, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+		}, {
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI41, .chan = 1 },
+			{ .csi = RVIN_CSI41, .chan = 0 },
+			{ .csi = RVIN_CSI41, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+		}, {
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_CSI41, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI41, .chan = 2 },
+			{ .csi = RVIN_CSI20, .chan = 2 },
+		}, {
+			{ .csi = RVIN_CSI41, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_CSI41, .chan = 3 },
+			{ .csi = RVIN_CSI20, .chan = 3 },
+		},
+	},
+};
+
+static const struct rvin_info rcar_info_r8a7795es1 = {
+	.chip = RCAR_GEN3,
+	.use_mc = true,
+	.max_width = 4096,
+	.max_height = 4096,
+
+	.num_chsels = 6,
+	.chsels = {
+		{
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI21, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI21, .chan = 0 },
+		}, {
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI21, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_CSI21, .chan = 1 },
+		}, {
+			{ .csi = RVIN_CSI21, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 2 },
+			{ .csi = RVIN_CSI20, .chan = 2 },
+			{ .csi = RVIN_CSI21, .chan = 2 },
+		}, {
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_CSI21, .chan = 1 },
+			{ .csi = RVIN_CSI40, .chan = 3 },
+			{ .csi = RVIN_CSI20, .chan = 3 },
+			{ .csi = RVIN_CSI21, .chan = 3 },
+		}, {
+			{ .csi = RVIN_CSI41, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI21, .chan = 0 },
+			{ .csi = RVIN_CSI41, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI21, .chan = 0 },
+		}, {
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI21, .chan = 0 },
+			{ .csi = RVIN_CSI41, .chan = 0 },
+			{ .csi = RVIN_CSI41, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_CSI21, .chan = 1 },
+		}, {
+			{ .csi = RVIN_CSI21, .chan = 0 },
+			{ .csi = RVIN_CSI41, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI41, .chan = 2 },
+			{ .csi = RVIN_CSI20, .chan = 2 },
+			{ .csi = RVIN_CSI21, .chan = 2 },
+		}, {
+			{ .csi = RVIN_CSI41, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_CSI21, .chan = 1 },
+			{ .csi = RVIN_CSI41, .chan = 3 },
+			{ .csi = RVIN_CSI20, .chan = 3 },
+			{ .csi = RVIN_CSI21, .chan = 3 },
+		},
+	},
+};
+
 static const struct of_device_id rvin_of_id_table[] = {
 	{
 		.compatible = "renesas,vin-r8a7778",
@@ -985,12 +1114,25 @@ static const struct of_device_id rvin_of_id_table[] = {
 		.compatible = "renesas,rcar-gen2-vin",
 		.data = &rcar_info_gen2,
 	},
+	{
+		.compatible = "renesas,vin-r8a7795",
+		.data = &rcar_info_r8a7795,
+	},
 	{ },
 };
 MODULE_DEVICE_TABLE(of, rvin_of_id_table);
 
+static const struct soc_device_attribute r8a7795es1[] = {
+	{
+		.soc_id = "r8a7795", .revision = "ES1.*",
+		.data = &rcar_info_r8a7795es1,
+	},
+	{ /* sentinel */ }
+};
+
 static int rcar_vin_probe(struct platform_device *pdev)
 {
+	const struct soc_device_attribute *attr;
 	struct rvin_dev *vin;
 	struct resource *mem;
 	int irq, ret;
@@ -1002,6 +1144,14 @@ static int rcar_vin_probe(struct platform_device *pdev)
 	vin->dev = &pdev->dev;
 	vin->info = of_device_get_match_data(&pdev->dev);
 
+	/*
+	 * Special care is needed on r8a7795 ES1.x since it
+	 * uses different routing than r8a7795 ES2.0.
+	 */
+	attr = soc_device_match(r8a7795es1);
+	if (attr)
+		vin->info = attr->data;
+
 	mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
 	if (mem == NULL)
 		return -EINVAL;
-- 
2.15.0

[PATCH v9 27/28] rcar-vin: enable support for r8a7796

[Niklas Söderlund]

Add the SoC specific information for Renesas r8a7796.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
Acked-by: Rob Herring <robh@kernel.org>
---
 .../devicetree/bindings/media/rcar_vin.txt         |  1 +
 drivers/media/platform/rcar-vin/rcar-core.c        | 64 ++++++++++++++++++++++
 2 files changed, 65 insertions(+)

diff --git a/Documentation/devicetree/bindings/media/rcar_vin.txt b/Documentation/devicetree/bindings/media/rcar_vin.txt
index 5a95d9668d2c7dfd..314743532bbb4523 100644
--- a/Documentation/devicetree/bindings/media/rcar_vin.txt
+++ b/Documentation/devicetree/bindings/media/rcar_vin.txt
@@ -20,6 +20,7 @@ on Gen3 to a CSI-2 receiver.
    - "renesas,vin-r8a7793" for the R8A7793 device
    - "renesas,vin-r8a7794" for the R8A7794 device
    - "renesas,vin-r8a7795" for the R8A7795 device
+   - "renesas,vin-r8a7796" for the R8A7796 device
    - "renesas,rcar-gen2-vin" for a generic R-Car Gen2 or RZ/G1 compatible
      device.
    - "renesas,rcar-gen3-vin" for a generic R-Car Gen3 compatible device.
diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index 66a8144fbba437d3..ed7fbb58ad6846c1 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -1085,6 +1085,66 @@ static const struct rvin_info rcar_info_r8a7795es1 = {
 	},
 };
 
+static const struct rvin_info rcar_info_r8a7796 = {
+	.chip = RCAR_GEN3,
+	.use_mc = true,
+	.max_width = 4096,
+	.max_height = 4096,
+
+	.num_chsels = 5,
+	.chsels = {
+		{
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_NC, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+		}, {
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_NC, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+		}, {
+			{ .csi = RVIN_NC, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 2 },
+			{ .csi = RVIN_CSI20, .chan = 2 },
+		}, {
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_NC, .chan = 1 },
+			{ .csi = RVIN_CSI40, .chan = 3 },
+			{ .csi = RVIN_CSI20, .chan = 3 },
+		}, {
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_NC, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+		}, {
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_NC, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+		}, {
+			{ .csi = RVIN_NC, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI20, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 2 },
+			{ .csi = RVIN_CSI20, .chan = 2 },
+		}, {
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_CSI20, .chan = 1 },
+			{ .csi = RVIN_NC, .chan = 1 },
+			{ .csi = RVIN_CSI40, .chan = 3 },
+			{ .csi = RVIN_CSI20, .chan = 3 },
+		},
+	},
+};
+
 static const struct of_device_id rvin_of_id_table[] = {
 	{
 		.compatible = "renesas,vin-r8a7778",
@@ -1118,6 +1178,10 @@ static const struct of_device_id rvin_of_id_table[] = {
 		.compatible = "renesas,vin-r8a7795",
 		.data = &rcar_info_r8a7795,
 	},
+	{
+		.compatible = "renesas,vin-r8a7796",
+		.data = &rcar_info_r8a7796,
+	},
 	{ },
 };
 MODULE_DEVICE_TABLE(of, rvin_of_id_table);
-- 
2.15.0

[PATCH v9 28/28] rcar-vin: enable support for r8a77970

[Niklas Söderlund]

Add the SoC specific information for Renesas r8a77970.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Tested-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
Acked-by: Rob Herring <robh@kernel.org>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 .../devicetree/bindings/media/rcar_vin.txt         |  1 +
 drivers/media/platform/rcar-vin/rcar-core.c        | 40 ++++++++++++++++++++++
 2 files changed, 41 insertions(+)

diff --git a/Documentation/devicetree/bindings/media/rcar_vin.txt b/Documentation/devicetree/bindings/media/rcar_vin.txt
index 314743532bbb4523..6b98f8a3398fa493 100644
--- a/Documentation/devicetree/bindings/media/rcar_vin.txt
+++ b/Documentation/devicetree/bindings/media/rcar_vin.txt
@@ -21,6 +21,7 @@ on Gen3 to a CSI-2 receiver.
    - "renesas,vin-r8a7794" for the R8A7794 device
    - "renesas,vin-r8a7795" for the R8A7795 device
    - "renesas,vin-r8a7796" for the R8A7796 device
+   - "renesas,vin-r8a77970" for the R8A77970 device
    - "renesas,rcar-gen2-vin" for a generic R-Car Gen2 or RZ/G1 compatible
      device.
    - "renesas,rcar-gen3-vin" for a generic R-Car Gen3 compatible device.
diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index ed7fbb58ad6846c1..136179ffeebf6862 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -1145,6 +1145,42 @@ static const struct rvin_info rcar_info_r8a7796 = {
 	},
 };
 
+static const struct rvin_info rcar_info_r8a77970 = {
+	.chip = RCAR_GEN3,
+	.use_mc = true,
+	.max_width = 4096,
+	.max_height = 4096,
+
+	.num_chsels = 5,
+	.chsels = {
+		{
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_NC, .chan = 0 },
+			{ .csi = RVIN_NC, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_NC, .chan = 0 },
+		}, {
+			{ .csi = RVIN_NC, .chan = 0 },
+			{ .csi = RVIN_NC, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_NC, .chan = 0 },
+		}, {
+			{ .csi = RVIN_NC, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 0 },
+			{ .csi = RVIN_NC, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 2 },
+			{ .csi = RVIN_NC, .chan = 0 },
+		}, {
+			{ .csi = RVIN_CSI40, .chan = 1 },
+			{ .csi = RVIN_NC, .chan = 0 },
+			{ .csi = RVIN_NC, .chan = 0 },
+			{ .csi = RVIN_CSI40, .chan = 3 },
+			{ .csi = RVIN_NC, .chan = 0 },
+		},
+	},
+};
+
 static const struct of_device_id rvin_of_id_table[] = {
 	{
 		.compatible = "renesas,vin-r8a7778",
@@ -1182,6 +1218,10 @@ static const struct of_device_id rvin_of_id_table[] = {
 		.compatible = "renesas,vin-r8a7796",
 		.data = &rcar_info_r8a7796,
 	},
+	{
+		.compatible = "renesas,vin-r8a77970",
+		.data = &rcar_info_r8a77970,
+	},
 	{ },
 };
 MODULE_DEVICE_TABLE(of, rvin_of_id_table);
-- 
2.15.0

Re: [PATCH v9 01/28] rcar-vin: add Gen3 devicetree bindings documentation

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:15 EET Niklas Söderlund wrote:
> Document the devicetree bindings for the CSI-2 inputs available on Gen3.
> 
> There is a need to add a custom property 'renesas,id' and to define
> which CSI-2 input is described in which endpoint under the port@1 node.
> This information is needed since there are a set of predefined routes
> between each VIN and CSI-2 block. This routing table will be kept
> inside the driver but in order for it to act on it it must know which
> VIN and CSI-2 is which.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Acked-by: Rob Herring <robh@kernel.org>
> ---
>  .../devicetree/bindings/media/rcar_vin.txt         | 116 +++++++++++++++---
>  1 file changed, 104 insertions(+), 12 deletions(-)
> 
> diff --git a/Documentation/devicetree/bindings/media/rcar_vin.txt
> b/Documentation/devicetree/bindings/media/rcar_vin.txt index
> ff9697ed81396e64..5a95d9668d2c7dfd 100644
> --- a/Documentation/devicetree/bindings/media/rcar_vin.txt
> +++ b/Documentation/devicetree/bindings/media/rcar_vin.txt
> @@ -2,8 +2,12 @@ Renesas R-Car Video Input driver (rcar_vin)
>  -------------------------------------------
> 
>  The rcar_vin device provides video input capabilities for the Renesas R-Car
> -family of devices. The current blocks are always slaves and suppot one
> input
> -channel which can be either RGB, YUYV or BT656.
> +family of devices.
> +
> +Each VIN instance has a single parallel input that supports RGB and YUV
> video,
> +with both external synchronization and BT.656 synchronization for the
> latter.
> +Depending on the instance the VIN input is connected to external SoC pins,
> or
> +on Gen3 to a CSI-2 receiver.
> 
>   - compatible: Must be one or more of the following
>     - "renesas,vin-r8a7743" for the R8A7743 device
> @@ -31,21 +35,38 @@ channel which can be either RGB, YUYV or BT656.
>  Additionally, an alias named vinX will need to be created to specify
>  which video input device this is.
> 
> -The per-board settings:
> +The per-board settings Gen2:

Nitpicking, s/Gen2/for Gen2 platforms/

(or Gen2 hardware, or Gen2 systems, pick the one you like best)

>   - port sub-node describing a single endpoint connected to the vin
>     as described in video-interfaces.txt[1]. Only the first one will
>     be considered as each vin interface has one input port.
> 
> -   These settings are used to work out video input format and widths
> -   into the system.
> +The per-board settings Gen3:

Ditto.

> +
> +Gen3 can support both a single connected parallel input source from
> +external SoC pins (port0) and/or multiple parallel input sources from
> +local SoC CSI-2 receivers (port1) depending on SoC.
> 
> +- renesas,id - ID number of the VIN, VINx in the documentation.
> +- ports
> +    - port0 - sub-node describing a single endpoint connected to the VIN
> +      from external SoC pins described in video-interfaces.txt[1]. Only
> +      the first one will be considered as each VIN interface has at most
> +      one set of SoC external input pins.

s/port0/port 0/ or s/port0/port@0/

I'd go further than that and make it invalid to have multiple endpoints 
instead of ignoring all but the first one.

I would also explicitly state that VIN instances not connected to external 
pins shall have no port 0.

> +    - port1 - sub-nodes describing one or more endpoints connected to
> +      the VIN from local SoC CSI-2 receivers. The endpoint numbers must
> +      use the following schema.

Nitpicking again, the Gen2-specific properties are indented above while the 
Gen3 properties are not indented here. Pick the one you prefer :-)

> -Device node example
> --------------------
> +        - Endpoint 0 - sub-node describing the endpoint which is CSI20
> +        - Endpoint 1 - sub-node describing the endpoint which is CSI21
> +        - Endpoint 2 - sub-node describing the endpoint which is CSI40
> +        - Endpoint 3 - sub-node describing the endpoint which is CSI41

How about s/which is/connected to/ ?

> -	aliases {
> -	       vin0 = &vin0;
> -	};
> +Device node example Gen2

s/Gen2/for Gen2 platforms/

and same in a few places below.

> +------------------------
> +
> +        aliases {
> +                vin0 = &vin0;
> +        };

This is unrelated, but do we need aliases ?

>          vin0: vin@0xe6ef0000 {
>                  compatible = "renesas,vin-r8a7790",
> "renesas,rcar-gen2-vin"; @@ -55,8 +76,8 @@ Device node example
>                  status = "disabled";
>          };
> 
> -Board setup example (vin1 composite video input)
> -------------------------------------------------
> +Board setup example Gen2 (vin1 composite video input)
> +-----------------------------------------------------
> 
>  &i2c2   {
>          status = "ok";
> @@ -95,6 +116,77 @@ Board setup example (vin1 composite video input)
>          };
>  };
> 
> +Device node example Gen3
> +------------------------
> +
> +        vin0: video@e6ef0000 {
> +                compatible = "renesas,vin-r8a7795";
> +                reg = <0 0xe6ef0000 0 0x1000>;
> +                interrupts = <GIC_SPI 188 IRQ_TYPE_LEVEL_HIGH>;
> +                clocks = <&cpg CPG_MOD 811>;
> +                power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
> +                resets = <&cpg 811>;
> +                renesas,id = <0>;
> +
> +                ports {
> +                        #address-cells = <1>;
> +                        #size-cells = <0>;
> +
> +                        port@1 {
> +                                #address-cells = <1>;
> +                                #size-cells = <0>;
> +
> +                                reg = <1>;
> +
> +                                vin0csi20: endpoint@0 {
> +                                        reg = <0>;
> +                                        remote-endpoint= <&csi20vin0>;
> +                                };
> +                                vin0csi21: endpoint@1 {
> +                                        reg = <1>;
> +                                        remote-endpoint= <&csi21vin0>;
> +                                };
> +                                vin0csi40: endpoint@2 {
> +                                        reg = <2>;
> +                                        remote-endpoint= <&csi40vin0>;
> +                                };
> +                        };
> +                };
> +        };
> +
> +        csi20: csi2@fea80000 {
> +                compatible = "renesas,r8a7795-csi2";
> +                reg = <0 0xfea80000 0 0x10000>;
> +                interrupts = <GIC_SPI 184 IRQ_TYPE_LEVEL_HIGH>;
> +                clocks = <&cpg CPG_MOD 714>;
> +                power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
> +                resets = <&cpg 714>;
> +
> +                ports {
> +                        #address-cells = <1>;
> +                        #size-cells = <0>;
> +
> +                        port@0 {
> +                                reg = <0>;
> +                                csi20_in: endpoint {
> +                                        clock-lanes = <0>;
> +                                        data-lanes = <1>;
> +                                        remote-endpoint = <&adv7482_txb>;
> +                                };
> +                        };
> +
> +                        port@1 {
> +                                #address-cells = <1>;
> +                                #size-cells = <0>;
> 
> +                                reg = <1>;
> +
> +                                csi20vin0: endpoint@0 {
> +                                        reg = <0>;
> +                                        remote-endpoint = <&vin0csi20>;
> +                                };
> +                        };
> +                };
> +        };
> 
>  [1] video-interfaces.txt common video media interface

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 02/28] rcar-vin: rename poorly named initialize and cleanup functions

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:16 EET Niklas Söderlund wrote:
> The functions to initialize and cleanup the hardware and video device
> where poorly named from the start. Rename them to better describe their
> intended function.

It's interesting that you describe the functions' purpose as initialize and 
cleanup here and name them register and unregister :-) It's not a big deal, 
but you might want some consistency between the commit message and the code.

> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Either way,

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  drivers/media/platform/rcar-vin/rcar-core.c | 10 +++++-----
>  drivers/media/platform/rcar-vin/rcar-dma.c  |  6 +++---
>  drivers/media/platform/rcar-vin/rcar-v4l2.c |  4 ++--
>  drivers/media/platform/rcar-vin/rcar-vin.h  |  8 ++++----
>  4 files changed, 14 insertions(+), 14 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
> b/drivers/media/platform/rcar-vin/rcar-core.c index
> 108d776f32651b27..f7a4c21909da6923 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -93,7 +93,7 @@ static int rvin_digital_notify_complete(struct
> v4l2_async_notifier *notifier) return ret;
>  	}
> 
> -	return rvin_v4l2_probe(vin);
> +	return rvin_v4l2_register(vin);
>  }
> 
>  static void rvin_digital_notify_unbind(struct v4l2_async_notifier
> *notifier, @@ -103,7 +103,7 @@ static void
> rvin_digital_notify_unbind(struct v4l2_async_notifier *notifier, struct
> rvin_dev *vin = notifier_to_vin(notifier);
> 
>  	vin_dbg(vin, "unbind digital subdev %s\n", subdev->name);
> -	rvin_v4l2_remove(vin);
> +	rvin_v4l2_unregister(vin);
>  	vin->digital->subdev = NULL;
>  }
> 
> @@ -245,7 +245,7 @@ static int rcar_vin_probe(struct platform_device *pdev)
>  	if (irq < 0)
>  		return irq;
> 
> -	ret = rvin_dma_probe(vin, irq);
> +	ret = rvin_dma_register(vin, irq);
>  	if (ret)
>  		return ret;
> 
> @@ -260,7 +260,7 @@ static int rcar_vin_probe(struct platform_device *pdev)
> 
>  	return 0;
>  error:
> -	rvin_dma_remove(vin);
> +	rvin_dma_unregister(vin);
>  	v4l2_async_notifier_cleanup(&vin->notifier);
> 
>  	return ret;
> @@ -275,7 +275,7 @@ static int rcar_vin_remove(struct platform_device *pdev)
> v4l2_async_notifier_unregister(&vin->notifier);
>  	v4l2_async_notifier_cleanup(&vin->notifier);
> 
> -	rvin_dma_remove(vin);
> +	rvin_dma_unregister(vin);
> 
>  	return 0;
>  }
> diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> b/drivers/media/platform/rcar-vin/rcar-dma.c index
> 23fdff7a7370842e..d701b52d198243b5 100644
> --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> @@ -1153,14 +1153,14 @@ static const struct vb2_ops rvin_qops = {
>  	.wait_finish		= vb2_ops_wait_finish,
>  };
> 
> -void rvin_dma_remove(struct rvin_dev *vin)
> +void rvin_dma_unregister(struct rvin_dev *vin)
>  {
>  	mutex_destroy(&vin->lock);
> 
>  	v4l2_device_unregister(&vin->v4l2_dev);
>  }
> 
> -int rvin_dma_probe(struct rvin_dev *vin, int irq)
> +int rvin_dma_register(struct rvin_dev *vin, int irq)
>  {
>  	struct vb2_queue *q = &vin->queue;
>  	int i, ret;
> @@ -1208,7 +1208,7 @@ int rvin_dma_probe(struct rvin_dev *vin, int irq)
> 
>  	return 0;
>  error:
> -	rvin_dma_remove(vin);
> +	rvin_dma_unregister(vin);
> 
>  	return ret;
>  }
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> b479b882da12f62d..178aecc94962abe2 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -839,7 +839,7 @@ static const struct v4l2_file_operations rvin_fops = {
>  	.read		= vb2_fop_read,
>  };
> 
> -void rvin_v4l2_remove(struct rvin_dev *vin)
> +void rvin_v4l2_unregister(struct rvin_dev *vin)
>  {
>  	v4l2_info(&vin->v4l2_dev, "Removing %s\n",
>  		  video_device_node_name(&vin->vdev));
> @@ -866,7 +866,7 @@ static void rvin_notify(struct v4l2_subdev *sd,
>  	}
>  }
> 
> -int rvin_v4l2_probe(struct rvin_dev *vin)
> +int rvin_v4l2_register(struct rvin_dev *vin)
>  {
>  	struct video_device *vdev = &vin->vdev;
>  	struct v4l2_subdev *sd = vin_to_source(vin);
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> b/drivers/media/platform/rcar-vin/rcar-vin.h index
> 5382078143fb3869..85cb7ec53d2b08b5 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -153,11 +153,11 @@ struct rvin_dev {
>  #define vin_warn(d, fmt, arg...)	dev_warn(d->dev, fmt, ##arg)
>  #define vin_err(d, fmt, arg...)		dev_err(d->dev, fmt, ##arg)
> 
> -int rvin_dma_probe(struct rvin_dev *vin, int irq);
> -void rvin_dma_remove(struct rvin_dev *vin);
> +int rvin_dma_register(struct rvin_dev *vin, int irq);
> +void rvin_dma_unregister(struct rvin_dev *vin);
> 
> -int rvin_v4l2_probe(struct rvin_dev *vin);
> -void rvin_v4l2_remove(struct rvin_dev *vin);
> +int rvin_v4l2_register(struct rvin_dev *vin);
> +void rvin_v4l2_unregister(struct rvin_dev *vin);
> 
>  const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 03/28] rcar-vin: unregister video device on driver removal

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:17 EET Niklas Söderlund wrote:
> If the video device was registered by the complete() callback it should
> be unregistered when the driver is removed.

The .remove() operation indicates device removal, not driver removal (or, the 
be more precise, it indicates that the device is unbound from the driver). I'd 
update the commit message accordingly.

> Protect from printing an uninitialized video device node name by adding a
> check in rvin_v4l2_unregister() to identify that the video device is
> registered.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/rcar-vin/rcar-core.c | 2 ++
>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 +++
>  2 files changed, 5 insertions(+)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
> b/drivers/media/platform/rcar-vin/rcar-core.c index
> f7a4c21909da6923..6d99542ec74b49a7 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -272,6 +272,8 @@ static int rcar_vin_remove(struct platform_device *pdev)
> 
>  	pm_runtime_disable(&pdev->dev);
> 
> +	rvin_v4l2_unregister(vin);

Unless I'm mistaken, you're unregistering the video device both here and in 
the unbound() function. That's messy, but it's not really your fault, the V4L2 
core is very messy in the first place, and registering video devices in the 
complete() handler is a bad idea. As that can't be fixed for now,

Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Hans, I still would like to hear your opinion on how this should be solved. 
You've voiced a few weeks ago that register video devices at probe() time 
isn't a good idea but you've never explained how we should fix the problem. I 
still firmly believe that video devices should be registered at probe time, 
and we need to reach an agreement on a technical solution to this problem.

>  	v4l2_async_notifier_unregister(&vin->notifier);
>  	v4l2_async_notifier_cleanup(&vin->notifier);
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> 178aecc94962abe2..32a658214f48fa49 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -841,6 +841,9 @@ static const struct v4l2_file_operations rvin_fops = {
> 
>  void rvin_v4l2_unregister(struct rvin_dev *vin)
>  {
> +	if (!video_is_registered(&vin->vdev))
> +		return;
> +
>  	v4l2_info(&vin->v4l2_dev, "Removing %s\n",
>  		  video_device_node_name(&vin->vdev));

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 04/28] rcar-vin: move subdevice handling to async callbacks

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:18 EET Niklas Söderlund wrote:
> In preparation for Gen3 support move the subdevice initialization and
> clean up from rvin_v4l2_{register,unregister}() directly to the async
> callbacks. This simplifies the addition of Gen3 support as the
> rvin_v4l2_register() can be shared for both Gen2 and Gen3 while direct
> subdevice control are only used on Gen2.
> 
> While moving this code drop a large comment which is copied from the
> framework documentation and fold rvin_mbus_supported() into its only
> caller.

I'd really move the initialization and cleanup code to two separate functions, 
it's getting hard to read. This is especially true for the initialization 
code, but I'd do the same for the cleanup code as well even if it's just a 
matter of calling v4l2_ctrl_handler_free().

> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/rcar-vin/rcar-core.c | 105 ++++++++++++++++--------
>  drivers/media/platform/rcar-vin/rcar-v4l2.c |  35 ----------
>  2 files changed, 67 insertions(+), 73 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
> b/drivers/media/platform/rcar-vin/rcar-core.c index
> 6d99542ec74b49a7..6ab51acd676641ec 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -46,47 +46,11 @@ static int rvin_find_pad(struct v4l2_subdev *sd, int
> direction) return -EINVAL;
>  }
> 
> -static bool rvin_mbus_supported(struct rvin_graph_entity *entity)
> -{
> -	struct v4l2_subdev *sd = entity->subdev;
> -	struct v4l2_subdev_mbus_code_enum code = {
> -		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
> -	};
> -
> -	code.index = 0;
> -	code.pad = entity->source_pad;
> -	while (!v4l2_subdev_call(sd, pad, enum_mbus_code, NULL, &code)) {
> -		code.index++;
> -		switch (code.code) {
> -		case MEDIA_BUS_FMT_YUYV8_1X16:
> -		case MEDIA_BUS_FMT_UYVY8_2X8:
> -		case MEDIA_BUS_FMT_UYVY10_2X10:
> -		case MEDIA_BUS_FMT_RGB888_1X24:
> -			entity->code = code.code;
> -			return true;
> -		default:
> -			break;
> -		}
> -	}
> -
> -	return false;
> -}
> -
>  static int rvin_digital_notify_complete(struct v4l2_async_notifier
> *notifier) {
>  	struct rvin_dev *vin = notifier_to_vin(notifier);
>  	int ret;
> 
> -	/* Verify subdevices mbus format */
> -	if (!rvin_mbus_supported(vin->digital)) {
> -		vin_err(vin, "Unsupported media bus format for %s\n",
> -			vin->digital->subdev->name);
> -		return -EINVAL;
> -	}
> -
> -	vin_dbg(vin, "Found media bus format for %s: %d\n",
> -		vin->digital->subdev->name, vin->digital->code);
> -
>  	ret = v4l2_device_register_subdev_nodes(&vin->v4l2_dev);
>  	if (ret < 0) {
>  		vin_err(vin, "Failed to register subdev nodes\n");
> @@ -103,8 +67,16 @@ static void rvin_digital_notify_unbind(struct
> v4l2_async_notifier *notifier, struct rvin_dev *vin =
> notifier_to_vin(notifier);
> 
>  	vin_dbg(vin, "unbind digital subdev %s\n", subdev->name);
> +
> +	mutex_lock(&vin->lock);
> +
>  	rvin_v4l2_unregister(vin);
> +	v4l2_ctrl_handler_free(&vin->ctrl_handler);
> +
> +	vin->vdev.ctrl_handler = NULL;
>  	vin->digital->subdev = NULL;
> +
> +	mutex_unlock(&vin->lock);
>  }
> 
>  static int rvin_digital_notify_bound(struct v4l2_async_notifier *notifier,
> @@ -112,12 +84,14 @@ static int rvin_digital_notify_bound(struct
> v4l2_async_notifier *notifier, struct v4l2_async_subdev *asd)
>  {
>  	struct rvin_dev *vin = notifier_to_vin(notifier);
> +	struct v4l2_subdev_mbus_code_enum code = {
> +		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
> +	};
>  	int ret;
> 
>  	v4l2_set_subdev_hostdata(subdev, vin);
> 
>  	/* Find source and sink pad of remote subdevice */
> -
>  	ret = rvin_find_pad(subdev, MEDIA_PAD_FL_SOURCE);
>  	if (ret < 0)
>  		return ret;
> @@ -126,21 +100,74 @@ static int rvin_digital_notify_bound(struct
> v4l2_async_notifier *notifier, ret = rvin_find_pad(subdev,
> MEDIA_PAD_FL_SINK);
>  	vin->digital->sink_pad = ret < 0 ? 0 : ret;
> 
> +	/* Find compatible subdevices mbus format */
> +	vin->digital->code = 0;
> +	code.index = 0;
> +	code.pad = vin->digital->source_pad;
> +	while (!vin->digital->code &&
> +	       !v4l2_subdev_call(subdev, pad, enum_mbus_code, NULL, &code)) {
> +		code.index++;
> +		switch (code.code) {
> +		case MEDIA_BUS_FMT_YUYV8_1X16:
> +		case MEDIA_BUS_FMT_UYVY8_2X8:
> +		case MEDIA_BUS_FMT_UYVY10_2X10:
> +		case MEDIA_BUS_FMT_RGB888_1X24:
> +			vin->digital->code = code.code;
> +			vin_dbg(vin, "Found media bus format for %s: %d\n",
> +				subdev->name, vin->digital->code);
> +			break;
> +		default:
> +			break;
> +		}
> +	}
> +
> +	if (!vin->digital->code) {
> +		vin_err(vin, "Unsupported media bus format for %s\n",
> +			subdev->name);
> +		return -EINVAL;
> +	}
> +
> +	/* Read tvnorms */
> +	ret = v4l2_subdev_call(subdev, video, g_tvnorms, &vin->vdev.tvnorms);
> +	if (ret < 0 && ret != -ENOIOCTLCMD && ret != -ENODEV)
> +		return ret;
> +
> +	mutex_lock(&vin->lock);
> +
> +	/* Add the controls */
> +	ret = v4l2_ctrl_handler_init(&vin->ctrl_handler, 16);
> +	if (ret < 0)
> +		goto err;
> +
> +	ret = v4l2_ctrl_add_handler(&vin->ctrl_handler, subdev->ctrl_handler,
> +				    NULL);
> +	if (ret < 0)
> +		goto err_ctrl;
> +
> +	vin->vdev.ctrl_handler = &vin->ctrl_handler;
> +
>  	vin->digital->subdev = subdev;
> 
> +	mutex_unlock(&vin->lock);
> +
>  	vin_dbg(vin, "bound subdev %s source pad: %u sink pad: %u\n",
>  		subdev->name, vin->digital->source_pad,
>  		vin->digital->sink_pad);
> 
>  	return 0;
> +err_ctrl:
> +	v4l2_ctrl_handler_free(&vin->ctrl_handler);
> +err:
> +	mutex_unlock(&vin->lock);
> +	return ret;
>  }
> +
>  static const struct v4l2_async_notifier_operations rvin_digital_notify_ops
> = { .bound = rvin_digital_notify_bound,
>  	.unbind = rvin_digital_notify_unbind,
>  	.complete = rvin_digital_notify_complete,
>  };
> 
> -
>  static int rvin_digital_parse_v4l2(struct device *dev,
>  				   struct v4l2_fwnode_endpoint *vep,
>  				   struct v4l2_async_subdev *asd)
> @@ -277,6 +304,8 @@ static int rcar_vin_remove(struct platform_device *pdev)
> v4l2_async_notifier_unregister(&vin->notifier);
>  	v4l2_async_notifier_cleanup(&vin->notifier);
> 
> +	v4l2_ctrl_handler_free(&vin->ctrl_handler);
> +
>  	rvin_dma_unregister(vin);
> 
>  	return 0;
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> 32a658214f48fa49..4a0610a6b4503501 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -847,9 +847,6 @@ void rvin_v4l2_unregister(struct rvin_dev *vin)
>  	v4l2_info(&vin->v4l2_dev, "Removing %s\n",
>  		  video_device_node_name(&vin->vdev));
> 
> -	/* Checks internaly if handlers have been init or not */
> -	v4l2_ctrl_handler_free(&vin->ctrl_handler);
> -
>  	/* Checks internaly if vdev have been init or not */
>  	video_unregister_device(&vin->vdev);
>  }
> @@ -872,41 +869,10 @@ static void rvin_notify(struct v4l2_subdev *sd,
>  int rvin_v4l2_register(struct rvin_dev *vin)
>  {
>  	struct video_device *vdev = &vin->vdev;
> -	struct v4l2_subdev *sd = vin_to_source(vin);
>  	int ret;
> 
> -	v4l2_set_subdev_hostdata(sd, vin);
> -
>  	vin->v4l2_dev.notify = rvin_notify;
> 
> -	ret = v4l2_subdev_call(sd, video, g_tvnorms, &vin->vdev.tvnorms);
> -	if (ret < 0 && ret != -ENOIOCTLCMD && ret != -ENODEV)
> -		return ret;
> -
> -	if (vin->vdev.tvnorms == 0) {
> -		/* Disable the STD API if there are no tvnorms defined */
> -		v4l2_disable_ioctl(&vin->vdev, VIDIOC_G_STD);
> -		v4l2_disable_ioctl(&vin->vdev, VIDIOC_S_STD);
> -		v4l2_disable_ioctl(&vin->vdev, VIDIOC_QUERYSTD);
> -		v4l2_disable_ioctl(&vin->vdev, VIDIOC_ENUMSTD);
> -	}
> -
> -	/* Add the controls */
> -	/*
> -	 * Currently the subdev with the largest number of controls (13) is
> -	 * ov6550. So let's pick 16 as a hint for the control handler. Note
> -	 * that this is a hint only: too large and you waste some memory, too
> -	 * small and there is a (very) small performance hit when looking up
> -	 * controls in the internal hash.
> -	 */
> -	ret = v4l2_ctrl_handler_init(&vin->ctrl_handler, 16);
> -	if (ret < 0)
> -		return ret;
> -
> -	ret = v4l2_ctrl_add_handler(&vin->ctrl_handler, sd->ctrl_handler, NULL);
> -	if (ret < 0)
> -		return ret;
> -
>  	/* video node */
>  	vdev->fops = &rvin_fops;
>  	vdev->v4l2_dev = &vin->v4l2_dev;
> @@ -915,7 +881,6 @@ int rvin_v4l2_register(struct rvin_dev *vin)
>  	vdev->release = video_device_release_empty;
>  	vdev->ioctl_ops = &rvin_ioctl_ops;
>  	vdev->lock = &vin->lock;
> -	vdev->ctrl_handler = &vin->ctrl_handler;
>  	vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING |
>  		V4L2_CAP_READWRITE;


-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 05/28] rcar-vin: move chip information to own struct

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:19 EET Niklas Söderlund wrote:
> When Gen3 support is added to the driver more than chip ID will be
> different for the different SoCs. To avoid a lot of if statements in the
> code create a struct chip_info to store this information.

The structure is called rvin_info.

> And while we are at it sort the compatible string entries and make use
> of of_device_get_match_data() which will always work as the driver is DT
> only, so there's always a valid match.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/rcar-vin/rcar-core.c | 54 +++++++++++++++++++++-----
>  drivers/media/platform/rcar-vin/rcar-v4l2.c |  3 +-
>  drivers/media/platform/rcar-vin/rcar-vin.h  | 12 +++++--
>  3 files changed, 53 insertions(+), 16 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
> b/drivers/media/platform/rcar-vin/rcar-core.c index
> 6ab51acd676641ec..73c1700a409bfd35 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -230,21 +230,53 @@ static int rvin_digital_graph_init(struct rvin_dev
> *vin) * Platform Device Driver
>   */
> 
> +static const struct rvin_info rcar_info_h1 = {
> +	.chip = RCAR_H1,
> +};
> +
> +static const struct rvin_info rcar_info_m1 = {
> +	.chip = RCAR_M1,
> +};
> +
> +static const struct rvin_info rcar_info_gen2 = {
> +	.chip = RCAR_GEN2,
> +};
> +
>  static const struct of_device_id rvin_of_id_table[] = {
> -	{ .compatible = "renesas,vin-r8a7794", .data = (void *)RCAR_GEN2 },
> -	{ .compatible = "renesas,vin-r8a7793", .data = (void *)RCAR_GEN2 },
> -	{ .compatible = "renesas,vin-r8a7791", .data = (void *)RCAR_GEN2 },
> -	{ .compatible = "renesas,vin-r8a7790", .data = (void *)RCAR_GEN2 },
> -	{ .compatible = "renesas,vin-r8a7779", .data = (void *)RCAR_H1 },
> -	{ .compatible = "renesas,vin-r8a7778", .data = (void *)RCAR_M1 },
> -	{ .compatible = "renesas,rcar-gen2-vin", .data = (void *)RCAR_GEN2 },
> +	{
> +		.compatible = "renesas,vin-r8a7778",
> +		.data = &rcar_info_m1,
> +	},
> +	{
> +		.compatible = "renesas,vin-r8a7779",
> +		.data = &rcar_info_h1,
> +	},
> +	{
> +		.compatible = "renesas,vin-r8a7790",
> +		.data = &rcar_info_gen2,
> +	},
> +	{
> +		.compatible = "renesas,vin-r8a7791",
> +		.data = &rcar_info_gen2,
> +	},
> +	{
> +		.compatible = "renesas,vin-r8a7793",
> +		.data = &rcar_info_gen2,
> +	},
> +	{
> +		.compatible = "renesas,vin-r8a7794",
> +		.data = &rcar_info_gen2,
> +	},
> +	{
> +		.compatible = "renesas,rcar-gen2-vin",
> +		.data = &rcar_info_gen2,
> +	},
>  	{ },

{ /* Sentinel */ } is usually used to emphasize the need of an empty entry.

>  };
>  MODULE_DEVICE_TABLE(of, rvin_of_id_table);
> 
>  static int rcar_vin_probe(struct platform_device *pdev)
>  {
> -	const struct of_device_id *match;
>  	struct rvin_dev *vin;
>  	struct resource *mem;
>  	int irq, ret;
> @@ -253,12 +285,8 @@ static int rcar_vin_probe(struct platform_device *pdev)
> if (!vin)
>  		return -ENOMEM;
> 
> -	match = of_match_device(of_match_ptr(rvin_of_id_table), &pdev->dev);
> -	if (!match)
> -		return -ENODEV;
> -
>  	vin->dev = &pdev->dev;
> -	vin->chip = (enum chip_id)match->data;
> +	vin->info = of_device_get_match_data(&pdev->dev);
> 
>  	mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
>  	if (mem == NULL)
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> 4a0610a6b4503501..b1caa04921aa23bb 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -266,7 +266,8 @@ static int __rvin_try_format(struct rvin_dev *vin,
>  	pix->sizeimage = max_t(u32, pix->sizeimage,
>  			       rvin_format_sizeimage(pix));
> 
> -	if (vin->chip == RCAR_M1 && pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
> +	if (vin->info->chip == RCAR_M1 &&
> +	    pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
>  		vin_err(vin, "pixel format XBGR32 not supported on M1\n");
>  		return -EINVAL;
>  	}
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> b/drivers/media/platform/rcar-vin/rcar-vin.h index
> 85cb7ec53d2b08b5..0d3949c8c08c8f63 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -88,11 +88,19 @@ struct rvin_graph_entity {
>  	unsigned int sink_pad;
>  };
> 
> +/**
> + * struct rvin_info - Information about the particular VIN implementation
> + * @chip:		type of VIN chip

While at it, how about also renaming "chip" to something more appropriate, as 
the VIN isn't a chip ? Maybe "model" ?

With this fixed,

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> + */
> +struct rvin_info {
> +	enum chip_id chip;
> +};
> +
>  /**
>   * struct rvin_dev - Renesas VIN device structure
>   * @dev:		(OF) device
>   * @base:		device I/O register space remapped to virtual memory
> - * @chip:		type of VIN chip
> + * @info:		info about VIN instance
>   *
>   * @vdev:		V4L2 video device associated with VIN
>   * @v4l2_dev:		V4L2 device
> @@ -120,7 +128,7 @@ struct rvin_graph_entity {
>  struct rvin_dev {
>  	struct device *dev;
>  	void __iomem *base;
> -	enum chip_id chip;
> +	const struct rvin_info *info;
> 
>  	struct video_device vdev;
>  	struct v4l2_device v4l2_dev;

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 06/28] rcar-vin: move max width and height information to chip information

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:20 EET Niklas Söderlund wrote:
> On Gen3 the max supported width and height will be different from Gen2.
> Move the limits to the struct rvin_info to prepare for Gen3 support.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/rcar-vin/rcar-core.c | 6 ++++++
>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 6 ++----
>  drivers/media/platform/rcar-vin/rcar-vin.h  | 6 ++++++
>  3 files changed, 14 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
> b/drivers/media/platform/rcar-vin/rcar-core.c index
> 73c1700a409bfd35..03d3cd63e38bee11 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -232,14 +232,20 @@ static int rvin_digital_graph_init(struct rvin_dev
> *vin)
> 
>  static const struct rvin_info rcar_info_h1 = {
>  	.chip = RCAR_H1,
> +	.max_width = 2048,
> +	.max_height = 2048,
>  };
> 
>  static const struct rvin_info rcar_info_m1 = {
>  	.chip = RCAR_M1,
> +	.max_width = 2048,
> +	.max_height = 2048,
>  };
> 
>  static const struct rvin_info rcar_info_gen2 = {
>  	.chip = RCAR_GEN2,
> +	.max_width = 2048,
> +	.max_height = 2048,
>  };
> 
>  static const struct of_device_id rvin_of_id_table[] = {
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> b1caa04921aa23bb..59ec6d3d119590aa 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -23,8 +23,6 @@
>  #include "rcar-vin.h"
> 
>  #define RVIN_DEFAULT_FORMAT	V4L2_PIX_FMT_YUYV
> -#define RVIN_MAX_WIDTH		2048
> -#define RVIN_MAX_HEIGHT		2048
> 
>  /*
> ---------------------------------------------------------------------------
> -- * Format Conversions
> @@ -258,8 +256,8 @@ static int __rvin_try_format(struct rvin_dev *vin,
>  	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
> 
>  	/* Limit to VIN capabilities */
> -	v4l_bound_align_image(&pix->width, 2, RVIN_MAX_WIDTH, walign,
> -			      &pix->height, 4, RVIN_MAX_HEIGHT, 2, 0);
> +	v4l_bound_align_image(&pix->width, 2, vin->info->max_width, walign,
> +			      &pix->height, 4, vin->info->max_height, 2, 0);
> 
>  	pix->bytesperline = max_t(u32, pix->bytesperline,
>  				  rvin_format_bytesperline(pix));
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> b/drivers/media/platform/rcar-vin/rcar-vin.h index
> 0d3949c8c08c8f63..646f897f5c05ec4e 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -91,9 +91,15 @@ struct rvin_graph_entity {
>  /**
>   * struct rvin_info - Information about the particular VIN implementation
>   * @chip:		type of VIN chip
> + *

Nitpicking, there's no need for a blank line here.

> + * max_width:		max input width the VIN supports
> + * max_height:		max input height the VIN supports

And you're missing the @ before the field names. Please make sure to compile 
the documentation to check for kerneldoc errors.

With this fixed,

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

>   */
>  struct rvin_info {
>  	enum chip_id chip;
> +
> +	unsigned int max_width;
> +	unsigned int max_height;
>  };
> 
>  /**

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 07/28] rcar-vin: change name of video device

[Laurent Pinchart]

Hi Niklas,

(CC'ing Sakari)

Thank you for the patch.

On Friday, 8 December 2017 03:08:21 EET Niklas Söderlund wrote:
> The rcar-vin driver needs to be part of a media controller to support
> Gen3. Give each VIN instance a unique name so it can be referenced from
> userspace.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> 59ec6d3d119590aa..19de99133f048960 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -876,7 +876,8 @@ int rvin_v4l2_register(struct rvin_dev *vin)
>  	vdev->fops = &rvin_fops;
>  	vdev->v4l2_dev = &vin->v4l2_dev;
>  	vdev->queue = &vin->queue;
> -	strlcpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name));
> +	snprintf(vdev->name, sizeof(vdev->name), "%s %s", KBUILD_MODNAME,
> +		 dev_name(vin->dev));

Do we need the module name here ? How about calling them "%s output", 
dev_name(vin->dev) to emphasize the fact that this is a video node and not a 
VIN subdev ? This is what the omap3isp and vsp1 drivers do.

We're suffering a bit from the fact that V4L2 has never standardized a naming 
scheme for the devices. It wouldn't be fair to ask you to fix that as a 
prerequisite to get the VIN driver merged, but we clearly have to work on that 
at some point.

>  	vdev->release = video_device_release_empty;
>  	vdev->ioctl_ops = &rvin_ioctl_ops;
>  	vdev->lock = &vin->lock;

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 08/28] rcar-vin: move functions regarding scaling

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:22 EET Niklas Söderlund wrote:
> In preparation of refactoring the scaling code move the code regarding
> scaling to to the top of the file to avoid the need to add forward
> declarations. No code is changed in this commit only whole functions
> moved inside the same file.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

The patch is awful to review from the e-mail as git has done a very bad job 
formatting it. If you have to resend it, use --patience for this patch, it 
will help a lot.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  drivers/media/platform/rcar-vin/rcar-dma.c | 806 ++++++++++++++------------
>  1 file changed, 405 insertions(+), 401 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> b/drivers/media/platform/rcar-vin/rcar-dma.c index
> d701b52d198243b5..a7cda3922cb74baa 100644
> --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> @@ -138,305 +138,6 @@ static u32 rvin_read(struct rvin_dev *vin, u32 offset)
> return ioread32(vin->base + offset);
>  }
> 
> -static int rvin_setup(struct rvin_dev *vin)
> -{
> -	u32 vnmc, dmr, dmr2, interrupts;
> -	v4l2_std_id std;
> -	bool progressive = false, output_is_yuv = false, input_is_yuv = false;
> -
> -	switch (vin->format.field) {
> -	case V4L2_FIELD_TOP:
> -		vnmc = VNMC_IM_ODD;
> -		break;
> -	case V4L2_FIELD_BOTTOM:
> -		vnmc = VNMC_IM_EVEN;
> -		break;
> -	case V4L2_FIELD_INTERLACED:
> -		/* Default to TB */
> -		vnmc = VNMC_IM_FULL;
> -		/* Use BT if video standard can be read and is 60 Hz format */
> -		if (!v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
> -			if (std & V4L2_STD_525_60)
> -				vnmc = VNMC_IM_FULL | VNMC_FOC;
> -		}
> -		break;
> -	case V4L2_FIELD_INTERLACED_TB:
> -		vnmc = VNMC_IM_FULL;
> -		break;
> -	case V4L2_FIELD_INTERLACED_BT:
> -		vnmc = VNMC_IM_FULL | VNMC_FOC;
> -		break;
> -	case V4L2_FIELD_ALTERNATE:
> -	case V4L2_FIELD_NONE:
> -		if (vin->continuous) {
> -			vnmc = VNMC_IM_ODD_EVEN;
> -			progressive = true;
> -		} else {
> -			vnmc = VNMC_IM_ODD;
> -		}
> -		break;
> -	default:
> -		vnmc = VNMC_IM_ODD;
> -		break;
> -	}
> -
> -	/*
> -	 * Input interface
> -	 */
> -	switch (vin->digital->code) {
> -	case MEDIA_BUS_FMT_YUYV8_1X16:
> -		/* BT.601/BT.1358 16bit YCbCr422 */
> -		vnmc |= VNMC_INF_YUV16;
> -		input_is_yuv = true;
> -		break;
> -	case MEDIA_BUS_FMT_UYVY8_2X8:
> -		/* BT.656 8bit YCbCr422 or BT.601 8bit YCbCr422 */
> -		vnmc |= vin->digital->mbus_cfg.type == V4L2_MBUS_BT656 ?
> -			VNMC_INF_YUV8_BT656 : VNMC_INF_YUV8_BT601;
> -		input_is_yuv = true;
> -		break;
> -	case MEDIA_BUS_FMT_RGB888_1X24:
> -		vnmc |= VNMC_INF_RGB888;
> -		break;
> -	case MEDIA_BUS_FMT_UYVY10_2X10:
> -		/* BT.656 10bit YCbCr422 or BT.601 10bit YCbCr422 */
> -		vnmc |= vin->digital->mbus_cfg.type == V4L2_MBUS_BT656 ?
> -			VNMC_INF_YUV10_BT656 : VNMC_INF_YUV10_BT601;
> -		input_is_yuv = true;
> -		break;
> -	default:
> -		break;
> -	}
> -
> -	/* Enable VSYNC Field Toogle mode after one VSYNC input */
> -	dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
> -
> -	/* Hsync Signal Polarity Select */
> -	if (!(vin->digital->mbus_cfg.flags & V4L2_MBUS_HSYNC_ACTIVE_LOW))
> -		dmr2 |= VNDMR2_HPS;
> -
> -	/* Vsync Signal Polarity Select */
> -	if (!(vin->digital->mbus_cfg.flags & V4L2_MBUS_VSYNC_ACTIVE_LOW))
> -		dmr2 |= VNDMR2_VPS;
> -
> -	/*
> -	 * Output format
> -	 */
> -	switch (vin->format.pixelformat) {
> -	case V4L2_PIX_FMT_NV16:
> -		rvin_write(vin,
> -			   ALIGN(vin->format.width * vin->format.height, 0x80),
> -			   VNUVAOF_REG);
> -		dmr = VNDMR_DTMD_YCSEP;
> -		output_is_yuv = true;
> -		break;
> -	case V4L2_PIX_FMT_YUYV:
> -		dmr = VNDMR_BPSM;
> -		output_is_yuv = true;
> -		break;
> -	case V4L2_PIX_FMT_UYVY:
> -		dmr = 0;
> -		output_is_yuv = true;
> -		break;
> -	case V4L2_PIX_FMT_XRGB555:
> -		dmr = VNDMR_DTMD_ARGB1555;
> -		break;
> -	case V4L2_PIX_FMT_RGB565:
> -		dmr = 0;
> -		break;
> -	case V4L2_PIX_FMT_XBGR32:
> -		/* Note: not supported on M1 */
> -		dmr = VNDMR_EXRGB;
> -		break;
> -	default:
> -		vin_err(vin, "Invalid pixelformat (0x%x)\n",
> -			vin->format.pixelformat);
> -		return -EINVAL;
> -	}
> -
> -	/* Always update on field change */
> -	vnmc |= VNMC_VUP;
> -
> -	/* If input and output use the same colorspace, use bypass mode */
> -	if (input_is_yuv == output_is_yuv)
> -		vnmc |= VNMC_BPS;
> -
> -	/* Progressive or interlaced mode */
> -	interrupts = progressive ? VNIE_FIE : VNIE_EFE;
> -
> -	/* Ack interrupts */
> -	rvin_write(vin, interrupts, VNINTS_REG);
> -	/* Enable interrupts */
> -	rvin_write(vin, interrupts, VNIE_REG);
> -	/* Start capturing */
> -	rvin_write(vin, dmr, VNDMR_REG);
> -	rvin_write(vin, dmr2, VNDMR2_REG);
> -
> -	/* Enable module */
> -	rvin_write(vin, vnmc | VNMC_ME, VNMC_REG);
> -
> -	return 0;
> -}
> -
> -static void rvin_disable_interrupts(struct rvin_dev *vin)
> -{
> -	rvin_write(vin, 0, VNIE_REG);
> -}
> -
> -static u32 rvin_get_interrupt_status(struct rvin_dev *vin)
> -{
> -	return rvin_read(vin, VNINTS_REG);
> -}
> -
> -static void rvin_ack_interrupt(struct rvin_dev *vin)
> -{
> -	rvin_write(vin, rvin_read(vin, VNINTS_REG), VNINTS_REG);
> -}
> -
> -static bool rvin_capture_active(struct rvin_dev *vin)
> -{
> -	return rvin_read(vin, VNMS_REG) & VNMS_CA;
> -}
> -
> -static int rvin_get_active_slot(struct rvin_dev *vin, u32 vnms)
> -{
> -	if (vin->continuous)
> -		return (vnms & VNMS_FBS_MASK) >> VNMS_FBS_SHIFT;
> -
> -	return 0;
> -}
> -
> -static enum v4l2_field rvin_get_active_field(struct rvin_dev *vin, u32
> vnms) -{
> -	if (vin->format.field == V4L2_FIELD_ALTERNATE) {
> -		/* If FS is set it's a Even field */
> -		if (vnms & VNMS_FS)
> -			return V4L2_FIELD_BOTTOM;
> -		return V4L2_FIELD_TOP;
> -	}
> -
> -	return vin->format.field;
> -}
> -
> -static void rvin_set_slot_addr(struct rvin_dev *vin, int slot, dma_addr_t
> addr) -{
> -	const struct rvin_video_format *fmt;
> -	int offsetx, offsety;
> -	dma_addr_t offset;
> -
> -	fmt = rvin_format_from_pixel(vin->format.pixelformat);
> -
> -	/*
> -	 * There is no HW support for composition do the beast we can
> -	 * by modifying the buffer offset
> -	 */
> -	offsetx = vin->compose.left * fmt->bpp;
> -	offsety = vin->compose.top * vin->format.bytesperline;
> -	offset = addr + offsetx + offsety;
> -
> -	/*
> -	 * The address needs to be 128 bytes aligned. Driver should never accept
> -	 * settings that do not satisfy this in the first place...
> -	 */
> -	if (WARN_ON((offsetx | offsety | offset) & HW_BUFFER_MASK))
> -		return;
> -
> -	rvin_write(vin, offset, VNMB_REG(slot));
> -}
> -
> -/* Moves a buffer from the queue to the HW slots */
> -static bool rvin_fill_hw_slot(struct rvin_dev *vin, int slot)
> -{
> -	struct rvin_buffer *buf;
> -	struct vb2_v4l2_buffer *vbuf;
> -	dma_addr_t phys_addr_top;
> -
> -	if (vin->queue_buf[slot] != NULL)
> -		return true;
> -
> -	if (list_empty(&vin->buf_list))
> -		return false;
> -
> -	vin_dbg(vin, "Filling HW slot: %d\n", slot);
> -
> -	/* Keep track of buffer we give to HW */
> -	buf = list_entry(vin->buf_list.next, struct rvin_buffer, list);
> -	vbuf = &buf->vb;
> -	list_del_init(to_buf_list(vbuf));
> -	vin->queue_buf[slot] = vbuf;
> -
> -	/* Setup DMA */
> -	phys_addr_top = vb2_dma_contig_plane_dma_addr(&vbuf->vb2_buf, 0);
> -	rvin_set_slot_addr(vin, slot, phys_addr_top);
> -
> -	return true;
> -}
> -
> -static bool rvin_fill_hw(struct rvin_dev *vin)
> -{
> -	int slot, limit;
> -
> -	limit = vin->continuous ? HW_BUFFER_NUM : 1;
> -
> -	for (slot = 0; slot < limit; slot++)
> -		if (!rvin_fill_hw_slot(vin, slot))
> -			return false;
> -	return true;
> -}
> -
> -static void rvin_capture_on(struct rvin_dev *vin)
> -{
> -	vin_dbg(vin, "Capture on in %s mode\n",
> -		vin->continuous ? "continuous" : "single");
> -
> -	if (vin->continuous)
> -		/* Continuous Frame Capture Mode */
> -		rvin_write(vin, VNFC_C_FRAME, VNFC_REG);
> -	else
> -		/* Single Frame Capture Mode */
> -		rvin_write(vin, VNFC_S_FRAME, VNFC_REG);
> -}
> -
> -static int rvin_capture_start(struct rvin_dev *vin)
> -{
> -	struct rvin_buffer *buf, *node;
> -	int bufs, ret;
> -
> -	/* Count number of free buffers */
> -	bufs = 0;
> -	list_for_each_entry_safe(buf, node, &vin->buf_list, list)
> -		bufs++;
> -
> -	/* Continuous capture requires more buffers then there are HW slots */
> -	vin->continuous = bufs > HW_BUFFER_NUM;
> -
> -	if (!rvin_fill_hw(vin)) {
> -		vin_err(vin, "HW not ready to start, not enough buffers available\n");
> -		return -EINVAL;
> -	}
> -
> -	rvin_crop_scale_comp(vin);
> -
> -	ret = rvin_setup(vin);
> -	if (ret)
> -		return ret;
> -
> -	rvin_capture_on(vin);
> -
> -	vin->state = RUNNING;
> -
> -	return 0;
> -}
> -
> -static void rvin_capture_stop(struct rvin_dev *vin)
> -{
> -	/* Set continuous & single transfer off */
> -	rvin_write(vin, 0, VNFC_REG);
> -
> -	/* Disable module */
> -	rvin_write(vin, rvin_read(vin, VNMC_REG) & ~VNMC_ME, VNMC_REG);
> -}
> -
>  /*
> ---------------------------------------------------------------------------
> -- * Crop and Scaling Gen2
>   */
> @@ -757,139 +458,442 @@ static const struct vin_coeff vin_coeff_set[] = {
>  			  0x0370e83b, 0x0310d439, 0x03a0f83d,
>  			  0x0370e83c, 0x0300d438, 0x03b0fc3c },
>  	}
> -};
> +};
> +
> +static void rvin_set_coeff(struct rvin_dev *vin, unsigned short xs)
> +{
> +	int i;
> +	const struct vin_coeff *p_prev_set = NULL;
> +	const struct vin_coeff *p_set = NULL;
> +
> +	/* Look for suitable coefficient values */
> +	for (i = 0; i < ARRAY_SIZE(vin_coeff_set); i++) {
> +		p_prev_set = p_set;
> +		p_set = &vin_coeff_set[i];
> +
> +		if (xs < p_set->xs_value)
> +			break;
> +	}
> +
> +	/* Use previous value if its XS value is closer */
> +	if (p_prev_set && p_set &&
> +	    xs - p_prev_set->xs_value < p_set->xs_value - xs)
> +		p_set = p_prev_set;
> +
> +	/* Set coefficient registers */
> +	rvin_write(vin, p_set->coeff_set[0], VNC1A_REG);
> +	rvin_write(vin, p_set->coeff_set[1], VNC1B_REG);
> +	rvin_write(vin, p_set->coeff_set[2], VNC1C_REG);
> +
> +	rvin_write(vin, p_set->coeff_set[3], VNC2A_REG);
> +	rvin_write(vin, p_set->coeff_set[4], VNC2B_REG);
> +	rvin_write(vin, p_set->coeff_set[5], VNC2C_REG);
> +
> +	rvin_write(vin, p_set->coeff_set[6], VNC3A_REG);
> +	rvin_write(vin, p_set->coeff_set[7], VNC3B_REG);
> +	rvin_write(vin, p_set->coeff_set[8], VNC3C_REG);
> +
> +	rvin_write(vin, p_set->coeff_set[9], VNC4A_REG);
> +	rvin_write(vin, p_set->coeff_set[10], VNC4B_REG);
> +	rvin_write(vin, p_set->coeff_set[11], VNC4C_REG);
> +
> +	rvin_write(vin, p_set->coeff_set[12], VNC5A_REG);
> +	rvin_write(vin, p_set->coeff_set[13], VNC5B_REG);
> +	rvin_write(vin, p_set->coeff_set[14], VNC5C_REG);
> +
> +	rvin_write(vin, p_set->coeff_set[15], VNC6A_REG);
> +	rvin_write(vin, p_set->coeff_set[16], VNC6B_REG);
> +	rvin_write(vin, p_set->coeff_set[17], VNC6C_REG);
> +
> +	rvin_write(vin, p_set->coeff_set[18], VNC7A_REG);
> +	rvin_write(vin, p_set->coeff_set[19], VNC7B_REG);
> +	rvin_write(vin, p_set->coeff_set[20], VNC7C_REG);
> +
> +	rvin_write(vin, p_set->coeff_set[21], VNC8A_REG);
> +	rvin_write(vin, p_set->coeff_set[22], VNC8B_REG);
> +	rvin_write(vin, p_set->coeff_set[23], VNC8C_REG);
> +}
> +
> +void rvin_crop_scale_comp(struct rvin_dev *vin)
> +{
> +	u32 xs, ys;
> +
> +	/* Set Start/End Pixel/Line Pre-Clip */
> +	rvin_write(vin, vin->crop.left, VNSPPRC_REG);
> +	rvin_write(vin, vin->crop.left + vin->crop.width - 1, VNEPPRC_REG);
> +	switch (vin->format.field) {
> +	case V4L2_FIELD_INTERLACED:
> +	case V4L2_FIELD_INTERLACED_TB:
> +	case V4L2_FIELD_INTERLACED_BT:
> +		rvin_write(vin, vin->crop.top / 2, VNSLPRC_REG);
> +		rvin_write(vin, (vin->crop.top + vin->crop.height) / 2 - 1,
> +			   VNELPRC_REG);
> +		break;
> +	default:
> +		rvin_write(vin, vin->crop.top, VNSLPRC_REG);
> +		rvin_write(vin, vin->crop.top + vin->crop.height - 1,
> +			   VNELPRC_REG);
> +		break;
> +	}
> +
> +	/* Set scaling coefficient */
> +	ys = 0;
> +	if (vin->crop.height != vin->compose.height)
> +		ys = (4096 * vin->crop.height) / vin->compose.height;
> +	rvin_write(vin, ys, VNYS_REG);
> +
> +	xs = 0;
> +	if (vin->crop.width != vin->compose.width)
> +		xs = (4096 * vin->crop.width) / vin->compose.width;
> +
> +	/* Horizontal upscaling is up to double size */
> +	if (xs > 0 && xs < 2048)
> +		xs = 2048;
> +
> +	rvin_write(vin, xs, VNXS_REG);
> +
> +	/* Horizontal upscaling is done out by scaling down from double size */
> +	if (xs < 4096)
> +		xs *= 2;
> +
> +	rvin_set_coeff(vin, xs);
> +
> +	/* Set Start/End Pixel/Line Post-Clip */
> +	rvin_write(vin, 0, VNSPPOC_REG);
> +	rvin_write(vin, 0, VNSLPOC_REG);
> +	rvin_write(vin, vin->format.width - 1, VNEPPOC_REG);
> +	switch (vin->format.field) {
> +	case V4L2_FIELD_INTERLACED:
> +	case V4L2_FIELD_INTERLACED_TB:
> +	case V4L2_FIELD_INTERLACED_BT:
> +		rvin_write(vin, vin->format.height / 2 - 1, VNELPOC_REG);
> +		break;
> +	default:
> +		rvin_write(vin, vin->format.height - 1, VNELPOC_REG);
> +		break;
> +	}
> +
> +	if (vin->format.pixelformat == V4L2_PIX_FMT_NV16)
> +		rvin_write(vin, ALIGN(vin->format.width, 0x20), VNIS_REG);
> +	else
> +		rvin_write(vin, ALIGN(vin->format.width, 0x10), VNIS_REG);
> +
> +	vin_dbg(vin,
> +		"Pre-Clip: %ux%u@%u:%u YS: %d XS: %d Post-Clip: %ux%u@%u:%u\n",
> +		vin->crop.width, vin->crop.height, vin->crop.left,
> +		vin->crop.top, ys, xs, vin->format.width, vin->format.height,
> +		0, 0);
> +}
> +
> +void rvin_scale_try(struct rvin_dev *vin, struct v4l2_pix_format *pix,
> +		    u32 width, u32 height)
> +{
> +	/* All VIN channels on Gen2 have scalers */
> +	pix->width = width;
> +	pix->height = height;
> +}
> +
> +/*
> ---------------------------------------------------------------------------
> -- + * Hardware setup
> + */
> +
> +static int rvin_setup(struct rvin_dev *vin)
> +{
> +	u32 vnmc, dmr, dmr2, interrupts;
> +	v4l2_std_id std;
> +	bool progressive = false, output_is_yuv = false, input_is_yuv = false;
> +
> +	switch (vin->format.field) {
> +	case V4L2_FIELD_TOP:
> +		vnmc = VNMC_IM_ODD;
> +		break;
> +	case V4L2_FIELD_BOTTOM:
> +		vnmc = VNMC_IM_EVEN;
> +		break;
> +	case V4L2_FIELD_INTERLACED:
> +		/* Default to TB */
> +		vnmc = VNMC_IM_FULL;
> +		/* Use BT if video standard can be read and is 60 Hz format */
> +		if (!v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
> +			if (std & V4L2_STD_525_60)
> +				vnmc = VNMC_IM_FULL | VNMC_FOC;
> +		}
> +		break;
> +	case V4L2_FIELD_INTERLACED_TB:
> +		vnmc = VNMC_IM_FULL;
> +		break;
> +	case V4L2_FIELD_INTERLACED_BT:
> +		vnmc = VNMC_IM_FULL | VNMC_FOC;
> +		break;
> +	case V4L2_FIELD_ALTERNATE:
> +	case V4L2_FIELD_NONE:
> +		if (vin->continuous) {
> +			vnmc = VNMC_IM_ODD_EVEN;
> +			progressive = true;
> +		} else {
> +			vnmc = VNMC_IM_ODD;
> +		}
> +		break;
> +	default:
> +		vnmc = VNMC_IM_ODD;
> +		break;
> +	}
> +
> +	/*
> +	 * Input interface
> +	 */
> +	switch (vin->digital->code) {
> +	case MEDIA_BUS_FMT_YUYV8_1X16:
> +		/* BT.601/BT.1358 16bit YCbCr422 */
> +		vnmc |= VNMC_INF_YUV16;
> +		input_is_yuv = true;
> +		break;
> +	case MEDIA_BUS_FMT_UYVY8_2X8:
> +		/* BT.656 8bit YCbCr422 or BT.601 8bit YCbCr422 */
> +		vnmc |= vin->digital->mbus_cfg.type == V4L2_MBUS_BT656 ?
> +			VNMC_INF_YUV8_BT656 : VNMC_INF_YUV8_BT601;
> +		input_is_yuv = true;
> +		break;
> +	case MEDIA_BUS_FMT_RGB888_1X24:
> +		vnmc |= VNMC_INF_RGB888;
> +		break;
> +	case MEDIA_BUS_FMT_UYVY10_2X10:
> +		/* BT.656 10bit YCbCr422 or BT.601 10bit YCbCr422 */
> +		vnmc |= vin->digital->mbus_cfg.type == V4L2_MBUS_BT656 ?
> +			VNMC_INF_YUV10_BT656 : VNMC_INF_YUV10_BT601;
> +		input_is_yuv = true;
> +		break;
> +	default:
> +		break;
> +	}
> +
> +	/* Enable VSYNC Field Toogle mode after one VSYNC input */
> +	dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
> +
> +	/* Hsync Signal Polarity Select */
> +	if (!(vin->digital->mbus_cfg.flags & V4L2_MBUS_HSYNC_ACTIVE_LOW))
> +		dmr2 |= VNDMR2_HPS;
> +
> +	/* Vsync Signal Polarity Select */
> +	if (!(vin->digital->mbus_cfg.flags & V4L2_MBUS_VSYNC_ACTIVE_LOW))
> +		dmr2 |= VNDMR2_VPS;
> +
> +	/*
> +	 * Output format
> +	 */
> +	switch (vin->format.pixelformat) {
> +	case V4L2_PIX_FMT_NV16:
> +		rvin_write(vin,
> +			   ALIGN(vin->format.width * vin->format.height, 0x80),
> +			   VNUVAOF_REG);
> +		dmr = VNDMR_DTMD_YCSEP;
> +		output_is_yuv = true;
> +		break;
> +	case V4L2_PIX_FMT_YUYV:
> +		dmr = VNDMR_BPSM;
> +		output_is_yuv = true;
> +		break;
> +	case V4L2_PIX_FMT_UYVY:
> +		dmr = 0;
> +		output_is_yuv = true;
> +		break;
> +	case V4L2_PIX_FMT_XRGB555:
> +		dmr = VNDMR_DTMD_ARGB1555;
> +		break;
> +	case V4L2_PIX_FMT_RGB565:
> +		dmr = 0;
> +		break;
> +	case V4L2_PIX_FMT_XBGR32:
> +		/* Note: not supported on M1 */
> +		dmr = VNDMR_EXRGB;
> +		break;
> +	default:
> +		vin_err(vin, "Invalid pixelformat (0x%x)\n",
> +			vin->format.pixelformat);
> +		return -EINVAL;
> +	}
> 
> -static void rvin_set_coeff(struct rvin_dev *vin, unsigned short xs)
> +	/* Always update on field change */
> +	vnmc |= VNMC_VUP;
> +
> +	/* If input and output use the same colorspace, use bypass mode */
> +	if (input_is_yuv == output_is_yuv)
> +		vnmc |= VNMC_BPS;
> +
> +	/* Progressive or interlaced mode */
> +	interrupts = progressive ? VNIE_FIE : VNIE_EFE;
> +
> +	/* Ack interrupts */
> +	rvin_write(vin, interrupts, VNINTS_REG);
> +	/* Enable interrupts */
> +	rvin_write(vin, interrupts, VNIE_REG);
> +	/* Start capturing */
> +	rvin_write(vin, dmr, VNDMR_REG);
> +	rvin_write(vin, dmr2, VNDMR2_REG);
> +
> +	/* Enable module */
> +	rvin_write(vin, vnmc | VNMC_ME, VNMC_REG);
> +
> +	return 0;
> +}
> +
> +static void rvin_disable_interrupts(struct rvin_dev *vin)
>  {
> -	int i;
> -	const struct vin_coeff *p_prev_set = NULL;
> -	const struct vin_coeff *p_set = NULL;
> +	rvin_write(vin, 0, VNIE_REG);
> +}
> 
> -	/* Look for suitable coefficient values */
> -	for (i = 0; i < ARRAY_SIZE(vin_coeff_set); i++) {
> -		p_prev_set = p_set;
> -		p_set = &vin_coeff_set[i];
> +static u32 rvin_get_interrupt_status(struct rvin_dev *vin)
> +{
> +	return rvin_read(vin, VNINTS_REG);
> +}
> 
> -		if (xs < p_set->xs_value)
> -			break;
> +static void rvin_ack_interrupt(struct rvin_dev *vin)
> +{
> +	rvin_write(vin, rvin_read(vin, VNINTS_REG), VNINTS_REG);
> +}
> +
> +static bool rvin_capture_active(struct rvin_dev *vin)
> +{
> +	return rvin_read(vin, VNMS_REG) & VNMS_CA;
> +}
> +
> +static int rvin_get_active_slot(struct rvin_dev *vin, u32 vnms)
> +{
> +	if (vin->continuous)
> +		return (vnms & VNMS_FBS_MASK) >> VNMS_FBS_SHIFT;
> +
> +	return 0;
> +}
> +
> +static enum v4l2_field rvin_get_active_field(struct rvin_dev *vin, u32
> vnms) +{
> +	if (vin->format.field == V4L2_FIELD_ALTERNATE) {
> +		/* If FS is set it's a Even field */
> +		if (vnms & VNMS_FS)
> +			return V4L2_FIELD_BOTTOM;
> +		return V4L2_FIELD_TOP;
>  	}
> 
> -	/* Use previous value if its XS value is closer */
> -	if (p_prev_set && p_set &&
> -	    xs - p_prev_set->xs_value < p_set->xs_value - xs)
> -		p_set = p_prev_set;
> +	return vin->format.field;
> +}
> 
> -	/* Set coefficient registers */
> -	rvin_write(vin, p_set->coeff_set[0], VNC1A_REG);
> -	rvin_write(vin, p_set->coeff_set[1], VNC1B_REG);
> -	rvin_write(vin, p_set->coeff_set[2], VNC1C_REG);
> +static void rvin_set_slot_addr(struct rvin_dev *vin, int slot, dma_addr_t
> addr) +{
> +	const struct rvin_video_format *fmt;
> +	int offsetx, offsety;
> +	dma_addr_t offset;
> 
> -	rvin_write(vin, p_set->coeff_set[3], VNC2A_REG);
> -	rvin_write(vin, p_set->coeff_set[4], VNC2B_REG);
> -	rvin_write(vin, p_set->coeff_set[5], VNC2C_REG);
> +	fmt = rvin_format_from_pixel(vin->format.pixelformat);
> 
> -	rvin_write(vin, p_set->coeff_set[6], VNC3A_REG);
> -	rvin_write(vin, p_set->coeff_set[7], VNC3B_REG);
> -	rvin_write(vin, p_set->coeff_set[8], VNC3C_REG);
> +	/*
> +	 * There is no HW support for composition do the beast we can
> +	 * by modifying the buffer offset
> +	 */
> +	offsetx = vin->compose.left * fmt->bpp;
> +	offsety = vin->compose.top * vin->format.bytesperline;
> +	offset = addr + offsetx + offsety;
> 
> -	rvin_write(vin, p_set->coeff_set[9], VNC4A_REG);
> -	rvin_write(vin, p_set->coeff_set[10], VNC4B_REG);
> -	rvin_write(vin, p_set->coeff_set[11], VNC4C_REG);
> +	/*
> +	 * The address needs to be 128 bytes aligned. Driver should never accept
> +	 * settings that do not satisfy this in the first place...
> +	 */
> +	if (WARN_ON((offsetx | offsety | offset) & HW_BUFFER_MASK))
> +		return;
> 
> -	rvin_write(vin, p_set->coeff_set[12], VNC5A_REG);
> -	rvin_write(vin, p_set->coeff_set[13], VNC5B_REG);
> -	rvin_write(vin, p_set->coeff_set[14], VNC5C_REG);
> +	rvin_write(vin, offset, VNMB_REG(slot));
> +}
> 
> -	rvin_write(vin, p_set->coeff_set[15], VNC6A_REG);
> -	rvin_write(vin, p_set->coeff_set[16], VNC6B_REG);
> -	rvin_write(vin, p_set->coeff_set[17], VNC6C_REG);
> +/* Moves a buffer from the queue to the HW slots */
> +static bool rvin_fill_hw_slot(struct rvin_dev *vin, int slot)
> +{
> +	struct rvin_buffer *buf;
> +	struct vb2_v4l2_buffer *vbuf;
> +	dma_addr_t phys_addr_top;
> 
> -	rvin_write(vin, p_set->coeff_set[18], VNC7A_REG);
> -	rvin_write(vin, p_set->coeff_set[19], VNC7B_REG);
> -	rvin_write(vin, p_set->coeff_set[20], VNC7C_REG);
> +	if (vin->queue_buf[slot] != NULL)
> +		return true;
> 
> -	rvin_write(vin, p_set->coeff_set[21], VNC8A_REG);
> -	rvin_write(vin, p_set->coeff_set[22], VNC8B_REG);
> -	rvin_write(vin, p_set->coeff_set[23], VNC8C_REG);
> +	if (list_empty(&vin->buf_list))
> +		return false;
> +
> +	vin_dbg(vin, "Filling HW slot: %d\n", slot);
> +
> +	/* Keep track of buffer we give to HW */
> +	buf = list_entry(vin->buf_list.next, struct rvin_buffer, list);
> +	vbuf = &buf->vb;
> +	list_del_init(to_buf_list(vbuf));
> +	vin->queue_buf[slot] = vbuf;
> +
> +	/* Setup DMA */
> +	phys_addr_top = vb2_dma_contig_plane_dma_addr(&vbuf->vb2_buf, 0);
> +	rvin_set_slot_addr(vin, slot, phys_addr_top);
> +
> +	return true;
>  }
> 
> -void rvin_crop_scale_comp(struct rvin_dev *vin)
> +static bool rvin_fill_hw(struct rvin_dev *vin)
>  {
> -	u32 xs, ys;
> +	int slot, limit;
> 
> -	/* Set Start/End Pixel/Line Pre-Clip */
> -	rvin_write(vin, vin->crop.left, VNSPPRC_REG);
> -	rvin_write(vin, vin->crop.left + vin->crop.width - 1, VNEPPRC_REG);
> -	switch (vin->format.field) {
> -	case V4L2_FIELD_INTERLACED:
> -	case V4L2_FIELD_INTERLACED_TB:
> -	case V4L2_FIELD_INTERLACED_BT:
> -		rvin_write(vin, vin->crop.top / 2, VNSLPRC_REG);
> -		rvin_write(vin, (vin->crop.top + vin->crop.height) / 2 - 1,
> -			   VNELPRC_REG);
> -		break;
> -	default:
> -		rvin_write(vin, vin->crop.top, VNSLPRC_REG);
> -		rvin_write(vin, vin->crop.top + vin->crop.height - 1,
> -			   VNELPRC_REG);
> -		break;
> -	}
> +	limit = vin->continuous ? HW_BUFFER_NUM : 1;
> 
> -	/* Set scaling coefficient */
> -	ys = 0;
> -	if (vin->crop.height != vin->compose.height)
> -		ys = (4096 * vin->crop.height) / vin->compose.height;
> -	rvin_write(vin, ys, VNYS_REG);
> +	for (slot = 0; slot < limit; slot++)
> +		if (!rvin_fill_hw_slot(vin, slot))
> +			return false;
> +	return true;
> +}
> 
> -	xs = 0;
> -	if (vin->crop.width != vin->compose.width)
> -		xs = (4096 * vin->crop.width) / vin->compose.width;
> +static void rvin_capture_on(struct rvin_dev *vin)
> +{
> +	vin_dbg(vin, "Capture on in %s mode\n",
> +		vin->continuous ? "continuous" : "single");
> 
> -	/* Horizontal upscaling is up to double size */
> -	if (xs > 0 && xs < 2048)
> -		xs = 2048;
> +	if (vin->continuous)
> +		/* Continuous Frame Capture Mode */
> +		rvin_write(vin, VNFC_C_FRAME, VNFC_REG);
> +	else
> +		/* Single Frame Capture Mode */
> +		rvin_write(vin, VNFC_S_FRAME, VNFC_REG);
> +}
> 
> -	rvin_write(vin, xs, VNXS_REG);
> +static int rvin_capture_start(struct rvin_dev *vin)
> +{
> +	struct rvin_buffer *buf, *node;
> +	int bufs, ret;
> 
> -	/* Horizontal upscaling is done out by scaling down from double size */
> -	if (xs < 4096)
> -		xs *= 2;
> +	/* Count number of free buffers */
> +	bufs = 0;
> +	list_for_each_entry_safe(buf, node, &vin->buf_list, list)
> +		bufs++;
> 
> -	rvin_set_coeff(vin, xs);
> +	/* Continuous capture requires more buffers then there are HW slots */
> +	vin->continuous = bufs > HW_BUFFER_NUM;
> 
> -	/* Set Start/End Pixel/Line Post-Clip */
> -	rvin_write(vin, 0, VNSPPOC_REG);
> -	rvin_write(vin, 0, VNSLPOC_REG);
> -	rvin_write(vin, vin->format.width - 1, VNEPPOC_REG);
> -	switch (vin->format.field) {
> -	case V4L2_FIELD_INTERLACED:
> -	case V4L2_FIELD_INTERLACED_TB:
> -	case V4L2_FIELD_INTERLACED_BT:
> -		rvin_write(vin, vin->format.height / 2 - 1, VNELPOC_REG);
> -		break;
> -	default:
> -		rvin_write(vin, vin->format.height - 1, VNELPOC_REG);
> -		break;
> +	if (!rvin_fill_hw(vin)) {
> +		vin_err(vin, "HW not ready to start, not enough buffers available\n");
> +		return -EINVAL;
>  	}
> 
> -	if (vin->format.pixelformat == V4L2_PIX_FMT_NV16)
> -		rvin_write(vin, ALIGN(vin->format.width, 0x20), VNIS_REG);
> -	else
> -		rvin_write(vin, ALIGN(vin->format.width, 0x10), VNIS_REG);
> +	rvin_crop_scale_comp(vin);
> 
> -	vin_dbg(vin,
> -		"Pre-Clip: %ux%u@%u:%u YS: %d XS: %d Post-Clip: %ux%u@%u:%u\n",
> -		vin->crop.width, vin->crop.height, vin->crop.left,
> -		vin->crop.top, ys, xs, vin->format.width, vin->format.height,
> -		0, 0);
> +	ret = rvin_setup(vin);
> +	if (ret)
> +		return ret;
> +
> +	rvin_capture_on(vin);
> +
> +	vin->state = RUNNING;
> +
> +	return 0;
>  }
> 
> -void rvin_scale_try(struct rvin_dev *vin, struct v4l2_pix_format *pix,
> -		    u32 width, u32 height)
> +static void rvin_capture_stop(struct rvin_dev *vin)
>  {
> -	/* All VIN channels on Gen2 have scalers */
> -	pix->width = width;
> -	pix->height = height;
> +	/* Set continuous & single transfer off */
> +	rvin_write(vin, 0, VNFC_REG);
> +
> +	/* Disable module */
> +	rvin_write(vin, rvin_read(vin, VNMC_REG) & ~VNMC_ME, VNMC_REG);
>  }
> 
>  /*
> ---------------------------------------------------------------------------
> --


-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 09/28] rcar-vin: all Gen2 boards can scale simplify logic

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:23 EET Niklas Söderlund wrote:
> The logic to preserve the requested format width and height are too
> complex and come from a premature optimization for Gen3. All Gen2 SoC
> can scale and the Gen3 implementation will not use these functions at
> all so simply preserve the width and height when interacting with the
> subdevice much like the field is preserved simplifies the logic quite a
> bit.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/rcar-vin/rcar-dma.c  |  8 --------
>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 22 ++++++++++------------
>  drivers/media/platform/rcar-vin/rcar-vin.h  |  2 --
>  3 files changed, 10 insertions(+), 22 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> b/drivers/media/platform/rcar-vin/rcar-dma.c index
> a7cda3922cb74baa..fd14be20a6604d7a 100644
> --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> @@ -585,14 +585,6 @@ void rvin_crop_scale_comp(struct rvin_dev *vin)
>  		0, 0);
>  }
> 
> -void rvin_scale_try(struct rvin_dev *vin, struct v4l2_pix_format *pix,
> -		    u32 width, u32 height)
> -{
> -	/* All VIN channels on Gen2 have scalers */
> -	pix->width = width;
> -	pix->height = height;
> -}
> -
>  /*
> ---------------------------------------------------------------------------
> -- * Hardware setup
>   */
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> 19de99133f048960..1c5e7f6d5b963740 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -166,6 +166,7 @@ static int __rvin_try_format_source(struct rvin_dev
> *vin, .which = which,
>  	};
>  	enum v4l2_field field;
> +	u32 width, height;
>  	int ret;
> 
>  	sd = vin_to_source(vin);
> @@ -178,7 +179,10 @@ static int __rvin_try_format_source(struct rvin_dev
> *vin,
> 
>  	format.pad = vin->digital->source_pad;
> 
> +	/* Allow the video device to override field and to scale */
>  	field = pix->field;
> +	width = pix->width;
> +	height = pix->height;
> 
>  	ret = v4l2_subdev_call(sd, pad, set_fmt, pad_cfg, &format);
>  	if (ret < 0 && ret != -ENOIOCTLCMD)
> @@ -191,6 +195,9 @@ static int __rvin_try_format_source(struct rvin_dev
> *vin, source->width = pix->width;
>  	source->height = pix->height;
> 

I would move the pix->field = field line not shown above to here.

Apart from that,

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> +	pix->width = width;
> +	pix->height = height;
> +
>  	vin_dbg(vin, "Source resolution: %ux%u\n", source->width,
>  		source->height);
> 
> @@ -204,13 +211,9 @@ static int __rvin_try_format(struct rvin_dev *vin,
>  			     struct v4l2_pix_format *pix,
>  			     struct rvin_source_fmt *source)
>  {
> -	u32 rwidth, rheight, walign;
> +	u32 walign;
>  	int ret;
> 
> -	/* Requested */
> -	rwidth = pix->width;
> -	rheight = pix->height;
> -
>  	/* Keep current field if no specific one is asked for */
>  	if (pix->field == V4L2_FIELD_ANY)
>  		pix->field = vin->format.field;
> @@ -248,10 +251,6 @@ static int __rvin_try_format(struct rvin_dev *vin,
>  		break;
>  	}
> 
> -	/* If source can't match format try if VIN can scale */
> -	if (source->width != rwidth || source->height != rheight)
> -		rvin_scale_try(vin, pix, rwidth, rheight);
> -
>  	/* HW limit width to a multiple of 32 (2^5) for NV16 else 2 (2^1) */
>  	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
> 
> @@ -270,9 +269,8 @@ static int __rvin_try_format(struct rvin_dev *vin,
>  		return -EINVAL;
>  	}
> 
> -	vin_dbg(vin, "Requested %ux%u Got %ux%u bpl: %d size: %d\n",
> -		rwidth, rheight, pix->width, pix->height,
> -		pix->bytesperline, pix->sizeimage);
> +	vin_dbg(vin, "Format %ux%u bpl: %d size: %d\n",
> +		pix->width, pix->height, pix->bytesperline, pix->sizeimage);
> 
>  	return 0;
>  }
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> b/drivers/media/platform/rcar-vin/rcar-vin.h index
> 646f897f5c05ec4e..36d0f0cc4ce01a6e 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -176,8 +176,6 @@ void rvin_v4l2_unregister(struct rvin_dev *vin);
>  const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
> 
>  /* Cropping, composing and scaling */
> -void rvin_scale_try(struct rvin_dev *vin, struct v4l2_pix_format *pix,
> -		    u32 width, u32 height);
>  void rvin_crop_scale_comp(struct rvin_dev *vin);
> 
>  #endif

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 03/28] rcar-vin: unregister video device on driver removal

[Hans Verkuil]

On 12/08/2017 08:54 AM, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:17 EET Niklas Söderlund wrote:
>> If the video device was registered by the complete() callback it should
>> be unregistered when the driver is removed.
> 
> The .remove() operation indicates device removal, not driver removal (or, the 
> be more precise, it indicates that the device is unbound from the driver). I'd 
> update the commit message accordingly.
> 
>> Protect from printing an uninitialized video device node name by adding a
>> check in rvin_v4l2_unregister() to identify that the video device is
>> registered.
>>
>> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
>> Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
>> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
>> ---
>>  drivers/media/platform/rcar-vin/rcar-core.c | 2 ++
>>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 +++
>>  2 files changed, 5 insertions(+)
>>
>> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
>> b/drivers/media/platform/rcar-vin/rcar-core.c index
>> f7a4c21909da6923..6d99542ec74b49a7 100644
>> --- a/drivers/media/platform/rcar-vin/rcar-core.c
>> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
>> @@ -272,6 +272,8 @@ static int rcar_vin_remove(struct platform_device *pdev)
>>
>>  	pm_runtime_disable(&pdev->dev);
>>
>> +	rvin_v4l2_unregister(vin);
> 
> Unless I'm mistaken, you're unregistering the video device both here and in 
> the unbound() function. That's messy, but it's not really your fault, the V4L2 
> core is very messy in the first place, and registering video devices in the 
> complete() handler is a bad idea. As that can't be fixed for now,
> 
> Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
> Hans, I still would like to hear your opinion on how this should be solved. 
> You've voiced a few weeks ago that register video devices at probe() time 
> isn't a good idea but you've never explained how we should fix the problem. I 
> still firmly believe that video devices should be registered at probe time, 
> and we need to reach an agreement on a technical solution to this problem.

I have tentatively planned to look into this next week. What will very likely
have to happen is that we need to split off allocation from the registration,
just as is done in most other subsystems. Allocation can be done at probe time,
but the final registration step should likely be in the complete().

To what extent that will resolve this specific issue I don't know. It will take
me time to understand this in more detail.

Regards,

	Hans

> 
>>  	v4l2_async_notifier_unregister(&vin->notifier);
>>  	v4l2_async_notifier_cleanup(&vin->notifier);
>>
>> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
>> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
>> 178aecc94962abe2..32a658214f48fa49 100644
>> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
>> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
>> @@ -841,6 +841,9 @@ static const struct v4l2_file_operations rvin_fops = {
>>
>>  void rvin_v4l2_unregister(struct rvin_dev *vin)
>>  {
>> +	if (!video_is_registered(&vin->vdev))
>> +		return;
>> +
>>  	v4l2_info(&vin->v4l2_dev, "Removing %s\n",
>>  		  video_device_node_name(&vin->vdev));
> 

Re: [PATCH v9 03/28] rcar-vin: unregister video device on driver removal

[Laurent Pinchart]

Hi Hans,

On Friday, 8 December 2017 10:46:34 EET Hans Verkuil wrote:
> On 12/08/2017 08:54 AM, Laurent Pinchart wrote:
> > On Friday, 8 December 2017 03:08:17 EET Niklas Söderlund wrote:
> >> If the video device was registered by the complete() callback it should
> >> be unregistered when the driver is removed.
> > 
> > The .remove() operation indicates device removal, not driver removal (or,
> > the be more precise, it indicates that the device is unbound from the
> > driver). I'd update the commit message accordingly.
> > 
> >> Protect from printing an uninitialized video device node name by adding a
> >> check in rvin_v4l2_unregister() to identify that the video device is
> >> registered.
> >> 
> >> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> >> Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> >> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> >> ---
> >> 
> >>  drivers/media/platform/rcar-vin/rcar-core.c | 2 ++
> >>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 +++
> >>  2 files changed, 5 insertions(+)
> >> 
> >> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
> >> b/drivers/media/platform/rcar-vin/rcar-core.c index
> >> f7a4c21909da6923..6d99542ec74b49a7 100644
> >> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> >> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> >> @@ -272,6 +272,8 @@ static int rcar_vin_remove(struct platform_device
> >> *pdev)>> 
> >>  	pm_runtime_disable(&pdev->dev);
> >> 
> >> +	rvin_v4l2_unregister(vin);
> > 
> > Unless I'm mistaken, you're unregistering the video device both here and
> > in the unbound() function. That's messy, but it's not really your fault,
> > the V4L2 core is very messy in the first place, and registering video
> > devices in the complete() handler is a bad idea. As that can't be fixed
> > for now,
> > 
> > Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > 
> > Hans, I still would like to hear your opinion on how this should be
> > solved. You've voiced a few weeks ago that register video devices at
> > probe() time isn't a good idea but you've never explained how we should
> > fix the problem. I still firmly believe that video devices should be
> > registered at probe time, and we need to reach an agreement on a technical
> > solution to this problem.
> 
> I have tentatively planned to look into this next week. What will very
> likely have to happen is that we need to split off allocation from the
> registration, just as is done in most other subsystems. Allocation can be
> done at probe time, but the final registration step should likely be in the
> complete().
> 
> To what extent that will resolve this specific issue I don't know. It will
> take me time to understand this in more detail.

I believe that splitting initialization from registration is a good idea, but 
I still believe that video nodes should be registered at probe time 
nonetheless. Let's discuss it again after next week when you'll have had time 
to think about it.

> >>  	v4l2_async_notifier_unregister(&vin->notifier);
> >>  	v4l2_async_notifier_cleanup(&vin->notifier);
> >> 
> >> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> >> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> >> 178aecc94962abe2..32a658214f48fa49 100644
> >> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> >> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> >> @@ -841,6 +841,9 @@ static const struct v4l2_file_operations rvin_fops =
> >> {
> >> 
> >>  void rvin_v4l2_unregister(struct rvin_dev *vin)
> >>  {
> >> +	if (!video_is_registered(&vin->vdev))
> >> +		return;
> >> +
> >> 
> >>  	v4l2_info(&vin->v4l2_dev, "Removing %s\n",
> >>  		  video_device_node_name(&vin->vdev));

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 11/28] rcar-vin: do not allow changing scaling and composing while streaming

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:25 EET Niklas Söderlund wrote:
> It is possible on Gen2 to change the registers controlling composing and
> scaling while the stream is running. It is however not a good idea to do
> so and could result in trouble. There are also no good reasons to allow
> this, remove immediate reflection in hardware registers from
> vidioc_s_selection and only configure scaling and composing when the
> stream starts.

There is a good reason: digital zoom.

> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/rcar-vin/rcar-dma.c  | 2 +-
>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ---
>  drivers/media/platform/rcar-vin/rcar-vin.h  | 3 ---
>  3 files changed, 1 insertion(+), 7 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> b/drivers/media/platform/rcar-vin/rcar-dma.c index
> fd14be20a6604d7a..7be5080f742825fb 100644
> --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> @@ -514,7 +514,7 @@ static void rvin_set_coeff(struct rvin_dev *vin,
> unsigned short xs) rvin_write(vin, p_set->coeff_set[23], VNC8C_REG);
>  }
> 
> -void rvin_crop_scale_comp(struct rvin_dev *vin)
> +static void rvin_crop_scale_comp(struct rvin_dev *vin)
>  {
>  	u32 xs, ys;
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> 254fa1c8770275a5..d6298c684ab2d731 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -436,9 +436,6 @@ static int rvin_s_selection(struct file *file, void *fh,
> return -EINVAL;
>  	}
> 
> -	/* HW supports modifying configuration while running */
> -	rvin_crop_scale_comp(vin);
> -
>  	return 0;
>  }
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> b/drivers/media/platform/rcar-vin/rcar-vin.h index
> 36d0f0cc4ce01a6e..67541b483ee43c52 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -175,7 +175,4 @@ void rvin_v4l2_unregister(struct rvin_dev *vin);
> 
>  const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
> 
> -/* Cropping, composing and scaling */
> -void rvin_crop_scale_comp(struct rvin_dev *vin);
> -
>  #endif

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 12/28] rcar-vin: read subdevice format for crop only when needed

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:26 EET Niklas Söderlund wrote:
> Instead of caching the subdevice format each time the video device
> format is set read it directly when it's needed. As it turns out the
> format is only needed when figuring out the max rectangle for cropping.
> 
> This simplifies the code and makes it clearer what the source format is
> used for.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 88 +++++++++++++-------------
>  drivers/media/platform/rcar-vin/rcar-vin.h  | 12 ----
>  2 files changed, 42 insertions(+), 58 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> d6298c684ab2d731..9cf9ff48ac1e2f4f 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -90,24 +90,30 @@ static u32 rvin_format_sizeimage(struct v4l2_pix_format
> *pix) * V4L2
>   */
> 
> -static int rvin_reset_format(struct rvin_dev *vin)
> +static int rvin_get_sd_format(struct rvin_dev *vin, struct v4l2_pix_format
> *pix)

What does sd stand for here ? How about rvin_get_source_format() ?

As the function retrieves the format on a subdev I'd rather use a 
v4l2_mbus_framefmt instead of a v4l2_pix_format, and convert in the callers if 
needed.

> {
>  	struct v4l2_subdev_format fmt = {
>  		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
> +		.pad = vin->digital->source_pad,
>  	};
> -	struct v4l2_mbus_framefmt *mf = &fmt.format;
>  	int ret;
> 
> -	fmt.pad = vin->digital->source_pad;
> -
>  	ret = v4l2_subdev_call(vin_to_source(vin), pad, get_fmt, NULL, &fmt);
>  	if (ret)
>  		return ret;
> 
> -	vin->format.width	= mf->width;
> -	vin->format.height	= mf->height;
> -	vin->format.colorspace	= mf->colorspace;
> -	vin->format.field	= mf->field;
> +	v4l2_fill_pix_format(pix, &fmt.format);
> +
> +	return 0;
> +}
> +
> +static int rvin_reset_format(struct rvin_dev *vin)
> +{
> +	int ret;
> +
> +	ret = rvin_get_sd_format(vin, &vin->format);
> +	if (ret)
> +		return ret;
> 
>  	/*
>  	 * If the subdevice uses ALTERNATE field mode and G_STD is
> @@ -137,12 +143,12 @@ static int rvin_reset_format(struct rvin_dev *vin)
>  	}
> 
>  	vin->crop.top = vin->crop.left = 0;
> -	vin->crop.width = mf->width;
> -	vin->crop.height = mf->height;
> +	vin->crop.width = vin->format.width;
> +	vin->crop.height = vin->format.height;
> 
>  	vin->compose.top = vin->compose.left = 0;
> -	vin->compose.width = mf->width;
> -	vin->compose.height = mf->height;
> +	vin->compose.width = vin->format.width;
> +	vin->compose.height = vin->format.height;
> 
>  	vin->format.bytesperline = rvin_format_bytesperline(&vin->format);
>  	vin->format.sizeimage = rvin_format_sizeimage(&vin->format);
> @@ -151,9 +157,7 @@ static int rvin_reset_format(struct rvin_dev *vin)
>  }
> 
>  static int __rvin_try_format_source(struct rvin_dev *vin,
> -				    u32 which,
> -				    struct v4l2_pix_format *pix,
> -				    struct rvin_source_fmt *source)
> +				    u32 which, struct v4l2_pix_format *pix)
>  {
>  	struct v4l2_subdev *sd;
>  	struct v4l2_subdev_pad_config *pad_cfg;
> @@ -186,25 +190,15 @@ static int __rvin_try_format_source(struct rvin_dev
> *vin, v4l2_fill_pix_format(pix, &format.format);
> 
>  	pix->field = field;
> -
> -	source->width = pix->width;
> -	source->height = pix->height;
> -
>  	pix->width = width;
>  	pix->height = height;
> -
> -	vin_dbg(vin, "Source resolution: %ux%u\n", source->width,
> -		source->height);
> -
>  done:
>  	v4l2_subdev_free_pad_config(pad_cfg);
>  	return ret;
>  }
> 
>  static int __rvin_try_format(struct rvin_dev *vin,
> -			     u32 which,
> -			     struct v4l2_pix_format *pix,
> -			     struct rvin_source_fmt *source)
> +			     u32 which, struct v4l2_pix_format *pix)
>  {
>  	u32 walign;
>  	int ret;
> @@ -225,7 +219,7 @@ static int __rvin_try_format(struct rvin_dev *vin,
>  	pix->sizeimage = 0;
> 
>  	/* Limit to source capabilities */
> -	ret = __rvin_try_format_source(vin, which, pix, source);
> +	ret = __rvin_try_format_source(vin, which, pix);
>  	if (ret)
>  		return ret;
> 
> @@ -234,7 +228,6 @@ static int __rvin_try_format(struct rvin_dev *vin,
>  	case V4L2_FIELD_BOTTOM:
>  	case V4L2_FIELD_ALTERNATE:
>  		pix->height /= 2;
> -		source->height /= 2;
>  		break;
>  	case V4L2_FIELD_NONE:
>  	case V4L2_FIELD_INTERLACED_TB:
> @@ -286,30 +279,23 @@ static int rvin_try_fmt_vid_cap(struct file *file,
> void *priv, struct v4l2_format *f)
>  {
>  	struct rvin_dev *vin = video_drvdata(file);
> -	struct rvin_source_fmt source;
> 
> -	return __rvin_try_format(vin, V4L2_SUBDEV_FORMAT_TRY, &f->fmt.pix,
> -				 &source);
> +	return __rvin_try_format(vin, V4L2_SUBDEV_FORMAT_TRY, &f->fmt.pix);
>  }
> 
>  static int rvin_s_fmt_vid_cap(struct file *file, void *priv,
>  			      struct v4l2_format *f)
>  {
>  	struct rvin_dev *vin = video_drvdata(file);
> -	struct rvin_source_fmt source;
>  	int ret;
> 
>  	if (vb2_is_busy(&vin->queue))
>  		return -EBUSY;
> 
> -	ret = __rvin_try_format(vin, V4L2_SUBDEV_FORMAT_ACTIVE, &f->fmt.pix,
> -				&source);
> +	ret = __rvin_try_format(vin, V4L2_SUBDEV_FORMAT_ACTIVE, &f->fmt.pix);
>  	if (ret)
>  		return ret;
> 
> -	vin->source.width = source.width;
> -	vin->source.height = source.height;
> -
>  	vin->format = f->fmt.pix;
> 
>  	return 0;
> @@ -340,6 +326,8 @@ static int rvin_g_selection(struct file *file, void *fh,
> struct v4l2_selection *s)
>  {
>  	struct rvin_dev *vin = video_drvdata(file);
> +	struct v4l2_pix_format pix;
> +	int ret;
> 
>  	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
>  		return -EINVAL;
> @@ -347,9 +335,12 @@ static int rvin_g_selection(struct file *file, void
> *fh, switch (s->target) {
>  	case V4L2_SEL_TGT_CROP_BOUNDS:
>  	case V4L2_SEL_TGT_CROP_DEFAULT:
> +		ret = rvin_get_sd_format(vin, &pix);
> +		if (ret)
> +			return ret;
>  		s->r.left = s->r.top = 0;
> -		s->r.width = vin->source.width;
> -		s->r.height = vin->source.height;
> +		s->r.width = pix.width;
> +		s->r.height = pix.height;
>  		break;
>  	case V4L2_SEL_TGT_CROP:
>  		s->r = vin->crop;
> @@ -375,12 +366,14 @@ static int rvin_s_selection(struct file *file, void
> *fh, {
>  	struct rvin_dev *vin = video_drvdata(file);
>  	const struct rvin_video_format *fmt;
> +	struct v4l2_pix_format pix;
>  	struct v4l2_rect r = s->r;
>  	struct v4l2_rect max_rect;
>  	struct v4l2_rect min_rect = {
>  		.width = 6,
>  		.height = 2,
>  	};
> +	int ret;
> 
>  	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
>  		return -EINVAL;
> @@ -390,22 +383,25 @@ static int rvin_s_selection(struct file *file, void
> *fh, switch (s->target) {
>  	case V4L2_SEL_TGT_CROP:
>  		/* Can't crop outside of source input */
> +		ret = rvin_get_sd_format(vin, &pix);
> +		if (ret)
> +			return ret;

I think you can move this code before the switch.

>  		max_rect.top = max_rect.left = 0;
> -		max_rect.width = vin->source.width;
> -		max_rect.height = vin->source.height;
> +		max_rect.width = pix.width;
> +		max_rect.height = pix.height;
>  		v4l2_rect_map_inside(&r, &max_rect);
> 
> -		v4l_bound_align_image(&r.width, 2, vin->source.width, 1,
> -				      &r.height, 4, vin->source.height, 2, 0);
> +		v4l_bound_align_image(&r.width, 2, pix.width, 1,
> +				      &r.height, 4, pix.height, 2, 0);
> 
> -		r.top  = clamp_t(s32, r.top, 0, vin->source.height - r.height);
> -		r.left = clamp_t(s32, r.left, 0, vin->source.width - r.width);
> +		r.top  = clamp_t(s32, r.top, 0, pix.height - r.height);
> +		r.left = clamp_t(s32, r.left, 0, pix.width - r.width);
> 
>  		vin->crop = s->r = r;
> 
>  		vin_dbg(vin, "Cropped %dx%d@%d:%d of %dx%d\n",
>  			r.width, r.height, r.left, r.top,
> -			vin->source.width, vin->source.height);
> +			pix.width, pix.height);
>  		break;
>  	case V4L2_SEL_TGT_COMPOSE:
>  		/* Make sure compose rect fits inside output format */
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> b/drivers/media/platform/rcar-vin/rcar-vin.h index
> 67541b483ee43c52..f8e0e7cedeaa6c38 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -48,16 +48,6 @@ enum rvin_dma_state {
>  	STOPPING,
>  };
> 
> -/**
> - * struct rvin_source_fmt - Source information
> - * @width:	Width from source
> - * @height:	Height from source
> - */
> -struct rvin_source_fmt {
> -	u32 width;
> -	u32 height;
> -};
> -
>  /**
>   * struct rvin_video_format - Data format stored in memory
>   * @fourcc:	Pixelformat
> @@ -125,7 +115,6 @@ struct rvin_info {
>   * @sequence:		V4L2 buffers sequence number
>   * @state:		keeps track of operation state
>   *
> - * @source:		active format from the video source
>   * @format:		active V4L2 pixel format
>   *
>   * @crop:		active cropping
> @@ -152,7 +141,6 @@ struct rvin_dev {
>  	unsigned int sequence;
>  	enum rvin_dma_state state;
> 
> -	struct rvin_source_fmt source;
>  	struct v4l2_pix_format format;
> 
>  	struct v4l2_rect crop;

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 13/28] rcar-vin: fix handling of single field frames (top, bottom and alternate fields)

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:27 EET Niklas Söderlund wrote:
> There was never proper support in the VIN driver to deliver ALTERNATING
> field format to user-space, remove this field option. For sources using
> this field format instead use the VIN hardware feature of combining the
> fields to an interlaced format. This mode of operation was previously
> the default behavior and ALTERNATING was only delivered to user-space if
> explicitly requested. Allowing this to be explicitly requested was a
> mistake and was never properly tested and never worked due to the
> constraints put on the field format when it comes to sequence numbers and
> timestamps etc.

I'm puzzled, why can't we support V4L2_FIELD_ALTERNATE if we can support 
V4L2_FIELD_TOP and V4L2_FIELD_BOTTOM ? I don't dispute the fact that the 
currently implemented logic might be wrong (although I haven't double-checked 
that), but what prevents us from implementing it correctly ?

> The height should not be cut in half for the format for TOP or BOTTOM
> fields settings. This was a mistake and it was made visible by the
> scaling refactoring. Correct behavior is that the user should request a
> frame size that fits the half height frame reflected in the field
> setting. If not the VIN will do its best to scale the top or bottom to
> the requested format and cropping and scaling do not work as expected.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/rcar-vin/rcar-dma.c  | 15 +--------
>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 48 ++++++++++----------------
>  2 files changed, 19 insertions(+), 44 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> b/drivers/media/platform/rcar-vin/rcar-dma.c index
> 7be5080f742825fb..e6478088d9464221 100644
> --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> @@ -617,7 +617,6 @@ static int rvin_setup(struct rvin_dev *vin)
>  	case V4L2_FIELD_INTERLACED_BT:
>  		vnmc = VNMC_IM_FULL | VNMC_FOC;
>  		break;
> -	case V4L2_FIELD_ALTERNATE:
>  	case V4L2_FIELD_NONE:
>  		if (vin->continuous) {
>  			vnmc = VNMC_IM_ODD_EVEN;
> @@ -757,18 +756,6 @@ static int rvin_get_active_slot(struct rvin_dev *vin,
> u32 vnms) return 0;
>  }
> 
> -static enum v4l2_field rvin_get_active_field(struct rvin_dev *vin, u32
> vnms)
> -{
> -	if (vin->format.field == V4L2_FIELD_ALTERNATE) {
> -		/* If FS is set it's a Even field */
> -		if (vnms & VNMS_FS)
> -			return V4L2_FIELD_BOTTOM;
> -		return V4L2_FIELD_TOP;
> -	}
> -
> -	return vin->format.field;
> -}
> -
>  static void rvin_set_slot_addr(struct rvin_dev *vin, int slot, dma_addr_t
> addr) {
>  	const struct rvin_video_format *fmt;
> @@ -941,7 +928,7 @@ static irqreturn_t rvin_irq(int irq, void *data)
>  		goto done;
> 
>  	/* Capture frame */
> -	vin->queue_buf[slot]->field = rvin_get_active_field(vin, vnms);
> +	vin->queue_buf[slot]->field = vin->format.field;
>  	vin->queue_buf[slot]->sequence = sequence;
>  	vin->queue_buf[slot]->vb2_buf.timestamp = ktime_get_ns();
>  	vb2_buffer_done(&vin->queue_buf[slot]->vb2_buf, VB2_BUF_STATE_DONE);
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> 9cf9ff48ac1e2f4f..37fe1f6c646b0ea3 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -102,6 +102,24 @@ static int rvin_get_sd_format(struct rvin_dev *vin,
> struct v4l2_pix_format *pix) if (ret)
>  		return ret;
> 
> +	switch (fmt.format.field) {
> +	case V4L2_FIELD_TOP:
> +	case V4L2_FIELD_BOTTOM:
> +	case V4L2_FIELD_NONE:
> +	case V4L2_FIELD_INTERLACED_TB:
> +	case V4L2_FIELD_INTERLACED_BT:
> +	case V4L2_FIELD_INTERLACED:
> +		break;
> +	case V4L2_FIELD_ALTERNATE:
> +		/* Use VIN hardware to combine the two fields */
> +		fmt.format.field = V4L2_FIELD_INTERLACED;
> +		fmt.format.height *= 2;
> +		break;

I don't think this is right. If V4L2_FIELD_ALTERNATE isn't supported it should 
be rejected in the set format handler, or rather this logic should be moved 
there. It doesn't belong here, rvin_get_sd_format() should only be called with 
a validated and supported field.

Furthermore treating the pix parameter of this function as both input and 
output seems very confusing to me. If you want to extend rvin_get_sd_format() 
beyond just getting the format from the subdev then please document the 
function with kerneldoc, and let's try to make its API clear.

> +	default:
> +		vin->format.field = V4L2_FIELD_NONE;
> +		break;
> +	}
> +
>  	v4l2_fill_pix_format(pix, &fmt.format);
> 
>  	return 0;
> @@ -115,33 +133,6 @@ static int rvin_reset_format(struct rvin_dev *vin)
>  	if (ret)
>  		return ret;
> 
> -	/*
> -	 * If the subdevice uses ALTERNATE field mode and G_STD is
> -	 * implemented use the VIN HW to combine the two fields to
> -	 * one INTERLACED frame. The ALTERNATE field mode can still
> -	 * be requested in S_FMT and be respected, this is just the
> -	 * default which is applied at probing or when S_STD is called.
> -	 */
> -	if (vin->format.field == V4L2_FIELD_ALTERNATE &&
> -	    v4l2_subdev_has_op(vin_to_source(vin), video, g_std))
> -		vin->format.field = V4L2_FIELD_INTERLACED;
> -
> -	switch (vin->format.field) {
> -	case V4L2_FIELD_TOP:
> -	case V4L2_FIELD_BOTTOM:
> -	case V4L2_FIELD_ALTERNATE:
> -		vin->format.height /= 2;
> -		break;
> -	case V4L2_FIELD_NONE:
> -	case V4L2_FIELD_INTERLACED_TB:
> -	case V4L2_FIELD_INTERLACED_BT:
> -	case V4L2_FIELD_INTERLACED:
> -		break;
> -	default:
> -		vin->format.field = V4L2_FIELD_NONE;
> -		break;
> -	}
> -
>  	vin->crop.top = vin->crop.left = 0;
>  	vin->crop.width = vin->format.width;
>  	vin->crop.height = vin->format.height;
> @@ -226,9 +217,6 @@ static int __rvin_try_format(struct rvin_dev *vin,
>  	switch (pix->field) {
>  	case V4L2_FIELD_TOP:
>  	case V4L2_FIELD_BOTTOM:
> -	case V4L2_FIELD_ALTERNATE:
> -		pix->height /= 2;
> -		break;
>  	case V4L2_FIELD_NONE:
>  	case V4L2_FIELD_INTERLACED_TB:
>  	case V4L2_FIELD_INTERLACED_BT:

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 14/28] rcar-vin: move media bus configuration to struct rvin_info

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:28 EET Niklas Söderlund wrote:
> Bus configuration will once the driver is extended to support Gen3
> contain information not specific to only the directly connected parallel
> subdevice. Move it to struct rvin_info to show it's not always coupled
> to the parallel subdevice.

You do realize that rvin_info is a const structure and that you have moved the 
fields to the rvin_dev structure, right ? :-)

> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/rcar-vin/rcar-core.c | 18 +++++++++---------
>  drivers/media/platform/rcar-vin/rcar-dma.c  | 11 ++++++-----
>  drivers/media/platform/rcar-vin/rcar-v4l2.c |  2 +-
>  drivers/media/platform/rcar-vin/rcar-vin.h  |  9 ++++-----
>  4 files changed, 20 insertions(+), 20 deletions(-)

[snip]

> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> b/drivers/media/platform/rcar-vin/rcar-vin.h index
> f8e0e7cedeaa6c38..118f45b656920d39 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -62,8 +62,6 @@ struct rvin_video_format {
>   * struct rvin_graph_entity - Video endpoint from async framework
>   * @asd:	sub-device descriptor for async framework
>   * @subdev:	subdevice matched using async framework
> - * @code:	Media bus format from source
> - * @mbus_cfg:	Media bus format from DT
>   * @source_pad:	source pad of remote subdevice
>   * @sink_pad:	sink pad of remote subdevice
>   */
> @@ -71,9 +69,6 @@ struct rvin_graph_entity {
>  	struct v4l2_async_subdev asd;
>  	struct v4l2_subdev *subdev;
> 
> -	u32 code;
> -	struct v4l2_mbus_config mbus_cfg;
> -
>  	unsigned int source_pad;
>  	unsigned int sink_pad;
>  };
> @@ -115,6 +110,8 @@ struct rvin_info {
>   * @sequence:		V4L2 buffers sequence number
>   * @state:		keeps track of operation state
>   *
> + * @mbus_cfg:		media bus format from DT

This isn't a format, is it ?

> + * @code:		media bus coide from subdevice

s/coide/code/

and I think you should actually rewrite the description of the field 
completely, it's not clear what it contains.

>   * @format:		active V4L2 pixel format
>   *
>   * @crop:		active cropping
> @@ -141,6 +138,8 @@ struct rvin_dev {
>  	unsigned int sequence;
>  	enum rvin_dma_state state;
> 
> +	struct v4l2_mbus_config mbus_cfg;
> +	u32 code;
>  	struct v4l2_pix_format format;
> 
>  	struct v4l2_rect crop;

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 15/28] rcar-vin: enable Gen3 hardware configuration

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:29 EET Niklas Söderlund wrote:
> Add the register needed to work with Gen3 hardware. This patch adds
> the logic for how to work with the Gen3 hardware. More work is required
> to enable the subdevice structure needed to configure capturing.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/rcar-vin/rcar-dma.c | 94 ++++++++++++++++++---------
>  drivers/media/platform/rcar-vin/rcar-vin.h |  1 +
>  2 files changed, 64 insertions(+), 31 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> b/drivers/media/platform/rcar-vin/rcar-dma.c index
> d7660f485a2df9e4..ace95d5b543a17e3 100644
> --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> @@ -33,21 +33,23 @@
>  #define VNELPRC_REG	0x10	/* Video n End Line Pre-Clip Register */
>  #define VNSPPRC_REG	0x14	/* Video n Start Pixel Pre-Clip Register */
>  #define VNEPPRC_REG	0x18	/* Video n End Pixel Pre-Clip Register */
> -#define VNSLPOC_REG	0x1C	/* Video n Start Line Post-Clip Register */
> -#define VNELPOC_REG	0x20	/* Video n End Line Post-Clip Register */
> -#define VNSPPOC_REG	0x24	/* Video n Start Pixel Post-Clip Register */
> -#define VNEPPOC_REG	0x28	/* Video n End Pixel Post-Clip Register */
>  #define VNIS_REG	0x2C	/* Video n Image Stride Register */
>  #define VNMB_REG(m)	(0x30 + ((m) << 2)) /* Video n Memory Base m Register
> */ #define VNIE_REG	0x40	/* Video n Interrupt Enable Register */
>  #define VNINTS_REG	0x44	/* Video n Interrupt Status Register */
>  #define VNSI_REG	0x48	/* Video n Scanline Interrupt Register */
>  #define VNMTC_REG	0x4C	/* Video n Memory Transfer Control Register */
> -#define VNYS_REG	0x50	/* Video n Y Scale Register */
> -#define VNXS_REG	0x54	/* Video n X Scale Register */
>  #define VNDMR_REG	0x58	/* Video n Data Mode Register */
>  #define VNDMR2_REG	0x5C	/* Video n Data Mode Register 2 */
>  #define VNUVAOF_REG	0x60	/* Video n UV Address Offset Register */
> +
> +/* Register offsets specific for Gen2 */
> +#define VNSLPOC_REG	0x1C	/* Video n Start Line Post-Clip Register */
> +#define VNELPOC_REG	0x20	/* Video n End Line Post-Clip Register */
> +#define VNSPPOC_REG	0x24	/* Video n Start Pixel Post-Clip Register */
> +#define VNEPPOC_REG	0x28	/* Video n End Pixel Post-Clip Register */
> +#define VNYS_REG	0x50	/* Video n Y Scale Register */
> +#define VNXS_REG	0x54	/* Video n X Scale Register */
>  #define VNC1A_REG	0x80	/* Video n Coefficient Set C1A Register */
>  #define VNC1B_REG	0x84	/* Video n Coefficient Set C1B Register */
>  #define VNC1C_REG	0x88	/* Video n Coefficient Set C1C Register */
> @@ -73,9 +75,13 @@
>  #define VNC8B_REG	0xF4	/* Video n Coefficient Set C8B Register */
>  #define VNC8C_REG	0xF8	/* Video n Coefficient Set C8C Register */
> 
> +/* Register offsets specific for Gen3 */
> +#define VNCSI_IFMD_REG		0x20 /* Video n CSI2 Interface Mode Register */
> 
>  /* Register bit fields for R-Car VIN */
>  /* Video n Main Control Register bits */
> +#define VNMC_DPINE		(1 << 27) /* Gen3 specific */
> +#define VNMC_SCLE		(1 << 26) /* Gen3 specific */
>  #define VNMC_FOC		(1 << 21)
>  #define VNMC_YCAL		(1 << 19)
>  #define VNMC_INF_YUV8_BT656	(0 << 16)
> @@ -119,6 +125,13 @@
>  #define VNDMR2_FTEV		(1 << 17)
>  #define VNDMR2_VLV(n)		((n & 0xf) << 12)
> 
> +/* Video n CSI2 Interface Mode Register (Gen3) */
> +#define VNCSI_IFMD_DES2		(1 << 27)
> +#define VNCSI_IFMD_DES1		(1 << 26)
> +#define VNCSI_IFMD_DES0		(1 << 25)
> +#define VNCSI_IFMD_CSI_CHSEL(n) ((n & 0xf) << 0)

*Always* enclose macro arguments in parentheses otherwise they are subject to 
side effects.

#define VNCSI_IFMD_CSI_CHSEL(n) (((n) & 0xf) << 0)

> +#define VNCSI_IFMD_CSI_CHSEL_MASK 0xf
> +
>  struct rvin_buffer {
>  	struct vb2_v4l2_buffer vb;
>  	struct list_head list;
> @@ -514,28 +527,10 @@ static void rvin_set_coeff(struct rvin_dev *vin,
> unsigned short xs) rvin_write(vin, p_set->coeff_set[23], VNC8C_REG);
>  }
> 
> -static void rvin_crop_scale_comp(struct rvin_dev *vin)
> +static void rvin_crop_scale_comp_gen2(struct rvin_dev *vin)
>  {
>  	u32 xs, ys;
> 
> -	/* Set Start/End Pixel/Line Pre-Clip */
> -	rvin_write(vin, vin->crop.left, VNSPPRC_REG);
> -	rvin_write(vin, vin->crop.left + vin->crop.width - 1, VNEPPRC_REG);
> -	switch (vin->format.field) {
> -	case V4L2_FIELD_INTERLACED:
> -	case V4L2_FIELD_INTERLACED_TB:
> -	case V4L2_FIELD_INTERLACED_BT:
> -		rvin_write(vin, vin->crop.top / 2, VNSLPRC_REG);
> -		rvin_write(vin, (vin->crop.top + vin->crop.height) / 2 - 1,
> -			   VNELPRC_REG);
> -		break;
> -	default:
> -		rvin_write(vin, vin->crop.top, VNSLPRC_REG);
> -		rvin_write(vin, vin->crop.top + vin->crop.height - 1,
> -			   VNELPRC_REG);
> -		break;
> -	}
> -
>  	/* Set scaling coefficient */
>  	ys = 0;
>  	if (vin->crop.height != vin->compose.height)
> @@ -573,11 +568,6 @@ static void rvin_crop_scale_comp(struct rvin_dev *vin)
>  		break;
>  	}
> 
> -	if (vin->format.pixelformat == V4L2_PIX_FMT_NV16)
> -		rvin_write(vin, ALIGN(vin->format.width, 0x20), VNIS_REG);
> -	else
> -		rvin_write(vin, ALIGN(vin->format.width, 0x10), VNIS_REG);
> -
>  	vin_dbg(vin,
>  		"Pre-Clip: %ux%u@%u:%u YS: %d XS: %d Post-Clip: %ux%u@%u:%u\n",
>  		vin->crop.width, vin->crop.height, vin->crop.left,

Would it make sense to keep the debug message in the common function, or ill 
cropping and composing be handled through subdevs for Gen3 ?

With these two issues addressed,

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> @@ -585,6 +575,37 @@ static void rvin_crop_scale_comp(struct rvin_dev *vin)
>  		0, 0);
>  }
> 
> +static void rvin_crop_scale_comp(struct rvin_dev *vin)
> +{
> +	/* Set Start/End Pixel/Line Pre-Clip */
> +	rvin_write(vin, vin->crop.left, VNSPPRC_REG);
> +	rvin_write(vin, vin->crop.left + vin->crop.width - 1, VNEPPRC_REG);
> +
> +	switch (vin->format.field) {
> +	case V4L2_FIELD_INTERLACED:
> +	case V4L2_FIELD_INTERLACED_TB:
> +	case V4L2_FIELD_INTERLACED_BT:
> +		rvin_write(vin, vin->crop.top / 2, VNSLPRC_REG);
> +		rvin_write(vin, (vin->crop.top + vin->crop.height) / 2 - 1,
> +			   VNELPRC_REG);
> +		break;
> +	default:
> +		rvin_write(vin, vin->crop.top, VNSLPRC_REG);
> +		rvin_write(vin, vin->crop.top + vin->crop.height - 1,
> +			   VNELPRC_REG);
> +		break;
> +	}
> +
> +	/* TODO: Add support for the UDS scaler. */
> +	if (vin->info->chip != RCAR_GEN3)
> +		rvin_crop_scale_comp_gen2(vin);
> +
> +	if (vin->format.pixelformat == V4L2_PIX_FMT_NV16)
> +		rvin_write(vin, ALIGN(vin->format.width, 0x20), VNIS_REG);
> +	else
> +		rvin_write(vin, ALIGN(vin->format.width, 0x10), VNIS_REG);
> +}
> +
>  /* ------------------------------------------------------------------------
>   * Hardware setup
>   */
> @@ -659,7 +680,10 @@ static int rvin_setup(struct rvin_dev *vin)
>  	}
> 
>  	/* Enable VSYNC Field Toogle mode after one VSYNC input */
> -	dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
> +	if (vin->info->chip == RCAR_GEN3)
> +		dmr2 = VNDMR2_FTEV;
> +	else
> +		dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
> 
>  	/* Hsync Signal Polarity Select */
>  	if (!(vin->mbus_cfg.flags & V4L2_MBUS_HSYNC_ACTIVE_LOW))
> @@ -711,6 +735,14 @@ static int rvin_setup(struct rvin_dev *vin)
>  	if (input_is_yuv == output_is_yuv)
>  		vnmc |= VNMC_BPS;
> 
> +	if (vin->info->chip == RCAR_GEN3) {
> +		/* Select between CSI-2 and Digital input */
> +		if (vin->mbus_cfg.type == V4L2_MBUS_CSI2)
> +			vnmc &= ~VNMC_DPINE;
> +		else
> +			vnmc |= VNMC_DPINE;
> +	}
> +
>  	/* Progressive or interlaced mode */
>  	interrupts = progressive ? VNIE_FIE : VNIE_EFE;
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> b/drivers/media/platform/rcar-vin/rcar-vin.h index
> 118f45b656920d39..a440effe4b86af31 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -33,6 +33,7 @@ enum chip_id {
>  	RCAR_H1,
>  	RCAR_M1,
>  	RCAR_GEN2,
> +	RCAR_GEN3,
>  };
> 
>  /**

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 16/28] rcar-vin: add function to manipulate Gen3 chsel value

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:30 EET Niklas Söderlund wrote:
> On Gen3 the CSI-2 routing is controlled by the VnCSI_IFMD register. One
> feature of this register is that it's only present in the VIN0 and VIN4
> instances. The register in VIN0 controls the routing for VIN0-3 and the
> register in VIN4 controls routing for VIN4-7.
> 
> To be able to control routing from a media device this function is need
> to control runtime PM for the subgroup master (VIN0 and VIN4). The
> subgroup master must be switched on before the register is manipulated,
> once the operation is complete it's safe to switch the master off and
> the new routing will still be in effect.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/rcar-vin/rcar-dma.c | 25 +++++++++++++++++++++++++
>  drivers/media/platform/rcar-vin/rcar-vin.h |  2 ++
>  2 files changed, 27 insertions(+)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> b/drivers/media/platform/rcar-vin/rcar-dma.c index
> ace95d5b543a17e3..d2788d8bb9565aaa 100644
> --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> @@ -16,6 +16,7 @@
> 
>  #include <linux/delay.h>
>  #include <linux/interrupt.h>
> +#include <linux/pm_runtime.h>
> 
>  #include <media/videobuf2-dma-contig.h>
> 
> @@ -1228,3 +1229,27 @@ int rvin_dma_register(struct rvin_dev *vin, int irq)
> 
>  	return ret;
>  }
> +
> +/* ------------------------------------------------------------------------
>   + * Gen3 CHSEL manipulation
> + */
> +
> +void rvin_set_chsel(struct rvin_dev *vin, u8 chsel)

How about naming the function a bit more explicitly, 
rvin_set_channel_routing() for instance ?

> +{
> +	u32 ifmd, vnmc;
> +
> +	pm_runtime_get_sync(vin->dev);

Shouldn't you check the return value of this function ?

> +
> +	/* Make register writes take effect immediately */
> +	vnmc = rvin_read(vin, VNMC_REG) & ~VNMC_VUP;
> +	rvin_write(vin, vnmc, VNMC_REG);

Shouldn't you restore the original value of VNMC at the end of the function ? 
What if this races with device access local to the VIN0 or VIN4 instance ?

> +	ifmd = VNCSI_IFMD_DES2 | VNCSI_IFMD_DES1 | VNCSI_IFMD_DES0 |
> +		VNCSI_IFMD_CSI_CHSEL(chsel);
> +
> +	rvin_write(vin, ifmd, VNCSI_IFMD_REG);
> +
> +	vin_dbg(vin, "Set IFMD 0x%x\n", ifmd);
> +
> +	pm_runtime_put(vin->dev);
> +}
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> b/drivers/media/platform/rcar-vin/rcar-vin.h index
> a440effe4b86af31..7819c760c2c13422 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -163,4 +163,6 @@ void rvin_v4l2_unregister(struct rvin_dev *vin);
> 
>  const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
> 
> +void rvin_set_chsel(struct rvin_dev *vin, u8 chsel);
> +
>  #endif

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 17/28] rcar-vin: add flag to switch to media controller mode

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:31 EET Niklas Söderlund wrote:
> On Gen3 a media controller API needs to be used to allow userspace to
> configure the subdevices in the pipeline instead of directly controlling
> a single source subdevice, which is and will continue to be the mode of
> operation on Gen2.
> 
> Prepare for these two modes of operation by adding a flag to struct
> rvin_info which will control which mode to use.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  drivers/media/platform/rcar-vin/rcar-core.c | 6 +++++-
>  drivers/media/platform/rcar-vin/rcar-vin.h  | 2 ++
>  2 files changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
> b/drivers/media/platform/rcar-vin/rcar-core.c index
> 7d49904cab9cb2d9..61f48ecc1ab815ec 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -232,18 +232,21 @@ static int rvin_digital_graph_init(struct rvin_dev
> *vin)
> 
>  static const struct rvin_info rcar_info_h1 = {
>  	.chip = RCAR_H1,
> +	.use_mc = false,
>  	.max_width = 2048,
>  	.max_height = 2048,
>  };
> 
>  static const struct rvin_info rcar_info_m1 = {
>  	.chip = RCAR_M1,
> +	.use_mc = false,
>  	.max_width = 2048,
>  	.max_height = 2048,
>  };
> 
>  static const struct rvin_info rcar_info_gen2 = {
>  	.chip = RCAR_GEN2,
> +	.use_mc = false,
>  	.max_width = 2048,
>  	.max_height = 2048,
>  };
> @@ -338,7 +341,8 @@ static int rcar_vin_remove(struct platform_device *pdev)
> v4l2_async_notifier_unregister(&vin->notifier);
>  	v4l2_async_notifier_cleanup(&vin->notifier);
> 
> -	v4l2_ctrl_handler_free(&vin->ctrl_handler);
> +	if (!vin->info->use_mc)
> +		v4l2_ctrl_handler_free(&vin->ctrl_handler);
> 
>  	rvin_dma_unregister(vin);
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> b/drivers/media/platform/rcar-vin/rcar-vin.h index
> 7819c760c2c13422..0747873c2b9cb74c 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -77,12 +77,14 @@ struct rvin_graph_entity {
>  /**
>   * struct rvin_info - Information about the particular VIN implementation
>   * @chip:		type of VIN chip
> + * @use_mc:		use media controller instead of controlling subdevice
>   *
>   * max_width:		max input width the VIN supports
>   * max_height:		max input height the VIN supports
>   */
>  struct rvin_info {
>  	enum chip_id chip;
> +	bool use_mc;
> 
>  	unsigned int max_width;
>  	unsigned int max_height;


-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 18/28] rcar-vin: break out format alignment and checking

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:32 EET Niklas Söderlund wrote:
> Part of the format alignment and checking can be shared with the Gen3
> format handling. Break that part out to its own function. While doing
> this clean up the checking and add more checks.

I'd split that in two patches, they are unrelated and should be reviewed 
separately.

> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 98 +++++++++++++------------
>  1 file changed, 51 insertions(+), 47 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> 56c5183f55922e1d..0ffbf0c16fb7b00e 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -86,6 +86,56 @@ static u32 rvin_format_sizeimage(struct v4l2_pix_format
> *pix) return pix->bytesperline * pix->height;
>  }
> 
> +static int rvin_format_align(struct rvin_dev *vin, struct v4l2_pix_format
> *pix)
> +{
> +	u32 walign;
> +
> +	/* If requested format is not supported fallback to the default */
> +	if (!rvin_format_from_pixel(pix->pixelformat)) {
> +		vin_dbg(vin, "Format 0x%x not found, using default 0x%x\n",
> +			pix->pixelformat, RVIN_DEFAULT_FORMAT);
> +		pix->pixelformat = RVIN_DEFAULT_FORMAT;
> +	}
> +
> +	switch (pix->field) {
> +	case V4L2_FIELD_TOP:
> +	case V4L2_FIELD_BOTTOM:
> +	case V4L2_FIELD_NONE:
> +	case V4L2_FIELD_INTERLACED_TB:
> +	case V4L2_FIELD_INTERLACED_BT:
> +	case V4L2_FIELD_INTERLACED:
> +		break;
> +	default:
> +		pix->field = V4L2_FIELD_NONE;
> +		break;
> +	}
> +
> +	/* Check that colorspace is reasonable, if not keep current */
> +	if (!pix->colorspace || pix->colorspace >= 0xff)

Where does 0xff come from ? It seems a bit random to me.

> +		pix->colorspace = vin->format.colorspace;

I don't think that's a good idea. You should pick a default if the colorspace 
can't be supported. Beside, what's the point in accepting colorspaces if 
they're not handled by the driver ? Why don't you just set a fixed value based 
on the colorspace reported by the source ?

> +	/* HW limit width to a multiple of 32 (2^5) for NV16 else 2 (2^1) */
> +	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
> +
> +	/* Limit to VIN capabilities */
> +	v4l_bound_align_image(&pix->width, 2, vin->info->max_width, walign,
> +			      &pix->height, 4, vin->info->max_height, 2, 0);
> +
> +	pix->bytesperline = rvin_format_bytesperline(pix);
> +	pix->sizeimage = rvin_format_sizeimage(pix);

You're now hardcoding those values instead of only enforcing a minimum. Why is 
that ?

> +
> +	if (vin->info->chip == RCAR_M1 &&
> +	    pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
> +		vin_err(vin, "pixel format XBGR32 not supported on M1\n");
> +		return -EINVAL;
> +	}

You should move this with the other format check at the beginning of the 
function. and selecting the default format instead of returning an error.

> +	vin_dbg(vin, "Format %ux%u bpl: %d size: %d\n",
> +		pix->width, pix->height, pix->bytesperline, pix->sizeimage);
> +
> +	return 0;
> +}
> +
>  /*
> ---------------------------------------------------------------------------
> -- * V4L2
>   */
> @@ -191,64 +241,18 @@ static int __rvin_try_format_source(struct rvin_dev
> *vin, static int __rvin_try_format(struct rvin_dev *vin,
>  			     u32 which, struct v4l2_pix_format *pix)
>  {
> -	u32 walign;
>  	int ret;
> 
>  	/* Keep current field if no specific one is asked for */
>  	if (pix->field == V4L2_FIELD_ANY)
>  		pix->field = vin->format.field;
> 
> -	/* If requested format is not supported fallback to the default */
> -	if (!rvin_format_from_pixel(pix->pixelformat)) {
> -		vin_dbg(vin, "Format 0x%x not found, using default 0x%x\n",
> -			pix->pixelformat, RVIN_DEFAULT_FORMAT);
> -		pix->pixelformat = RVIN_DEFAULT_FORMAT;
> -	}
> -
> -	/* Always recalculate */
> -	pix->bytesperline = 0;
> -	pix->sizeimage = 0;
> -
>  	/* Limit to source capabilities */
>  	ret = __rvin_try_format_source(vin, which, pix);
>  	if (ret)
>  		return ret;
> 
> -	switch (pix->field) {
> -	case V4L2_FIELD_TOP:
> -	case V4L2_FIELD_BOTTOM:
> -	case V4L2_FIELD_NONE:
> -	case V4L2_FIELD_INTERLACED_TB:
> -	case V4L2_FIELD_INTERLACED_BT:
> -	case V4L2_FIELD_INTERLACED:
> -		break;
> -	default:
> -		pix->field = V4L2_FIELD_NONE;
> -		break;
> -	}
> -
> -	/* HW limit width to a multiple of 32 (2^5) for NV16 else 2 (2^1) */
> -	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
> -
> -	/* Limit to VIN capabilities */
> -	v4l_bound_align_image(&pix->width, 2, vin->info->max_width, walign,
> -			      &pix->height, 4, vin->info->max_height, 2, 0);
> -
> -	pix->bytesperline = max_t(u32, pix->bytesperline,
> -				  rvin_format_bytesperline(pix));
> -	pix->sizeimage = max_t(u32, pix->sizeimage,
> -			       rvin_format_sizeimage(pix));
> -
> -	if (vin->info->chip == RCAR_M1 &&
> -	    pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
> -		vin_err(vin, "pixel format XBGR32 not supported on M1\n");
> -		return -EINVAL;
> -	}
> -
> -	vin_dbg(vin, "Format %ux%u bpl: %d size: %d\n",
> -		pix->width, pix->height, pix->bytesperline, pix->sizeimage);
> -
> -	return 0;
> +	return rvin_format_align(vin, pix);
>  }
> 
>  static int rvin_querycap(struct file *file, void *priv,


-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 19/28] rcar-vin: use different v4l2 operations in media controller mode

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:33 EET Niklas Söderlund wrote:
> When the driver runs in media controller mode it should not directly
> control the subdevice instead userspace will be responsible for
> configuring the pipeline. To be able to run in this mode a different set
> of v4l2 operations needs to be used.
> 
> Add a new set of v4l2 operations to support the running without directly

Maybe s/the running/operation/ ?

> interacting with the source subdevice.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/rcar-vin/rcar-dma.c  |   3 +-
>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 155 ++++++++++++++++++++++++-
>  drivers/media/platform/rcar-vin/rcar-vin.h  |   1 +
>  3 files changed, 155 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> b/drivers/media/platform/rcar-vin/rcar-dma.c index
> d2788d8bb9565aaa..6c5df13b30d6dd14 100644
> --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> @@ -628,7 +628,8 @@ static int rvin_setup(struct rvin_dev *vin)
>  		/* Default to TB */
>  		vnmc = VNMC_IM_FULL;
>  		/* Use BT if video standard can be read and is 60 Hz format */
> -		if (!v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
> +		if (!vin->info->use_mc &&
> +		    !v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
>  			if (std & V4L2_STD_525_60)
>  				vnmc = VNMC_IM_FULL | VNMC_FOC;
>  		}

I think the subdev should be queried in rcar-v4l2.c and the information cached 
in the rvin_dev structure instead of queried here. Interactions with the 
subdev should be minimized in rvin-dma.c. You can fix this in a separate patch 
if you prefer.

> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> 0ffbf0c16fb7b00e..5fea2856fd61030f 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -23,6 +23,9 @@
>  #include "rcar-vin.h"
> 
>  #define RVIN_DEFAULT_FORMAT	V4L2_PIX_FMT_YUYV
> +#define RVIN_DEFAULT_WIDTH	800
> +#define RVIN_DEFAULT_HEIGHT	600
> +#define RVIN_DEFAULT_COLORSPACE	V4L2_COLORSPACE_SRGB
> 
>  /* ------------------------------------------------------------------------
>   * Format Conversions
> @@ -671,6 +674,84 @@ static const struct v4l2_ioctl_ops rvin_ioctl_ops = {
>  	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
>  };
> 
> +/* ------------------------------------------------------------------------
>   + * V4L2 Media Controller
> + */
> +
> +static int __rvin_mc_try_format(struct rvin_dev *vin,
> +				struct v4l2_pix_format *pix)
> +{
> +	/* Keep current field if no specific one is asked for */
> +	if (pix->field == V4L2_FIELD_ANY)
> +		pix->field = vin->format.field;

You should set a default value instead, trying a format should return the same 
result regardless of the current state of the device.

> +	return rvin_format_align(vin, pix);
> +}
> +
> +static int rvin_mc_try_fmt_vid_cap(struct file *file, void *priv,
> +				   struct v4l2_format *f)
> +{
> +	struct rvin_dev *vin = video_drvdata(file);
> +
> +	return __rvin_mc_try_format(vin, &f->fmt.pix);
> +}
> +
> +static int rvin_mc_s_fmt_vid_cap(struct file *file, void *priv,
> +				 struct v4l2_format *f)
> +{
> +	struct rvin_dev *vin = video_drvdata(file);
> +	int ret;
> +
> +	if (vb2_is_busy(&vin->queue))
> +		return -EBUSY;
> +
> +	ret = __rvin_mc_try_format(vin, &f->fmt.pix);
> +	if (ret)
> +		return ret;
> +
> +	vin->format = f->fmt.pix;
> +
> +	return 0;
> +}
> +
> +static int rvin_mc_enum_input(struct file *file, void *priv,
> +			      struct v4l2_input *i)
> +{
> +	if (i->index != 0)
> +		return -EINVAL;
> +
> +	i->type = V4L2_INPUT_TYPE_CAMERA;
> +	strlcpy(i->name, "Camera", sizeof(i->name));
> +
> +	return 0;
> +}
> + 
> +static const struct v4l2_ioctl_ops rvin_mc_ioctl_ops = {
> +	.vidioc_querycap		= rvin_querycap,
> +	.vidioc_try_fmt_vid_cap		= rvin_mc_try_fmt_vid_cap,
> +	.vidioc_g_fmt_vid_cap		= rvin_g_fmt_vid_cap,
> +	.vidioc_s_fmt_vid_cap		= rvin_mc_s_fmt_vid_cap,
> +	.vidioc_enum_fmt_vid_cap	= rvin_enum_fmt_vid_cap,
> +
> +	.vidioc_enum_input		= rvin_mc_enum_input,
> +	.vidioc_g_input			= rvin_g_input,
> +	.vidioc_s_input			= rvin_s_input,

The input API makes no sense for MC-based devices.

> +	.vidioc_reqbufs			= vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs		= vb2_ioctl_create_bufs,
> +	.vidioc_querybuf		= vb2_ioctl_querybuf,
> +	.vidioc_qbuf			= vb2_ioctl_qbuf,
> +	.vidioc_dqbuf			= vb2_ioctl_dqbuf,
> +	.vidioc_expbuf			= vb2_ioctl_expbuf,
> +	.vidioc_prepare_buf		= vb2_ioctl_prepare_buf,
> +	.vidioc_streamon		= vb2_ioctl_streamon,
> +	.vidioc_streamoff		= vb2_ioctl_streamoff,
> +
> +	.vidioc_log_status		= v4l2_ctrl_log_status,
> +	.vidioc_subscribe_event		= rvin_subscribe_event,
> +	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
> +};
> +
>  /* ------------------------------------------------------------------------
>   * File Operations
>   */
> @@ -814,6 +895,60 @@ static const struct v4l2_file_operations rvin_fops = {
>  	.read		= vb2_fop_read,
>  };
> 
> +/* ------------------------------------------------------------------------
> + * Media controller file Operations

I'm personally fine with capitalizing all words in a title, or only the first 
one, but I don't see a reason to capitalize the first and last only :-)

> + */
> +
> +static int rvin_mc_open(struct file *file)
> +{
> +	struct rvin_dev *vin = video_drvdata(file);
> +	int ret;
> +
> +	mutex_lock(&vin->lock);
> +
> +	file->private_data = vin;
> +
> +	ret = v4l2_fh_open(file);
> +	if (ret)
> +		goto unlock;
> +
> +	pm_runtime_get_sync(vin->dev);

Can't this function return an error ?

> +	v4l2_pipeline_pm_use(&vin->vdev.entity, 1);
> +
> +unlock:
> +	mutex_unlock(&vin->lock);
> +
> +	return ret;
> +}
> +
> +static int rvin_mc_release(struct file *file)
> +{
> +	struct rvin_dev *vin = video_drvdata(file);
> +	int ret;
> +
> +	mutex_lock(&vin->lock);
> +
> +	/* the release helper will cleanup any on-going streaming */
> +	ret = _vb2_fop_release(file, NULL);
> +
> +	v4l2_pipeline_pm_use(&vin->vdev.entity, 0);
> +	pm_runtime_put(vin->dev);
> +
> +	mutex_unlock(&vin->lock);
> +
> +	return ret;
> +}
> +
> +static const struct v4l2_file_operations rvin_mc_fops = {
> +	.owner		= THIS_MODULE,
> +	.unlocked_ioctl	= video_ioctl2,
> +	.open		= rvin_mc_open,
> +	.release	= rvin_mc_release,
> +	.poll		= vb2_fop_poll,
> +	.mmap		= vb2_fop_mmap,
> +	.read		= vb2_fop_read,
> +};
> +
>  void rvin_v4l2_unregister(struct rvin_dev *vin)
>  {
>  	if (!video_is_registered(&vin->vdev))
> @@ -849,19 +984,33 @@ int rvin_v4l2_register(struct rvin_dev *vin)
>  	vin->v4l2_dev.notify = rvin_notify;
> 
>  	/* video node */
> -	vdev->fops = &rvin_fops;
>  	vdev->v4l2_dev = &vin->v4l2_dev;
>  	vdev->queue = &vin->queue;
>  	snprintf(vdev->name, sizeof(vdev->name), "%s %s", KBUILD_MODNAME,
>  		 dev_name(vin->dev));
>  	vdev->release = video_device_release_empty;
> -	vdev->ioctl_ops = &rvin_ioctl_ops;
>  	vdev->lock = &vin->lock;
>  	vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING |
>  		V4L2_CAP_READWRITE;
> 
> +	/* Set some form of default format */

"Some form of" ? :-)

>  	vin->format.pixelformat	= RVIN_DEFAULT_FORMAT;
> -	rvin_reset_format(vin);
> +	vin->format.width = RVIN_DEFAULT_WIDTH;
> +	vin->format.height = RVIN_DEFAULT_HEIGHT;
> +	vin->format.colorspace = RVIN_DEFAULT_COLORSPACE;
> +
> +	if (vin->info->use_mc) {
> +		vdev->fops = &rvin_mc_fops;
> +		vdev->ioctl_ops = &rvin_mc_ioctl_ops;
> +	} else {
> +		vdev->fops = &rvin_fops;
> +		vdev->ioctl_ops = &rvin_ioctl_ops;
> +		rvin_reset_format(vin);
> +	}
> +
> +	ret = rvin_format_align(vin, &vin->format);
> +	if (ret)
> +		return ret;
> 
>  	ret = video_register_device(&vin->vdev, VFL_TYPE_GRABBER, -1);
>  	if (ret) {
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> b/drivers/media/platform/rcar-vin/rcar-vin.h index
> 0747873c2b9cb74c..fd3cd781be0ab1cf 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -21,6 +21,7 @@
>  #include <media/v4l2-ctrls.h>
>  #include <media/v4l2-dev.h>
>  #include <media/v4l2-device.h>
> +#include <media/v4l2-mc.h>

Can't you include this header in the .c files instead ? Minimizing the headers 
included by other headers lowers (pre)compilation time.

>  #include <media/videobuf2-v4l2.h>
> 
>  /* Number of HW buffers */

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 20/28] rcar-vin: prepare for media controller mode initialization

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:34 EET Niklas Söderlund wrote:
> When running in media controller mode a media pad is needed, register
> one. Also set the media bus format to CSI-2.

This sounds a bit unclear to me. We don't need a pad for the sake of it, what 
we need to do is to initialize the entity of the device device. I'd rephrase 
the commit message accordingly.

> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/rcar-vin/rcar-core.c | 24 ++++++++++++++++++++++--
>  drivers/media/platform/rcar-vin/rcar-vin.h  |  4 ++++
>  2 files changed, 26 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
> b/drivers/media/platform/rcar-vin/rcar-core.c index
> 61f48ecc1ab815ec..45de4079fd835759 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -46,6 +46,10 @@ static int rvin_find_pad(struct v4l2_subdev *sd, int
> direction) return -EINVAL;
>  }
> 
> +/* ------------------------------------------------------------------------
> + * Digital async notifier
> + */
> +
>  static int rvin_digital_notify_complete(struct v4l2_async_notifier
> *notifier) {
>  	struct rvin_dev *vin = notifier_to_vin(notifier);
> @@ -226,6 +230,20 @@ static int rvin_digital_graph_init(struct rvin_dev
> *vin) return 0;
>  }
> 
> +/* ------------------------------------------------------------------------
> + * Group async notifier

The function below isn't related to async notifiers. This might change in 
latter patches, but I'd make the section name consistent with the 
implementation here, and change it later if needed.

> + */
> +
> +static int rvin_group_init(struct rvin_dev *vin)
> +{
> +	/* All our sources are CSI-2 */
> +	vin->mbus_cfg.type = V4L2_MBUS_CSI2;
> +	vin->mbus_cfg.flags = 0;
> +
> +	vin->pad.flags = MEDIA_PAD_FL_SINK;
> +	return media_entity_pads_init(&vin->vdev.entity, 1, &vin->pad);
> +}
> +
>  /* ------------------------------------------------------------------------
>   * Platform Device Driver
>   */
> @@ -314,8 +332,10 @@ static int rcar_vin_probe(struct platform_device *pdev)
> return ret;
> 
>  	platform_set_drvdata(pdev, vin);
> -
> -	ret = rvin_digital_graph_init(vin);
> +	if (vin->info->use_mc)
> +		ret = rvin_group_init(vin);
> +	else
> +		ret = rvin_digital_graph_init(vin);
>  	if (ret < 0)
>  		goto error;
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> b/drivers/media/platform/rcar-vin/rcar-vin.h index
> fd3cd781be0ab1cf..07d270a976893cdb 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -103,6 +103,8 @@ struct rvin_info {
>   * @notifier:		V4L2 asynchronous subdevs notifier
>   * @digital:		entity in the DT for local digital subdevice
>   *
> + * @pad:		pad for media controller

I'd rephrase this too based on the comment I made regarding the commit 
message.

>   * @lock:		protects @queue
>   * @queue:		vb2 buffers queue
>   *
> @@ -132,6 +134,8 @@ struct rvin_dev {
>  	struct v4l2_async_notifier notifier;
>  	struct rvin_graph_entity *digital;
> 
> +	struct media_pad pad;
> +
>  	struct mutex lock;
>  	struct vb2_queue queue;

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 26/28] rcar-vin: enable support for r8a7795

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:40 EET Niklas Söderlund wrote:
> Add the SoC specific information for Renesas r8a7795 ES1.x and ES2.0.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>

I'll trust that the routing table is correct. Apart from that,

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  drivers/media/platform/rcar-vin/Kconfig     |   2 +-
>  drivers/media/platform/rcar-vin/rcar-core.c | 150 +++++++++++++++++++++++++
>  2 files changed, 151 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/Kconfig
> b/drivers/media/platform/rcar-vin/Kconfig index
> af4c98b44d2e22cb..8fa7ee468c63afb9 100644
> --- a/drivers/media/platform/rcar-vin/Kconfig
> +++ b/drivers/media/platform/rcar-vin/Kconfig
> @@ -6,7 +6,7 @@ config VIDEO_RCAR_VIN
>  	select V4L2_FWNODE
>  	---help---
>  	  Support for Renesas R-Car Video Input (VIN) driver.
> -	  Supports R-Car Gen2 SoCs.
> +	  Supports R-Car Gen2 and Gen3 SoCs.
> 
>  	  To compile this driver as a module, choose M here: the
>  	  module will be called rcar-vin.
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
> b/drivers/media/platform/rcar-vin/rcar-core.c index
> 09ebeff1580556dc..66a8144fbba437d3 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -21,6 +21,7 @@
>  #include <linux/platform_device.h>
>  #include <linux/pm_runtime.h>
>  #include <linux/slab.h>
> +#include <linux/sys_soc.h>
> 
>  #include <media/v4l2-async.h>
>  #include <media/v4l2-fwnode.h>
> @@ -956,6 +957,134 @@ static const struct rvin_info rcar_info_gen2 = {
>  	.max_height = 2048,
>  };
> 
> +static const struct rvin_info rcar_info_r8a7795 = {
> +	.chip = RCAR_GEN3,
> +	.use_mc = true,
> +	.max_width = 4096,
> +	.max_height = 4096,
> +
> +	.num_chsels = 5,
> +	.chsels = {
> +		{
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 1 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +		}, {
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 1 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +		}, {
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 2 },
> +			{ .csi = RVIN_CSI20, .chan = 2 },
> +		}, {
> +			{ .csi = RVIN_CSI40, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_CSI40, .chan = 3 },
> +			{ .csi = RVIN_CSI20, .chan = 3 },
> +		}, {
> +			{ .csi = RVIN_CSI41, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI41, .chan = 1 },
> +			{ .csi = RVIN_CSI41, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +		}, {
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI41, .chan = 1 },
> +			{ .csi = RVIN_CSI41, .chan = 0 },
> +			{ .csi = RVIN_CSI41, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +		}, {
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_CSI41, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI41, .chan = 2 },
> +			{ .csi = RVIN_CSI20, .chan = 2 },
> +		}, {
> +			{ .csi = RVIN_CSI41, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_CSI41, .chan = 3 },
> +			{ .csi = RVIN_CSI20, .chan = 3 },
> +		},
> +	},
> +};
> +
> +static const struct rvin_info rcar_info_r8a7795es1 = {
> +	.chip = RCAR_GEN3,
> +	.use_mc = true,
> +	.max_width = 4096,
> +	.max_height = 4096,
> +
> +	.num_chsels = 6,
> +	.chsels = {
> +		{
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI21, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI21, .chan = 0 },
> +		}, {
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI21, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_CSI21, .chan = 1 },
> +		}, {
> +			{ .csi = RVIN_CSI21, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 2 },
> +			{ .csi = RVIN_CSI20, .chan = 2 },
> +			{ .csi = RVIN_CSI21, .chan = 2 },
> +		}, {
> +			{ .csi = RVIN_CSI40, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_CSI21, .chan = 1 },
> +			{ .csi = RVIN_CSI40, .chan = 3 },
> +			{ .csi = RVIN_CSI20, .chan = 3 },
> +			{ .csi = RVIN_CSI21, .chan = 3 },
> +		}, {
> +			{ .csi = RVIN_CSI41, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI21, .chan = 0 },
> +			{ .csi = RVIN_CSI41, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI21, .chan = 0 },
> +		}, {
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI21, .chan = 0 },
> +			{ .csi = RVIN_CSI41, .chan = 0 },
> +			{ .csi = RVIN_CSI41, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_CSI21, .chan = 1 },
> +		}, {
> +			{ .csi = RVIN_CSI21, .chan = 0 },
> +			{ .csi = RVIN_CSI41, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI41, .chan = 2 },
> +			{ .csi = RVIN_CSI20, .chan = 2 },
> +			{ .csi = RVIN_CSI21, .chan = 2 },
> +		}, {
> +			{ .csi = RVIN_CSI41, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_CSI21, .chan = 1 },
> +			{ .csi = RVIN_CSI41, .chan = 3 },
> +			{ .csi = RVIN_CSI20, .chan = 3 },
> +			{ .csi = RVIN_CSI21, .chan = 3 },
> +		},
> +	},
> +};
> +
>  static const struct of_device_id rvin_of_id_table[] = {
>  	{
>  		.compatible = "renesas,vin-r8a7778",
> @@ -985,12 +1114,25 @@ static const struct of_device_id rvin_of_id_table[] =
> { .compatible = "renesas,rcar-gen2-vin",
>  		.data = &rcar_info_gen2,
>  	},
> +	{
> +		.compatible = "renesas,vin-r8a7795",
> +		.data = &rcar_info_r8a7795,
> +	},
>  	{ },
>  };
>  MODULE_DEVICE_TABLE(of, rvin_of_id_table);
> 
> +static const struct soc_device_attribute r8a7795es1[] = {
> +	{
> +		.soc_id = "r8a7795", .revision = "ES1.*",
> +		.data = &rcar_info_r8a7795es1,
> +	},
> +	{ /* sentinel */ }
> +};
> +
>  static int rcar_vin_probe(struct platform_device *pdev)
>  {
> +	const struct soc_device_attribute *attr;
>  	struct rvin_dev *vin;
>  	struct resource *mem;
>  	int irq, ret;
> @@ -1002,6 +1144,14 @@ static int rcar_vin_probe(struct platform_device
> *pdev) vin->dev = &pdev->dev;
>  	vin->info = of_device_get_match_data(&pdev->dev);
> 
> +	/*
> +	 * Special care is needed on r8a7795 ES1.x since it
> +	 * uses different routing than r8a7795 ES2.0.
> +	 */
> +	attr = soc_device_match(r8a7795es1);
> +	if (attr)
> +		vin->info = attr->data;
> +
>  	mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
>  	if (mem == NULL)
>  		return -EINVAL;


-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 19/28] rcar-vin: use different v4l2 operations in media controller mode

[Hans Verkuil]

Hi Laurent,

>> +static const struct v4l2_ioctl_ops rvin_mc_ioctl_ops = {
>> +	.vidioc_querycap		= rvin_querycap,
>> +	.vidioc_try_fmt_vid_cap		= rvin_mc_try_fmt_vid_cap,
>> +	.vidioc_g_fmt_vid_cap		= rvin_g_fmt_vid_cap,
>> +	.vidioc_s_fmt_vid_cap		= rvin_mc_s_fmt_vid_cap,
>> +	.vidioc_enum_fmt_vid_cap	= rvin_enum_fmt_vid_cap,
>> +
>> +	.vidioc_enum_input		= rvin_mc_enum_input,
>> +	.vidioc_g_input			= rvin_g_input,
>> +	.vidioc_s_input			= rvin_s_input,
> 
> The input API makes no sense for MC-based devices.

We've had this discussion before:

https://patchwork.linuxtv.org/patch/41857/

There was never a v3 of that patch, so nothing was done with it.

The issue here is that the spec requires G/S_INPUT to be present for
video nodes. There currently is no exception for MC devices.

Regards,

	Hans

Re: [PATCH v9 27/28] rcar-vin: enable support for r8a7796

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:41 EET Niklas Söderlund wrote:
> Add the SoC specific information for Renesas r8a7796.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> Acked-by: Rob Herring <robh@kernel.org>
> ---
>  .../devicetree/bindings/media/rcar_vin.txt         |  1 +
>  drivers/media/platform/rcar-vin/rcar-core.c        | 64 +++++++++++++++++++
>  2 files changed, 65 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/media/rcar_vin.txt
> b/Documentation/devicetree/bindings/media/rcar_vin.txt index
> 5a95d9668d2c7dfd..314743532bbb4523 100644
> --- a/Documentation/devicetree/bindings/media/rcar_vin.txt
> +++ b/Documentation/devicetree/bindings/media/rcar_vin.txt
> @@ -20,6 +20,7 @@ on Gen3 to a CSI-2 receiver.
>     - "renesas,vin-r8a7793" for the R8A7793 device
>     - "renesas,vin-r8a7794" for the R8A7794 device
>     - "renesas,vin-r8a7795" for the R8A7795 device
> +   - "renesas,vin-r8a7796" for the R8A7796 device

I would move this to patch 01/28, it would nicely separate code changes from 
DT bindings changes.

Apart from that, and the fact that I'll trust that the routing table is 
correct,

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

All this applies to patch 28/28 as well.

>     - "renesas,rcar-gen2-vin" for a generic R-Car Gen2 or RZ/G1 compatible
>       device.
>     - "renesas,rcar-gen3-vin" for a generic R-Car Gen3 compatible device.
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
> b/drivers/media/platform/rcar-vin/rcar-core.c index
> 66a8144fbba437d3..ed7fbb58ad6846c1 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -1085,6 +1085,66 @@ static const struct rvin_info rcar_info_r8a7795es1 =
> { },
>  };
> 
> +static const struct rvin_info rcar_info_r8a7796 = {
> +	.chip = RCAR_GEN3,
> +	.use_mc = true,
> +	.max_width = 4096,
> +	.max_height = 4096,
> +
> +	.num_chsels = 5,
> +	.chsels = {
> +		{
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_NC, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +		}, {
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_NC, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +		}, {
> +			{ .csi = RVIN_NC, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 2 },
> +			{ .csi = RVIN_CSI20, .chan = 2 },
> +		}, {
> +			{ .csi = RVIN_CSI40, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_NC, .chan = 1 },
> +			{ .csi = RVIN_CSI40, .chan = 3 },
> +			{ .csi = RVIN_CSI20, .chan = 3 },
> +		}, {
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_NC, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +		}, {
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_NC, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +		}, {
> +			{ .csi = RVIN_NC, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 0 },
> +			{ .csi = RVIN_CSI20, .chan = 0 },
> +			{ .csi = RVIN_CSI40, .chan = 2 },
> +			{ .csi = RVIN_CSI20, .chan = 2 },
> +		}, {
> +			{ .csi = RVIN_CSI40, .chan = 1 },
> +			{ .csi = RVIN_CSI20, .chan = 1 },
> +			{ .csi = RVIN_NC, .chan = 1 },
> +			{ .csi = RVIN_CSI40, .chan = 3 },
> +			{ .csi = RVIN_CSI20, .chan = 3 },
> +		},
> +	},
> +};
> +
>  static const struct of_device_id rvin_of_id_table[] = {
>  	{
>  		.compatible = "renesas,vin-r8a7778",
> @@ -1118,6 +1178,10 @@ static const struct of_device_id rvin_of_id_table[] =
> { .compatible = "renesas,vin-r8a7795",
>  		.data = &rcar_info_r8a7795,
>  	},
> +	{
> +		.compatible = "renesas,vin-r8a7796",
> +		.data = &rcar_info_r8a7796,
> +	},
>  	{ },
>  };
>  MODULE_DEVICE_TABLE(of, rvin_of_id_table);

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 01/28] rcar-vin: add Gen3 devicetree bindings documentation

[Niklas Söderlund]

Hi Laurent,

Thanks for your comments!

On 2017-12-08 09:46:24 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:15 EET Niklas Söderlund wrote:
> > Document the devicetree bindings for the CSI-2 inputs available on Gen3.
> > 
> > There is a need to add a custom property 'renesas,id' and to define
> > which CSI-2 input is described in which endpoint under the port@1 node.
> > This information is needed since there are a set of predefined routes
> > between each VIN and CSI-2 block. This routing table will be kept
> > inside the driver but in order for it to act on it it must know which
> > VIN and CSI-2 is which.
> > 
> > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > Acked-by: Rob Herring <robh@kernel.org>
> > ---
> >  .../devicetree/bindings/media/rcar_vin.txt         | 116 +++++++++++++++---
> >  1 file changed, 104 insertions(+), 12 deletions(-)
> > 
> > diff --git a/Documentation/devicetree/bindings/media/rcar_vin.txt
> > b/Documentation/devicetree/bindings/media/rcar_vin.txt index
> > ff9697ed81396e64..5a95d9668d2c7dfd 100644
> > --- a/Documentation/devicetree/bindings/media/rcar_vin.txt
> > +++ b/Documentation/devicetree/bindings/media/rcar_vin.txt
> > @@ -2,8 +2,12 @@ Renesas R-Car Video Input driver (rcar_vin)
> >  -------------------------------------------
> > 
> >  The rcar_vin device provides video input capabilities for the Renesas R-Car
> > -family of devices. The current blocks are always slaves and suppot one
> > input
> > -channel which can be either RGB, YUYV or BT656.
> > +family of devices.
> > +
> > +Each VIN instance has a single parallel input that supports RGB and YUV
> > video,
> > +with both external synchronization and BT.656 synchronization for the
> > latter.
> > +Depending on the instance the VIN input is connected to external SoC pins,
> > or
> > +on Gen3 to a CSI-2 receiver.
> > 
> >   - compatible: Must be one or more of the following
> >     - "renesas,vin-r8a7743" for the R8A7743 device
> > @@ -31,21 +35,38 @@ channel which can be either RGB, YUYV or BT656.
> >  Additionally, an alias named vinX will need to be created to specify
> >  which video input device this is.
> > 
> > -The per-board settings:
> > +The per-board settings Gen2:
> 
> Nitpicking, s/Gen2/for Gen2 platforms/
> 
> (or Gen2 hardware, or Gen2 systems, pick the one you like best)

Good catch, I guess I'm to caught up in it all and it's obvious for me 
what Gen2 is :-) I will update and use "Gen2 platform" throughout this 
document.

> 
> >   - port sub-node describing a single endpoint connected to the vin
> >     as described in video-interfaces.txt[1]. Only the first one will
> >     be considered as each vin interface has one input port.
> > 
> > -   These settings are used to work out video input format and widths
> > -   into the system.
> > +The per-board settings Gen3:
> 
> Ditto.
> 
> > +
> > +Gen3 can support both a single connected parallel input source from
> > +external SoC pins (port0) and/or multiple parallel input sources from
> > +local SoC CSI-2 receivers (port1) depending on SoC.
> > 
> > +- renesas,id - ID number of the VIN, VINx in the documentation.
> > +- ports
> > +    - port0 - sub-node describing a single endpoint connected to the VIN
> > +      from external SoC pins described in video-interfaces.txt[1]. Only
> > +      the first one will be considered as each VIN interface has at most
> > +      one set of SoC external input pins.
> 
> s/port0/port 0/ or s/port0/port@0/

I will use 'port 0' as I think it makes more sens when reading.

> 
> I'd go further than that and make it invalid to have multiple endpoints 
> instead of ignoring all but the first one.

Good catch, I'm describing what the current rcar-vin driver dose and not 
the generic case where in fact only one endpoint in port 0 is valid.  
Will update.

> 
> I would also explicitly state that VIN instances not connected to external 
> pins shall have no port 0.

It's good to be explicit, will add this.

> 
> > +    - port1 - sub-nodes describing one or more endpoints connected to
> > +      the VIN from local SoC CSI-2 receivers. The endpoint numbers must
> > +      use the following schema.
> 
> Nitpicking again, the Gen2-specific properties are indented above while the 
> Gen3 properties are not indented here. Pick the one you prefer :-)

I like it they way it is :-)

The Gen2-specific properties are all on the same level in the dt 
description. While some Gen3-specific properties (port 0 and port 1) are 
indented one level in the dt description and this is why I indented them 
one level in the documentation. To make it clear that they should be 
children of the ports node.

I don't feel strongly about this but I feel it adds to the readability.  
If you don't agree I'm happy to remove the indentation for these two 
properties.

> 
> > -Device node example
> > --------------------
> > +        - Endpoint 0 - sub-node describing the endpoint which is CSI20
> > +        - Endpoint 1 - sub-node describing the endpoint which is CSI21
> > +        - Endpoint 2 - sub-node describing the endpoint which is CSI40
> > +        - Endpoint 3 - sub-node describing the endpoint which is CSI41
> 
> How about s/which is/connected to/ ?

Sounds better, will update.


> 
> > -	aliases {
> > -	       vin0 = &vin0;
> > -	};
> > +Device node example Gen2
> 
> s/Gen2/for Gen2 platforms/
> 
> and same in a few places below.

Will fix.

> 
> > +------------------------
> > +
> > +        aliases {
> > +                vin0 = &vin0;
> > +        };
> 
> This is unrelated, but do we need aliases ?

We don't have aliases for Gen3 but they exists in DTS for Gen2. My 
feeling is that we don't need them. My plan is to once Gen3 support is 
done try to sort this out, both in the documentation and in the dts 
files for Gen2.

> 
> >          vin0: vin@0xe6ef0000 {
> >                  compatible = "renesas,vin-r8a7790",
> > "renesas,rcar-gen2-vin"; @@ -55,8 +76,8 @@ Device node example
> >                  status = "disabled";
> >          };
> > 
> > -Board setup example (vin1 composite video input)
> > -------------------------------------------------
> > +Board setup example Gen2 (vin1 composite video input)
> > +-----------------------------------------------------
> > 
> >  &i2c2   {
> >          status = "ok";
> > @@ -95,6 +116,77 @@ Board setup example (vin1 composite video input)
> >          };
> >  };
> > 
> > +Device node example Gen3
> > +------------------------
> > +
> > +        vin0: video@e6ef0000 {
> > +                compatible = "renesas,vin-r8a7795";
> > +                reg = <0 0xe6ef0000 0 0x1000>;
> > +                interrupts = <GIC_SPI 188 IRQ_TYPE_LEVEL_HIGH>;
> > +                clocks = <&cpg CPG_MOD 811>;
> > +                power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
> > +                resets = <&cpg 811>;
> > +                renesas,id = <0>;
> > +
> > +                ports {
> > +                        #address-cells = <1>;
> > +                        #size-cells = <0>;
> > +
> > +                        port@1 {
> > +                                #address-cells = <1>;
> > +                                #size-cells = <0>;
> > +
> > +                                reg = <1>;
> > +
> > +                                vin0csi20: endpoint@0 {
> > +                                        reg = <0>;
> > +                                        remote-endpoint= <&csi20vin0>;
> > +                                };
> > +                                vin0csi21: endpoint@1 {
> > +                                        reg = <1>;
> > +                                        remote-endpoint= <&csi21vin0>;
> > +                                };
> > +                                vin0csi40: endpoint@2 {
> > +                                        reg = <2>;
> > +                                        remote-endpoint= <&csi40vin0>;
> > +                                };
> > +                        };
> > +                };
> > +        };
> > +
> > +        csi20: csi2@fea80000 {
> > +                compatible = "renesas,r8a7795-csi2";
> > +                reg = <0 0xfea80000 0 0x10000>;
> > +                interrupts = <GIC_SPI 184 IRQ_TYPE_LEVEL_HIGH>;
> > +                clocks = <&cpg CPG_MOD 714>;
> > +                power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
> > +                resets = <&cpg 714>;
> > +
> > +                ports {
> > +                        #address-cells = <1>;
> > +                        #size-cells = <0>;
> > +
> > +                        port@0 {
> > +                                reg = <0>;
> > +                                csi20_in: endpoint {
> > +                                        clock-lanes = <0>;
> > +                                        data-lanes = <1>;
> > +                                        remote-endpoint = <&adv7482_txb>;
> > +                                };
> > +                        };
> > +
> > +                        port@1 {
> > +                                #address-cells = <1>;
> > +                                #size-cells = <0>;
> > 
> > +                                reg = <1>;
> > +
> > +                                csi20vin0: endpoint@0 {
> > +                                        reg = <0>;
> > +                                        remote-endpoint = <&vin0csi20>;
> > +                                };
> > +                        };
> > +                };
> > +        };
> > 
> >  [1] video-interfaces.txt common video media interface
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas Söderlund

Re: [PATCH v9 01/28] rcar-vin: add Gen3 devicetree bindings documentation

[Niklas Söderlund]

Hi Laurent,

Thanks for your comments!

On 2017-12-08 09:46:24 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:15 EET Niklas S�derlund wrote:
> > Document the devicetree bindings for the CSI-2 inputs available on Gen3.
> > 
> > There is a need to add a custom property 'renesas,id' and to define
> > which CSI-2 input is described in which endpoint under the port@1 node.
> > This information is needed since there are a set of predefined routes
> > between each VIN and CSI-2 block. This routing table will be kept
> > inside the driver but in order for it to act on it it must know which
> > VIN and CSI-2 is which.
> > 
> > Signed-off-by: Niklas S�derlund <niklas.soderlund+renesas@ragnatech.se>
> > Acked-by: Rob Herring <robh@kernel.org>
> > ---
> >  .../devicetree/bindings/media/rcar_vin.txt         | 116 +++++++++++++++---
> >  1 file changed, 104 insertions(+), 12 deletions(-)
> > 
> > diff --git a/Documentation/devicetree/bindings/media/rcar_vin.txt
> > b/Documentation/devicetree/bindings/media/rcar_vin.txt index
> > ff9697ed81396e64..5a95d9668d2c7dfd 100644
> > --- a/Documentation/devicetree/bindings/media/rcar_vin.txt
> > +++ b/Documentation/devicetree/bindings/media/rcar_vin.txt
> > @@ -2,8 +2,12 @@ Renesas R-Car Video Input driver (rcar_vin)
> >  -------------------------------------------
> > 
> >  The rcar_vin device provides video input capabilities for the Renesas R-Car
> > -family of devices. The current blocks are always slaves and suppot one
> > input
> > -channel which can be either RGB, YUYV or BT656.
> > +family of devices.
> > +
> > +Each VIN instance has a single parallel input that supports RGB and YUV
> > video,
> > +with both external synchronization and BT.656 synchronization for the
> > latter.
> > +Depending on the instance the VIN input is connected to external SoC pins,
> > or
> > +on Gen3 to a CSI-2 receiver.
> > 
> >   - compatible: Must be one or more of the following
> >     - "renesas,vin-r8a7743" for the R8A7743 device
> > @@ -31,21 +35,38 @@ channel which can be either RGB, YUYV or BT656.
> >  Additionally, an alias named vinX will need to be created to specify
> >  which video input device this is.
> > 
> > -The per-board settings:
> > +The per-board settings Gen2:
> 
> Nitpicking, s/Gen2/for Gen2 platforms/
> 
> (or Gen2 hardware, or Gen2 systems, pick the one you like best)

Good catch, I guess I'm to caught up in it all and it's obvious for me 
what Gen2 is :-) I will update and use "Gen2 platform" throughout this 
document.

> 
> >   - port sub-node describing a single endpoint connected to the vin
> >     as described in video-interfaces.txt[1]. Only the first one will
> >     be considered as each vin interface has one input port.
> > 
> > -   These settings are used to work out video input format and widths
> > -   into the system.
> > +The per-board settings Gen3:
> 
> Ditto.
> 
> > +
> > +Gen3 can support both a single connected parallel input source from
> > +external SoC pins (port0) and/or multiple parallel input sources from
> > +local SoC CSI-2 receivers (port1) depending on SoC.
> > 
> > +- renesas,id - ID number of the VIN, VINx in the documentation.
> > +- ports
> > +    - port0 - sub-node describing a single endpoint connected to the VIN
> > +      from external SoC pins described in video-interfaces.txt[1]. Only
> > +      the first one will be considered as each VIN interface has at most
> > +      one set of SoC external input pins.
> 
> s/port0/port 0/ or s/port0/port@0/

I will use 'port 0' as I think it makes more sens when reading.

> 
> I'd go further than that and make it invalid to have multiple endpoints 
> instead of ignoring all but the first one.

Good catch, I'm describing what the current rcar-vin driver dose and not 
the generic case where in fact only one endpoint in port 0 is valid.  
Will update.

> 
> I would also explicitly state that VIN instances not connected to external 
> pins shall have no port 0.

It's good to be explicit, will add this.

> 
> > +    - port1 - sub-nodes describing one or more endpoints connected to
> > +      the VIN from local SoC CSI-2 receivers. The endpoint numbers must
> > +      use the following schema.
> 
> Nitpicking again, the Gen2-specific properties are indented above while the 
> Gen3 properties are not indented here. Pick the one you prefer :-)

I like it they way it is :-)

The Gen2-specific properties are all on the same level in the dt 
description. While some Gen3-specific properties (port 0 and port 1) are 
indented one level in the dt description and this is why I indented them 
one level in the documentation. To make it clear that they should be 
children of the ports node.

I don't feel strongly about this but I feel it adds to the readability.  
If you don't agree I'm happy to remove the indentation for these two 
properties.

> 
> > -Device node example
> > --------------------
> > +        - Endpoint 0 - sub-node describing the endpoint which is CSI20
> > +        - Endpoint 1 - sub-node describing the endpoint which is CSI21
> > +        - Endpoint 2 - sub-node describing the endpoint which is CSI40
> > +        - Endpoint 3 - sub-node describing the endpoint which is CSI41
> 
> How about s/which is/connected to/ ?

Sounds better, will update.


> 
> > -	aliases {
> > -	       vin0 = &vin0;
> > -	};
> > +Device node example Gen2
> 
> s/Gen2/for Gen2 platforms/
> 
> and same in a few places below.

Will fix.

> 
> > +------------------------
> > +
> > +        aliases {
> > +                vin0 = &vin0;
> > +        };
> 
> This is unrelated, but do we need aliases ?

We don't have aliases for Gen3 but they exists in DTS for Gen2. My 
feeling is that we don't need them. My plan is to once Gen3 support is 
done try to sort this out, both in the documentation and in the dts 
files for Gen2.

> 
> >          vin0: vin@0xe6ef0000 {
> >                  compatible = "renesas,vin-r8a7790",
> > "renesas,rcar-gen2-vin"; @@ -55,8 +76,8 @@ Device node example
> >                  status = "disabled";
> >          };
> > 
> > -Board setup example (vin1 composite video input)
> > -------------------------------------------------
> > +Board setup example Gen2 (vin1 composite video input)
> > +-----------------------------------------------------
> > 
> >  &i2c2   {
> >          status = "ok";
> > @@ -95,6 +116,77 @@ Board setup example (vin1 composite video input)
> >          };
> >  };
> > 
> > +Device node example Gen3
> > +------------------------
> > +
> > +        vin0: video@e6ef0000 {
> > +                compatible = "renesas,vin-r8a7795";
> > +                reg = <0 0xe6ef0000 0 0x1000>;
> > +                interrupts = <GIC_SPI 188 IRQ_TYPE_LEVEL_HIGH>;
> > +                clocks = <&cpg CPG_MOD 811>;
> > +                power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
> > +                resets = <&cpg 811>;
> > +                renesas,id = <0>;
> > +
> > +                ports {
> > +                        #address-cells = <1>;
> > +                        #size-cells = <0>;
> > +
> > +                        port@1 {
> > +                                #address-cells = <1>;
> > +                                #size-cells = <0>;
> > +
> > +                                reg = <1>;
> > +
> > +                                vin0csi20: endpoint@0 {
> > +                                        reg = <0>;
> > +                                        remote-endpoint= <&csi20vin0>;
> > +                                };
> > +                                vin0csi21: endpoint@1 {
> > +                                        reg = <1>;
> > +                                        remote-endpoint= <&csi21vin0>;
> > +                                };
> > +                                vin0csi40: endpoint@2 {
> > +                                        reg = <2>;
> > +                                        remote-endpoint= <&csi40vin0>;
> > +                                };
> > +                        };
> > +                };
> > +        };
> > +
> > +        csi20: csi2@fea80000 {
> > +                compatible = "renesas,r8a7795-csi2";
> > +                reg = <0 0xfea80000 0 0x10000>;
> > +                interrupts = <GIC_SPI 184 IRQ_TYPE_LEVEL_HIGH>;
> > +                clocks = <&cpg CPG_MOD 714>;
> > +                power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
> > +                resets = <&cpg 714>;
> > +
> > +                ports {
> > +                        #address-cells = <1>;
> > +                        #size-cells = <0>;
> > +
> > +                        port@0 {
> > +                                reg = <0>;
> > +                                csi20_in: endpoint {
> > +                                        clock-lanes = <0>;
> > +                                        data-lanes = <1>;
> > +                                        remote-endpoint = <&adv7482_txb>;
> > +                                };
> > +                        };
> > +
> > +                        port@1 {
> > +                                #address-cells = <1>;
> > +                                #size-cells = <0>;
> > 
> > +                                reg = <1>;
> > +
> > +                                csi20vin0: endpoint@0 {
> > +                                        reg = <0>;
> > +                                        remote-endpoint = <&vin0csi20>;
> > +                                };
> > +                        };
> > +                };
> > +        };
> > 
> >  [1] video-interfaces.txt common video media interface
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas S�derlund

Re: [PATCH v9 02/28] rcar-vin: rename poorly named initialize and cleanup functions

[Niklas Söderlund]

Hi Laurent,

Thanks for your comments.

On 2017-12-08 09:49:55 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:16 EET Niklas Söderlund wrote:
> > The functions to initialize and cleanup the hardware and video device
> > where poorly named from the start. Rename them to better describe their
> > intended function.
> 
> It's interesting that you describe the functions' purpose as initialize and 
> cleanup here and name them register and unregister :-) It's not a big deal, 
> but you might want some consistency between the commit message and the code.

Agreed, will fix this as I need to resend anyway.

> 
> > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> 
> Either way,
> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Thanks.

> 
> > ---
> >  drivers/media/platform/rcar-vin/rcar-core.c | 10 +++++-----
> >  drivers/media/platform/rcar-vin/rcar-dma.c  |  6 +++---
> >  drivers/media/platform/rcar-vin/rcar-v4l2.c |  4 ++--
> >  drivers/media/platform/rcar-vin/rcar-vin.h  |  8 ++++----
> >  4 files changed, 14 insertions(+), 14 deletions(-)
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
> > b/drivers/media/platform/rcar-vin/rcar-core.c index
> > 108d776f32651b27..f7a4c21909da6923 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-core.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> > @@ -93,7 +93,7 @@ static int rvin_digital_notify_complete(struct
> > v4l2_async_notifier *notifier) return ret;
> >  	}
> > 
> > -	return rvin_v4l2_probe(vin);
> > +	return rvin_v4l2_register(vin);
> >  }
> > 
> >  static void rvin_digital_notify_unbind(struct v4l2_async_notifier
> > *notifier, @@ -103,7 +103,7 @@ static void
> > rvin_digital_notify_unbind(struct v4l2_async_notifier *notifier, struct
> > rvin_dev *vin = notifier_to_vin(notifier);
> > 
> >  	vin_dbg(vin, "unbind digital subdev %s\n", subdev->name);
> > -	rvin_v4l2_remove(vin);
> > +	rvin_v4l2_unregister(vin);
> >  	vin->digital->subdev = NULL;
> >  }
> > 
> > @@ -245,7 +245,7 @@ static int rcar_vin_probe(struct platform_device *pdev)
> >  	if (irq < 0)
> >  		return irq;
> > 
> > -	ret = rvin_dma_probe(vin, irq);
> > +	ret = rvin_dma_register(vin, irq);
> >  	if (ret)
> >  		return ret;
> > 
> > @@ -260,7 +260,7 @@ static int rcar_vin_probe(struct platform_device *pdev)
> > 
> >  	return 0;
> >  error:
> > -	rvin_dma_remove(vin);
> > +	rvin_dma_unregister(vin);
> >  	v4l2_async_notifier_cleanup(&vin->notifier);
> > 
> >  	return ret;
> > @@ -275,7 +275,7 @@ static int rcar_vin_remove(struct platform_device *pdev)
> > v4l2_async_notifier_unregister(&vin->notifier);
> >  	v4l2_async_notifier_cleanup(&vin->notifier);
> > 
> > -	rvin_dma_remove(vin);
> > +	rvin_dma_unregister(vin);
> > 
> >  	return 0;
> >  }
> > diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> > b/drivers/media/platform/rcar-vin/rcar-dma.c index
> > 23fdff7a7370842e..d701b52d198243b5 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> > @@ -1153,14 +1153,14 @@ static const struct vb2_ops rvin_qops = {
> >  	.wait_finish		= vb2_ops_wait_finish,
> >  };
> > 
> > -void rvin_dma_remove(struct rvin_dev *vin)
> > +void rvin_dma_unregister(struct rvin_dev *vin)
> >  {
> >  	mutex_destroy(&vin->lock);
> > 
> >  	v4l2_device_unregister(&vin->v4l2_dev);
> >  }
> > 
> > -int rvin_dma_probe(struct rvin_dev *vin, int irq)
> > +int rvin_dma_register(struct rvin_dev *vin, int irq)
> >  {
> >  	struct vb2_queue *q = &vin->queue;
> >  	int i, ret;
> > @@ -1208,7 +1208,7 @@ int rvin_dma_probe(struct rvin_dev *vin, int irq)
> > 
> >  	return 0;
> >  error:
> > -	rvin_dma_remove(vin);
> > +	rvin_dma_unregister(vin);
> > 
> >  	return ret;
> >  }
> > diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> > b479b882da12f62d..178aecc94962abe2 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > @@ -839,7 +839,7 @@ static const struct v4l2_file_operations rvin_fops = {
> >  	.read		= vb2_fop_read,
> >  };
> > 
> > -void rvin_v4l2_remove(struct rvin_dev *vin)
> > +void rvin_v4l2_unregister(struct rvin_dev *vin)
> >  {
> >  	v4l2_info(&vin->v4l2_dev, "Removing %s\n",
> >  		  video_device_node_name(&vin->vdev));
> > @@ -866,7 +866,7 @@ static void rvin_notify(struct v4l2_subdev *sd,
> >  	}
> >  }
> > 
> > -int rvin_v4l2_probe(struct rvin_dev *vin)
> > +int rvin_v4l2_register(struct rvin_dev *vin)
> >  {
> >  	struct video_device *vdev = &vin->vdev;
> >  	struct v4l2_subdev *sd = vin_to_source(vin);
> > diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> > b/drivers/media/platform/rcar-vin/rcar-vin.h index
> > 5382078143fb3869..85cb7ec53d2b08b5 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> > +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> > @@ -153,11 +153,11 @@ struct rvin_dev {
> >  #define vin_warn(d, fmt, arg...)	dev_warn(d->dev, fmt, ##arg)
> >  #define vin_err(d, fmt, arg...)		dev_err(d->dev, fmt, ##arg)
> > 
> > -int rvin_dma_probe(struct rvin_dev *vin, int irq);
> > -void rvin_dma_remove(struct rvin_dev *vin);
> > +int rvin_dma_register(struct rvin_dev *vin, int irq);
> > +void rvin_dma_unregister(struct rvin_dev *vin);
> > 
> > -int rvin_v4l2_probe(struct rvin_dev *vin);
> > -void rvin_v4l2_remove(struct rvin_dev *vin);
> > +int rvin_v4l2_register(struct rvin_dev *vin);
> > +void rvin_v4l2_unregister(struct rvin_dev *vin);
> > 
> >  const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas Söderlund

Re: [PATCH v9 02/28] rcar-vin: rename poorly named initialize and cleanup functions

[Niklas Söderlund]

Hi Laurent,

Thanks for your comments.

On 2017-12-08 09:49:55 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:16 EET Niklas S�derlund wrote:
> > The functions to initialize and cleanup the hardware and video device
> > where poorly named from the start. Rename them to better describe their
> > intended function.
> 
> It's interesting that you describe the functions' purpose as initialize and 
> cleanup here and name them register and unregister :-) It's not a big deal, 
> but you might want some consistency between the commit message and the code.

Agreed, will fix this as I need to resend anyway.

> 
> > Signed-off-by: Niklas S�derlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> 
> Either way,
> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Thanks.

> 
> > ---
> >  drivers/media/platform/rcar-vin/rcar-core.c | 10 +++++-----
> >  drivers/media/platform/rcar-vin/rcar-dma.c  |  6 +++---
> >  drivers/media/platform/rcar-vin/rcar-v4l2.c |  4 ++--
> >  drivers/media/platform/rcar-vin/rcar-vin.h  |  8 ++++----
> >  4 files changed, 14 insertions(+), 14 deletions(-)
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
> > b/drivers/media/platform/rcar-vin/rcar-core.c index
> > 108d776f32651b27..f7a4c21909da6923 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-core.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> > @@ -93,7 +93,7 @@ static int rvin_digital_notify_complete(struct
> > v4l2_async_notifier *notifier) return ret;
> >  	}
> > 
> > -	return rvin_v4l2_probe(vin);
> > +	return rvin_v4l2_register(vin);
> >  }
> > 
> >  static void rvin_digital_notify_unbind(struct v4l2_async_notifier
> > *notifier, @@ -103,7 +103,7 @@ static void
> > rvin_digital_notify_unbind(struct v4l2_async_notifier *notifier, struct
> > rvin_dev *vin = notifier_to_vin(notifier);
> > 
> >  	vin_dbg(vin, "unbind digital subdev %s\n", subdev->name);
> > -	rvin_v4l2_remove(vin);
> > +	rvin_v4l2_unregister(vin);
> >  	vin->digital->subdev = NULL;
> >  }
> > 
> > @@ -245,7 +245,7 @@ static int rcar_vin_probe(struct platform_device *pdev)
> >  	if (irq < 0)
> >  		return irq;
> > 
> > -	ret = rvin_dma_probe(vin, irq);
> > +	ret = rvin_dma_register(vin, irq);
> >  	if (ret)
> >  		return ret;
> > 
> > @@ -260,7 +260,7 @@ static int rcar_vin_probe(struct platform_device *pdev)
> > 
> >  	return 0;
> >  error:
> > -	rvin_dma_remove(vin);
> > +	rvin_dma_unregister(vin);
> >  	v4l2_async_notifier_cleanup(&vin->notifier);
> > 
> >  	return ret;
> > @@ -275,7 +275,7 @@ static int rcar_vin_remove(struct platform_device *pdev)
> > v4l2_async_notifier_unregister(&vin->notifier);
> >  	v4l2_async_notifier_cleanup(&vin->notifier);
> > 
> > -	rvin_dma_remove(vin);
> > +	rvin_dma_unregister(vin);
> > 
> >  	return 0;
> >  }
> > diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> > b/drivers/media/platform/rcar-vin/rcar-dma.c index
> > 23fdff7a7370842e..d701b52d198243b5 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> > @@ -1153,14 +1153,14 @@ static const struct vb2_ops rvin_qops = {
> >  	.wait_finish		= vb2_ops_wait_finish,
> >  };
> > 
> > -void rvin_dma_remove(struct rvin_dev *vin)
> > +void rvin_dma_unregister(struct rvin_dev *vin)
> >  {
> >  	mutex_destroy(&vin->lock);
> > 
> >  	v4l2_device_unregister(&vin->v4l2_dev);
> >  }
> > 
> > -int rvin_dma_probe(struct rvin_dev *vin, int irq)
> > +int rvin_dma_register(struct rvin_dev *vin, int irq)
> >  {
> >  	struct vb2_queue *q = &vin->queue;
> >  	int i, ret;
> > @@ -1208,7 +1208,7 @@ int rvin_dma_probe(struct rvin_dev *vin, int irq)
> > 
> >  	return 0;
> >  error:
> > -	rvin_dma_remove(vin);
> > +	rvin_dma_unregister(vin);
> > 
> >  	return ret;
> >  }
> > diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> > b479b882da12f62d..178aecc94962abe2 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > @@ -839,7 +839,7 @@ static const struct v4l2_file_operations rvin_fops = {
> >  	.read		= vb2_fop_read,
> >  };
> > 
> > -void rvin_v4l2_remove(struct rvin_dev *vin)
> > +void rvin_v4l2_unregister(struct rvin_dev *vin)
> >  {
> >  	v4l2_info(&vin->v4l2_dev, "Removing %s\n",
> >  		  video_device_node_name(&vin->vdev));
> > @@ -866,7 +866,7 @@ static void rvin_notify(struct v4l2_subdev *sd,
> >  	}
> >  }
> > 
> > -int rvin_v4l2_probe(struct rvin_dev *vin)
> > +int rvin_v4l2_register(struct rvin_dev *vin)
> >  {
> >  	struct video_device *vdev = &vin->vdev;
> >  	struct v4l2_subdev *sd = vin_to_source(vin);
> > diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> > b/drivers/media/platform/rcar-vin/rcar-vin.h index
> > 5382078143fb3869..85cb7ec53d2b08b5 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> > +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> > @@ -153,11 +153,11 @@ struct rvin_dev {
> >  #define vin_warn(d, fmt, arg...)	dev_warn(d->dev, fmt, ##arg)
> >  #define vin_err(d, fmt, arg...)		dev_err(d->dev, fmt, ##arg)
> > 
> > -int rvin_dma_probe(struct rvin_dev *vin, int irq);
> > -void rvin_dma_remove(struct rvin_dev *vin);
> > +int rvin_dma_register(struct rvin_dev *vin, int irq);
> > +void rvin_dma_unregister(struct rvin_dev *vin);
> > 
> > -int rvin_v4l2_probe(struct rvin_dev *vin);
> > -void rvin_v4l2_remove(struct rvin_dev *vin);
> > +int rvin_v4l2_register(struct rvin_dev *vin);
> > +void rvin_v4l2_unregister(struct rvin_dev *vin);
> > 
> >  const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas S�derlund

Re: [PATCH v9 03/28] rcar-vin: unregister video device on driver removal

[Niklas Söderlund]

Hi Laurent,

Thanks for your comments.

On 2017-12-08 09:54:31 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:17 EET Niklas Söderlund wrote:
> > If the video device was registered by the complete() callback it should
> > be unregistered when the driver is removed.
> 
> The .remove() operation indicates device removal, not driver removal (or, the 
> be more precise, it indicates that the device is unbound from the driver). I'd 
> update the commit message accordingly.

I'm not sure I fully understand this comment.

My take is that .remove() indicates that the device is removed and not 
the driver itself, as the driver might be used by multiple devices and 
the .remove() function is therefor not an indication that the driver is 
being unloaded.

So if I understood you correctly the following would be a better to go 
in the commit message:

"If the video device was registered by the complete() callback it should 
be unregistered when a device is unbound from the driver."

> 
> > Protect from printing an uninitialized video device node name by adding a
> > check in rvin_v4l2_unregister() to identify that the video device is
> > registered.
> > 
> > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >  drivers/media/platform/rcar-vin/rcar-core.c | 2 ++
> >  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 +++
> >  2 files changed, 5 insertions(+)
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
> > b/drivers/media/platform/rcar-vin/rcar-core.c index
> > f7a4c21909da6923..6d99542ec74b49a7 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-core.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> > @@ -272,6 +272,8 @@ static int rcar_vin_remove(struct platform_device *pdev)
> > 
> >  	pm_runtime_disable(&pdev->dev);
> > 
> > +	rvin_v4l2_unregister(vin);
> 
> Unless I'm mistaken, you're unregistering the video device both here and in 
> the unbound() function. That's messy, but it's not really your fault, the V4L2 
> core is very messy in the first place, and registering video devices in the 
> complete() handler is a bad idea. As that can't be fixed for now,
> 
> Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Big thanks for this :-)

> 
> Hans, I still would like to hear your opinion on how this should be solved. 
> You've voiced a few weeks ago that register video devices at probe() time 
> isn't a good idea but you've never explained how we should fix the problem. I 
> still firmly believe that video devices should be registered at probe time, 
> and we need to reach an agreement on a technical solution to this problem.
> 
> >  	v4l2_async_notifier_unregister(&vin->notifier);
> >  	v4l2_async_notifier_cleanup(&vin->notifier);
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> > 178aecc94962abe2..32a658214f48fa49 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > @@ -841,6 +841,9 @@ static const struct v4l2_file_operations rvin_fops = {
> > 
> >  void rvin_v4l2_unregister(struct rvin_dev *vin)
> >  {
> > +	if (!video_is_registered(&vin->vdev))
> > +		return;
> > +
> >  	v4l2_info(&vin->v4l2_dev, "Removing %s\n",
> >  		  video_device_node_name(&vin->vdev));
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas Söderlund

Re: [PATCH v9 03/28] rcar-vin: unregister video device on driver removal

[Niklas Söderlund]

Hi Laurent,

Thanks for your comments.

On 2017-12-08 09:54:31 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:17 EET Niklas S�derlund wrote:
> > If the video device was registered by the complete() callback it should
> > be unregistered when the driver is removed.
> 
> The .remove() operation indicates device removal, not driver removal (or, the 
> be more precise, it indicates that the device is unbound from the driver). I'd 
> update the commit message accordingly.

I'm not sure I fully understand this comment.

My take is that .remove() indicates that the device is removed and not 
the driver itself, as the driver might be used by multiple devices and 
the .remove() function is therefor not an indication that the driver is 
being unloaded.

So if I understood you correctly the following would be a better to go 
in the commit message:

"If the video device was registered by the complete() callback it should 
be unregistered when a device is unbound from the driver."

> 
> > Protect from printing an uninitialized video device node name by adding a
> > check in rvin_v4l2_unregister() to identify that the video device is
> > registered.
> > 
> > Signed-off-by: Niklas S�derlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >  drivers/media/platform/rcar-vin/rcar-core.c | 2 ++
> >  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 +++
> >  2 files changed, 5 insertions(+)
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
> > b/drivers/media/platform/rcar-vin/rcar-core.c index
> > f7a4c21909da6923..6d99542ec74b49a7 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-core.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> > @@ -272,6 +272,8 @@ static int rcar_vin_remove(struct platform_device *pdev)
> > 
> >  	pm_runtime_disable(&pdev->dev);
> > 
> > +	rvin_v4l2_unregister(vin);
> 
> Unless I'm mistaken, you're unregistering the video device both here and in 
> the unbound() function. That's messy, but it's not really your fault, the V4L2 
> core is very messy in the first place, and registering video devices in the 
> complete() handler is a bad idea. As that can't be fixed for now,
> 
> Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Big thanks for this :-)

> 
> Hans, I still would like to hear your opinion on how this should be solved. 
> You've voiced a few weeks ago that register video devices at probe() time 
> isn't a good idea but you've never explained how we should fix the problem. I 
> still firmly believe that video devices should be registered at probe time, 
> and we need to reach an agreement on a technical solution to this problem.
> 
> >  	v4l2_async_notifier_unregister(&vin->notifier);
> >  	v4l2_async_notifier_cleanup(&vin->notifier);
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> > 178aecc94962abe2..32a658214f48fa49 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > @@ -841,6 +841,9 @@ static const struct v4l2_file_operations rvin_fops = {
> > 
> >  void rvin_v4l2_unregister(struct rvin_dev *vin)
> >  {
> > +	if (!video_is_registered(&vin->vdev))
> > +		return;
> > +
> >  	v4l2_info(&vin->v4l2_dev, "Removing %s\n",
> >  		  video_device_node_name(&vin->vdev));
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas S�derlund

Re: [PATCH v9 13/28] rcar-vin: fix handling of single field frames (top, bottom and alternate fields)

[Niklas Söderlund]

Hi Laurent,

Thanks for your feedback.

On 2017-12-08 11:35:18 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:27 EET Niklas Söderlund wrote:
> > There was never proper support in the VIN driver to deliver ALTERNATING
> > field format to user-space, remove this field option. For sources using
> > this field format instead use the VIN hardware feature of combining the
> > fields to an interlaced format. This mode of operation was previously
> > the default behavior and ALTERNATING was only delivered to user-space if
> > explicitly requested. Allowing this to be explicitly requested was a
> > mistake and was never properly tested and never worked due to the
> > constraints put on the field format when it comes to sequence numbers and
> > timestamps etc.
> 
> I'm puzzled, why can't we support V4L2_FIELD_ALTERNATE if we can support 
> V4L2_FIELD_TOP and V4L2_FIELD_BOTTOM ? I don't dispute the fact that the 
> currently implemented logic might be wrong (although I haven't double-checked 
> that), but what prevents us from implementing it correctly ?

Maybe my commit message is fuzzy. We can support V4L2_FIELD_ALTERNATE as 
a source to the VIN but we can't (yet) support delivering it to 
user-space in a good way. So if we have a video source which outputs 
V4L2_FIELD_ALTERNATE we are fine as we can use the hardware to interlace 
that or only capture the TOP or BOTTOM fields.

But the driver logic to capture frames (the whole dance with single and 
continues capture modes) to be able to deal with situations where 
buffers are not queued fast enough currently prevents us from delivering 
V4L2_FIELD_ALTERNATE to user-space. The problem is we can only capture 
(correctly) ALTERNATE if we run in continues mode, if the driver is feed 
buffers to slow and switches to single capture mode we can't live up to 
the specification of the field order from the documentation:

"If fields are successive, without any dropped fields between them 
(fields can drop individually), can be determined from the struct 
v4l2_buffer sequence field."

So even if in single capture mode we switch between TOP and BOTTOM for 
each capture the sequence number would always be sequential but the 
fields would in temporal time potentially be far apparat (depending on 
how fast user-space queues buffers + the time it takes to shutdown and 
startup the VIN capture).

So instead of badly supporting this field order now I feel it's better 
to not support it and once we tackle the issue of trying to remove 
single capture mode (if at all possible) add support for it. But this is 
a task for a different patch-set as this one is quiet large already and 
it's focus is to add Gen3 support.

> 
> > The height should not be cut in half for the format for TOP or BOTTOM
> > fields settings. This was a mistake and it was made visible by the
> > scaling refactoring. Correct behavior is that the user should request a
> > frame size that fits the half height frame reflected in the field
> > setting. If not the VIN will do its best to scale the top or bottom to
> > the requested format and cropping and scaling do not work as expected.
> > 
> > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >  drivers/media/platform/rcar-vin/rcar-dma.c  | 15 +--------
> >  drivers/media/platform/rcar-vin/rcar-v4l2.c | 48 ++++++++++----------------
> >  2 files changed, 19 insertions(+), 44 deletions(-)
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> > b/drivers/media/platform/rcar-vin/rcar-dma.c index
> > 7be5080f742825fb..e6478088d9464221 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> > @@ -617,7 +617,6 @@ static int rvin_setup(struct rvin_dev *vin)
> >  	case V4L2_FIELD_INTERLACED_BT:
> >  		vnmc = VNMC_IM_FULL | VNMC_FOC;
> >  		break;
> > -	case V4L2_FIELD_ALTERNATE:
> >  	case V4L2_FIELD_NONE:
> >  		if (vin->continuous) {
> >  			vnmc = VNMC_IM_ODD_EVEN;
> > @@ -757,18 +756,6 @@ static int rvin_get_active_slot(struct rvin_dev *vin,
> > u32 vnms) return 0;
> >  }
> > 
> > -static enum v4l2_field rvin_get_active_field(struct rvin_dev *vin, u32
> > vnms)
> > -{
> > -	if (vin->format.field == V4L2_FIELD_ALTERNATE) {
> > -		/* If FS is set it's a Even field */
> > -		if (vnms & VNMS_FS)
> > -			return V4L2_FIELD_BOTTOM;
> > -		return V4L2_FIELD_TOP;
> > -	}
> > -
> > -	return vin->format.field;
> > -}
> > -
> >  static void rvin_set_slot_addr(struct rvin_dev *vin, int slot, dma_addr_t
> > addr) {
> >  	const struct rvin_video_format *fmt;
> > @@ -941,7 +928,7 @@ static irqreturn_t rvin_irq(int irq, void *data)
> >  		goto done;
> > 
> >  	/* Capture frame */
> > -	vin->queue_buf[slot]->field = rvin_get_active_field(vin, vnms);
> > +	vin->queue_buf[slot]->field = vin->format.field;
> >  	vin->queue_buf[slot]->sequence = sequence;
> >  	vin->queue_buf[slot]->vb2_buf.timestamp = ktime_get_ns();
> >  	vb2_buffer_done(&vin->queue_buf[slot]->vb2_buf, VB2_BUF_STATE_DONE);
> > diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> > 9cf9ff48ac1e2f4f..37fe1f6c646b0ea3 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > @@ -102,6 +102,24 @@ static int rvin_get_sd_format(struct rvin_dev *vin,
> > struct v4l2_pix_format *pix) if (ret)
> >  		return ret;
> > 
> > +	switch (fmt.format.field) {
> > +	case V4L2_FIELD_TOP:
> > +	case V4L2_FIELD_BOTTOM:
> > +	case V4L2_FIELD_NONE:
> > +	case V4L2_FIELD_INTERLACED_TB:
> > +	case V4L2_FIELD_INTERLACED_BT:
> > +	case V4L2_FIELD_INTERLACED:
> > +		break;
> > +	case V4L2_FIELD_ALTERNATE:
> > +		/* Use VIN hardware to combine the two fields */
> > +		fmt.format.field = V4L2_FIELD_INTERLACED;
> > +		fmt.format.height *= 2;
> > +		break;
> 
> I don't think this is right. If V4L2_FIELD_ALTERNATE isn't supported it should 
> be rejected in the set format handler, or rather this logic should be moved 
> there. It doesn't belong here, rvin_get_sd_format() should only be called with 
> a validated and supported field.

I might misunderstand you here, fmt.format.field comes from a the 
subdevice, just above this:

    struct v4l2_subdev_format fmt = {
	    .which = V4L2_SUBDEV_FORMAT_ACTIVE,
	    .pad = vin->digital->source_pad,
    };
    int ret;

    ret = v4l2_subdev_call(vin_to_source(vin), pad, get_fmt, NULL, &fmt);
    if (ret)
	    return ret;

    switch (fmt.format.field) {
        ...
    }

So the format acted on here is the one from the subdevice, and if it is 
V4L2_FIELD_ALTERNATE it is supported as a source format, just not for 
output to user-space.

> 
> Furthermore treating the pix parameter of this function as both input and 
> output seems very confusing to me. If you want to extend rvin_get_sd_format() 
> beyond just getting the format from the subdev then please document the 
> function with kerneldoc, and let's try to make its API clear.

This comment confuses me, are we looking at the same change? The only 
reference I have to the pix parameter in rvin_get_sd_format() is just 
before the function returns and it's:

   v4l2_fill_pix_format(pix, &fmt.format);

So it's only used as an output for this function.

> 
> > +	default:
> > +		vin->format.field = V4L2_FIELD_NONE;
> > +		break;
> > +	}
> > +
> >  	v4l2_fill_pix_format(pix, &fmt.format);
> > 
> >  	return 0;
> > @@ -115,33 +133,6 @@ static int rvin_reset_format(struct rvin_dev *vin)
> >  	if (ret)
> >  		return ret;
> > 
> > -	/*
> > -	 * If the subdevice uses ALTERNATE field mode and G_STD is
> > -	 * implemented use the VIN HW to combine the two fields to
> > -	 * one INTERLACED frame. The ALTERNATE field mode can still
> > -	 * be requested in S_FMT and be respected, this is just the
> > -	 * default which is applied at probing or when S_STD is called.
> > -	 */
> > -	if (vin->format.field == V4L2_FIELD_ALTERNATE &&
> > -	    v4l2_subdev_has_op(vin_to_source(vin), video, g_std))
> > -		vin->format.field = V4L2_FIELD_INTERLACED;
> > -
> > -	switch (vin->format.field) {
> > -	case V4L2_FIELD_TOP:
> > -	case V4L2_FIELD_BOTTOM:
> > -	case V4L2_FIELD_ALTERNATE:
> > -		vin->format.height /= 2;
> > -		break;
> > -	case V4L2_FIELD_NONE:
> > -	case V4L2_FIELD_INTERLACED_TB:
> > -	case V4L2_FIELD_INTERLACED_BT:
> > -	case V4L2_FIELD_INTERLACED:
> > -		break;
> > -	default:
> > -		vin->format.field = V4L2_FIELD_NONE;
> > -		break;
> > -	}
> > -
> >  	vin->crop.top = vin->crop.left = 0;
> >  	vin->crop.width = vin->format.width;
> >  	vin->crop.height = vin->format.height;
> > @@ -226,9 +217,6 @@ static int __rvin_try_format(struct rvin_dev *vin,
> >  	switch (pix->field) {
> >  	case V4L2_FIELD_TOP:
> >  	case V4L2_FIELD_BOTTOM:
> > -	case V4L2_FIELD_ALTERNATE:
> > -		pix->height /= 2;
> > -		break;
> >  	case V4L2_FIELD_NONE:
> >  	case V4L2_FIELD_INTERLACED_TB:
> >  	case V4L2_FIELD_INTERLACED_BT:
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas Söderlund

Re: [PATCH v9 13/28] rcar-vin: fix handling of single field frames (top, bottom and alternate fields)

[Niklas Söderlund]

Hi Laurent,

Thanks for your feedback.

On 2017-12-08 11:35:18 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:27 EET Niklas S�derlund wrote:
> > There was never proper support in the VIN driver to deliver ALTERNATING
> > field format to user-space, remove this field option. For sources using
> > this field format instead use the VIN hardware feature of combining the
> > fields to an interlaced format. This mode of operation was previously
> > the default behavior and ALTERNATING was only delivered to user-space if
> > explicitly requested. Allowing this to be explicitly requested was a
> > mistake and was never properly tested and never worked due to the
> > constraints put on the field format when it comes to sequence numbers and
> > timestamps etc.
> 
> I'm puzzled, why can't we support V4L2_FIELD_ALTERNATE if we can support 
> V4L2_FIELD_TOP and V4L2_FIELD_BOTTOM ? I don't dispute the fact that the 
> currently implemented logic might be wrong (although I haven't double-checked 
> that), but what prevents us from implementing it correctly ?

Maybe my commit message is fuzzy. We can support V4L2_FIELD_ALTERNATE as 
a source to the VIN but we can't (yet) support delivering it to 
user-space in a good way. So if we have a video source which outputs 
V4L2_FIELD_ALTERNATE we are fine as we can use the hardware to interlace 
that or only capture the TOP or BOTTOM fields.

But the driver logic to capture frames (the whole dance with single and 
continues capture modes) to be able to deal with situations where 
buffers are not queued fast enough currently prevents us from delivering 
V4L2_FIELD_ALTERNATE to user-space. The problem is we can only capture 
(correctly) ALTERNATE if we run in continues mode, if the driver is feed 
buffers to slow and switches to single capture mode we can't live up to 
the specification of the field order from the documentation:

"If fields are successive, without any dropped fields between them 
(fields can drop individually), can be determined from the struct 
v4l2_buffer sequence field."

So even if in single capture mode we switch between TOP and BOTTOM for 
each capture the sequence number would always be sequential but the 
fields would in temporal time potentially be far apparat (depending on 
how fast user-space queues buffers + the time it takes to shutdown and 
startup the VIN capture).

So instead of badly supporting this field order now I feel it's better 
to not support it and once we tackle the issue of trying to remove 
single capture mode (if at all possible) add support for it. But this is 
a task for a different patch-set as this one is quiet large already and 
it's focus is to add Gen3 support.

> 
> > The height should not be cut in half for the format for TOP or BOTTOM
> > fields settings. This was a mistake and it was made visible by the
> > scaling refactoring. Correct behavior is that the user should request a
> > frame size that fits the half height frame reflected in the field
> > setting. If not the VIN will do its best to scale the top or bottom to
> > the requested format and cropping and scaling do not work as expected.
> > 
> > Signed-off-by: Niklas S�derlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >  drivers/media/platform/rcar-vin/rcar-dma.c  | 15 +--------
> >  drivers/media/platform/rcar-vin/rcar-v4l2.c | 48 ++++++++++----------------
> >  2 files changed, 19 insertions(+), 44 deletions(-)
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> > b/drivers/media/platform/rcar-vin/rcar-dma.c index
> > 7be5080f742825fb..e6478088d9464221 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> > @@ -617,7 +617,6 @@ static int rvin_setup(struct rvin_dev *vin)
> >  	case V4L2_FIELD_INTERLACED_BT:
> >  		vnmc = VNMC_IM_FULL | VNMC_FOC;
> >  		break;
> > -	case V4L2_FIELD_ALTERNATE:
> >  	case V4L2_FIELD_NONE:
> >  		if (vin->continuous) {
> >  			vnmc = VNMC_IM_ODD_EVEN;
> > @@ -757,18 +756,6 @@ static int rvin_get_active_slot(struct rvin_dev *vin,
> > u32 vnms) return 0;
> >  }
> > 
> > -static enum v4l2_field rvin_get_active_field(struct rvin_dev *vin, u32
> > vnms)
> > -{
> > -	if (vin->format.field == V4L2_FIELD_ALTERNATE) {
> > -		/* If FS is set it's a Even field */
> > -		if (vnms & VNMS_FS)
> > -			return V4L2_FIELD_BOTTOM;
> > -		return V4L2_FIELD_TOP;
> > -	}
> > -
> > -	return vin->format.field;
> > -}
> > -
> >  static void rvin_set_slot_addr(struct rvin_dev *vin, int slot, dma_addr_t
> > addr) {
> >  	const struct rvin_video_format *fmt;
> > @@ -941,7 +928,7 @@ static irqreturn_t rvin_irq(int irq, void *data)
> >  		goto done;
> > 
> >  	/* Capture frame */
> > -	vin->queue_buf[slot]->field = rvin_get_active_field(vin, vnms);
> > +	vin->queue_buf[slot]->field = vin->format.field;
> >  	vin->queue_buf[slot]->sequence = sequence;
> >  	vin->queue_buf[slot]->vb2_buf.timestamp = ktime_get_ns();
> >  	vb2_buffer_done(&vin->queue_buf[slot]->vb2_buf, VB2_BUF_STATE_DONE);
> > diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> > 9cf9ff48ac1e2f4f..37fe1f6c646b0ea3 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > @@ -102,6 +102,24 @@ static int rvin_get_sd_format(struct rvin_dev *vin,
> > struct v4l2_pix_format *pix) if (ret)
> >  		return ret;
> > 
> > +	switch (fmt.format.field) {
> > +	case V4L2_FIELD_TOP:
> > +	case V4L2_FIELD_BOTTOM:
> > +	case V4L2_FIELD_NONE:
> > +	case V4L2_FIELD_INTERLACED_TB:
> > +	case V4L2_FIELD_INTERLACED_BT:
> > +	case V4L2_FIELD_INTERLACED:
> > +		break;
> > +	case V4L2_FIELD_ALTERNATE:
> > +		/* Use VIN hardware to combine the two fields */
> > +		fmt.format.field = V4L2_FIELD_INTERLACED;
> > +		fmt.format.height *= 2;
> > +		break;
> 
> I don't think this is right. If V4L2_FIELD_ALTERNATE isn't supported it should 
> be rejected in the set format handler, or rather this logic should be moved 
> there. It doesn't belong here, rvin_get_sd_format() should only be called with 
> a validated and supported field.

I might misunderstand you here, fmt.format.field comes from a the 
subdevice, just above this:

    struct v4l2_subdev_format fmt = {
	    .which = V4L2_SUBDEV_FORMAT_ACTIVE,
	    .pad = vin->digital->source_pad,
    };
    int ret;

    ret = v4l2_subdev_call(vin_to_source(vin), pad, get_fmt, NULL, &fmt);
    if (ret)
	    return ret;

    switch (fmt.format.field) {
        ...
    }

So the format acted on here is the one from the subdevice, and if it is 
V4L2_FIELD_ALTERNATE it is supported as a source format, just not for 
output to user-space.

> 
> Furthermore treating the pix parameter of this function as both input and 
> output seems very confusing to me. If you want to extend rvin_get_sd_format() 
> beyond just getting the format from the subdev then please document the 
> function with kerneldoc, and let's try to make its API clear.

This comment confuses me, are we looking at the same change? The only 
reference I have to the pix parameter in rvin_get_sd_format() is just 
before the function returns and it's:

   v4l2_fill_pix_format(pix, &fmt.format);

So it's only used as an output for this function.

> 
> > +	default:
> > +		vin->format.field = V4L2_FIELD_NONE;
> > +		break;
> > +	}
> > +
> >  	v4l2_fill_pix_format(pix, &fmt.format);
> > 
> >  	return 0;
> > @@ -115,33 +133,6 @@ static int rvin_reset_format(struct rvin_dev *vin)
> >  	if (ret)
> >  		return ret;
> > 
> > -	/*
> > -	 * If the subdevice uses ALTERNATE field mode and G_STD is
> > -	 * implemented use the VIN HW to combine the two fields to
> > -	 * one INTERLACED frame. The ALTERNATE field mode can still
> > -	 * be requested in S_FMT and be respected, this is just the
> > -	 * default which is applied at probing or when S_STD is called.
> > -	 */
> > -	if (vin->format.field == V4L2_FIELD_ALTERNATE &&
> > -	    v4l2_subdev_has_op(vin_to_source(vin), video, g_std))
> > -		vin->format.field = V4L2_FIELD_INTERLACED;
> > -
> > -	switch (vin->format.field) {
> > -	case V4L2_FIELD_TOP:
> > -	case V4L2_FIELD_BOTTOM:
> > -	case V4L2_FIELD_ALTERNATE:
> > -		vin->format.height /= 2;
> > -		break;
> > -	case V4L2_FIELD_NONE:
> > -	case V4L2_FIELD_INTERLACED_TB:
> > -	case V4L2_FIELD_INTERLACED_BT:
> > -	case V4L2_FIELD_INTERLACED:
> > -		break;
> > -	default:
> > -		vin->format.field = V4L2_FIELD_NONE;
> > -		break;
> > -	}
> > -
> >  	vin->crop.top = vin->crop.left = 0;
> >  	vin->crop.width = vin->format.width;
> >  	vin->crop.height = vin->format.height;
> > @@ -226,9 +217,6 @@ static int __rvin_try_format(struct rvin_dev *vin,
> >  	switch (pix->field) {
> >  	case V4L2_FIELD_TOP:
> >  	case V4L2_FIELD_BOTTOM:
> > -	case V4L2_FIELD_ALTERNATE:
> > -		pix->height /= 2;
> > -		break;
> >  	case V4L2_FIELD_NONE:
> >  	case V4L2_FIELD_INTERLACED_TB:
> >  	case V4L2_FIELD_INTERLACED_BT:
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas S�derlund

Re: [PATCH v9 11/28] rcar-vin: do not allow changing scaling and composing while streaming

[Niklas Söderlund]

Hi Laurent,

Thanks for your feedback.

On 2017-12-08 11:04:26 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:25 EET Niklas Söderlund wrote:
> > It is possible on Gen2 to change the registers controlling composing and
> > scaling while the stream is running. It is however not a good idea to do
> > so and could result in trouble. There are also no good reasons to allow
> > this, remove immediate reflection in hardware registers from
> > vidioc_s_selection and only configure scaling and composing when the
> > stream starts.
> 
> There is a good reason: digital zoom.

OK, so you would recommend me to drop this patch to keep the current 
behavior?

> 
> > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >  drivers/media/platform/rcar-vin/rcar-dma.c  | 2 +-
> >  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ---
> >  drivers/media/platform/rcar-vin/rcar-vin.h  | 3 ---
> >  3 files changed, 1 insertion(+), 7 deletions(-)
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> > b/drivers/media/platform/rcar-vin/rcar-dma.c index
> > fd14be20a6604d7a..7be5080f742825fb 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> > @@ -514,7 +514,7 @@ static void rvin_set_coeff(struct rvin_dev *vin,
> > unsigned short xs) rvin_write(vin, p_set->coeff_set[23], VNC8C_REG);
> >  }
> > 
> > -void rvin_crop_scale_comp(struct rvin_dev *vin)
> > +static void rvin_crop_scale_comp(struct rvin_dev *vin)
> >  {
> >  	u32 xs, ys;
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> > 254fa1c8770275a5..d6298c684ab2d731 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > @@ -436,9 +436,6 @@ static int rvin_s_selection(struct file *file, void *fh,
> > return -EINVAL;
> >  	}
> > 
> > -	/* HW supports modifying configuration while running */
> > -	rvin_crop_scale_comp(vin);
> > -
> >  	return 0;
> >  }
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> > b/drivers/media/platform/rcar-vin/rcar-vin.h index
> > 36d0f0cc4ce01a6e..67541b483ee43c52 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> > +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> > @@ -175,7 +175,4 @@ void rvin_v4l2_unregister(struct rvin_dev *vin);
> > 
> >  const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
> > 
> > -/* Cropping, composing and scaling */
> > -void rvin_crop_scale_comp(struct rvin_dev *vin);
> > -
> >  #endif
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas Söderlund

Re: [PATCH v9 11/28] rcar-vin: do not allow changing scaling and composing while streaming

[Niklas Söderlund]

Hi Laurent,

Thanks for your feedback.

On 2017-12-08 11:04:26 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:25 EET Niklas S�derlund wrote:
> > It is possible on Gen2 to change the registers controlling composing and
> > scaling while the stream is running. It is however not a good idea to do
> > so and could result in trouble. There are also no good reasons to allow
> > this, remove immediate reflection in hardware registers from
> > vidioc_s_selection and only configure scaling and composing when the
> > stream starts.
> 
> There is a good reason: digital zoom.

OK, so you would recommend me to drop this patch to keep the current 
behavior?

> 
> > Signed-off-by: Niklas S�derlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >  drivers/media/platform/rcar-vin/rcar-dma.c  | 2 +-
> >  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ---
> >  drivers/media/platform/rcar-vin/rcar-vin.h  | 3 ---
> >  3 files changed, 1 insertion(+), 7 deletions(-)
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> > b/drivers/media/platform/rcar-vin/rcar-dma.c index
> > fd14be20a6604d7a..7be5080f742825fb 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> > @@ -514,7 +514,7 @@ static void rvin_set_coeff(struct rvin_dev *vin,
> > unsigned short xs) rvin_write(vin, p_set->coeff_set[23], VNC8C_REG);
> >  }
> > 
> > -void rvin_crop_scale_comp(struct rvin_dev *vin)
> > +static void rvin_crop_scale_comp(struct rvin_dev *vin)
> >  {
> >  	u32 xs, ys;
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> > 254fa1c8770275a5..d6298c684ab2d731 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > @@ -436,9 +436,6 @@ static int rvin_s_selection(struct file *file, void *fh,
> > return -EINVAL;
> >  	}
> > 
> > -	/* HW supports modifying configuration while running */
> > -	rvin_crop_scale_comp(vin);
> > -
> >  	return 0;
> >  }
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> > b/drivers/media/platform/rcar-vin/rcar-vin.h index
> > 36d0f0cc4ce01a6e..67541b483ee43c52 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> > +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> > @@ -175,7 +175,4 @@ void rvin_v4l2_unregister(struct rvin_dev *vin);
> > 
> >  const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
> > 
> > -/* Cropping, composing and scaling */
> > -void rvin_crop_scale_comp(struct rvin_dev *vin);
> > -
> >  #endif
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas S�derlund

Re: [PATCH v9 03/28] rcar-vin: unregister video device on driver removal

[Laurent Pinchart]

Hi Niklas,

On Friday, 8 December 2017 15:09:21 EET Niklas Söderlund wrote:
> On 2017-12-08 09:54:31 +0200, Laurent Pinchart wrote:
> > On Friday, 8 December 2017 03:08:17 EET Niklas Söderlund wrote:
> >> If the video device was registered by the complete() callback it should
> >> be unregistered when the driver is removed.
> > 
> > The .remove() operation indicates device removal, not driver removal (or,
> > the be more precise, it indicates that the device is unbound from the
> > driver). I'd update the commit message accordingly.
> 
> I'm not sure I fully understand this comment.
> 
> My take is that .remove() indicates that the device is removed and not
> the driver itself, as the driver might be used by multiple devices and
> the .remove() function is therefor not an indication that the driver is
> being unloaded.
> 
> So if I understood you correctly the following would be a better to go
> in the commit message:
> 
> "If the video device was registered by the complete() callback it should
> be unregistered when a device is unbound from the driver."

Perfect :-)

> >> Protect from printing an uninitialized video device node name by adding
> >> a check in rvin_v4l2_unregister() to identify that the video device is
> >> registered.
> >> 
> >> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> >> Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> >> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> >> ---
> >> 
> >>  drivers/media/platform/rcar-vin/rcar-core.c | 2 ++
> >>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 +++
> >>  2 files changed, 5 insertions(+)
> >> 
> >> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
> >> b/drivers/media/platform/rcar-vin/rcar-core.c index
> >> f7a4c21909da6923..6d99542ec74b49a7 100644
> >> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> >> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> >> @@ -272,6 +272,8 @@ static int rcar_vin_remove(struct platform_device
> >> *pdev)> > 
> >>  	pm_runtime_disable(&pdev->dev);
> >> 
> >> +	rvin_v4l2_unregister(vin);
> > 
> > Unless I'm mistaken, you're unregistering the video device both here and
> > in the unbound() function. That's messy, but it's not really your fault,
> > the V4L2 core is very messy in the first place, and registering video
> > devices in the complete() handler is a bad idea. As that can't be fixed
> > for now,
> > 
> > Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
> Big thanks for this :-)
> 
> > Hans, I still would like to hear your opinion on how this should be
> > solved.
> > You've voiced a few weeks ago that register video devices at probe() time
> > isn't a good idea but you've never explained how we should fix the
> > problem. I still firmly believe that video devices should be registered
> > at probe time, and we need to reach an agreement on a technical solution
> > to this problem.
> > 
> >>  	v4l2_async_notifier_unregister(&vin->notifier);
> >>  	v4l2_async_notifier_cleanup(&vin->notifier);
> >> 
> >> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> >> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> >> 178aecc94962abe2..32a658214f48fa49 100644
> >> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> >> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> >> @@ -841,6 +841,9 @@ static const struct v4l2_file_operations rvin_fops =
> >> {
> >> 
> >>  void rvin_v4l2_unregister(struct rvin_dev *vin)
> >>  {
> >> +	if (!video_is_registered(&vin->vdev))
> >> +		return;
> >> +
> >>  	v4l2_info(&vin->v4l2_dev, "Removing %s\n",
> >>  		  video_device_node_name(&vin->vdev));

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 11/28] rcar-vin: do not allow changing scaling and composing while streaming

[Laurent Pinchart]

Hi Niklas,

On Friday, 8 December 2017 16:14:23 EET Niklas Söderlund wrote:
> On 2017-12-08 11:04:26 +0200, Laurent Pinchart wrote:
> > On Friday, 8 December 2017 03:08:25 EET Niklas Söderlund wrote:
> >> It is possible on Gen2 to change the registers controlling composing and
> >> scaling while the stream is running. It is however not a good idea to do
> >> so and could result in trouble. There are also no good reasons to allow
> >> this, remove immediate reflection in hardware registers from
> >> vidioc_s_selection and only configure scaling and composing when the
> >> stream starts.
> > 
> > There is a good reason: digital zoom.
> 
> OK, so you would recommend me to drop this patch to keep the current
> behavior?

Yes, unless you don't care about breaking use cases for Gen2, but in that case 
I'd recommend dropping Gen2 support altogether :-)

> >> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> >> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> >> ---
> >> 
> >>  drivers/media/platform/rcar-vin/rcar-dma.c  | 2 +-
> >>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ---
> >>  drivers/media/platform/rcar-vin/rcar-vin.h  | 3 ---
> >>  3 files changed, 1 insertion(+), 7 deletions(-)

[snip]

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 13/28] rcar-vin: fix handling of single field frames (top, bottom and alternate fields)

[Laurent Pinchart]

Hi Niklas,

On Friday, 8 December 2017 16:06:58 EET Niklas Söderlund wrote:
> On 2017-12-08 11:35:18 +0200, Laurent Pinchart wrote:
> > On Friday, 8 December 2017 03:08:27 EET Niklas Söderlund wrote:
> >> There was never proper support in the VIN driver to deliver ALTERNATING
> >> field format to user-space, remove this field option. For sources using
> >> this field format instead use the VIN hardware feature of combining the
> >> fields to an interlaced format. This mode of operation was previously
> >> the default behavior and ALTERNATING was only delivered to user-space if
> >> explicitly requested. Allowing this to be explicitly requested was a
> >> mistake and was never properly tested and never worked due to the
> >> constraints put on the field format when it comes to sequence numbers
> >> and timestamps etc.
> > 
> > I'm puzzled, why can't we support V4L2_FIELD_ALTERNATE if we can support
> > V4L2_FIELD_TOP and V4L2_FIELD_BOTTOM ? I don't dispute the fact that the
> > currently implemented logic might be wrong (although I haven't
> > double-checked that), but what prevents us from implementing it correctly
> > ?
> 
> Maybe my commit message is fuzzy. We can support V4L2_FIELD_ALTERNATE as
> a source to the VIN but we can't (yet) support delivering it to
> user-space in a good way. So if we have a video source which outputs
> V4L2_FIELD_ALTERNATE we are fine as we can use the hardware to interlace
> that or only capture the TOP or BOTTOM fields.
> 
> But the driver logic to capture frames (the whole dance with single and
> continues capture modes) to be able to deal with situations where
> buffers are not queued fast enough currently prevents us from delivering
> V4L2_FIELD_ALTERNATE to user-space. The problem is we can only capture
> (correctly) ALTERNATE if we run in continues mode, if the driver is feed
> buffers to slow and switches to single capture mode we can't live up to
> the specification of the field order from the documentation:
> 
> "If fields are successive, without any dropped fields between them
> (fields can drop individually), can be determined from the struct
> v4l2_buffer sequence field."
> 
> So even if in single capture mode we switch between TOP and BOTTOM for
> each capture the sequence number would always be sequential but the
> fields would in temporal time potentially be far apparat (depending on
> how fast user-space queues buffers + the time it takes to shutdown and
> startup the VIN capture).
> 
> So instead of badly supporting this field order now I feel it's better
> to not support it and once we tackle the issue of trying to remove
> single capture mode (if at all possible) add support for it. But this is
> a task for a different patch-set as this one is quiet large already and
> it's focus is to add Gen3 support.

OK, so we could support capturing alternating fields, but in that case we 
wouldn't be able to provide accurate sequence numbers. I'm fine with dropping 
support for ALTERNATE, but I would capture that information in the commit 
message, and probably as well in a comment in the code.

> >> The height should not be cut in half for the format for TOP or BOTTOM
> >> fields settings. This was a mistake and it was made visible by the
> >> scaling refactoring. Correct behavior is that the user should request a
> >> frame size that fits the half height frame reflected in the field
> >> setting. If not the VIN will do its best to scale the top or bottom to
> >> the requested format and cropping and scaling do not work as expected.
> >> 
> >> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> >> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> >> ---
> >> 
> >>  drivers/media/platform/rcar-vin/rcar-dma.c  | 15 +--------
> >>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 48 +++++++++--------------
> >>  2 files changed, 19 insertions(+), 44 deletions(-)

[snip]

> >> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> >> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> >> 9cf9ff48ac1e2f4f..37fe1f6c646b0ea3 100644
> >> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> >> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> >> @@ -102,6 +102,24 @@ static int rvin_get_sd_format(struct rvin_dev *vin,
> >> struct v4l2_pix_format *pix)
> >> 	if (ret)
> >>  		return ret;
> >> 
> >> +	switch (fmt.format.field) {
> >> +	case V4L2_FIELD_TOP:
> >> +	case V4L2_FIELD_BOTTOM:
> >> +	case V4L2_FIELD_NONE:
> >> +	case V4L2_FIELD_INTERLACED_TB:
> >> +	case V4L2_FIELD_INTERLACED_BT:
> >> +	case V4L2_FIELD_INTERLACED:
> >> +		break;
> >> +	case V4L2_FIELD_ALTERNATE:
> >> +		/* Use VIN hardware to combine the two fields */
> >> +		fmt.format.field = V4L2_FIELD_INTERLACED;
> >> +		fmt.format.height *= 2;
> >> +		break;
> > 
> > I don't think this is right. If V4L2_FIELD_ALTERNATE isn't supported it
> > should be rejected in the set format handler, or rather this logic should
> > be moved there. It doesn't belong here, rvin_get_sd_format() should only
> > be called with a validated and supported field.
> 
> I might misunderstand you here, fmt.format.field comes from a the
> subdevice, just above this:
> 
>     struct v4l2_subdev_format fmt = {
> 	    .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> 	    .pad = vin->digital->source_pad,
>     };
>     int ret;
> 
>     ret = v4l2_subdev_call(vin_to_source(vin), pad, get_fmt, NULL, &fmt);
>     if (ret)
> 	    return ret;
> 
>     switch (fmt.format.field) {
>         ...
>     }
> 
> So the format acted on here is the one from the subdevice, and if it is
> V4L2_FIELD_ALTERNATE it is supported as a source format, just not for
> output to user-space.
> 
> > Furthermore treating the pix parameter of this function as both input and
> > output seems very confusing to me. If you want to extend
> > rvin_get_sd_format() beyond just getting the format from the subdev then
> > please document the function with kerneldoc, and let's try to make its
> > API clear.
> 
> This comment confuses me, are we looking at the same change? The only
> reference I have to the pix parameter in rvin_get_sd_format() is just
> before the function returns and it's:
> 
>    v4l2_fill_pix_format(pix, &fmt.format);
> 
> So it's only used as an output for this function.

I had mistakenly read the switch statement as operating on the pix function 
parameter. My bad, sorry about the noise.

However, V4L2_FIELD_ALTERNATE should still be rejected in the set format 
handler, and I don't think you do so in this patch.

It looks like the field handling logic needs a rewrite :-)

> >> +	default:
> >> +		vin->format.field = V4L2_FIELD_NONE;
> >> +		break;
> >> +	}
> >> +
> >>  	v4l2_fill_pix_format(pix, &fmt.format);
> >>  	
> >>  	return 0;

[snip]

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 19/28] rcar-vin: use different v4l2 operations in media controller mode

[Laurent Pinchart]

Hi Hans,

On Friday, 8 December 2017 12:24:26 EET Hans Verkuil wrote:
> Hi Laurent,
> 
> >> +static const struct v4l2_ioctl_ops rvin_mc_ioctl_ops = {
> >> +	.vidioc_querycap		= rvin_querycap,
> >> +	.vidioc_try_fmt_vid_cap		= rvin_mc_try_fmt_vid_cap,
> >> +	.vidioc_g_fmt_vid_cap		= rvin_g_fmt_vid_cap,
> >> +	.vidioc_s_fmt_vid_cap		= rvin_mc_s_fmt_vid_cap,
> >> +	.vidioc_enum_fmt_vid_cap	= rvin_enum_fmt_vid_cap,
> >> +
> >> +	.vidioc_enum_input		= rvin_mc_enum_input,
> >> +	.vidioc_g_input			= rvin_g_input,
> >> +	.vidioc_s_input			= rvin_s_input,
> > 
> > The input API makes no sense for MC-based devices.
> 
> We've had this discussion before:
> 
> https://patchwork.linuxtv.org/patch/41857/
> 
> There was never a v3 of that patch, so nothing was done with it.
> 
> The issue here is that the spec requires G/S_INPUT to be present for
> video nodes. There currently is no exception for MC devices.

I think we both agree that we should fix the spec :-) It shouldn't be a big 
deal as MC-enabled applications running with an MC-enabled driver don't use 
the input API anyway.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 21/28] rcar-vin: add group allocator functions

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:35 EET Niklas Söderlund wrote:
> In media controller mode all VIN instances needs to be part of the same
> media graph. There is also a need to each VIN instance to know and in
> some cases be able to communicate with other VIN instances.
> 
> Add an allocator framework where the first VIN instance to be probed
> creates a shared data structure and creates a media device.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/rcar-vin/rcar-core.c | 179 ++++++++++++++++++++++++-
>  drivers/media/platform/rcar-vin/rcar-vin.h  |  38 ++++++
>  2 files changed, 215 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
> b/drivers/media/platform/rcar-vin/rcar-core.c index
> 45de4079fd835759..a6713fd61dd87a88 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -20,12 +20,170 @@
>  #include <linux/of_graph.h>
>  #include <linux/platform_device.h>
>  #include <linux/pm_runtime.h>
> +#include <linux/slab.h>
> 
>  #include <media/v4l2-async.h>
>  #include <media/v4l2-fwnode.h>
> 
>  #include "rcar-vin.h"
> 
> +/* ------------------------------------------------------------------------
>  + * Gen3 CSI2 Group Allocator
> + */
> +
> +static int rvin_group_read_id(struct rvin_dev *vin, struct device_node *np)
> +{
> +	u32 val;
> +	int ret;
> +
> +	ret = of_property_read_u32(np, "renesas,id", &val);
> +	if (ret) {
> +		vin_err(vin, "%pOF: No renesas,id property found\n", np);
> +		return -EINVAL;
> +	}
> +
> +	if (val >= RCAR_VIN_NUM) {
> +		vin_err(vin, "%pOF: Invalid renesas,id '%u'\n", np, val);
> +		return -EINVAL;
> +	}

I'd move all this to the main DT parsing function.

> +	return val;
> +}
> +
> +static DEFINE_MUTEX(rvin_group_lock);
> +static struct rvin_group *rvin_group_data;

Nitpicking, static variables are often defined at the beginning of the C file, 
before any function. I'll let you decide whether you want to move them.

> +static void rvin_group_release(struct kref *kref)
> +{
> +	struct rvin_group *group =
> +		container_of(kref, struct rvin_group, refcount);
> +
> +	mutex_lock(&rvin_group_lock);
> +
> +	media_device_unregister(&group->mdev);
> +	media_device_cleanup(&group->mdev);
> +
> +	rvin_group_data = NULL;
> +
> +	mutex_unlock(&rvin_group_lock);
> +
> +	kfree(group);
> +}
> +
> +static struct rvin_group *__rvin_group_allocate(struct rvin_dev *vin)
> +{
> +	struct rvin_group *group;
> +
> +	if (rvin_group_data) {
> +		group = rvin_group_data;
> +		kref_get(&group->refcount);
> +		vin_dbg(vin, "%s: get group=%p\n", __func__, group);
> +		return group;
> +	}
> +
> +	group = kzalloc(sizeof(*group), GFP_KERNEL);
> +	if (!group)
> +		return NULL;
> +
> +	kref_init(&group->refcount);
> +	rvin_group_data = group;

Ouch. While I agree with the global mutex, a single global group variable 
reminds me of the days when per-device data was happily stored in global 
variables because, you know, we will never have more than one instance of that 
device, right ? (Or, sometimes, because the driver author didn't know what an 
instance was.)

Ideally we'd want a linked list of groups, and this function would either 
retrieve the group that the VIN instance is part of, or allocate a new one.

> +	vin_dbg(vin, "%s: alloc group=%p\n", __func__, group);

Do you still need those two debug statements (and all of the other ones below) 
?

> +	return group;
> +}
> +
> +static int rvin_group_add_vin(struct rvin_dev *vin)
> +{
> +	int ret;
> +
> +	ret = rvin_group_read_id(vin, vin->dev->of_node);
> +	if (ret < 0)
> +		return ret;
> +
> +	mutex_lock(&vin->group->lock);
> +
> +	if (vin->group->vin[ret]) {
> +		mutex_unlock(&vin->group->lock);
> +		vin_err(vin, "VIN number %d already occupied\n", ret);
> +		return -EINVAL;

Can this happen ?

> +	}
> +
> +	vin->group->vin[ret] = vin;
> +
> +	mutex_unlock(&vin->group->lock);
> +
> +	vin_dbg(vin, "I'm VIN number %d", ret);
> +
> +	return 0;
> +}
> +
> +static int rvin_group_allocate(struct rvin_dev *vin)
> +{
> +	struct rvin_group *group;
> +	struct media_device *mdev;
> +	int ret;
> +
> +	mutex_lock(&rvin_group_lock);
> +
> +	group = __rvin_group_allocate(vin);
> +	if (!group) {
> +		mutex_unlock(&rvin_group_lock);

I'd use a goto unlock instead of spreading mutex_unlock() calls through the 
function. It's easier to locate a misplaced return in a function that should 
only use gotos than a missing mutex_unlock().

> +		return -ENOMEM;
> +	}
> +
> +	/* Init group data if it is not already initialized */
> +	mdev = &group->mdev;
> +	if (!mdev->dev) {
> +		mutex_init(&group->lock);
> +		mdev->dev = vin->dev;
> +
> +		strlcpy(mdev->driver_name, "Renesas VIN",
> +			sizeof(mdev->driver_name));

How about using the module name ?

> +		strlcpy(mdev->model, vin->dev->of_node->name,
> +			sizeof(mdev->model));

I wonder whether you shouldn't somehow use the compatible string to create the 
model name, in order to accurately report the device model.

> +		snprintf(mdev->bus_info, sizeof(mdev->bus_info), "platform:%s",
> +			 dev_name(mdev->dev));
> +		media_device_init(mdev);
> +
> +		ret = media_device_register(mdev);
> +		if (ret) {
> +			vin_err(vin, "Failed to register media device\n");
> +			kref_put(&group->refcount, rvin_group_release);

This will deadlock if you're releasing the last reference. There's an 
identical issue below.

> +			mutex_unlock(&rvin_group_lock);
> +			return ret;
> +		}
> +	}
> +
> +	vin->group = group;
> +	vin->v4l2_dev.mdev = mdev;
> +
> +	ret = rvin_group_add_vin(vin);

I'd inline the function here.

> +	if (ret) {
> +		kref_put(&group->refcount, rvin_group_release);
> +		mutex_unlock(&rvin_group_lock);
> +		return ret;
> +	}
> +
> +	mutex_unlock(&rvin_group_lock);
> +
> +	return 0;
> +}
> +
> +static void rvin_group_delete(struct rvin_dev *vin)

This function doesn't completely delete the group, it only drops one 
reference. How about rvin_group_put() ? rvin_group_allocate() could then be 
called rvin_group_get().

> +{
> +	unsigned int i;
> +
> +	mutex_lock(&vin->group->lock);
> +	for (i = 0; i < RCAR_VIN_NUM; i++)
> +		if (vin->group->vin[i] == vin)
> +			vin->group->vin[i] = NULL;
> +	mutex_unlock(&vin->group->lock);
> +
> +	vin_dbg(vin, "%s: group=%p\n", __func__, &vin->group);
> +	kref_put(&vin->group->refcount, rvin_group_release);
> +}
> +
>  /* ------------------------------------------------------------------------
>   * Async notifier
>   */
> @@ -236,12 +394,27 @@ static int rvin_digital_graph_init(struct rvin_dev
> *vin)
> 
>  static int rvin_group_init(struct rvin_dev *vin)
>  {
> +	int ret;
> +
> +	ret = rvin_group_allocate(vin);
> +	if (ret)
> +		return ret;
> +
>  	/* All our sources are CSI-2 */
>  	vin->mbus_cfg.type = V4L2_MBUS_CSI2;
>  	vin->mbus_cfg.flags = 0;
> 
>  	vin->pad.flags = MEDIA_PAD_FL_SINK;
> -	return media_entity_pads_init(&vin->vdev.entity, 1, &vin->pad);
> +	ret = media_entity_pads_init(&vin->vdev.entity, 1, &vin->pad);
> +	if (ret)
> +		goto error_group;
> +
> +	return 0;
> +
> +error_group:
> +	rvin_group_delete(vin);
> +
> +	return ret;
>  }
> 
>  /* ------------------------------------------------------------------------
> @@ -361,7 +534,9 @@ static int rcar_vin_remove(struct platform_device
> *pdev) v4l2_async_notifier_unregister(&vin->notifier);
>  	v4l2_async_notifier_cleanup(&vin->notifier);
> 
> -	if (!vin->info->use_mc)
> +	if (vin->info->use_mc)
> +		rvin_group_delete(vin);
> +	else
>  		v4l2_ctrl_handler_free(&vin->ctrl_handler);
> 
>  	rvin_dma_unregister(vin);
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> b/drivers/media/platform/rcar-vin/rcar-vin.h index
> 07d270a976893cdb..5f736a3500b6e10f 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -17,6 +17,8 @@
>  #ifndef __RCAR_VIN__
>  #define __RCAR_VIN__
> 
> +#include <linux/kref.h>
> +
>  #include <media/v4l2-async.h>
>  #include <media/v4l2-ctrls.h>
>  #include <media/v4l2-dev.h>
> @@ -30,6 +32,9 @@
>  /* Address alignment mask for HW buffers */
>  #define HW_BUFFER_MASK 0x7f
> 
> +/* Max number on VIN instances that can be in a system */
> +#define RCAR_VIN_NUM 8
> +
>  enum chip_id {
>  	RCAR_H1,
>  	RCAR_M1,
> @@ -37,6 +42,15 @@ enum chip_id {
>  	RCAR_GEN3,
>  };
> 
> +enum rvin_csi_id {
> +	RVIN_CSI20,
> +	RVIN_CSI21,
> +	RVIN_CSI40,
> +	RVIN_CSI41,
> +	RVIN_CSI_MAX,
> +	RVIN_NC, /* Not Connected */
> +};
> +
>  /**
>   * STOPPED  - No operation in progress
>   * RUNNING  - Operation in progress have buffers
> @@ -75,6 +89,8 @@ struct rvin_graph_entity {
>  	unsigned int sink_pad;
>  };
> 
> +struct rvin_group;

Forward declarations are usually grouped at the beginning of the header.

>  /**
>   * struct rvin_info - Information about the particular VIN implementation
>   * @chip:		type of VIN chip
> @@ -103,6 +119,7 @@ struct rvin_info {
>   * @notifier:		V4L2 asynchronous subdevs notifier
>   * @digital:		entity in the DT for local digital subdevice
>   *
> + * @group:		Gen3 CSI group
>   * @pad:		pad for media controller
>   *
>   * @lock:		protects @queue
> @@ -134,6 +151,7 @@ struct rvin_dev {
>  	struct v4l2_async_notifier notifier;
>  	struct rvin_graph_entity *digital;
> 
> +	struct rvin_group *group;
>  	struct media_pad pad;
> 
>  	struct mutex lock;
> @@ -162,6 +180,26 @@ struct rvin_dev {
>  #define vin_warn(d, fmt, arg...)	dev_warn(d->dev, fmt, ##arg)
>  #define vin_err(d, fmt, arg...)		dev_err(d->dev, fmt, ##arg)
> 
> +/**
> + * struct rvin_group - VIN CSI2 group information
> + * @refcount:		number of VIN instances using the group
> + *
> + * @mdev:		media device which represents the group
> + *
> + * @lock:		protects the vin and csi members
> + * @vin:		VIN instances which are part of the group
> + * @csi:		CSI-2 entities that are part of the group
> + */
> +struct rvin_group {
> +	struct kref refcount;
> +
> +	struct media_device mdev;
> +
> +	struct mutex lock;
> +	struct rvin_dev *vin[RCAR_VIN_NUM];
> +	struct rvin_graph_entity csi[RVIN_CSI_MAX];

Given that the number and types of CSI receivers varies quite a bit between 
SoCs I wonder whether this couldn't be a linked list. If csi was an array of 
pointers it would be less of an issue, but an array of rvin_graph_entity can 
grow large.

> +};
> +
>  int rvin_dma_register(struct rvin_dev *vin, int irq);
>  void rvin_dma_unregister(struct rvin_dev *vin);

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 22/28] rcar-vin: add chsel information to rvin_info

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:36 EET Niklas Söderlund wrote:
> Each Gen3 SoC has a limited set of predefined routing possibilities for
> which CSI-2 device and virtual channel can be routed to which VIN
> instance. Prepare to store this information in the struct rvin_info.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/rcar-vin/rcar-vin.h | 25 +++++++++++++++++++++++++
>  1 file changed, 25 insertions(+)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> b/drivers/media/platform/rcar-vin/rcar-vin.h index
> 5f736a3500b6e10f..41bf24aa8a1a0aed 100644
> --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> @@ -35,6 +35,9 @@
>  /* Max number on VIN instances that can be in a system */
>  #define RCAR_VIN_NUM 8
> 
> +/* Max number of CHSEL values for any Gen3 SoC */
> +#define RCAR_CHSEL_MAX 6
> +
>  enum chip_id {
>  	RCAR_H1,
>  	RCAR_M1,
> @@ -91,6 +94,22 @@ struct rvin_graph_entity {
> 
>  struct rvin_group;
> 
> +/** struct rvin_group_chsel - Map a CSI-2 receiver and channel to a CHSEL
> value
> + * @csi:		VIN internal number for CSI-2 device
> + * @chan:		Output channel of the CSI-2 receiver. Each R-Car CSI-2
> + *			receiver has four output channels facing the VIN
> + *			devices, each channel can carry one CSI-2 Virtual
> + *			Channel (VC) and there are no correlation between
> + *			output channel number and CSI-2 VC. It's up to the
> + *			CSI-2 receiver driver to configure which VC is
> + *			outputted on which channel, the VIN devices only
> + *			cares about output channels.
> + */
> +struct rvin_group_chsel {
> +	enum rvin_csi_id csi;
> +	unsigned int chan;
> +};
> +
>  /**
>   * struct rvin_info - Information about the particular VIN implementation
>   * @chip:		type of VIN chip
> @@ -98,6 +117,9 @@ struct rvin_group;
>   *
>   * max_width:		max input width the VIN supports
>   * max_height:		max input height the VIN supports
> + *
> + * num_chsels:		number of possible chsel values for this VIN
> + * chsels:		routing table VIN <-> CSI-2 for the chsel values
>   */
>  struct rvin_info {
>  	enum chip_id chip;
> @@ -105,6 +127,9 @@ struct rvin_info {
> 
>  	unsigned int max_width;
>  	unsigned int max_height;
> +
> +	unsigned int num_chsels;
> +	struct rvin_group_chsel chsels[RCAR_VIN_NUM][RCAR_CHSEL_MAX];

That will result in a quite sparse array. I wonder whether we couldn't instead 
have an array of num_chsels elements storing the CSI receiver ID, the CSI 
receiver output channel, and the VIN ID.

Furthermore, shouldn't the CSI receiver ID be specified in DT using the 
renesas,id property like we do for the VIN instances, instead of through the 
endpoint number ? I know this will require a bit of refactoring, but I think 
it would stimplify both the DT bindings and the code as we wouldn't hardcode a 
fixed set of CSI receivers.

>  };
> 
>  /**

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 25/28] rcar-vin: extend {start,stop}_streaming to work with media controller

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Friday, 8 December 2017 03:08:39 EET Niklas Söderlund wrote:
> The procedure to start or stop streaming using the non-MC single
> subdevice and the MC graph and multiple subdevices are quite different.
> Create a new function to abstract which method is used based on which
> mode the driver is running in and add logic to start the MC graph.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/rcar-vin/rcar-dma.c | 112 ++++++++++++++++++++++++--
>  1 file changed, 105 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> b/drivers/media/platform/rcar-vin/rcar-dma.c index
> 6c5df13b30d6dd14..8a6674a891aab357 100644
> --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> @@ -1087,15 +1087,115 @@ static void rvin_buffer_queue(struct vb2_buffer
> *vb) spin_unlock_irqrestore(&vin->qlock, flags);
>  }
> 
> +static int rvin_set_stream(struct rvin_dev *vin, int on)
> +{
> +	struct v4l2_subdev_format fmt = {
> +		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
> +	};
> +	struct media_pipeline *pipe;
> +	struct  v4l2_subdev *sd;
> +	struct media_pad *pad;
> +	int ret;
> +
> +	/* No media controller used, simply pass operation to subdevice */
> +	if (!vin->info->use_mc) {
> +		ret = v4l2_subdev_call(vin->digital->subdev, video, s_stream,
> +				       on);
> +
> +		return ret == -ENOIOCTLCMD ? 0 : ret;
> +	}
> +
> +	pad = media_entity_remote_pad(&vin->pad);
> +	if (!pad)
> +		return -EPIPE;
> +
> +	sd = media_entity_to_v4l2_subdev(pad->entity);
> +	if (!sd)
> +		return -EPIPE;

Can a pad have no entity ?

> +	if (!on) {
> +		media_pipeline_stop(&vin->vdev.entity);
> +		return v4l2_subdev_call(sd, video, s_stream, 0);
> +	}
> +
> +	fmt.pad = pad->index;
> +	if (v4l2_subdev_call(sd, pad, get_fmt, NULL, &fmt))
> +		return -EPIPE;
> +
> +	switch (fmt.format.code) {
> +	case MEDIA_BUS_FMT_YUYV8_1X16:
> +	case MEDIA_BUS_FMT_UYVY8_2X8:
> +	case MEDIA_BUS_FMT_UYVY10_2X10:
> +	case MEDIA_BUS_FMT_RGB888_1X24:
> +		vin->code = fmt.format.code;
> +		break;
> +	default:
> +		return -EPIPE;
> +	}
> +
> +	switch (fmt.format.field) {
> +	case V4L2_FIELD_TOP:
> +	case V4L2_FIELD_BOTTOM:
> +	case V4L2_FIELD_NONE:
> +	case V4L2_FIELD_INTERLACED_TB:
> +	case V4L2_FIELD_INTERLACED_BT:
> +	case V4L2_FIELD_INTERLACED:
> +	case V4L2_FIELD_SEQ_TB:
> +	case V4L2_FIELD_SEQ_BT:
> +		/* Supported natively */
> +		break;
> +	case V4L2_FIELD_ALTERNATE:
> +		switch (vin->format.field) {
> +		case V4L2_FIELD_TOP:
> +		case V4L2_FIELD_BOTTOM:
> +		case V4L2_FIELD_NONE:
> +			break;
> +		case V4L2_FIELD_INTERLACED_TB:
> +		case V4L2_FIELD_INTERLACED_BT:
> +		case V4L2_FIELD_INTERLACED:
> +		case V4L2_FIELD_SEQ_TB:
> +		case V4L2_FIELD_SEQ_BT:
> +			/* Use VIN hardware to combine the two fields */
> +			fmt.format.height *= 2;
> +			break;
> +		default:
> +			return -EPIPE;
> +		}
> +		break;
> +	default:
> +		return -EPIPE;
> +	}
> +
> +	if (fmt.format.width != vin->format.width ||
> +	    fmt.format.height != vin->format.height)
> +		return -EPIPE;

Shouldn't you also validate the media bus pixel code against the pixel format 
?

> +	pipe = sd->entity.pipe ? sd->entity.pipe : &vin->vdev.pipe;

How is this protected against race conditions if you call VIDIOC_STREAMON on 
two VIN instances connected to the same CSI-2 receiver ?

And how does this handle two VIN instances connected to two different CSI-2 
receivers that are both connected to the same source ? We will use two 
different pipeline objects, is that a problem ?

> +	if (media_pipeline_start(&vin->vdev.entity, pipe))
> +		return -EPIPE;
> +
> +	ret = v4l2_subdev_call(sd, video, s_stream, 1);
> +	if (ret == -ENOIOCTLCMD)
> +		ret = 0;
> +	if (ret)
> +		media_pipeline_stop(&vin->vdev.entity);
> +
> +	return ret;
> +}
> +
>  static int rvin_start_streaming(struct vb2_queue *vq, unsigned int count)
>  {
>  	struct rvin_dev *vin = vb2_get_drv_priv(vq);
> -	struct v4l2_subdev *sd;
>  	unsigned long flags;
>  	int ret;
> 
> -	sd = vin_to_source(vin);
> -	v4l2_subdev_call(sd, video, s_stream, 1);
> +	ret = rvin_set_stream(vin, 1);
> +	if (ret) {
> +		spin_lock_irqsave(&vin->qlock, flags);
> +		return_all_buffers(vin, VB2_BUF_STATE_QUEUED);
> +		spin_unlock_irqrestore(&vin->qlock, flags);
> +		return ret;
> +	}
> 
>  	spin_lock_irqsave(&vin->qlock, flags);
> 
> @@ -1104,7 +1204,7 @@ static int rvin_start_streaming(struct vb2_queue *vq,
> unsigned int count) ret = rvin_capture_start(vin);
>  	if (ret) {
>  		return_all_buffers(vin, VB2_BUF_STATE_QUEUED);
> -		v4l2_subdev_call(sd, video, s_stream, 0);
> +		rvin_set_stream(vin, 0);
>  	}
> 
>  	spin_unlock_irqrestore(&vin->qlock, flags);
> @@ -1115,7 +1215,6 @@ static int rvin_start_streaming(struct vb2_queue *vq,
> unsigned int count) static void rvin_stop_streaming(struct vb2_queue *vq)
>  {
>  	struct rvin_dev *vin = vb2_get_drv_priv(vq);
> -	struct v4l2_subdev *sd;
>  	unsigned long flags;
>  	int retries = 0;
> 
> @@ -1154,8 +1253,7 @@ static void rvin_stop_streaming(struct vb2_queue *vq)
> 
>  	spin_unlock_irqrestore(&vin->qlock, flags);
> 
> -	sd = vin_to_source(vin);
> -	v4l2_subdev_call(sd, video, s_stream, 0);
> +	rvin_set_stream(vin, 0);
> 
>  	/* disable interrupts */
>  	rvin_disable_interrupts(vin);

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 07/28] rcar-vin: change name of video device

[Sakari Ailus]

On Fri, Dec 08, 2017 at 10:17:36AM +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> (CC'ing Sakari)
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:21 EET Niklas Söderlund wrote:
> > The rcar-vin driver needs to be part of a media controller to support
> > Gen3. Give each VIN instance a unique name so it can be referenced from
> > userspace.
> > 
> > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> > 59ec6d3d119590aa..19de99133f048960 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > @@ -876,7 +876,8 @@ int rvin_v4l2_register(struct rvin_dev *vin)
> >  	vdev->fops = &rvin_fops;
> >  	vdev->v4l2_dev = &vin->v4l2_dev;
> >  	vdev->queue = &vin->queue;
> > -	strlcpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name));
> > +	snprintf(vdev->name, sizeof(vdev->name), "%s %s", KBUILD_MODNAME,
> > +		 dev_name(vin->dev));
> 
> Do we need the module name here ? How about calling them "%s output", 
> dev_name(vin->dev) to emphasize the fact that this is a video node and not a 
> VIN subdev ? This is what the omap3isp and vsp1 drivers do.
> 
> We're suffering a bit from the fact that V4L2 has never standardized a naming 
> scheme for the devices. It wouldn't be fair to ask you to fix that as a 
> prerequisite to get the VIN driver merged, but we clearly have to work on that 
> at some point.

Agreed, this needs to be stable and I think aligning to what omap3isp or
vsp1 do would be a good fix here.

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi

Re: [PATCH v9 07/28] rcar-vin: change name of video device

[Sakari Ailus]

On Fri, Dec 08, 2017 at 10:17:36AM +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> (CC'ing Sakari)
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:21 EET Niklas S�derlund wrote:
> > The rcar-vin driver needs to be part of a media controller to support
> > Gen3. Give each VIN instance a unique name so it can be referenced from
> > userspace.
> > 
> > Signed-off-by: Niklas S�derlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> > 59ec6d3d119590aa..19de99133f048960 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > @@ -876,7 +876,8 @@ int rvin_v4l2_register(struct rvin_dev *vin)
> >  	vdev->fops = &rvin_fops;
> >  	vdev->v4l2_dev = &vin->v4l2_dev;
> >  	vdev->queue = &vin->queue;
> > -	strlcpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name));
> > +	snprintf(vdev->name, sizeof(vdev->name), "%s %s", KBUILD_MODNAME,
> > +		 dev_name(vin->dev));
> 
> Do we need the module name here ? How about calling them "%s output", 
> dev_name(vin->dev) to emphasize the fact that this is a video node and not a 
> VIN subdev ? This is what the omap3isp and vsp1 drivers do.
> 
> We're suffering a bit from the fact that V4L2 has never standardized a naming 
> scheme for the devices. It wouldn't be fair to ask you to fix that as a 
> prerequisite to get the VIN driver merged, but we clearly have to work on that 
> at some point.

Agreed, this needs to be stable and I think aligning to what omap3isp or
vsp1 do would be a good fix here.

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi

Re: [PATCH v9 07/28] rcar-vin: change name of video device

[Laurent Pinchart]

Hi Sakari,

On Thursday, 14 December 2017 16:25:00 EET Sakari Ailus wrote:
> On Fri, Dec 08, 2017 at 10:17:36AM +0200, Laurent Pinchart wrote:
> > On Friday, 8 December 2017 03:08:21 EET Niklas Söderlund wrote:
> > > The rcar-vin driver needs to be part of a media controller to support
> > > Gen3. Give each VIN instance a unique name so it can be referenced from
> > > userspace.
> > > 
> > > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > > Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> > > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > > ---
> > > 
> > >  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ++-
> > >  1 file changed, 2 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > > b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> > > 59ec6d3d119590aa..19de99133f048960 100644
> > > --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > > +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > > @@ -876,7 +876,8 @@ int rvin_v4l2_register(struct rvin_dev *vin)
> > > 
> > >  	vdev->fops = &rvin_fops;
> > >  	vdev->v4l2_dev = &vin->v4l2_dev;
> > >  	vdev->queue = &vin->queue;
> > > 
> > > -	strlcpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name));
> > > +	snprintf(vdev->name, sizeof(vdev->name), "%s %s", KBUILD_MODNAME,
> > > +		 dev_name(vin->dev));
> > 
> > Do we need the module name here ? How about calling them "%s output",
> > dev_name(vin->dev) to emphasize the fact that this is a video node and not
> > a VIN subdev ? This is what the omap3isp and vsp1 drivers do.
> > 
> > We're suffering a bit from the fact that V4L2 has never standardized a
> > naming scheme for the devices. It wouldn't be fair to ask you to fix that
> > as a prerequisite to get the VIN driver merged, but we clearly have to
> > work on that at some point.
> 
> Agreed, this needs to be stable and I think aligning to what omap3isp or
> vsp1 do would be a good fix here.

Even omap3isp and vsp1 are not fully aligned, so I think we need to design a 
naming policy and document it.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 07/28] rcar-vin: change name of video device

[Niklas Söderlund]

Hi Laurent and Sakari,

Thanks for your comments.

On 2017-12-14 17:50:24 +0200, Laurent Pinchart wrote:
> Hi Sakari,
> 
> On Thursday, 14 December 2017 16:25:00 EET Sakari Ailus wrote:
> > On Fri, Dec 08, 2017 at 10:17:36AM +0200, Laurent Pinchart wrote:
> > > On Friday, 8 December 2017 03:08:21 EET Niklas Söderlund wrote:
> > > > The rcar-vin driver needs to be part of a media controller to support
> > > > Gen3. Give each VIN instance a unique name so it can be referenced from
> > > > userspace.
> > > > 
> > > > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > > > Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> > > > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > > > ---
> > > > 
> > > >  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ++-
> > > >  1 file changed, 2 insertions(+), 1 deletion(-)
> > > > 
> > > > diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > > > b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> > > > 59ec6d3d119590aa..19de99133f048960 100644
> > > > --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > > > +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > > > @@ -876,7 +876,8 @@ int rvin_v4l2_register(struct rvin_dev *vin)
> > > > 
> > > >  	vdev->fops = &rvin_fops;
> > > >  	vdev->v4l2_dev = &vin->v4l2_dev;
> > > >  	vdev->queue = &vin->queue;
> > > > 
> > > > -	strlcpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name));
> > > > +	snprintf(vdev->name, sizeof(vdev->name), "%s %s", KBUILD_MODNAME,
> > > > +		 dev_name(vin->dev));
> > > 
> > > Do we need the module name here ? How about calling them "%s output",
> > > dev_name(vin->dev) to emphasize the fact that this is a video node and not
> > > a VIN subdev ? This is what the omap3isp and vsp1 drivers do.
> > > 
> > > We're suffering a bit from the fact that V4L2 has never standardized a
> > > naming scheme for the devices. It wouldn't be fair to ask you to fix that
> > > as a prerequisite to get the VIN driver merged, but we clearly have to
> > > work on that at some point.
> > 
> > Agreed, this needs to be stable and I think aligning to what omap3isp or
> > vsp1 do would be a good fix here.
> 
> Even omap3isp and vsp1 are not fully aligned, so I think we need to design a 
> naming policy and document it.

I agree that align this is a good idea. And for this reason I chosen to 
update this patch as such:

"%s output", dev_name(vin->dev)

I hope this is a step in the correct direction. If not please let me 
know as soon as possible so I can minimize the trouble for the other 
developers doing stuff on-top of this series and there test scripts :-)

> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas Söderlund

Re: [PATCH v9 07/28] rcar-vin: change name of video device

[Niklas Söderlund]

Hi Laurent and Sakari,

Thanks for your comments.

On 2017-12-14 17:50:24 +0200, Laurent Pinchart wrote:
> Hi Sakari,
> 
> On Thursday, 14 December 2017 16:25:00 EET Sakari Ailus wrote:
> > On Fri, Dec 08, 2017 at 10:17:36AM +0200, Laurent Pinchart wrote:
> > > On Friday, 8 December 2017 03:08:21 EET Niklas S�derlund wrote:
> > > > The rcar-vin driver needs to be part of a media controller to support
> > > > Gen3. Give each VIN instance a unique name so it can be referenced from
> > > > userspace.
> > > > 
> > > > Signed-off-by: Niklas S�derlund <niklas.soderlund+renesas@ragnatech.se>
> > > > Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
> > > > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > > > ---
> > > > 
> > > >  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ++-
> > > >  1 file changed, 2 insertions(+), 1 deletion(-)
> > > > 
> > > > diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > > > b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> > > > 59ec6d3d119590aa..19de99133f048960 100644
> > > > --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > > > +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > > > @@ -876,7 +876,8 @@ int rvin_v4l2_register(struct rvin_dev *vin)
> > > > 
> > > >  	vdev->fops = &rvin_fops;
> > > >  	vdev->v4l2_dev = &vin->v4l2_dev;
> > > >  	vdev->queue = &vin->queue;
> > > > 
> > > > -	strlcpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name));
> > > > +	snprintf(vdev->name, sizeof(vdev->name), "%s %s", KBUILD_MODNAME,
> > > > +		 dev_name(vin->dev));
> > > 
> > > Do we need the module name here ? How about calling them "%s output",
> > > dev_name(vin->dev) to emphasize the fact that this is a video node and not
> > > a VIN subdev ? This is what the omap3isp and vsp1 drivers do.
> > > 
> > > We're suffering a bit from the fact that V4L2 has never standardized a
> > > naming scheme for the devices. It wouldn't be fair to ask you to fix that
> > > as a prerequisite to get the VIN driver merged, but we clearly have to
> > > work on that at some point.
> > 
> > Agreed, this needs to be stable and I think aligning to what omap3isp or
> > vsp1 do would be a good fix here.
> 
> Even omap3isp and vsp1 are not fully aligned, so I think we need to design a 
> naming policy and document it.

I agree that align this is a good idea. And for this reason I chosen to 
update this patch as such:

"%s output", dev_name(vin->dev)

I hope this is a step in the correct direction. If not please let me 
know as soon as possible so I can minimize the trouble for the other 
developers doing stuff on-top of this series and there test scripts :-)

> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas S�derlund

Re: [PATCH v9 09/28] rcar-vin: all Gen2 boards can scale simplify logic

[Niklas Söderlund]

Hi Laurent,

Thanks for your comment.

On 2017-12-08 10:33:32 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:23 EET Niklas Söderlund wrote:
> > The logic to preserve the requested format width and height are too
> > complex and come from a premature optimization for Gen3. All Gen2 SoC
> > can scale and the Gen3 implementation will not use these functions at
> > all so simply preserve the width and height when interacting with the
> > subdevice much like the field is preserved simplifies the logic quite a
> > bit.
> > 
> > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >  drivers/media/platform/rcar-vin/rcar-dma.c  |  8 --------
> >  drivers/media/platform/rcar-vin/rcar-v4l2.c | 22 ++++++++++------------
> >  drivers/media/platform/rcar-vin/rcar-vin.h  |  2 --
> >  3 files changed, 10 insertions(+), 22 deletions(-)
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> > b/drivers/media/platform/rcar-vin/rcar-dma.c index
> > a7cda3922cb74baa..fd14be20a6604d7a 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> > @@ -585,14 +585,6 @@ void rvin_crop_scale_comp(struct rvin_dev *vin)
> >  		0, 0);
> >  }
> > 
> > -void rvin_scale_try(struct rvin_dev *vin, struct v4l2_pix_format *pix,
> > -		    u32 width, u32 height)
> > -{
> > -	/* All VIN channels on Gen2 have scalers */
> > -	pix->width = width;
> > -	pix->height = height;
> > -}
> > -
> >  /*
> > ---------------------------------------------------------------------------
> > -- * Hardware setup
> >   */
> > diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> > 19de99133f048960..1c5e7f6d5b963740 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > @@ -166,6 +166,7 @@ static int __rvin_try_format_source(struct rvin_dev
> > *vin, .which = which,
> >  	};
> >  	enum v4l2_field field;
> > +	u32 width, height;
> >  	int ret;
> > 
> >  	sd = vin_to_source(vin);
> > @@ -178,7 +179,10 @@ static int __rvin_try_format_source(struct rvin_dev
> > *vin,
> > 
> >  	format.pad = vin->digital->source_pad;
> > 
> > +	/* Allow the video device to override field and to scale */
> >  	field = pix->field;
> > +	width = pix->width;
> > +	height = pix->height;
> > 
> >  	ret = v4l2_subdev_call(sd, pad, set_fmt, pad_cfg, &format);
> >  	if (ret < 0 && ret != -ENOIOCTLCMD)
> > @@ -191,6 +195,9 @@ static int __rvin_try_format_source(struct rvin_dev
> > *vin, source->width = pix->width;
> >  	source->height = pix->height;
> > 
> 
> I would move the pix->field = field line not shown above to here.

Agree, thanks I will do so.

> 
> Apart from that,
> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Thanks.

> 
> > +	pix->width = width;
> > +	pix->height = height;
> > +
> >  	vin_dbg(vin, "Source resolution: %ux%u\n", source->width,
> >  		source->height);
> > 
> > @@ -204,13 +211,9 @@ static int __rvin_try_format(struct rvin_dev *vin,
> >  			     struct v4l2_pix_format *pix,
> >  			     struct rvin_source_fmt *source)
> >  {
> > -	u32 rwidth, rheight, walign;
> > +	u32 walign;
> >  	int ret;
> > 
> > -	/* Requested */
> > -	rwidth = pix->width;
> > -	rheight = pix->height;
> > -
> >  	/* Keep current field if no specific one is asked for */
> >  	if (pix->field == V4L2_FIELD_ANY)
> >  		pix->field = vin->format.field;
> > @@ -248,10 +251,6 @@ static int __rvin_try_format(struct rvin_dev *vin,
> >  		break;
> >  	}
> > 
> > -	/* If source can't match format try if VIN can scale */
> > -	if (source->width != rwidth || source->height != rheight)
> > -		rvin_scale_try(vin, pix, rwidth, rheight);
> > -
> >  	/* HW limit width to a multiple of 32 (2^5) for NV16 else 2 (2^1) */
> >  	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
> > 
> > @@ -270,9 +269,8 @@ static int __rvin_try_format(struct rvin_dev *vin,
> >  		return -EINVAL;
> >  	}
> > 
> > -	vin_dbg(vin, "Requested %ux%u Got %ux%u bpl: %d size: %d\n",
> > -		rwidth, rheight, pix->width, pix->height,
> > -		pix->bytesperline, pix->sizeimage);
> > +	vin_dbg(vin, "Format %ux%u bpl: %d size: %d\n",
> > +		pix->width, pix->height, pix->bytesperline, pix->sizeimage);
> > 
> >  	return 0;
> >  }
> > diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> > b/drivers/media/platform/rcar-vin/rcar-vin.h index
> > 646f897f5c05ec4e..36d0f0cc4ce01a6e 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> > +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> > @@ -176,8 +176,6 @@ void rvin_v4l2_unregister(struct rvin_dev *vin);
> >  const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
> > 
> >  /* Cropping, composing and scaling */
> > -void rvin_scale_try(struct rvin_dev *vin, struct v4l2_pix_format *pix,
> > -		    u32 width, u32 height);
> >  void rvin_crop_scale_comp(struct rvin_dev *vin);
> > 
> >  #endif
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas Söderlund

Re: [PATCH v9 09/28] rcar-vin: all Gen2 boards can scale simplify logic

[Niklas Söderlund]

Hi Laurent,

Thanks for your comment.

On 2017-12-08 10:33:32 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:23 EET Niklas S�derlund wrote:
> > The logic to preserve the requested format width and height are too
> > complex and come from a premature optimization for Gen3. All Gen2 SoC
> > can scale and the Gen3 implementation will not use these functions at
> > all so simply preserve the width and height when interacting with the
> > subdevice much like the field is preserved simplifies the logic quite a
> > bit.
> > 
> > Signed-off-by: Niklas S�derlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >  drivers/media/platform/rcar-vin/rcar-dma.c  |  8 --------
> >  drivers/media/platform/rcar-vin/rcar-v4l2.c | 22 ++++++++++------------
> >  drivers/media/platform/rcar-vin/rcar-vin.h  |  2 --
> >  3 files changed, 10 insertions(+), 22 deletions(-)
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> > b/drivers/media/platform/rcar-vin/rcar-dma.c index
> > a7cda3922cb74baa..fd14be20a6604d7a 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> > @@ -585,14 +585,6 @@ void rvin_crop_scale_comp(struct rvin_dev *vin)
> >  		0, 0);
> >  }
> > 
> > -void rvin_scale_try(struct rvin_dev *vin, struct v4l2_pix_format *pix,
> > -		    u32 width, u32 height)
> > -{
> > -	/* All VIN channels on Gen2 have scalers */
> > -	pix->width = width;
> > -	pix->height = height;
> > -}
> > -
> >  /*
> > ---------------------------------------------------------------------------
> > -- * Hardware setup
> >   */
> > diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> > 19de99133f048960..1c5e7f6d5b963740 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > @@ -166,6 +166,7 @@ static int __rvin_try_format_source(struct rvin_dev
> > *vin, .which = which,
> >  	};
> >  	enum v4l2_field field;
> > +	u32 width, height;
> >  	int ret;
> > 
> >  	sd = vin_to_source(vin);
> > @@ -178,7 +179,10 @@ static int __rvin_try_format_source(struct rvin_dev
> > *vin,
> > 
> >  	format.pad = vin->digital->source_pad;
> > 
> > +	/* Allow the video device to override field and to scale */
> >  	field = pix->field;
> > +	width = pix->width;
> > +	height = pix->height;
> > 
> >  	ret = v4l2_subdev_call(sd, pad, set_fmt, pad_cfg, &format);
> >  	if (ret < 0 && ret != -ENOIOCTLCMD)
> > @@ -191,6 +195,9 @@ static int __rvin_try_format_source(struct rvin_dev
> > *vin, source->width = pix->width;
> >  	source->height = pix->height;
> > 
> 
> I would move the pix->field = field line not shown above to here.

Agree, thanks I will do so.

> 
> Apart from that,
> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Thanks.

> 
> > +	pix->width = width;
> > +	pix->height = height;
> > +
> >  	vin_dbg(vin, "Source resolution: %ux%u\n", source->width,
> >  		source->height);
> > 
> > @@ -204,13 +211,9 @@ static int __rvin_try_format(struct rvin_dev *vin,
> >  			     struct v4l2_pix_format *pix,
> >  			     struct rvin_source_fmt *source)
> >  {
> > -	u32 rwidth, rheight, walign;
> > +	u32 walign;
> >  	int ret;
> > 
> > -	/* Requested */
> > -	rwidth = pix->width;
> > -	rheight = pix->height;
> > -
> >  	/* Keep current field if no specific one is asked for */
> >  	if (pix->field == V4L2_FIELD_ANY)
> >  		pix->field = vin->format.field;
> > @@ -248,10 +251,6 @@ static int __rvin_try_format(struct rvin_dev *vin,
> >  		break;
> >  	}
> > 
> > -	/* If source can't match format try if VIN can scale */
> > -	if (source->width != rwidth || source->height != rheight)
> > -		rvin_scale_try(vin, pix, rwidth, rheight);
> > -
> >  	/* HW limit width to a multiple of 32 (2^5) for NV16 else 2 (2^1) */
> >  	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
> > 
> > @@ -270,9 +269,8 @@ static int __rvin_try_format(struct rvin_dev *vin,
> >  		return -EINVAL;
> >  	}
> > 
> > -	vin_dbg(vin, "Requested %ux%u Got %ux%u bpl: %d size: %d\n",
> > -		rwidth, rheight, pix->width, pix->height,
> > -		pix->bytesperline, pix->sizeimage);
> > +	vin_dbg(vin, "Format %ux%u bpl: %d size: %d\n",
> > +		pix->width, pix->height, pix->bytesperline, pix->sizeimage);
> > 
> >  	return 0;
> >  }
> > diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> > b/drivers/media/platform/rcar-vin/rcar-vin.h index
> > 646f897f5c05ec4e..36d0f0cc4ce01a6e 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> > +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> > @@ -176,8 +176,6 @@ void rvin_v4l2_unregister(struct rvin_dev *vin);
> >  const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
> > 
> >  /* Cropping, composing and scaling */
> > -void rvin_scale_try(struct rvin_dev *vin, struct v4l2_pix_format *pix,
> > -		    u32 width, u32 height);
> >  void rvin_crop_scale_comp(struct rvin_dev *vin);
> > 
> >  #endif
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas S�derlund

Re: [PATCH v9 11/28] rcar-vin: do not allow changing scaling and composing while streaming

[Niklas Söderlund]

Hi Laurent,

On 2017-12-08 21:20:48 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> On Friday, 8 December 2017 16:14:23 EET Niklas Söderlund wrote:
> > On 2017-12-08 11:04:26 +0200, Laurent Pinchart wrote:
> > > On Friday, 8 December 2017 03:08:25 EET Niklas Söderlund wrote:
> > >> It is possible on Gen2 to change the registers controlling composing and
> > >> scaling while the stream is running. It is however not a good idea to do
> > >> so and could result in trouble. There are also no good reasons to allow
> > >> this, remove immediate reflection in hardware registers from
> > >> vidioc_s_selection and only configure scaling and composing when the
> > >> stream starts.
> > > 
> > > There is a good reason: digital zoom.
> > 
> > OK, so you would recommend me to drop this patch to keep the current
> > behavior?
> 
> Yes, unless you don't care about breaking use cases for Gen2, but in that case 
> I'd recommend dropping Gen2 support altogether :-)

Well I don't want to do that so I will drop this patch for the next 
version. Thanks for clarifying the use-case for this.

> 
> > >> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > >> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > >> ---
> > >> 
> > >>  drivers/media/platform/rcar-vin/rcar-dma.c  | 2 +-
> > >>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ---
> > >>  drivers/media/platform/rcar-vin/rcar-vin.h  | 3 ---
> > >>  3 files changed, 1 insertion(+), 7 deletions(-)
> 
> [snip]
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas Söderlund

Re: [PATCH v9 11/28] rcar-vin: do not allow changing scaling and composing while streaming

[Niklas Söderlund]

Hi Laurent,

On 2017-12-08 21:20:48 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> On Friday, 8 December 2017 16:14:23 EET Niklas S�derlund wrote:
> > On 2017-12-08 11:04:26 +0200, Laurent Pinchart wrote:
> > > On Friday, 8 December 2017 03:08:25 EET Niklas S�derlund wrote:
> > >> It is possible on Gen2 to change the registers controlling composing and
> > >> scaling while the stream is running. It is however not a good idea to do
> > >> so and could result in trouble. There are also no good reasons to allow
> > >> this, remove immediate reflection in hardware registers from
> > >> vidioc_s_selection and only configure scaling and composing when the
> > >> stream starts.
> > > 
> > > There is a good reason: digital zoom.
> > 
> > OK, so you would recommend me to drop this patch to keep the current
> > behavior?
> 
> Yes, unless you don't care about breaking use cases for Gen2, but in that case 
> I'd recommend dropping Gen2 support altogether :-)

Well I don't want to do that so I will drop this patch for the next 
version. Thanks for clarifying the use-case for this.

> 
> > >> Signed-off-by: Niklas S�derlund <niklas.soderlund+renesas@ragnatech.se>
> > >> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > >> ---
> > >> 
> > >>  drivers/media/platform/rcar-vin/rcar-dma.c  | 2 +-
> > >>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ---
> > >>  drivers/media/platform/rcar-vin/rcar-vin.h  | 3 ---
> > >>  3 files changed, 1 insertion(+), 7 deletions(-)
> 
> [snip]
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas S�derlund

Re: [PATCH v9 13/28] rcar-vin: fix handling of single field frames (top, bottom and alternate fields)

[Niklas Söderlund]

Hi Laurent,

Thanks for your comments.

On 2017-12-08 21:30:20 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> On Friday, 8 December 2017 16:06:58 EET Niklas Söderlund wrote:
> > On 2017-12-08 11:35:18 +0200, Laurent Pinchart wrote:
> > > On Friday, 8 December 2017 03:08:27 EET Niklas Söderlund wrote:
> > >> There was never proper support in the VIN driver to deliver ALTERNATING
> > >> field format to user-space, remove this field option. For sources using
> > >> this field format instead use the VIN hardware feature of combining the
> > >> fields to an interlaced format. This mode of operation was previously
> > >> the default behavior and ALTERNATING was only delivered to user-space if
> > >> explicitly requested. Allowing this to be explicitly requested was a
> > >> mistake and was never properly tested and never worked due to the
> > >> constraints put on the field format when it comes to sequence numbers
> > >> and timestamps etc.
> > > 
> > > I'm puzzled, why can't we support V4L2_FIELD_ALTERNATE if we can support
> > > V4L2_FIELD_TOP and V4L2_FIELD_BOTTOM ? I don't dispute the fact that the
> > > currently implemented logic might be wrong (although I haven't
> > > double-checked that), but what prevents us from implementing it correctly
> > > ?
> > 
> > Maybe my commit message is fuzzy. We can support V4L2_FIELD_ALTERNATE as
> > a source to the VIN but we can't (yet) support delivering it to
> > user-space in a good way. So if we have a video source which outputs
> > V4L2_FIELD_ALTERNATE we are fine as we can use the hardware to interlace
> > that or only capture the TOP or BOTTOM fields.
> > 
> > But the driver logic to capture frames (the whole dance with single and
> > continues capture modes) to be able to deal with situations where
> > buffers are not queued fast enough currently prevents us from delivering
> > V4L2_FIELD_ALTERNATE to user-space. The problem is we can only capture
> > (correctly) ALTERNATE if we run in continues mode, if the driver is feed
> > buffers to slow and switches to single capture mode we can't live up to
> > the specification of the field order from the documentation:
> > 
> > "If fields are successive, without any dropped fields between them
> > (fields can drop individually), can be determined from the struct
> > v4l2_buffer sequence field."
> > 
> > So even if in single capture mode we switch between TOP and BOTTOM for
> > each capture the sequence number would always be sequential but the
> > fields would in temporal time potentially be far apparat (depending on
> > how fast user-space queues buffers + the time it takes to shutdown and
> > startup the VIN capture).
> > 
> > So instead of badly supporting this field order now I feel it's better
> > to not support it and once we tackle the issue of trying to remove
> > single capture mode (if at all possible) add support for it. But this is
> > a task for a different patch-set as this one is quiet large already and
> > it's focus is to add Gen3 support.
> 
> OK, so we could support capturing alternating fields, but in that case we 
> wouldn't be able to provide accurate sequence numbers. I'm fine with dropping 
> support for ALTERNATE, but I would capture that information in the commit 
> message, and probably as well in a comment in the code.

Agree, I tried to capture this in the commit message. But as You did not 
understand it I need to make a better job. I will expand on this in the 
commit message and add a comment in the code. Thanks.

> 
> > >> The height should not be cut in half for the format for TOP or BOTTOM
> > >> fields settings. This was a mistake and it was made visible by the
> > >> scaling refactoring. Correct behavior is that the user should request a
> > >> frame size that fits the half height frame reflected in the field
> > >> setting. If not the VIN will do its best to scale the top or bottom to
> > >> the requested format and cropping and scaling do not work as expected.
> > >> 
> > >> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > >> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > >> ---
> > >> 
> > >>  drivers/media/platform/rcar-vin/rcar-dma.c  | 15 +--------
> > >>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 48 +++++++++--------------
> > >>  2 files changed, 19 insertions(+), 44 deletions(-)
> 
> [snip]
> 
> > >> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > >> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> > >> 9cf9ff48ac1e2f4f..37fe1f6c646b0ea3 100644
> > >> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > >> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > >> @@ -102,6 +102,24 @@ static int rvin_get_sd_format(struct rvin_dev *vin,
> > >> struct v4l2_pix_format *pix)
> > >> 	if (ret)
> > >>  		return ret;
> > >> 
> > >> +	switch (fmt.format.field) {
> > >> +	case V4L2_FIELD_TOP:
> > >> +	case V4L2_FIELD_BOTTOM:
> > >> +	case V4L2_FIELD_NONE:
> > >> +	case V4L2_FIELD_INTERLACED_TB:
> > >> +	case V4L2_FIELD_INTERLACED_BT:
> > >> +	case V4L2_FIELD_INTERLACED:
> > >> +		break;
> > >> +	case V4L2_FIELD_ALTERNATE:
> > >> +		/* Use VIN hardware to combine the two fields */
> > >> +		fmt.format.field = V4L2_FIELD_INTERLACED;
> > >> +		fmt.format.height *= 2;
> > >> +		break;
> > > 
> > > I don't think this is right. If V4L2_FIELD_ALTERNATE isn't supported it
> > > should be rejected in the set format handler, or rather this logic should
> > > be moved there. It doesn't belong here, rvin_get_sd_format() should only
> > > be called with a validated and supported field.
> > 
> > I might misunderstand you here, fmt.format.field comes from a the
> > subdevice, just above this:
> > 
> >     struct v4l2_subdev_format fmt = {
> > 	    .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> > 	    .pad = vin->digital->source_pad,
> >     };
> >     int ret;
> > 
> >     ret = v4l2_subdev_call(vin_to_source(vin), pad, get_fmt, NULL, &fmt);
> >     if (ret)
> > 	    return ret;
> > 
> >     switch (fmt.format.field) {
> >         ...
> >     }
> > 
> > So the format acted on here is the one from the subdevice, and if it is
> > V4L2_FIELD_ALTERNATE it is supported as a source format, just not for
> > output to user-space.
> > 
> > > Furthermore treating the pix parameter of this function as both input and
> > > output seems very confusing to me. If you want to extend
> > > rvin_get_sd_format() beyond just getting the format from the subdev then
> > > please document the function with kerneldoc, and let's try to make its
> > > API clear.
> > 
> > This comment confuses me, are we looking at the same change? The only
> > reference I have to the pix parameter in rvin_get_sd_format() is just
> > before the function returns and it's:
> > 
> >    v4l2_fill_pix_format(pix, &fmt.format);
> > 
> > So it's only used as an output for this function.
> 
> I had mistakenly read the switch statement as operating on the pix function 
> parameter. My bad, sorry about the noise.

No problem :-)

> 
> However, V4L2_FIELD_ALTERNATE should still be rejected in the set format 
> handler, and I don't think you do so in this patch.
> 
> It looks like the field handling logic needs a rewrite :-)

The V4L2_FIELD_ALTERNATE is rejected in the set format handler, or 
rather if the user tries to request it will get V4L2_FIELD_NONE, see the 
latest change of this patch ;-)

But I will add a comment there to explain why ALTERNATE is rejected.

> 
> > >> +	default:
> > >> +		vin->format.field = V4L2_FIELD_NONE;
> > >> +		break;
> > >> +	}
> > >> +
> > >>  	v4l2_fill_pix_format(pix, &fmt.format);
> > >>  	
> > >>  	return 0;
> 
> [snip]
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas Söderlund

Re: [PATCH v9 13/28] rcar-vin: fix handling of single field frames (top, bottom and alternate fields)

[Niklas Söderlund]

Hi Laurent,

Thanks for your comments.

On 2017-12-08 21:30:20 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> On Friday, 8 December 2017 16:06:58 EET Niklas S�derlund wrote:
> > On 2017-12-08 11:35:18 +0200, Laurent Pinchart wrote:
> > > On Friday, 8 December 2017 03:08:27 EET Niklas S�derlund wrote:
> > >> There was never proper support in the VIN driver to deliver ALTERNATING
> > >> field format to user-space, remove this field option. For sources using
> > >> this field format instead use the VIN hardware feature of combining the
> > >> fields to an interlaced format. This mode of operation was previously
> > >> the default behavior and ALTERNATING was only delivered to user-space if
> > >> explicitly requested. Allowing this to be explicitly requested was a
> > >> mistake and was never properly tested and never worked due to the
> > >> constraints put on the field format when it comes to sequence numbers
> > >> and timestamps etc.
> > > 
> > > I'm puzzled, why can't we support V4L2_FIELD_ALTERNATE if we can support
> > > V4L2_FIELD_TOP and V4L2_FIELD_BOTTOM ? I don't dispute the fact that the
> > > currently implemented logic might be wrong (although I haven't
> > > double-checked that), but what prevents us from implementing it correctly
> > > ?
> > 
> > Maybe my commit message is fuzzy. We can support V4L2_FIELD_ALTERNATE as
> > a source to the VIN but we can't (yet) support delivering it to
> > user-space in a good way. So if we have a video source which outputs
> > V4L2_FIELD_ALTERNATE we are fine as we can use the hardware to interlace
> > that or only capture the TOP or BOTTOM fields.
> > 
> > But the driver logic to capture frames (the whole dance with single and
> > continues capture modes) to be able to deal with situations where
> > buffers are not queued fast enough currently prevents us from delivering
> > V4L2_FIELD_ALTERNATE to user-space. The problem is we can only capture
> > (correctly) ALTERNATE if we run in continues mode, if the driver is feed
> > buffers to slow and switches to single capture mode we can't live up to
> > the specification of the field order from the documentation:
> > 
> > "If fields are successive, without any dropped fields between them
> > (fields can drop individually), can be determined from the struct
> > v4l2_buffer sequence field."
> > 
> > So even if in single capture mode we switch between TOP and BOTTOM for
> > each capture the sequence number would always be sequential but the
> > fields would in temporal time potentially be far apparat (depending on
> > how fast user-space queues buffers + the time it takes to shutdown and
> > startup the VIN capture).
> > 
> > So instead of badly supporting this field order now I feel it's better
> > to not support it and once we tackle the issue of trying to remove
> > single capture mode (if at all possible) add support for it. But this is
> > a task for a different patch-set as this one is quiet large already and
> > it's focus is to add Gen3 support.
> 
> OK, so we could support capturing alternating fields, but in that case we 
> wouldn't be able to provide accurate sequence numbers. I'm fine with dropping 
> support for ALTERNATE, but I would capture that information in the commit 
> message, and probably as well in a comment in the code.

Agree, I tried to capture this in the commit message. But as You did not 
understand it I need to make a better job. I will expand on this in the 
commit message and add a comment in the code. Thanks.

> 
> > >> The height should not be cut in half for the format for TOP or BOTTOM
> > >> fields settings. This was a mistake and it was made visible by the
> > >> scaling refactoring. Correct behavior is that the user should request a
> > >> frame size that fits the half height frame reflected in the field
> > >> setting. If not the VIN will do its best to scale the top or bottom to
> > >> the requested format and cropping and scaling do not work as expected.
> > >> 
> > >> Signed-off-by: Niklas S�derlund <niklas.soderlund+renesas@ragnatech.se>
> > >> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > >> ---
> > >> 
> > >>  drivers/media/platform/rcar-vin/rcar-dma.c  | 15 +--------
> > >>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 48 +++++++++--------------
> > >>  2 files changed, 19 insertions(+), 44 deletions(-)
> 
> [snip]
> 
> > >> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > >> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> > >> 9cf9ff48ac1e2f4f..37fe1f6c646b0ea3 100644
> > >> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > >> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > >> @@ -102,6 +102,24 @@ static int rvin_get_sd_format(struct rvin_dev *vin,
> > >> struct v4l2_pix_format *pix)
> > >> 	if (ret)
> > >>  		return ret;
> > >> 
> > >> +	switch (fmt.format.field) {
> > >> +	case V4L2_FIELD_TOP:
> > >> +	case V4L2_FIELD_BOTTOM:
> > >> +	case V4L2_FIELD_NONE:
> > >> +	case V4L2_FIELD_INTERLACED_TB:
> > >> +	case V4L2_FIELD_INTERLACED_BT:
> > >> +	case V4L2_FIELD_INTERLACED:
> > >> +		break;
> > >> +	case V4L2_FIELD_ALTERNATE:
> > >> +		/* Use VIN hardware to combine the two fields */
> > >> +		fmt.format.field = V4L2_FIELD_INTERLACED;
> > >> +		fmt.format.height *= 2;
> > >> +		break;
> > > 
> > > I don't think this is right. If V4L2_FIELD_ALTERNATE isn't supported it
> > > should be rejected in the set format handler, or rather this logic should
> > > be moved there. It doesn't belong here, rvin_get_sd_format() should only
> > > be called with a validated and supported field.
> > 
> > I might misunderstand you here, fmt.format.field comes from a the
> > subdevice, just above this:
> > 
> >     struct v4l2_subdev_format fmt = {
> > 	    .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> > 	    .pad = vin->digital->source_pad,
> >     };
> >     int ret;
> > 
> >     ret = v4l2_subdev_call(vin_to_source(vin), pad, get_fmt, NULL, &fmt);
> >     if (ret)
> > 	    return ret;
> > 
> >     switch (fmt.format.field) {
> >         ...
> >     }
> > 
> > So the format acted on here is the one from the subdevice, and if it is
> > V4L2_FIELD_ALTERNATE it is supported as a source format, just not for
> > output to user-space.
> > 
> > > Furthermore treating the pix parameter of this function as both input and
> > > output seems very confusing to me. If you want to extend
> > > rvin_get_sd_format() beyond just getting the format from the subdev then
> > > please document the function with kerneldoc, and let's try to make its
> > > API clear.
> > 
> > This comment confuses me, are we looking at the same change? The only
> > reference I have to the pix parameter in rvin_get_sd_format() is just
> > before the function returns and it's:
> > 
> >    v4l2_fill_pix_format(pix, &fmt.format);
> > 
> > So it's only used as an output for this function.
> 
> I had mistakenly read the switch statement as operating on the pix function 
> parameter. My bad, sorry about the noise.

No problem :-)

> 
> However, V4L2_FIELD_ALTERNATE should still be rejected in the set format 
> handler, and I don't think you do so in this patch.
> 
> It looks like the field handling logic needs a rewrite :-)

The V4L2_FIELD_ALTERNATE is rejected in the set format handler, or 
rather if the user tries to request it will get V4L2_FIELD_NONE, see the 
latest change of this patch ;-)

But I will add a comment there to explain why ALTERNATE is rejected.

> 
> > >> +	default:
> > >> +		vin->format.field = V4L2_FIELD_NONE;
> > >> +		break;
> > >> +	}
> > >> +
> > >>  	v4l2_fill_pix_format(pix, &fmt.format);
> > >>  	
> > >>  	return 0;
> 
> [snip]
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas S�derlund

Re: [PATCH v9 15/28] rcar-vin: enable Gen3 hardware configuration

[Niklas Söderlund]

Hi Laurent,

Thanks for your comments.

On 2017-12-08 11:47:13 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:29 EET Niklas Söderlund wrote:
> > Add the register needed to work with Gen3 hardware. This patch adds
> > the logic for how to work with the Gen3 hardware. More work is required
> > to enable the subdevice structure needed to configure capturing.
> > 
> > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >  drivers/media/platform/rcar-vin/rcar-dma.c | 94 ++++++++++++++++++---------
> >  drivers/media/platform/rcar-vin/rcar-vin.h |  1 +
> >  2 files changed, 64 insertions(+), 31 deletions(-)
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> > b/drivers/media/platform/rcar-vin/rcar-dma.c index
> > d7660f485a2df9e4..ace95d5b543a17e3 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> > @@ -33,21 +33,23 @@
> >  #define VNELPRC_REG	0x10	/* Video n End Line Pre-Clip Register */
> >  #define VNSPPRC_REG	0x14	/* Video n Start Pixel Pre-Clip Register */
> >  #define VNEPPRC_REG	0x18	/* Video n End Pixel Pre-Clip Register */
> > -#define VNSLPOC_REG	0x1C	/* Video n Start Line Post-Clip Register */
> > -#define VNELPOC_REG	0x20	/* Video n End Line Post-Clip Register */
> > -#define VNSPPOC_REG	0x24	/* Video n Start Pixel Post-Clip Register */
> > -#define VNEPPOC_REG	0x28	/* Video n End Pixel Post-Clip Register */
> >  #define VNIS_REG	0x2C	/* Video n Image Stride Register */
> >  #define VNMB_REG(m)	(0x30 + ((m) << 2)) /* Video n Memory Base m Register
> > */ #define VNIE_REG	0x40	/* Video n Interrupt Enable Register */
> >  #define VNINTS_REG	0x44	/* Video n Interrupt Status Register */
> >  #define VNSI_REG	0x48	/* Video n Scanline Interrupt Register */
> >  #define VNMTC_REG	0x4C	/* Video n Memory Transfer Control Register */
> > -#define VNYS_REG	0x50	/* Video n Y Scale Register */
> > -#define VNXS_REG	0x54	/* Video n X Scale Register */
> >  #define VNDMR_REG	0x58	/* Video n Data Mode Register */
> >  #define VNDMR2_REG	0x5C	/* Video n Data Mode Register 2 */
> >  #define VNUVAOF_REG	0x60	/* Video n UV Address Offset Register */
> > +
> > +/* Register offsets specific for Gen2 */
> > +#define VNSLPOC_REG	0x1C	/* Video n Start Line Post-Clip Register */
> > +#define VNELPOC_REG	0x20	/* Video n End Line Post-Clip Register */
> > +#define VNSPPOC_REG	0x24	/* Video n Start Pixel Post-Clip Register */
> > +#define VNEPPOC_REG	0x28	/* Video n End Pixel Post-Clip Register */
> > +#define VNYS_REG	0x50	/* Video n Y Scale Register */
> > +#define VNXS_REG	0x54	/* Video n X Scale Register */
> >  #define VNC1A_REG	0x80	/* Video n Coefficient Set C1A Register */
> >  #define VNC1B_REG	0x84	/* Video n Coefficient Set C1B Register */
> >  #define VNC1C_REG	0x88	/* Video n Coefficient Set C1C Register */
> > @@ -73,9 +75,13 @@
> >  #define VNC8B_REG	0xF4	/* Video n Coefficient Set C8B Register */
> >  #define VNC8C_REG	0xF8	/* Video n Coefficient Set C8C Register */
> > 
> > +/* Register offsets specific for Gen3 */
> > +#define VNCSI_IFMD_REG		0x20 /* Video n CSI2 Interface Mode Register */
> > 
> >  /* Register bit fields for R-Car VIN */
> >  /* Video n Main Control Register bits */
> > +#define VNMC_DPINE		(1 << 27) /* Gen3 specific */
> > +#define VNMC_SCLE		(1 << 26) /* Gen3 specific */
> >  #define VNMC_FOC		(1 << 21)
> >  #define VNMC_YCAL		(1 << 19)
> >  #define VNMC_INF_YUV8_BT656	(0 << 16)
> > @@ -119,6 +125,13 @@
> >  #define VNDMR2_FTEV		(1 << 17)
> >  #define VNDMR2_VLV(n)		((n & 0xf) << 12)
> > 
> > +/* Video n CSI2 Interface Mode Register (Gen3) */
> > +#define VNCSI_IFMD_DES2		(1 << 27)
> > +#define VNCSI_IFMD_DES1		(1 << 26)
> > +#define VNCSI_IFMD_DES0		(1 << 25)
> > +#define VNCSI_IFMD_CSI_CHSEL(n) ((n & 0xf) << 0)
> 
> *Always* enclose macro arguments in parentheses otherwise they are subject to 
> side effects.
> 
> #define VNCSI_IFMD_CSI_CHSEL(n) (((n) & 0xf) << 0)

Thanks for spotting this.

> 
> > +#define VNCSI_IFMD_CSI_CHSEL_MASK 0xf
> > +
> >  struct rvin_buffer {
> >  	struct vb2_v4l2_buffer vb;
> >  	struct list_head list;
> > @@ -514,28 +527,10 @@ static void rvin_set_coeff(struct rvin_dev *vin,
> > unsigned short xs) rvin_write(vin, p_set->coeff_set[23], VNC8C_REG);
> >  }
> > 
> > -static void rvin_crop_scale_comp(struct rvin_dev *vin)
> > +static void rvin_crop_scale_comp_gen2(struct rvin_dev *vin)
> >  {
> >  	u32 xs, ys;
> > 
> > -	/* Set Start/End Pixel/Line Pre-Clip */
> > -	rvin_write(vin, vin->crop.left, VNSPPRC_REG);
> > -	rvin_write(vin, vin->crop.left + vin->crop.width - 1, VNEPPRC_REG);
> > -	switch (vin->format.field) {
> > -	case V4L2_FIELD_INTERLACED:
> > -	case V4L2_FIELD_INTERLACED_TB:
> > -	case V4L2_FIELD_INTERLACED_BT:
> > -		rvin_write(vin, vin->crop.top / 2, VNSLPRC_REG);
> > -		rvin_write(vin, (vin->crop.top + vin->crop.height) / 2 - 1,
> > -			   VNELPRC_REG);
> > -		break;
> > -	default:
> > -		rvin_write(vin, vin->crop.top, VNSLPRC_REG);
> > -		rvin_write(vin, vin->crop.top + vin->crop.height - 1,
> > -			   VNELPRC_REG);
> > -		break;
> > -	}
> > -
> >  	/* Set scaling coefficient */
> >  	ys = 0;
> >  	if (vin->crop.height != vin->compose.height)
> > @@ -573,11 +568,6 @@ static void rvin_crop_scale_comp(struct rvin_dev *vin)
> >  		break;
> >  	}
> > 
> > -	if (vin->format.pixelformat == V4L2_PIX_FMT_NV16)
> > -		rvin_write(vin, ALIGN(vin->format.width, 0x20), VNIS_REG);
> > -	else
> > -		rvin_write(vin, ALIGN(vin->format.width, 0x10), VNIS_REG);
> > -
> >  	vin_dbg(vin,
> >  		"Pre-Clip: %ux%u@%u:%u YS: %d XS: %d Post-Clip: %ux%u@%u:%u\n",
> >  		vin->crop.width, vin->crop.height, vin->crop.left,
> 
> Would it make sense to keep the debug message in the common function, or ill 
> cropping and composing be handled through subdevs for Gen3 ?

Yes, my current idea is to model the scaler as a subdevice. As I'm sure 
you are aware not all VIN instances have a scaler. And those who do 
share it with one other VIN instance, so both of them can't use the 
scaler at the same time (see register VnMC.SCLE).

Modeling it as a separate subdevice and using media links ensure only 
one VIN is using it at a time therefor I think is the best solution.

> 
> With these two issues addressed,
> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
> > @@ -585,6 +575,37 @@ static void rvin_crop_scale_comp(struct rvin_dev *vin)
> >  		0, 0);
> >  }
> > 
> > +static void rvin_crop_scale_comp(struct rvin_dev *vin)
> > +{
> > +	/* Set Start/End Pixel/Line Pre-Clip */
> > +	rvin_write(vin, vin->crop.left, VNSPPRC_REG);
> > +	rvin_write(vin, vin->crop.left + vin->crop.width - 1, VNEPPRC_REG);
> > +
> > +	switch (vin->format.field) {
> > +	case V4L2_FIELD_INTERLACED:
> > +	case V4L2_FIELD_INTERLACED_TB:
> > +	case V4L2_FIELD_INTERLACED_BT:
> > +		rvin_write(vin, vin->crop.top / 2, VNSLPRC_REG);
> > +		rvin_write(vin, (vin->crop.top + vin->crop.height) / 2 - 1,
> > +			   VNELPRC_REG);
> > +		break;
> > +	default:
> > +		rvin_write(vin, vin->crop.top, VNSLPRC_REG);
> > +		rvin_write(vin, vin->crop.top + vin->crop.height - 1,
> > +			   VNELPRC_REG);
> > +		break;
> > +	}
> > +
> > +	/* TODO: Add support for the UDS scaler. */
> > +	if (vin->info->chip != RCAR_GEN3)
> > +		rvin_crop_scale_comp_gen2(vin);
> > +
> > +	if (vin->format.pixelformat == V4L2_PIX_FMT_NV16)
> > +		rvin_write(vin, ALIGN(vin->format.width, 0x20), VNIS_REG);
> > +	else
> > +		rvin_write(vin, ALIGN(vin->format.width, 0x10), VNIS_REG);
> > +}
> > +
> >  /* ------------------------------------------------------------------------
> >   * Hardware setup
> >   */
> > @@ -659,7 +680,10 @@ static int rvin_setup(struct rvin_dev *vin)
> >  	}
> > 
> >  	/* Enable VSYNC Field Toogle mode after one VSYNC input */
> > -	dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
> > +	if (vin->info->chip == RCAR_GEN3)
> > +		dmr2 = VNDMR2_FTEV;
> > +	else
> > +		dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
> > 
> >  	/* Hsync Signal Polarity Select */
> >  	if (!(vin->mbus_cfg.flags & V4L2_MBUS_HSYNC_ACTIVE_LOW))
> > @@ -711,6 +735,14 @@ static int rvin_setup(struct rvin_dev *vin)
> >  	if (input_is_yuv == output_is_yuv)
> >  		vnmc |= VNMC_BPS;
> > 
> > +	if (vin->info->chip == RCAR_GEN3) {
> > +		/* Select between CSI-2 and Digital input */
> > +		if (vin->mbus_cfg.type == V4L2_MBUS_CSI2)
> > +			vnmc &= ~VNMC_DPINE;
> > +		else
> > +			vnmc |= VNMC_DPINE;
> > +	}
> > +
> >  	/* Progressive or interlaced mode */
> >  	interrupts = progressive ? VNIE_FIE : VNIE_EFE;
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> > b/drivers/media/platform/rcar-vin/rcar-vin.h index
> > 118f45b656920d39..a440effe4b86af31 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> > +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> > @@ -33,6 +33,7 @@ enum chip_id {
> >  	RCAR_H1,
> >  	RCAR_M1,
> >  	RCAR_GEN2,
> > +	RCAR_GEN3,
> >  };
> > 
> >  /**
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas Söderlund

Re: [PATCH v9 15/28] rcar-vin: enable Gen3 hardware configuration

[Niklas Söderlund]

Hi Laurent,

Thanks for your comments.

On 2017-12-08 11:47:13 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:29 EET Niklas S�derlund wrote:
> > Add the register needed to work with Gen3 hardware. This patch adds
> > the logic for how to work with the Gen3 hardware. More work is required
> > to enable the subdevice structure needed to configure capturing.
> > 
> > Signed-off-by: Niklas S�derlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >  drivers/media/platform/rcar-vin/rcar-dma.c | 94 ++++++++++++++++++---------
> >  drivers/media/platform/rcar-vin/rcar-vin.h |  1 +
> >  2 files changed, 64 insertions(+), 31 deletions(-)
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> > b/drivers/media/platform/rcar-vin/rcar-dma.c index
> > d7660f485a2df9e4..ace95d5b543a17e3 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> > @@ -33,21 +33,23 @@
> >  #define VNELPRC_REG	0x10	/* Video n End Line Pre-Clip Register */
> >  #define VNSPPRC_REG	0x14	/* Video n Start Pixel Pre-Clip Register */
> >  #define VNEPPRC_REG	0x18	/* Video n End Pixel Pre-Clip Register */
> > -#define VNSLPOC_REG	0x1C	/* Video n Start Line Post-Clip Register */
> > -#define VNELPOC_REG	0x20	/* Video n End Line Post-Clip Register */
> > -#define VNSPPOC_REG	0x24	/* Video n Start Pixel Post-Clip Register */
> > -#define VNEPPOC_REG	0x28	/* Video n End Pixel Post-Clip Register */
> >  #define VNIS_REG	0x2C	/* Video n Image Stride Register */
> >  #define VNMB_REG(m)	(0x30 + ((m) << 2)) /* Video n Memory Base m Register
> > */ #define VNIE_REG	0x40	/* Video n Interrupt Enable Register */
> >  #define VNINTS_REG	0x44	/* Video n Interrupt Status Register */
> >  #define VNSI_REG	0x48	/* Video n Scanline Interrupt Register */
> >  #define VNMTC_REG	0x4C	/* Video n Memory Transfer Control Register */
> > -#define VNYS_REG	0x50	/* Video n Y Scale Register */
> > -#define VNXS_REG	0x54	/* Video n X Scale Register */
> >  #define VNDMR_REG	0x58	/* Video n Data Mode Register */
> >  #define VNDMR2_REG	0x5C	/* Video n Data Mode Register 2 */
> >  #define VNUVAOF_REG	0x60	/* Video n UV Address Offset Register */
> > +
> > +/* Register offsets specific for Gen2 */
> > +#define VNSLPOC_REG	0x1C	/* Video n Start Line Post-Clip Register */
> > +#define VNELPOC_REG	0x20	/* Video n End Line Post-Clip Register */
> > +#define VNSPPOC_REG	0x24	/* Video n Start Pixel Post-Clip Register */
> > +#define VNEPPOC_REG	0x28	/* Video n End Pixel Post-Clip Register */
> > +#define VNYS_REG	0x50	/* Video n Y Scale Register */
> > +#define VNXS_REG	0x54	/* Video n X Scale Register */
> >  #define VNC1A_REG	0x80	/* Video n Coefficient Set C1A Register */
> >  #define VNC1B_REG	0x84	/* Video n Coefficient Set C1B Register */
> >  #define VNC1C_REG	0x88	/* Video n Coefficient Set C1C Register */
> > @@ -73,9 +75,13 @@
> >  #define VNC8B_REG	0xF4	/* Video n Coefficient Set C8B Register */
> >  #define VNC8C_REG	0xF8	/* Video n Coefficient Set C8C Register */
> > 
> > +/* Register offsets specific for Gen3 */
> > +#define VNCSI_IFMD_REG		0x20 /* Video n CSI2 Interface Mode Register */
> > 
> >  /* Register bit fields for R-Car VIN */
> >  /* Video n Main Control Register bits */
> > +#define VNMC_DPINE		(1 << 27) /* Gen3 specific */
> > +#define VNMC_SCLE		(1 << 26) /* Gen3 specific */
> >  #define VNMC_FOC		(1 << 21)
> >  #define VNMC_YCAL		(1 << 19)
> >  #define VNMC_INF_YUV8_BT656	(0 << 16)
> > @@ -119,6 +125,13 @@
> >  #define VNDMR2_FTEV		(1 << 17)
> >  #define VNDMR2_VLV(n)		((n & 0xf) << 12)
> > 
> > +/* Video n CSI2 Interface Mode Register (Gen3) */
> > +#define VNCSI_IFMD_DES2		(1 << 27)
> > +#define VNCSI_IFMD_DES1		(1 << 26)
> > +#define VNCSI_IFMD_DES0		(1 << 25)
> > +#define VNCSI_IFMD_CSI_CHSEL(n) ((n & 0xf) << 0)
> 
> *Always* enclose macro arguments in parentheses otherwise they are subject to 
> side effects.
> 
> #define VNCSI_IFMD_CSI_CHSEL(n) (((n) & 0xf) << 0)

Thanks for spotting this.

> 
> > +#define VNCSI_IFMD_CSI_CHSEL_MASK 0xf
> > +
> >  struct rvin_buffer {
> >  	struct vb2_v4l2_buffer vb;
> >  	struct list_head list;
> > @@ -514,28 +527,10 @@ static void rvin_set_coeff(struct rvin_dev *vin,
> > unsigned short xs) rvin_write(vin, p_set->coeff_set[23], VNC8C_REG);
> >  }
> > 
> > -static void rvin_crop_scale_comp(struct rvin_dev *vin)
> > +static void rvin_crop_scale_comp_gen2(struct rvin_dev *vin)
> >  {
> >  	u32 xs, ys;
> > 
> > -	/* Set Start/End Pixel/Line Pre-Clip */
> > -	rvin_write(vin, vin->crop.left, VNSPPRC_REG);
> > -	rvin_write(vin, vin->crop.left + vin->crop.width - 1, VNEPPRC_REG);
> > -	switch (vin->format.field) {
> > -	case V4L2_FIELD_INTERLACED:
> > -	case V4L2_FIELD_INTERLACED_TB:
> > -	case V4L2_FIELD_INTERLACED_BT:
> > -		rvin_write(vin, vin->crop.top / 2, VNSLPRC_REG);
> > -		rvin_write(vin, (vin->crop.top + vin->crop.height) / 2 - 1,
> > -			   VNELPRC_REG);
> > -		break;
> > -	default:
> > -		rvin_write(vin, vin->crop.top, VNSLPRC_REG);
> > -		rvin_write(vin, vin->crop.top + vin->crop.height - 1,
> > -			   VNELPRC_REG);
> > -		break;
> > -	}
> > -
> >  	/* Set scaling coefficient */
> >  	ys = 0;
> >  	if (vin->crop.height != vin->compose.height)
> > @@ -573,11 +568,6 @@ static void rvin_crop_scale_comp(struct rvin_dev *vin)
> >  		break;
> >  	}
> > 
> > -	if (vin->format.pixelformat == V4L2_PIX_FMT_NV16)
> > -		rvin_write(vin, ALIGN(vin->format.width, 0x20), VNIS_REG);
> > -	else
> > -		rvin_write(vin, ALIGN(vin->format.width, 0x10), VNIS_REG);
> > -
> >  	vin_dbg(vin,
> >  		"Pre-Clip: %ux%u@%u:%u YS: %d XS: %d Post-Clip: %ux%u@%u:%u\n",
> >  		vin->crop.width, vin->crop.height, vin->crop.left,
> 
> Would it make sense to keep the debug message in the common function, or ill 
> cropping and composing be handled through subdevs for Gen3 ?

Yes, my current idea is to model the scaler as a subdevice. As I'm sure 
you are aware not all VIN instances have a scaler. And those who do 
share it with one other VIN instance, so both of them can't use the 
scaler at the same time (see register VnMC.SCLE).

Modeling it as a separate subdevice and using media links ensure only 
one VIN is using it at a time therefor I think is the best solution.

> 
> With these two issues addressed,
> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
> > @@ -585,6 +575,37 @@ static void rvin_crop_scale_comp(struct rvin_dev *vin)
> >  		0, 0);
> >  }
> > 
> > +static void rvin_crop_scale_comp(struct rvin_dev *vin)
> > +{
> > +	/* Set Start/End Pixel/Line Pre-Clip */
> > +	rvin_write(vin, vin->crop.left, VNSPPRC_REG);
> > +	rvin_write(vin, vin->crop.left + vin->crop.width - 1, VNEPPRC_REG);
> > +
> > +	switch (vin->format.field) {
> > +	case V4L2_FIELD_INTERLACED:
> > +	case V4L2_FIELD_INTERLACED_TB:
> > +	case V4L2_FIELD_INTERLACED_BT:
> > +		rvin_write(vin, vin->crop.top / 2, VNSLPRC_REG);
> > +		rvin_write(vin, (vin->crop.top + vin->crop.height) / 2 - 1,
> > +			   VNELPRC_REG);
> > +		break;
> > +	default:
> > +		rvin_write(vin, vin->crop.top, VNSLPRC_REG);
> > +		rvin_write(vin, vin->crop.top + vin->crop.height - 1,
> > +			   VNELPRC_REG);
> > +		break;
> > +	}
> > +
> > +	/* TODO: Add support for the UDS scaler. */
> > +	if (vin->info->chip != RCAR_GEN3)
> > +		rvin_crop_scale_comp_gen2(vin);
> > +
> > +	if (vin->format.pixelformat == V4L2_PIX_FMT_NV16)
> > +		rvin_write(vin, ALIGN(vin->format.width, 0x20), VNIS_REG);
> > +	else
> > +		rvin_write(vin, ALIGN(vin->format.width, 0x10), VNIS_REG);
> > +}
> > +
> >  /* ------------------------------------------------------------------------
> >   * Hardware setup
> >   */
> > @@ -659,7 +680,10 @@ static int rvin_setup(struct rvin_dev *vin)
> >  	}
> > 
> >  	/* Enable VSYNC Field Toogle mode after one VSYNC input */
> > -	dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
> > +	if (vin->info->chip == RCAR_GEN3)
> > +		dmr2 = VNDMR2_FTEV;
> > +	else
> > +		dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
> > 
> >  	/* Hsync Signal Polarity Select */
> >  	if (!(vin->mbus_cfg.flags & V4L2_MBUS_HSYNC_ACTIVE_LOW))
> > @@ -711,6 +735,14 @@ static int rvin_setup(struct rvin_dev *vin)
> >  	if (input_is_yuv == output_is_yuv)
> >  		vnmc |= VNMC_BPS;
> > 
> > +	if (vin->info->chip == RCAR_GEN3) {
> > +		/* Select between CSI-2 and Digital input */
> > +		if (vin->mbus_cfg.type == V4L2_MBUS_CSI2)
> > +			vnmc &= ~VNMC_DPINE;
> > +		else
> > +			vnmc |= VNMC_DPINE;
> > +	}
> > +
> >  	/* Progressive or interlaced mode */
> >  	interrupts = progressive ? VNIE_FIE : VNIE_EFE;
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> > b/drivers/media/platform/rcar-vin/rcar-vin.h index
> > 118f45b656920d39..a440effe4b86af31 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> > +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> > @@ -33,6 +33,7 @@ enum chip_id {
> >  	RCAR_H1,
> >  	RCAR_M1,
> >  	RCAR_GEN2,
> > +	RCAR_GEN3,
> >  };
> > 
> >  /**
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas S�derlund

Re: [PATCH v9 16/28] rcar-vin: add function to manipulate Gen3 chsel value

[Niklas Söderlund]

Hi Laurent,

Thanks for your comments.

On 2017-12-08 11:52:01 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:30 EET Niklas Söderlund wrote:
> > On Gen3 the CSI-2 routing is controlled by the VnCSI_IFMD register. One
> > feature of this register is that it's only present in the VIN0 and VIN4
> > instances. The register in VIN0 controls the routing for VIN0-3 and the
> > register in VIN4 controls routing for VIN4-7.
> > 
> > To be able to control routing from a media device this function is need
> > to control runtime PM for the subgroup master (VIN0 and VIN4). The
> > subgroup master must be switched on before the register is manipulated,
> > once the operation is complete it's safe to switch the master off and
> > the new routing will still be in effect.
> > 
> > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >  drivers/media/platform/rcar-vin/rcar-dma.c | 25 +++++++++++++++++++++++++
> >  drivers/media/platform/rcar-vin/rcar-vin.h |  2 ++
> >  2 files changed, 27 insertions(+)
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> > b/drivers/media/platform/rcar-vin/rcar-dma.c index
> > ace95d5b543a17e3..d2788d8bb9565aaa 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> > @@ -16,6 +16,7 @@
> > 
> >  #include <linux/delay.h>
> >  #include <linux/interrupt.h>
> > +#include <linux/pm_runtime.h>
> > 
> >  #include <media/videobuf2-dma-contig.h>
> > 
> > @@ -1228,3 +1229,27 @@ int rvin_dma_register(struct rvin_dev *vin, int irq)
> > 
> >  	return ret;
> >  }
> > +
> > +/* ------------------------------------------------------------------------
> >   + * Gen3 CHSEL manipulation
> > + */
> > +
> > +void rvin_set_chsel(struct rvin_dev *vin, u8 chsel)
> 
> How about naming the function a bit more explicitly, 
> rvin_set_channel_routing() for instance ?

I agree, it's a much better name.

> 
> > +{
> > +	u32 ifmd, vnmc;
> > +
> > +	pm_runtime_get_sync(vin->dev);
> 
> Shouldn't you check the return value of this function ?

Sakari asked the same thing in v4 :-)

In short no its not needed please see Geert's response [1]. If I recall 
correctly this was also discussed in more detail in another thread for 
some other driver whit a bit longer answer saying that it 
pm_runtime_get_sync() fails you have big problems but I can't find that 
thread now :-(

1. https://www.spinics.net/lists/linux-media/msg115241.html

> 
> > +
> > +	/* Make register writes take effect immediately */
> > +	vnmc = rvin_read(vin, VNMC_REG) & ~VNMC_VUP;
> > +	rvin_write(vin, vnmc, VNMC_REG);
> 
> Shouldn't you restore the original value of VNMC at the end of the function ? 
> What if this races with device access local to the VIN0 or VIN4 instance ?

Media link changes are not allowed when any VIN in the group are 
streaming so this should not be an issue. But I agree it's good form to 
restore the value anyhow so I will update this for the next version.

> 
> > +	ifmd = VNCSI_IFMD_DES2 | VNCSI_IFMD_DES1 | VNCSI_IFMD_DES0 |
> > +		VNCSI_IFMD_CSI_CHSEL(chsel);
> > +
> > +	rvin_write(vin, ifmd, VNCSI_IFMD_REG);
> > +
> > +	vin_dbg(vin, "Set IFMD 0x%x\n", ifmd);
> > +
> > +	pm_runtime_put(vin->dev);
> > +}
> > diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> > b/drivers/media/platform/rcar-vin/rcar-vin.h index
> > a440effe4b86af31..7819c760c2c13422 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> > +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> > @@ -163,4 +163,6 @@ void rvin_v4l2_unregister(struct rvin_dev *vin);
> > 
> >  const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
> > 
> > +void rvin_set_chsel(struct rvin_dev *vin, u8 chsel);
> > +
> >  #endif
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas Söderlund

Re: [PATCH v9 16/28] rcar-vin: add function to manipulate Gen3 chsel value

[Niklas Söderlund]

Hi Laurent,

Thanks for your comments.

On 2017-12-08 11:52:01 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:30 EET Niklas S�derlund wrote:
> > On Gen3 the CSI-2 routing is controlled by the VnCSI_IFMD register. One
> > feature of this register is that it's only present in the VIN0 and VIN4
> > instances. The register in VIN0 controls the routing for VIN0-3 and the
> > register in VIN4 controls routing for VIN4-7.
> > 
> > To be able to control routing from a media device this function is need
> > to control runtime PM for the subgroup master (VIN0 and VIN4). The
> > subgroup master must be switched on before the register is manipulated,
> > once the operation is complete it's safe to switch the master off and
> > the new routing will still be in effect.
> > 
> > Signed-off-by: Niklas S�derlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >  drivers/media/platform/rcar-vin/rcar-dma.c | 25 +++++++++++++++++++++++++
> >  drivers/media/platform/rcar-vin/rcar-vin.h |  2 ++
> >  2 files changed, 27 insertions(+)
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> > b/drivers/media/platform/rcar-vin/rcar-dma.c index
> > ace95d5b543a17e3..d2788d8bb9565aaa 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> > @@ -16,6 +16,7 @@
> > 
> >  #include <linux/delay.h>
> >  #include <linux/interrupt.h>
> > +#include <linux/pm_runtime.h>
> > 
> >  #include <media/videobuf2-dma-contig.h>
> > 
> > @@ -1228,3 +1229,27 @@ int rvin_dma_register(struct rvin_dev *vin, int irq)
> > 
> >  	return ret;
> >  }
> > +
> > +/* ------------------------------------------------------------------------
> >   + * Gen3 CHSEL manipulation
> > + */
> > +
> > +void rvin_set_chsel(struct rvin_dev *vin, u8 chsel)
> 
> How about naming the function a bit more explicitly, 
> rvin_set_channel_routing() for instance ?

I agree, it's a much better name.

> 
> > +{
> > +	u32 ifmd, vnmc;
> > +
> > +	pm_runtime_get_sync(vin->dev);
> 
> Shouldn't you check the return value of this function ?

Sakari asked the same thing in v4 :-)

In short no its not needed please see Geert's response [1]. If I recall 
correctly this was also discussed in more detail in another thread for 
some other driver whit a bit longer answer saying that it 
pm_runtime_get_sync() fails you have big problems but I can't find that 
thread now :-(

1. https://www.spinics.net/lists/linux-media/msg115241.html

> 
> > +
> > +	/* Make register writes take effect immediately */
> > +	vnmc = rvin_read(vin, VNMC_REG) & ~VNMC_VUP;
> > +	rvin_write(vin, vnmc, VNMC_REG);
> 
> Shouldn't you restore the original value of VNMC at the end of the function ? 
> What if this races with device access local to the VIN0 or VIN4 instance ?

Media link changes are not allowed when any VIN in the group are 
streaming so this should not be an issue. But I agree it's good form to 
restore the value anyhow so I will update this for the next version.

> 
> > +	ifmd = VNCSI_IFMD_DES2 | VNCSI_IFMD_DES1 | VNCSI_IFMD_DES0 |
> > +		VNCSI_IFMD_CSI_CHSEL(chsel);
> > +
> > +	rvin_write(vin, ifmd, VNCSI_IFMD_REG);
> > +
> > +	vin_dbg(vin, "Set IFMD 0x%x\n", ifmd);
> > +
> > +	pm_runtime_put(vin->dev);
> > +}
> > diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> > b/drivers/media/platform/rcar-vin/rcar-vin.h index
> > a440effe4b86af31..7819c760c2c13422 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> > +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> > @@ -163,4 +163,6 @@ void rvin_v4l2_unregister(struct rvin_dev *vin);
> > 
> >  const struct rvin_video_format *rvin_format_from_pixel(u32 pixelformat);
> > 
> > +void rvin_set_chsel(struct rvin_dev *vin, u8 chsel);
> > +
> >  #endif
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas S�derlund

Re: [PATCH v9 18/28] rcar-vin: break out format alignment and checking

[Niklas Söderlund]

Hi Laurent,

Thanks for your comments.

On 2017-12-08 12:01:08 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:32 EET Niklas Söderlund wrote:
> > Part of the format alignment and checking can be shared with the Gen3
> > format handling. Break that part out to its own function. While doing
> > this clean up the checking and add more checks.
> 
> I'd split that in two patches, they are unrelated and should be reviewed 
> separately.

Good point, I will do so. In fact I will split this in to three patches.
- Move things to a separate new function
- Deal with the update bytesperline and sizeimage calculation
- Add the new check on pixelformat

> 
> > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >  drivers/media/platform/rcar-vin/rcar-v4l2.c | 98 +++++++++++++------------
> >  1 file changed, 51 insertions(+), 47 deletions(-)
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> > 56c5183f55922e1d..0ffbf0c16fb7b00e 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > @@ -86,6 +86,56 @@ static u32 rvin_format_sizeimage(struct v4l2_pix_format
> > *pix) return pix->bytesperline * pix->height;
> >  }
> > 
> > +static int rvin_format_align(struct rvin_dev *vin, struct v4l2_pix_format
> > *pix)
> > +{
> > +	u32 walign;
> > +
> > +	/* If requested format is not supported fallback to the default */
> > +	if (!rvin_format_from_pixel(pix->pixelformat)) {
> > +		vin_dbg(vin, "Format 0x%x not found, using default 0x%x\n",
> > +			pix->pixelformat, RVIN_DEFAULT_FORMAT);
> > +		pix->pixelformat = RVIN_DEFAULT_FORMAT;
> > +	}
> > +
> > +	switch (pix->field) {
> > +	case V4L2_FIELD_TOP:
> > +	case V4L2_FIELD_BOTTOM:
> > +	case V4L2_FIELD_NONE:
> > +	case V4L2_FIELD_INTERLACED_TB:
> > +	case V4L2_FIELD_INTERLACED_BT:
> > +	case V4L2_FIELD_INTERLACED:
> > +		break;
> > +	default:
> > +		pix->field = V4L2_FIELD_NONE;
> > +		break;
> > +	}
> > +
> > +	/* Check that colorspace is reasonable, if not keep current */
> > +	if (!pix->colorspace || pix->colorspace >= 0xff)
> 
> Where does 0xff come from ? It seems a bit random to me.

It comes from v4l2-compliance source code, is that not how we are 
suppose to pass a compliance test? :-) Exact location is 
testColorspace() in utils/v4l2-compliance/v4l2-test-formats.cpp.  

The only other way I can think of is to list all formats from enum 
v4l2_colorspace in a swtich statement, but then if a new colorspace is 
added this will fall behind.  Other drivers just check for 
!pix->colorspace but that is not enough to pass the test.

On Gen2 this is not an issue as as you describe bellow the colorspace is 
always set to that of the source. But on Gen3 it's up to the user to set 
to colorspace if the default one is not right. Then this check is needed 
to not freak out v4l2-compliance.

I will keep this for now, and if I can think of something better I will 
switch to that. But I need to think more on this, in the mean time break 
it out to a separate patch.

> 
> > +		pix->colorspace = vin->format.colorspace;
> 
> I don't think that's a good idea. You should pick a default if the colorspace 
> can't be supported. Beside, what's the point in accepting colorspaces if 
> they're not handled by the driver ? Why don't you just set a fixed value based 
> on the colorspace reported by the source ?

Yes here I agree, it should set it to a default colorspace.

> 
> > +	/* HW limit width to a multiple of 32 (2^5) for NV16 else 2 (2^1) */
> > +	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
> > +
> > +	/* Limit to VIN capabilities */
> > +	v4l_bound_align_image(&pix->width, 2, vin->info->max_width, walign,
> > +			      &pix->height, 4, vin->info->max_height, 2, 0);
> > +
> > +	pix->bytesperline = rvin_format_bytesperline(pix);
> > +	pix->sizeimage = rvin_format_sizeimage(pix);
> 
> You're now hardcoding those values instead of only enforcing a minimum. Why is 
> that ?

Hum I don't understand this comment I think. I checked other drivers and 
this looks like how most of them are calculating this (vimc, xilinx, 
etc).  This change is mostly to get rid of things that never should have 
made it out of soc_camera. It worked for Gen2 as bytesperline and 
sizeimage where updated when querying the sensor. While on Gen3 this 
will break as there is no sensor to update this and it is therefore 
impossible to reduce these values.

> 
> > +
> > +	if (vin->info->chip == RCAR_M1 &&
> > +	    pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
> > +		vin_err(vin, "pixel format XBGR32 not supported on M1\n");
> > +		return -EINVAL;
> > +	}
> 
> You should move this with the other format check at the beginning of the 
> function. and selecting the default format instead of returning an error.

Yes will do.

> 
> > +	vin_dbg(vin, "Format %ux%u bpl: %d size: %d\n",
> > +		pix->width, pix->height, pix->bytesperline, pix->sizeimage);
> > +
> > +	return 0;
> > +}
> > +
> >  /*
> > ---------------------------------------------------------------------------
> > -- * V4L2
> >   */
> > @@ -191,64 +241,18 @@ static int __rvin_try_format_source(struct rvin_dev
> > *vin, static int __rvin_try_format(struct rvin_dev *vin,
> >  			     u32 which, struct v4l2_pix_format *pix)
> >  {
> > -	u32 walign;
> >  	int ret;
> > 
> >  	/* Keep current field if no specific one is asked for */
> >  	if (pix->field == V4L2_FIELD_ANY)
> >  		pix->field = vin->format.field;
> > 
> > -	/* If requested format is not supported fallback to the default */
> > -	if (!rvin_format_from_pixel(pix->pixelformat)) {
> > -		vin_dbg(vin, "Format 0x%x not found, using default 0x%x\n",
> > -			pix->pixelformat, RVIN_DEFAULT_FORMAT);
> > -		pix->pixelformat = RVIN_DEFAULT_FORMAT;
> > -	}
> > -
> > -	/* Always recalculate */
> > -	pix->bytesperline = 0;
> > -	pix->sizeimage = 0;
> > -
> >  	/* Limit to source capabilities */
> >  	ret = __rvin_try_format_source(vin, which, pix);
> >  	if (ret)
> >  		return ret;
> > 
> > -	switch (pix->field) {
> > -	case V4L2_FIELD_TOP:
> > -	case V4L2_FIELD_BOTTOM:
> > -	case V4L2_FIELD_NONE:
> > -	case V4L2_FIELD_INTERLACED_TB:
> > -	case V4L2_FIELD_INTERLACED_BT:
> > -	case V4L2_FIELD_INTERLACED:
> > -		break;
> > -	default:
> > -		pix->field = V4L2_FIELD_NONE;
> > -		break;
> > -	}
> > -
> > -	/* HW limit width to a multiple of 32 (2^5) for NV16 else 2 (2^1) */
> > -	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
> > -
> > -	/* Limit to VIN capabilities */
> > -	v4l_bound_align_image(&pix->width, 2, vin->info->max_width, walign,
> > -			      &pix->height, 4, vin->info->max_height, 2, 0);
> > -
> > -	pix->bytesperline = max_t(u32, pix->bytesperline,
> > -				  rvin_format_bytesperline(pix));
> > -	pix->sizeimage = max_t(u32, pix->sizeimage,
> > -			       rvin_format_sizeimage(pix));
> > -
> > -	if (vin->info->chip == RCAR_M1 &&
> > -	    pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
> > -		vin_err(vin, "pixel format XBGR32 not supported on M1\n");
> > -		return -EINVAL;
> > -	}
> > -
> > -	vin_dbg(vin, "Format %ux%u bpl: %d size: %d\n",
> > -		pix->width, pix->height, pix->bytesperline, pix->sizeimage);
> > -
> > -	return 0;
> > +	return rvin_format_align(vin, pix);
> >  }
> > 
> >  static int rvin_querycap(struct file *file, void *priv,
> 
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas Söderlund

Re: [PATCH v9 18/28] rcar-vin: break out format alignment and checking

[Niklas Söderlund]

Hi Laurent,

Thanks for your comments.

On 2017-12-08 12:01:08 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:32 EET Niklas S�derlund wrote:
> > Part of the format alignment and checking can be shared with the Gen3
> > format handling. Break that part out to its own function. While doing
> > this clean up the checking and add more checks.
> 
> I'd split that in two patches, they are unrelated and should be reviewed 
> separately.

Good point, I will do so. In fact I will split this in to three patches.
- Move things to a separate new function
- Deal with the update bytesperline and sizeimage calculation
- Add the new check on pixelformat

> 
> > Signed-off-by: Niklas S�derlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >  drivers/media/platform/rcar-vin/rcar-v4l2.c | 98 +++++++++++++------------
> >  1 file changed, 51 insertions(+), 47 deletions(-)
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> > 56c5183f55922e1d..0ffbf0c16fb7b00e 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > @@ -86,6 +86,56 @@ static u32 rvin_format_sizeimage(struct v4l2_pix_format
> > *pix) return pix->bytesperline * pix->height;
> >  }
> > 
> > +static int rvin_format_align(struct rvin_dev *vin, struct v4l2_pix_format
> > *pix)
> > +{
> > +	u32 walign;
> > +
> > +	/* If requested format is not supported fallback to the default */
> > +	if (!rvin_format_from_pixel(pix->pixelformat)) {
> > +		vin_dbg(vin, "Format 0x%x not found, using default 0x%x\n",
> > +			pix->pixelformat, RVIN_DEFAULT_FORMAT);
> > +		pix->pixelformat = RVIN_DEFAULT_FORMAT;
> > +	}
> > +
> > +	switch (pix->field) {
> > +	case V4L2_FIELD_TOP:
> > +	case V4L2_FIELD_BOTTOM:
> > +	case V4L2_FIELD_NONE:
> > +	case V4L2_FIELD_INTERLACED_TB:
> > +	case V4L2_FIELD_INTERLACED_BT:
> > +	case V4L2_FIELD_INTERLACED:
> > +		break;
> > +	default:
> > +		pix->field = V4L2_FIELD_NONE;
> > +		break;
> > +	}
> > +
> > +	/* Check that colorspace is reasonable, if not keep current */
> > +	if (!pix->colorspace || pix->colorspace >= 0xff)
> 
> Where does 0xff come from ? It seems a bit random to me.

It comes from v4l2-compliance source code, is that not how we are 
suppose to pass a compliance test? :-) Exact location is 
testColorspace() in utils/v4l2-compliance/v4l2-test-formats.cpp.  

The only other way I can think of is to list all formats from enum 
v4l2_colorspace in a swtich statement, but then if a new colorspace is 
added this will fall behind.  Other drivers just check for 
!pix->colorspace but that is not enough to pass the test.

On Gen2 this is not an issue as as you describe bellow the colorspace is 
always set to that of the source. But on Gen3 it's up to the user to set 
to colorspace if the default one is not right. Then this check is needed 
to not freak out v4l2-compliance.

I will keep this for now, and if I can think of something better I will 
switch to that. But I need to think more on this, in the mean time break 
it out to a separate patch.

> 
> > +		pix->colorspace = vin->format.colorspace;
> 
> I don't think that's a good idea. You should pick a default if the colorspace 
> can't be supported. Beside, what's the point in accepting colorspaces if 
> they're not handled by the driver ? Why don't you just set a fixed value based 
> on the colorspace reported by the source ?

Yes here I agree, it should set it to a default colorspace.

> 
> > +	/* HW limit width to a multiple of 32 (2^5) for NV16 else 2 (2^1) */
> > +	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
> > +
> > +	/* Limit to VIN capabilities */
> > +	v4l_bound_align_image(&pix->width, 2, vin->info->max_width, walign,
> > +			      &pix->height, 4, vin->info->max_height, 2, 0);
> > +
> > +	pix->bytesperline = rvin_format_bytesperline(pix);
> > +	pix->sizeimage = rvin_format_sizeimage(pix);
> 
> You're now hardcoding those values instead of only enforcing a minimum. Why is 
> that ?

Hum I don't understand this comment I think. I checked other drivers and 
this looks like how most of them are calculating this (vimc, xilinx, 
etc).  This change is mostly to get rid of things that never should have 
made it out of soc_camera. It worked for Gen2 as bytesperline and 
sizeimage where updated when querying the sensor. While on Gen3 this 
will break as there is no sensor to update this and it is therefore 
impossible to reduce these values.

> 
> > +
> > +	if (vin->info->chip == RCAR_M1 &&
> > +	    pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
> > +		vin_err(vin, "pixel format XBGR32 not supported on M1\n");
> > +		return -EINVAL;
> > +	}
> 
> You should move this with the other format check at the beginning of the 
> function. and selecting the default format instead of returning an error.

Yes will do.

> 
> > +	vin_dbg(vin, "Format %ux%u bpl: %d size: %d\n",
> > +		pix->width, pix->height, pix->bytesperline, pix->sizeimage);
> > +
> > +	return 0;
> > +}
> > +
> >  /*
> > ---------------------------------------------------------------------------
> > -- * V4L2
> >   */
> > @@ -191,64 +241,18 @@ static int __rvin_try_format_source(struct rvin_dev
> > *vin, static int __rvin_try_format(struct rvin_dev *vin,
> >  			     u32 which, struct v4l2_pix_format *pix)
> >  {
> > -	u32 walign;
> >  	int ret;
> > 
> >  	/* Keep current field if no specific one is asked for */
> >  	if (pix->field == V4L2_FIELD_ANY)
> >  		pix->field = vin->format.field;
> > 
> > -	/* If requested format is not supported fallback to the default */
> > -	if (!rvin_format_from_pixel(pix->pixelformat)) {
> > -		vin_dbg(vin, "Format 0x%x not found, using default 0x%x\n",
> > -			pix->pixelformat, RVIN_DEFAULT_FORMAT);
> > -		pix->pixelformat = RVIN_DEFAULT_FORMAT;
> > -	}
> > -
> > -	/* Always recalculate */
> > -	pix->bytesperline = 0;
> > -	pix->sizeimage = 0;
> > -
> >  	/* Limit to source capabilities */
> >  	ret = __rvin_try_format_source(vin, which, pix);
> >  	if (ret)
> >  		return ret;
> > 
> > -	switch (pix->field) {
> > -	case V4L2_FIELD_TOP:
> > -	case V4L2_FIELD_BOTTOM:
> > -	case V4L2_FIELD_NONE:
> > -	case V4L2_FIELD_INTERLACED_TB:
> > -	case V4L2_FIELD_INTERLACED_BT:
> > -	case V4L2_FIELD_INTERLACED:
> > -		break;
> > -	default:
> > -		pix->field = V4L2_FIELD_NONE;
> > -		break;
> > -	}
> > -
> > -	/* HW limit width to a multiple of 32 (2^5) for NV16 else 2 (2^1) */
> > -	walign = vin->format.pixelformat == V4L2_PIX_FMT_NV16 ? 5 : 1;
> > -
> > -	/* Limit to VIN capabilities */
> > -	v4l_bound_align_image(&pix->width, 2, vin->info->max_width, walign,
> > -			      &pix->height, 4, vin->info->max_height, 2, 0);
> > -
> > -	pix->bytesperline = max_t(u32, pix->bytesperline,
> > -				  rvin_format_bytesperline(pix));
> > -	pix->sizeimage = max_t(u32, pix->sizeimage,
> > -			       rvin_format_sizeimage(pix));
> > -
> > -	if (vin->info->chip == RCAR_M1 &&
> > -	    pix->pixelformat == V4L2_PIX_FMT_XBGR32) {
> > -		vin_err(vin, "pixel format XBGR32 not supported on M1\n");
> > -		return -EINVAL;
> > -	}
> > -
> > -	vin_dbg(vin, "Format %ux%u bpl: %d size: %d\n",
> > -		pix->width, pix->height, pix->bytesperline, pix->sizeimage);
> > -
> > -	return 0;
> > +	return rvin_format_align(vin, pix);
> >  }
> > 
> >  static int rvin_querycap(struct file *file, void *priv,
> 
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas S�derlund

Re: [PATCH v9 07/28] rcar-vin: change name of video device

[Laurent Pinchart]

Hi Niklas,

On Wednesday, 20 December 2017 17:20:55 EET Niklas Söderlund wrote:
> On 2017-12-14 17:50:24 +0200, Laurent Pinchart wrote:
> > On Thursday, 14 December 2017 16:25:00 EET Sakari Ailus wrote:
> >> On Fri, Dec 08, 2017 at 10:17:36AM +0200, Laurent Pinchart wrote:
> >>> On Friday, 8 December 2017 03:08:21 EET Niklas Söderlund wrote:
> >>>> The rcar-vin driver needs to be part of a media controller to
> >>>> support Gen3. Give each VIN instance a unique name so it can be
> >>>> referenced from userspace.
> >>>> 
> >>>> Signed-off-by: Niklas Söderlund
> >>>> <niklas.soderlund+renesas@ragnatech.se>
> >>>> Reviewed-by: Kieran Bingham
> >>>> <kieran.bingham+renesas@ideasonboard.com>
> >>>> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> >>>> ---
> >>>> 
> >>>>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ++-
> >>>>  1 file changed, 2 insertions(+), 1 deletion(-)
> >>>> 
> >>>> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> >>>> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> >>>> 59ec6d3d119590aa..19de99133f048960 100644
> >>>> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> >>>> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> >>>> @@ -876,7 +876,8 @@ int rvin_v4l2_register(struct rvin_dev *vin)
> >>>>  	vdev->fops = &rvin_fops;
> >>>>  	vdev->v4l2_dev = &vin->v4l2_dev;
> >>>>  	vdev->queue = &vin->queue;
> >>>> -	strlcpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name));
> >>>> +	snprintf(vdev->name, sizeof(vdev->name), "%s %s", KBUILD_MODNAME,
> >>>> +		 dev_name(vin->dev));
> >>> 
> >>> Do we need the module name here ? How about calling them "%s output",
> >>> dev_name(vin->dev) to emphasize the fact that this is a video node and
> >>> not a VIN subdev ? This is what the omap3isp and vsp1 drivers do.
> >>> 
> >>> We're suffering a bit from the fact that V4L2 has never standardized a
> >>> naming scheme for the devices. It wouldn't be fair to ask you to fix
> >>> that as a prerequisite to get the VIN driver merged, but we clearly have
> >>> to work on that at some point.
> >> 
> >> Agreed, this needs to be stable and I think aligning to what omap3isp or
> >> vsp1 do would be a good fix here.
> > 
> > Even omap3isp and vsp1 are not fully aligned, so I think we need to design
> > a naming policy and document it.
> 
> I agree that align this is a good idea. And for this reason I chosen to
> update this patch as such:
> 
> "%s output", dev_name(vin->dev)

Wouldn't it be easier for userspace to use

	"VIN%u output", index

where index is the VIN index as specified in DT ?

> I hope this is a step in the correct direction. If not please let me
> know as soon as possible so I can minimize the trouble for the other
> developers doing stuff on-top of this series and there test scripts :-)

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 07/28] rcar-vin: change name of video device

[Niklas Söderlund]

Hi Laurent,

On 2018-01-08 18:35:23 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> On Wednesday, 20 December 2017 17:20:55 EET Niklas Söderlund wrote:
> > On 2017-12-14 17:50:24 +0200, Laurent Pinchart wrote:
> > > On Thursday, 14 December 2017 16:25:00 EET Sakari Ailus wrote:
> > >> On Fri, Dec 08, 2017 at 10:17:36AM +0200, Laurent Pinchart wrote:
> > >>> On Friday, 8 December 2017 03:08:21 EET Niklas Söderlund wrote:
> > >>>> The rcar-vin driver needs to be part of a media controller to
> > >>>> support Gen3. Give each VIN instance a unique name so it can be
> > >>>> referenced from userspace.
> > >>>> 
> > >>>> Signed-off-by: Niklas Söderlund
> > >>>> <niklas.soderlund+renesas@ragnatech.se>
> > >>>> Reviewed-by: Kieran Bingham
> > >>>> <kieran.bingham+renesas@ideasonboard.com>
> > >>>> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > >>>> ---
> > >>>> 
> > >>>>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ++-
> > >>>>  1 file changed, 2 insertions(+), 1 deletion(-)
> > >>>> 
> > >>>> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > >>>> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> > >>>> 59ec6d3d119590aa..19de99133f048960 100644
> > >>>> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > >>>> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > >>>> @@ -876,7 +876,8 @@ int rvin_v4l2_register(struct rvin_dev *vin)
> > >>>>  	vdev->fops = &rvin_fops;
> > >>>>  	vdev->v4l2_dev = &vin->v4l2_dev;
> > >>>>  	vdev->queue = &vin->queue;
> > >>>> -	strlcpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name));
> > >>>> +	snprintf(vdev->name, sizeof(vdev->name), "%s %s", KBUILD_MODNAME,
> > >>>> +		 dev_name(vin->dev));
> > >>> 
> > >>> Do we need the module name here ? How about calling them "%s output",
> > >>> dev_name(vin->dev) to emphasize the fact that this is a video node and
> > >>> not a VIN subdev ? This is what the omap3isp and vsp1 drivers do.
> > >>> 
> > >>> We're suffering a bit from the fact that V4L2 has never standardized a
> > >>> naming scheme for the devices. It wouldn't be fair to ask you to fix
> > >>> that as a prerequisite to get the VIN driver merged, but we clearly have
> > >>> to work on that at some point.
> > >> 
> > >> Agreed, this needs to be stable and I think aligning to what omap3isp or
> > >> vsp1 do would be a good fix here.
> > > 
> > > Even omap3isp and vsp1 are not fully aligned, so I think we need to design
> > > a naming policy and document it.
> > 
> > I agree that align this is a good idea. And for this reason I chosen to
> > update this patch as such:
> > 
> > "%s output", dev_name(vin->dev)
> 
> Wouldn't it be easier for userspace to use
> 
> 	"VIN%u output", index
> 
> where index is the VIN index as specified in DT ?

I would be OK whit this, but do this agree with the idea above that we 
should try to align this name across drivers?

> 
> > I hope this is a step in the correct direction. If not please let me
> > know as soon as possible so I can minimize the trouble for the other
> > developers doing stuff on-top of this series and there test scripts :-)
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas Söderlund

Re: [PATCH v9 07/28] rcar-vin: change name of video device

[Niklas Söderlund]

Hi Laurent,

On 2018-01-08 18:35:23 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> On Wednesday, 20 December 2017 17:20:55 EET Niklas S�derlund wrote:
> > On 2017-12-14 17:50:24 +0200, Laurent Pinchart wrote:
> > > On Thursday, 14 December 2017 16:25:00 EET Sakari Ailus wrote:
> > >> On Fri, Dec 08, 2017 at 10:17:36AM +0200, Laurent Pinchart wrote:
> > >>> On Friday, 8 December 2017 03:08:21 EET Niklas S�derlund wrote:
> > >>>> The rcar-vin driver needs to be part of a media controller to
> > >>>> support Gen3. Give each VIN instance a unique name so it can be
> > >>>> referenced from userspace.
> > >>>> 
> > >>>> Signed-off-by: Niklas S�derlund
> > >>>> <niklas.soderlund+renesas@ragnatech.se>
> > >>>> Reviewed-by: Kieran Bingham
> > >>>> <kieran.bingham+renesas@ideasonboard.com>
> > >>>> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > >>>> ---
> > >>>> 
> > >>>>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ++-
> > >>>>  1 file changed, 2 insertions(+), 1 deletion(-)
> > >>>> 
> > >>>> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > >>>> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> > >>>> 59ec6d3d119590aa..19de99133f048960 100644
> > >>>> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > >>>> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> > >>>> @@ -876,7 +876,8 @@ int rvin_v4l2_register(struct rvin_dev *vin)
> > >>>>  	vdev->fops = &rvin_fops;
> > >>>>  	vdev->v4l2_dev = &vin->v4l2_dev;
> > >>>>  	vdev->queue = &vin->queue;
> > >>>> -	strlcpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name));
> > >>>> +	snprintf(vdev->name, sizeof(vdev->name), "%s %s", KBUILD_MODNAME,
> > >>>> +		 dev_name(vin->dev));
> > >>> 
> > >>> Do we need the module name here ? How about calling them "%s output",
> > >>> dev_name(vin->dev) to emphasize the fact that this is a video node and
> > >>> not a VIN subdev ? This is what the omap3isp and vsp1 drivers do.
> > >>> 
> > >>> We're suffering a bit from the fact that V4L2 has never standardized a
> > >>> naming scheme for the devices. It wouldn't be fair to ask you to fix
> > >>> that as a prerequisite to get the VIN driver merged, but we clearly have
> > >>> to work on that at some point.
> > >> 
> > >> Agreed, this needs to be stable and I think aligning to what omap3isp or
> > >> vsp1 do would be a good fix here.
> > > 
> > > Even omap3isp and vsp1 are not fully aligned, so I think we need to design
> > > a naming policy and document it.
> > 
> > I agree that align this is a good idea. And for this reason I chosen to
> > update this patch as such:
> > 
> > "%s output", dev_name(vin->dev)
> 
> Wouldn't it be easier for userspace to use
> 
> 	"VIN%u output", index
> 
> where index is the VIN index as specified in DT ?

I would be OK whit this, but do this agree with the idea above that we 
should try to align this name across drivers?

> 
> > I hope this is a step in the correct direction. If not please let me
> > know as soon as possible so I can minimize the trouble for the other
> > developers doing stuff on-top of this series and there test scripts :-)
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas S�derlund

Re: [PATCH v9 21/28] rcar-vin: add group allocator functions

[Niklas Söderlund]

Hi Laurent,

Thanks for your comments.

On 2017-12-08 22:12:56 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:35 EET Niklas Söderlund wrote:
> > In media controller mode all VIN instances needs to be part of the same
> > media graph. There is also a need to each VIN instance to know and in
> > some cases be able to communicate with other VIN instances.
> > 
> > Add an allocator framework where the first VIN instance to be probed
> > creates a shared data structure and creates a media device.
> > 
> > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >  drivers/media/platform/rcar-vin/rcar-core.c | 179 ++++++++++++++++++++++++-
> >  drivers/media/platform/rcar-vin/rcar-vin.h  |  38 ++++++
> >  2 files changed, 215 insertions(+), 2 deletions(-)
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
> > b/drivers/media/platform/rcar-vin/rcar-core.c index
> > 45de4079fd835759..a6713fd61dd87a88 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-core.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> > @@ -20,12 +20,170 @@
> >  #include <linux/of_graph.h>
> >  #include <linux/platform_device.h>
> >  #include <linux/pm_runtime.h>
> > +#include <linux/slab.h>
> > 
> >  #include <media/v4l2-async.h>
> >  #include <media/v4l2-fwnode.h>
> > 
> >  #include "rcar-vin.h"
> > 
> > +/* ------------------------------------------------------------------------
> >  + * Gen3 CSI2 Group Allocator
> > + */
> > +
> > +static int rvin_group_read_id(struct rvin_dev *vin, struct device_node *np)
> > +{
> > +	u32 val;
> > +	int ret;
> > +
> > +	ret = of_property_read_u32(np, "renesas,id", &val);
> > +	if (ret) {
> > +		vin_err(vin, "%pOF: No renesas,id property found\n", np);
> > +		return -EINVAL;
> > +	}
> > +
> > +	if (val >= RCAR_VIN_NUM) {
> > +		vin_err(vin, "%pOF: Invalid renesas,id '%u'\n", np, val);
> > +		return -EINVAL;
> > +	}
> 
> I'd move all this to the main DT parsing function.

Will do.

> 
> > +	return val;
> > +}
> > +
> > +static DEFINE_MUTEX(rvin_group_lock);
> > +static struct rvin_group *rvin_group_data;
> 
> Nitpicking, static variables are often defined at the beginning of the C file, 
> before any function. I'll let you decide whether you want to move them.

Will move.

> 
> > +static void rvin_group_release(struct kref *kref)
> > +{
> > +	struct rvin_group *group =
> > +		container_of(kref, struct rvin_group, refcount);
> > +
> > +	mutex_lock(&rvin_group_lock);
> > +
> > +	media_device_unregister(&group->mdev);
> > +	media_device_cleanup(&group->mdev);
> > +
> > +	rvin_group_data = NULL;
> > +
> > +	mutex_unlock(&rvin_group_lock);
> > +
> > +	kfree(group);
> > +}
> > +
> > +static struct rvin_group *__rvin_group_allocate(struct rvin_dev *vin)
> > +{
> > +	struct rvin_group *group;
> > +
> > +	if (rvin_group_data) {
> > +		group = rvin_group_data;
> > +		kref_get(&group->refcount);
> > +		vin_dbg(vin, "%s: get group=%p\n", __func__, group);
> > +		return group;
> > +	}
> > +
> > +	group = kzalloc(sizeof(*group), GFP_KERNEL);
> > +	if (!group)
> > +		return NULL;
> > +
> > +	kref_init(&group->refcount);
> > +	rvin_group_data = group;
> 
> Ouch. While I agree with the global mutex, a single global group variable 
> reminds me of the days when per-device data was happily stored in global 
> variables because, you know, we will never have more than one instance of that 
> device, right ? (Or, sometimes, because the driver author didn't know what an 
> instance was.)
> 
> Ideally we'd want a linked list of groups, and this function would either 
> retrieve the group that the VIN instance is part of, or allocate a new one.

I agree that removing the global group variable would be for the better, 
and I was hoping to be able to use the device allocator API for this 
once it's ready. However for now on all supported hardware (I know of) 
there would only be one group for the whole system.

- On H3 and M3-W CSI20 is connected to all VINs.
- On V3M there is only one CSI40 and it is connected to all VINs.

Is this satisfactory for you or do you believe it would be better to 
complete the device allocator first. Or implement parts of the ideas 
from that API locally to this driver? If I understand things correctly 
there are still obstacles blocking that API which are not trivial. And 
even if they are cleared it have no DT support AFIK so functionality to 
iterate over all nodes in a graph would be needed which in itself is no 
trivial task.

> 
> > +	vin_dbg(vin, "%s: alloc group=%p\n", __func__, group);
> 
> Do you still need those two debug statements (and all of the other ones below) 
> ?

Wops, no they can be dropped.

> 
> > +	return group;
> > +}
> > +
> > +static int rvin_group_add_vin(struct rvin_dev *vin)
> > +{
> > +	int ret;
> > +
> > +	ret = rvin_group_read_id(vin, vin->dev->of_node);
> > +	if (ret < 0)
> > +		return ret;
> > +
> > +	mutex_lock(&vin->group->lock);
> > +
> > +	if (vin->group->vin[ret]) {
> > +		mutex_unlock(&vin->group->lock);
> > +		vin_err(vin, "VIN number %d already occupied\n", ret);
> > +		return -EINVAL;
> 
> Can this happen ?

Sure, the values are read from DT so if the renesas,id property have the 
same value for more then one node. This is a incorrect DT of course but 
better to handle and warn for such a case I think?

> 
> > +	}
> > +
> > +	vin->group->vin[ret] = vin;
> > +
> > +	mutex_unlock(&vin->group->lock);
> > +
> > +	vin_dbg(vin, "I'm VIN number %d", ret);
> > +
> > +	return 0;
> > +}
> > +
> > +static int rvin_group_allocate(struct rvin_dev *vin)
> > +{
> > +	struct rvin_group *group;
> > +	struct media_device *mdev;
> > +	int ret;
> > +
> > +	mutex_lock(&rvin_group_lock);
> > +
> > +	group = __rvin_group_allocate(vin);
> > +	if (!group) {
> > +		mutex_unlock(&rvin_group_lock);
> 
> I'd use a goto unlock instead of spreading mutex_unlock() calls through the 
> function. It's easier to locate a misplaced return in a function that should 
> only use gotos than a missing mutex_unlock().

Will fix.

> 
> > +		return -ENOMEM;
> > +	}
> > +
> > +	/* Init group data if it is not already initialized */
> > +	mdev = &group->mdev;
> > +	if (!mdev->dev) {
> > +		mutex_init(&group->lock);
> > +		mdev->dev = vin->dev;
> > +
> > +		strlcpy(mdev->driver_name, "Renesas VIN",
> > +			sizeof(mdev->driver_name));
> 
> How about using the module name ?

Sure.

> 
> > +		strlcpy(mdev->model, vin->dev->of_node->name,
> > +			sizeof(mdev->model));
> 
> I wonder whether you shouldn't somehow use the compatible string to create the 
> model name, in order to accurately report the device model.

I will try and use the compatible string here.

> 
> > +		snprintf(mdev->bus_info, sizeof(mdev->bus_info), "platform:%s",
> > +			 dev_name(mdev->dev));
> > +		media_device_init(mdev);
> > +
> > +		ret = media_device_register(mdev);
> > +		if (ret) {
> > +			vin_err(vin, "Failed to register media device\n");
> > +			kref_put(&group->refcount, rvin_group_release);
> 
> This will deadlock if you're releasing the last reference. There's an 
> identical issue below.

Wops, thanks for pointing this out.

> 
> > +			mutex_unlock(&rvin_group_lock);
> > +			return ret;
> > +		}
> > +	}
> > +
> > +	vin->group = group;
> > +	vin->v4l2_dev.mdev = mdev;
> > +
> > +	ret = rvin_group_add_vin(vin);
> 
> I'd inline the function here.
> 
> > +	if (ret) {
> > +		kref_put(&group->refcount, rvin_group_release);
> > +		mutex_unlock(&rvin_group_lock);
> > +		return ret;
> > +	}
> > +
> > +	mutex_unlock(&rvin_group_lock);
> > +
> > +	return 0;
> > +}
> > +
> > +static void rvin_group_delete(struct rvin_dev *vin)
> 
> This function doesn't completely delete the group, it only drops one 
> reference. How about rvin_group_put() ? rvin_group_allocate() could then be 
> called rvin_group_get().

Good point.

> 
> > +{
> > +	unsigned int i;
> > +
> > +	mutex_lock(&vin->group->lock);
> > +	for (i = 0; i < RCAR_VIN_NUM; i++)
> > +		if (vin->group->vin[i] == vin)
> > +			vin->group->vin[i] = NULL;
> > +	mutex_unlock(&vin->group->lock);
> > +
> > +	vin_dbg(vin, "%s: group=%p\n", __func__, &vin->group);
> > +	kref_put(&vin->group->refcount, rvin_group_release);
> > +}
> > +
> >  /* ------------------------------------------------------------------------
> >   * Async notifier
> >   */
> > @@ -236,12 +394,27 @@ static int rvin_digital_graph_init(struct rvin_dev
> > *vin)
> > 
> >  static int rvin_group_init(struct rvin_dev *vin)
> >  {
> > +	int ret;
> > +
> > +	ret = rvin_group_allocate(vin);
> > +	if (ret)
> > +		return ret;
> > +
> >  	/* All our sources are CSI-2 */
> >  	vin->mbus_cfg.type = V4L2_MBUS_CSI2;
> >  	vin->mbus_cfg.flags = 0;
> > 
> >  	vin->pad.flags = MEDIA_PAD_FL_SINK;
> > -	return media_entity_pads_init(&vin->vdev.entity, 1, &vin->pad);
> > +	ret = media_entity_pads_init(&vin->vdev.entity, 1, &vin->pad);
> > +	if (ret)
> > +		goto error_group;
> > +
> > +	return 0;
> > +
> > +error_group:
> > +	rvin_group_delete(vin);
> > +
> > +	return ret;
> >  }
> > 
> >  /* ------------------------------------------------------------------------
> > @@ -361,7 +534,9 @@ static int rcar_vin_remove(struct platform_device
> > *pdev) v4l2_async_notifier_unregister(&vin->notifier);
> >  	v4l2_async_notifier_cleanup(&vin->notifier);
> > 
> > -	if (!vin->info->use_mc)
> > +	if (vin->info->use_mc)
> > +		rvin_group_delete(vin);
> > +	else
> >  		v4l2_ctrl_handler_free(&vin->ctrl_handler);
> > 
> >  	rvin_dma_unregister(vin);
> > diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> > b/drivers/media/platform/rcar-vin/rcar-vin.h index
> > 07d270a976893cdb..5f736a3500b6e10f 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> > +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> > @@ -17,6 +17,8 @@
> >  #ifndef __RCAR_VIN__
> >  #define __RCAR_VIN__
> > 
> > +#include <linux/kref.h>
> > +
> >  #include <media/v4l2-async.h>
> >  #include <media/v4l2-ctrls.h>
> >  #include <media/v4l2-dev.h>
> > @@ -30,6 +32,9 @@
> >  /* Address alignment mask for HW buffers */
> >  #define HW_BUFFER_MASK 0x7f
> > 
> > +/* Max number on VIN instances that can be in a system */
> > +#define RCAR_VIN_NUM 8
> > +
> >  enum chip_id {
> >  	RCAR_H1,
> >  	RCAR_M1,
> > @@ -37,6 +42,15 @@ enum chip_id {
> >  	RCAR_GEN3,
> >  };
> > 
> > +enum rvin_csi_id {
> > +	RVIN_CSI20,
> > +	RVIN_CSI21,
> > +	RVIN_CSI40,
> > +	RVIN_CSI41,
> > +	RVIN_CSI_MAX,
> > +	RVIN_NC, /* Not Connected */
> > +};
> > +
> >  /**
> >   * STOPPED  - No operation in progress
> >   * RUNNING  - Operation in progress have buffers
> > @@ -75,6 +89,8 @@ struct rvin_graph_entity {
> >  	unsigned int sink_pad;
> >  };
> > 
> > +struct rvin_group;
> 
> Forward declarations are usually grouped at the beginning of the header.

Will fix.

> 
> >  /**
> >   * struct rvin_info - Information about the particular VIN implementation
> >   * @chip:		type of VIN chip
> > @@ -103,6 +119,7 @@ struct rvin_info {
> >   * @notifier:		V4L2 asynchronous subdevs notifier
> >   * @digital:		entity in the DT for local digital subdevice
> >   *
> > + * @group:		Gen3 CSI group
> >   * @pad:		pad for media controller
> >   *
> >   * @lock:		protects @queue
> > @@ -134,6 +151,7 @@ struct rvin_dev {
> >  	struct v4l2_async_notifier notifier;
> >  	struct rvin_graph_entity *digital;
> > 
> > +	struct rvin_group *group;
> >  	struct media_pad pad;
> > 
> >  	struct mutex lock;
> > @@ -162,6 +180,26 @@ struct rvin_dev {
> >  #define vin_warn(d, fmt, arg...)	dev_warn(d->dev, fmt, ##arg)
> >  #define vin_err(d, fmt, arg...)		dev_err(d->dev, fmt, ##arg)
> > 
> > +/**
> > + * struct rvin_group - VIN CSI2 group information
> > + * @refcount:		number of VIN instances using the group
> > + *
> > + * @mdev:		media device which represents the group
> > + *
> > + * @lock:		protects the vin and csi members
> > + * @vin:		VIN instances which are part of the group
> > + * @csi:		CSI-2 entities that are part of the group
> > + */
> > +struct rvin_group {
> > +	struct kref refcount;
> > +
> > +	struct media_device mdev;
> > +
> > +	struct mutex lock;
> > +	struct rvin_dev *vin[RCAR_VIN_NUM];
> > +	struct rvin_graph_entity csi[RVIN_CSI_MAX];
> 
> Given that the number and types of CSI receivers varies quite a bit between 
> SoCs I wonder whether this couldn't be a linked list. If csi was an array of 
> pointers it would be less of an issue, but an array of rvin_graph_entity can 
> grow large.

I will see what makes most sens, a linked list or making it a array of 
pointers.

> 
> > +};
> > +
> >  int rvin_dma_register(struct rvin_dev *vin, int irq);
> >  void rvin_dma_unregister(struct rvin_dev *vin);
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas Söderlund

Re: [PATCH v9 21/28] rcar-vin: add group allocator functions

[Niklas Söderlund]

Hi Laurent,

Thanks for your comments.

On 2017-12-08 22:12:56 +0200, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Friday, 8 December 2017 03:08:35 EET Niklas S�derlund wrote:
> > In media controller mode all VIN instances needs to be part of the same
> > media graph. There is also a need to each VIN instance to know and in
> > some cases be able to communicate with other VIN instances.
> > 
> > Add an allocator framework where the first VIN instance to be probed
> > creates a shared data structure and creates a media device.
> > 
> > Signed-off-by: Niklas S�derlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >  drivers/media/platform/rcar-vin/rcar-core.c | 179 ++++++++++++++++++++++++-
> >  drivers/media/platform/rcar-vin/rcar-vin.h  |  38 ++++++
> >  2 files changed, 215 insertions(+), 2 deletions(-)
> > 
> > diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
> > b/drivers/media/platform/rcar-vin/rcar-core.c index
> > 45de4079fd835759..a6713fd61dd87a88 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-core.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> > @@ -20,12 +20,170 @@
> >  #include <linux/of_graph.h>
> >  #include <linux/platform_device.h>
> >  #include <linux/pm_runtime.h>
> > +#include <linux/slab.h>
> > 
> >  #include <media/v4l2-async.h>
> >  #include <media/v4l2-fwnode.h>
> > 
> >  #include "rcar-vin.h"
> > 
> > +/* ------------------------------------------------------------------------
> >  + * Gen3 CSI2 Group Allocator
> > + */
> > +
> > +static int rvin_group_read_id(struct rvin_dev *vin, struct device_node *np)
> > +{
> > +	u32 val;
> > +	int ret;
> > +
> > +	ret = of_property_read_u32(np, "renesas,id", &val);
> > +	if (ret) {
> > +		vin_err(vin, "%pOF: No renesas,id property found\n", np);
> > +		return -EINVAL;
> > +	}
> > +
> > +	if (val >= RCAR_VIN_NUM) {
> > +		vin_err(vin, "%pOF: Invalid renesas,id '%u'\n", np, val);
> > +		return -EINVAL;
> > +	}
> 
> I'd move all this to the main DT parsing function.

Will do.

> 
> > +	return val;
> > +}
> > +
> > +static DEFINE_MUTEX(rvin_group_lock);
> > +static struct rvin_group *rvin_group_data;
> 
> Nitpicking, static variables are often defined at the beginning of the C file, 
> before any function. I'll let you decide whether you want to move them.

Will move.

> 
> > +static void rvin_group_release(struct kref *kref)
> > +{
> > +	struct rvin_group *group =
> > +		container_of(kref, struct rvin_group, refcount);
> > +
> > +	mutex_lock(&rvin_group_lock);
> > +
> > +	media_device_unregister(&group->mdev);
> > +	media_device_cleanup(&group->mdev);
> > +
> > +	rvin_group_data = NULL;
> > +
> > +	mutex_unlock(&rvin_group_lock);
> > +
> > +	kfree(group);
> > +}
> > +
> > +static struct rvin_group *__rvin_group_allocate(struct rvin_dev *vin)
> > +{
> > +	struct rvin_group *group;
> > +
> > +	if (rvin_group_data) {
> > +		group = rvin_group_data;
> > +		kref_get(&group->refcount);
> > +		vin_dbg(vin, "%s: get group=%p\n", __func__, group);
> > +		return group;
> > +	}
> > +
> > +	group = kzalloc(sizeof(*group), GFP_KERNEL);
> > +	if (!group)
> > +		return NULL;
> > +
> > +	kref_init(&group->refcount);
> > +	rvin_group_data = group;
> 
> Ouch. While I agree with the global mutex, a single global group variable 
> reminds me of the days when per-device data was happily stored in global 
> variables because, you know, we will never have more than one instance of that 
> device, right ? (Or, sometimes, because the driver author didn't know what an 
> instance was.)
> 
> Ideally we'd want a linked list of groups, and this function would either 
> retrieve the group that the VIN instance is part of, or allocate a new one.

I agree that removing the global group variable would be for the better, 
and I was hoping to be able to use the device allocator API for this 
once it's ready. However for now on all supported hardware (I know of) 
there would only be one group for the whole system.

- On H3 and M3-W CSI20 is connected to all VINs.
- On V3M there is only one CSI40 and it is connected to all VINs.

Is this satisfactory for you or do you believe it would be better to 
complete the device allocator first. Or implement parts of the ideas 
from that API locally to this driver? If I understand things correctly 
there are still obstacles blocking that API which are not trivial. And 
even if they are cleared it have no DT support AFIK so functionality to 
iterate over all nodes in a graph would be needed which in itself is no 
trivial task.

> 
> > +	vin_dbg(vin, "%s: alloc group=%p\n", __func__, group);
> 
> Do you still need those two debug statements (and all of the other ones below) 
> ?

Wops, no they can be dropped.

> 
> > +	return group;
> > +}
> > +
> > +static int rvin_group_add_vin(struct rvin_dev *vin)
> > +{
> > +	int ret;
> > +
> > +	ret = rvin_group_read_id(vin, vin->dev->of_node);
> > +	if (ret < 0)
> > +		return ret;
> > +
> > +	mutex_lock(&vin->group->lock);
> > +
> > +	if (vin->group->vin[ret]) {
> > +		mutex_unlock(&vin->group->lock);
> > +		vin_err(vin, "VIN number %d already occupied\n", ret);
> > +		return -EINVAL;
> 
> Can this happen ?

Sure, the values are read from DT so if the renesas,id property have the 
same value for more then one node. This is a incorrect DT of course but 
better to handle and warn for such a case I think?

> 
> > +	}
> > +
> > +	vin->group->vin[ret] = vin;
> > +
> > +	mutex_unlock(&vin->group->lock);
> > +
> > +	vin_dbg(vin, "I'm VIN number %d", ret);
> > +
> > +	return 0;
> > +}
> > +
> > +static int rvin_group_allocate(struct rvin_dev *vin)
> > +{
> > +	struct rvin_group *group;
> > +	struct media_device *mdev;
> > +	int ret;
> > +
> > +	mutex_lock(&rvin_group_lock);
> > +
> > +	group = __rvin_group_allocate(vin);
> > +	if (!group) {
> > +		mutex_unlock(&rvin_group_lock);
> 
> I'd use a goto unlock instead of spreading mutex_unlock() calls through the 
> function. It's easier to locate a misplaced return in a function that should 
> only use gotos than a missing mutex_unlock().

Will fix.

> 
> > +		return -ENOMEM;
> > +	}
> > +
> > +	/* Init group data if it is not already initialized */
> > +	mdev = &group->mdev;
> > +	if (!mdev->dev) {
> > +		mutex_init(&group->lock);
> > +		mdev->dev = vin->dev;
> > +
> > +		strlcpy(mdev->driver_name, "Renesas VIN",
> > +			sizeof(mdev->driver_name));
> 
> How about using the module name ?

Sure.

> 
> > +		strlcpy(mdev->model, vin->dev->of_node->name,
> > +			sizeof(mdev->model));
> 
> I wonder whether you shouldn't somehow use the compatible string to create the 
> model name, in order to accurately report the device model.

I will try and use the compatible string here.

> 
> > +		snprintf(mdev->bus_info, sizeof(mdev->bus_info), "platform:%s",
> > +			 dev_name(mdev->dev));
> > +		media_device_init(mdev);
> > +
> > +		ret = media_device_register(mdev);
> > +		if (ret) {
> > +			vin_err(vin, "Failed to register media device\n");
> > +			kref_put(&group->refcount, rvin_group_release);
> 
> This will deadlock if you're releasing the last reference. There's an 
> identical issue below.

Wops, thanks for pointing this out.

> 
> > +			mutex_unlock(&rvin_group_lock);
> > +			return ret;
> > +		}
> > +	}
> > +
> > +	vin->group = group;
> > +	vin->v4l2_dev.mdev = mdev;
> > +
> > +	ret = rvin_group_add_vin(vin);
> 
> I'd inline the function here.
> 
> > +	if (ret) {
> > +		kref_put(&group->refcount, rvin_group_release);
> > +		mutex_unlock(&rvin_group_lock);
> > +		return ret;
> > +	}
> > +
> > +	mutex_unlock(&rvin_group_lock);
> > +
> > +	return 0;
> > +}
> > +
> > +static void rvin_group_delete(struct rvin_dev *vin)
> 
> This function doesn't completely delete the group, it only drops one 
> reference. How about rvin_group_put() ? rvin_group_allocate() could then be 
> called rvin_group_get().

Good point.

> 
> > +{
> > +	unsigned int i;
> > +
> > +	mutex_lock(&vin->group->lock);
> > +	for (i = 0; i < RCAR_VIN_NUM; i++)
> > +		if (vin->group->vin[i] == vin)
> > +			vin->group->vin[i] = NULL;
> > +	mutex_unlock(&vin->group->lock);
> > +
> > +	vin_dbg(vin, "%s: group=%p\n", __func__, &vin->group);
> > +	kref_put(&vin->group->refcount, rvin_group_release);
> > +}
> > +
> >  /* ------------------------------------------------------------------------
> >   * Async notifier
> >   */
> > @@ -236,12 +394,27 @@ static int rvin_digital_graph_init(struct rvin_dev
> > *vin)
> > 
> >  static int rvin_group_init(struct rvin_dev *vin)
> >  {
> > +	int ret;
> > +
> > +	ret = rvin_group_allocate(vin);
> > +	if (ret)
> > +		return ret;
> > +
> >  	/* All our sources are CSI-2 */
> >  	vin->mbus_cfg.type = V4L2_MBUS_CSI2;
> >  	vin->mbus_cfg.flags = 0;
> > 
> >  	vin->pad.flags = MEDIA_PAD_FL_SINK;
> > -	return media_entity_pads_init(&vin->vdev.entity, 1, &vin->pad);
> > +	ret = media_entity_pads_init(&vin->vdev.entity, 1, &vin->pad);
> > +	if (ret)
> > +		goto error_group;
> > +
> > +	return 0;
> > +
> > +error_group:
> > +	rvin_group_delete(vin);
> > +
> > +	return ret;
> >  }
> > 
> >  /* ------------------------------------------------------------------------
> > @@ -361,7 +534,9 @@ static int rcar_vin_remove(struct platform_device
> > *pdev) v4l2_async_notifier_unregister(&vin->notifier);
> >  	v4l2_async_notifier_cleanup(&vin->notifier);
> > 
> > -	if (!vin->info->use_mc)
> > +	if (vin->info->use_mc)
> > +		rvin_group_delete(vin);
> > +	else
> >  		v4l2_ctrl_handler_free(&vin->ctrl_handler);
> > 
> >  	rvin_dma_unregister(vin);
> > diff --git a/drivers/media/platform/rcar-vin/rcar-vin.h
> > b/drivers/media/platform/rcar-vin/rcar-vin.h index
> > 07d270a976893cdb..5f736a3500b6e10f 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-vin.h
> > +++ b/drivers/media/platform/rcar-vin/rcar-vin.h
> > @@ -17,6 +17,8 @@
> >  #ifndef __RCAR_VIN__
> >  #define __RCAR_VIN__
> > 
> > +#include <linux/kref.h>
> > +
> >  #include <media/v4l2-async.h>
> >  #include <media/v4l2-ctrls.h>
> >  #include <media/v4l2-dev.h>
> > @@ -30,6 +32,9 @@
> >  /* Address alignment mask for HW buffers */
> >  #define HW_BUFFER_MASK 0x7f
> > 
> > +/* Max number on VIN instances that can be in a system */
> > +#define RCAR_VIN_NUM 8
> > +
> >  enum chip_id {
> >  	RCAR_H1,
> >  	RCAR_M1,
> > @@ -37,6 +42,15 @@ enum chip_id {
> >  	RCAR_GEN3,
> >  };
> > 
> > +enum rvin_csi_id {
> > +	RVIN_CSI20,
> > +	RVIN_CSI21,
> > +	RVIN_CSI40,
> > +	RVIN_CSI41,
> > +	RVIN_CSI_MAX,
> > +	RVIN_NC, /* Not Connected */
> > +};
> > +
> >  /**
> >   * STOPPED  - No operation in progress
> >   * RUNNING  - Operation in progress have buffers
> > @@ -75,6 +89,8 @@ struct rvin_graph_entity {
> >  	unsigned int sink_pad;
> >  };
> > 
> > +struct rvin_group;
> 
> Forward declarations are usually grouped at the beginning of the header.

Will fix.

> 
> >  /**
> >   * struct rvin_info - Information about the particular VIN implementation
> >   * @chip:		type of VIN chip
> > @@ -103,6 +119,7 @@ struct rvin_info {
> >   * @notifier:		V4L2 asynchronous subdevs notifier
> >   * @digital:		entity in the DT for local digital subdevice
> >   *
> > + * @group:		Gen3 CSI group
> >   * @pad:		pad for media controller
> >   *
> >   * @lock:		protects @queue
> > @@ -134,6 +151,7 @@ struct rvin_dev {
> >  	struct v4l2_async_notifier notifier;
> >  	struct rvin_graph_entity *digital;
> > 
> > +	struct rvin_group *group;
> >  	struct media_pad pad;
> > 
> >  	struct mutex lock;
> > @@ -162,6 +180,26 @@ struct rvin_dev {
> >  #define vin_warn(d, fmt, arg...)	dev_warn(d->dev, fmt, ##arg)
> >  #define vin_err(d, fmt, arg...)		dev_err(d->dev, fmt, ##arg)
> > 
> > +/**
> > + * struct rvin_group - VIN CSI2 group information
> > + * @refcount:		number of VIN instances using the group
> > + *
> > + * @mdev:		media device which represents the group
> > + *
> > + * @lock:		protects the vin and csi members
> > + * @vin:		VIN instances which are part of the group
> > + * @csi:		CSI-2 entities that are part of the group
> > + */
> > +struct rvin_group {
> > +	struct kref refcount;
> > +
> > +	struct media_device mdev;
> > +
> > +	struct mutex lock;
> > +	struct rvin_dev *vin[RCAR_VIN_NUM];
> > +	struct rvin_graph_entity csi[RVIN_CSI_MAX];
> 
> Given that the number and types of CSI receivers varies quite a bit between 
> SoCs I wonder whether this couldn't be a linked list. If csi was an array of 
> pointers it would be less of an issue, but an array of rvin_graph_entity can 
> grow large.

I will see what makes most sens, a linked list or making it a array of 
pointers.

> 
> > +};
> > +
> >  int rvin_dma_register(struct rvin_dev *vin, int irq);
> >  void rvin_dma_unregister(struct rvin_dev *vin);
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

-- 
Regards,
Niklas S�derlund

Re: [PATCH v9 07/28] rcar-vin: change name of video device

[Laurent Pinchart]

Hi Niklas,

On Monday, 8 January 2018 18:42:05 EET Niklas Söderlund wrote:
> On 2018-01-08 18:35:23 +0200, Laurent Pinchart wrote:
> > On Wednesday, 20 December 2017 17:20:55 EET Niklas Söderlund wrote:
> >> On 2017-12-14 17:50:24 +0200, Laurent Pinchart wrote:
> >>> On Thursday, 14 December 2017 16:25:00 EET Sakari Ailus wrote:
> >>>> On Fri, Dec 08, 2017 at 10:17:36AM +0200, Laurent Pinchart wrote:
> >>>>> On Friday, 8 December 2017 03:08:21 EET Niklas Söderlund wrote:
> >>>>>> The rcar-vin driver needs to be part of a media controller to
> >>>>>> support Gen3. Give each VIN instance a unique name so it can be
> >>>>>> referenced from userspace.
> >>>>>> 
> >>>>>> Signed-off-by: Niklas Söderlund
> >>>>>> <niklas.soderlund+renesas@ragnatech.se>
> >>>>>> Reviewed-by: Kieran Bingham
> >>>>>> <kieran.bingham+renesas@ideasonboard.com>
> >>>>>> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> >>>>>> ---
> >>>>>> 
> >>>>>>  drivers/media/platform/rcar-vin/rcar-v4l2.c | 3 ++-
> >>>>>>  1 file changed, 2 insertions(+), 1 deletion(-)
> >>>>>> 
> >>>>>> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> >>>>>> b/drivers/media/platform/rcar-vin/rcar-v4l2.c index
> >>>>>> 59ec6d3d119590aa..19de99133f048960 100644
> >>>>>> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> >>>>>> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> >>>>>> @@ -876,7 +876,8 @@ int rvin_v4l2_register(struct rvin_dev *vin)
> >>>>>>  	vdev->fops = &rvin_fops;
> >>>>>>  	vdev->v4l2_dev = &vin->v4l2_dev;
> >>>>>>  	vdev->queue = &vin->queue;
> >>>>>> -	strlcpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name));
> >>>>>> +	snprintf(vdev->name, sizeof(vdev->name), "%s %s", KBUILD_MODNAME,
> >>>>>> +		 dev_name(vin->dev));
> >>>>> 
> >>>>> Do we need the module name here ? How about calling them "%s
> >>>>> output", dev_name(vin->dev) to emphasize the fact that this is a video
> >>>>> node and not a VIN subdev ? This is what the omap3isp and vsp1 drivers
> >>>>> do.
> >>>>> 
> >>>>> We're suffering a bit from the fact that V4L2 has never standardized
> >>>>> a naming scheme for the devices. It wouldn't be fair to ask you to fix
> >>>>> that as a prerequisite to get the VIN driver merged, but we clearly
> >>>>> have to work on that at some point.
> >>>> 
> >>>> Agreed, this needs to be stable and I think aligning to what omap3isp
> >>>> or vsp1 do would be a good fix here.
> >>> 
> >>> Even omap3isp and vsp1 are not fully aligned, so I think we need to
> >>> design a naming policy and document it.
> >> 
> >> I agree that align this is a good idea. And for this reason I chosen to
> >> update this patch as such:
> >> 
> >> "%s output", dev_name(vin->dev)
> > 
> > Wouldn't it be easier for userspace to use
> > 
> > 	"VIN%u output", index
> > 
> > where index is the VIN index as specified in DT ?
> 
> I would be OK whit this, but do this agree with the idea above that we
> should try to align this name across drivers?

Do you mean does this align with the naming policy that we haven't documented, 
or even designed, yet ? :-)

If you want to start designing that policy, here are a few ideas (that may or 
may not reflect general consensus).

First of all, a few general considerations about the entity name. Entity names 
shall be unique within the context of a media graph. There have been 
discussions in the past to move to a single media graph that would span the 
whole system but no concrete proposal has been submitted. Names thus don't 
have to be unique system-wide at the moment but this constraint should still 
be kept in mind. One option would be to simply prepend all entity names with 
the name of the media device (as defined today) if we later move to a single 
media graph.

Another important point is that entity names should be consumable by both 
machines and humans (at least by developers). They should thus describe the 
entity to some extend. A case could be made to make them parsable by machines 
(for instance using the "input" or "output" suffix to infer both the nature of 
the entity and its direction) but I don't think that's a good idea. A more 
formal description of the entity through structured properties would be a 
better API in my opinion.

It could be argued that an "input" or "output" suffix would duplicate 
information conveyed through those structured properties. While I can't deny 
that, the need to have unique names readable by developers is met by adding 
such suffixes.

Another point to consider is the prefix. Using the device name (as in 
dev_name(dev)) is an option but makes reading the media graph by humans more 
difficult. Using a more ad-hoc naming scheme (that would then be driver-
specific to some extend, as is "VIN%u", index for the VIN driver) would create 
names that are easier to read and interpret, and that would also vary less 
between SoCs of the same family.

I believe there's a balance to consider between readability and avoidance of 
duplicated information. One could argue that using "VIN" in the name would 
duplicate information conveyed at the media device level: only the VIN has 
video nodes, so there's need to make that explicit. Similarly the "OMAP3 ISP" 
prefix in the OMAP3 ISP media graph could be argued to be redundant, and is to 
some extend.

If I had to redesign the OMAP3 ISP entity naming scheme today I would consider 
dropping the "OMAP3 ISP" prefix. All of the ISP entities have a name of their 
own (CCP2, CSI2, CCDC, preview, resizer) that is unique already. The media 
device reports the device name through the info model field as "TI OMAP3 ISP" 
so I could argue that the prefix is not needed.

However, the graph can also contain external entities (mostly camera sensors 
in this case). Keeping the "OMAP3 ISP" prefix for all ISP entities would make 
it clear what the entities belong to, and would also help avoiding duplicate 
names. In the R-Car case we face a similar situation, as the graph contains, 
in addition to external camera sensors and video decoders, CSI-2 receivers 
that are in the SoC but not part of the VIN as such.

One possible way to solve this would be to report an entity hierarchy, or at 
least make it possible to group entities in named groups. The name of the 
group wouldn't need to be duplicated as a prefix in every entity.

> >> I hope this is a step in the correct direction. If not please let me
> >> know as soon as possible so I can minimize the trouble for the other
> >> developers doing stuff on-top of this series and there test scripts :-)

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 21/28] rcar-vin: add group allocator functions

[Laurent Pinchart]

Hi Niklas,

On Monday, 8 January 2018 19:24:47 EET Niklas Söderlund wrote:
> On 2017-12-08 22:12:56 +0200, Laurent Pinchart wrote:
> > On Friday, 8 December 2017 03:08:35 EET Niklas Söderlund wrote:
> >> In media controller mode all VIN instances needs to be part of the same
> >> media graph. There is also a need to each VIN instance to know and in
> >> some cases be able to communicate with other VIN instances.
> >> 
> >> Add an allocator framework where the first VIN instance to be probed
> >> creates a shared data structure and creates a media device.
> >> 
> >> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> >> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> >> ---
> >> 
> >>  drivers/media/platform/rcar-vin/rcar-core.c | 179 +++++++++++++++++++++-
> >>  drivers/media/platform/rcar-vin/rcar-vin.h  |  38 ++++++
> >>  2 files changed, 215 insertions(+), 2 deletions(-)
> >> 
> >> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c
> >> b/drivers/media/platform/rcar-vin/rcar-core.c index
> >> 45de4079fd835759..a6713fd61dd87a88 100644
> >> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> >> +++ b/drivers/media/platform/rcar-vin/rcar-core.c

[snip]

> >> +static struct rvin_group *__rvin_group_allocate(struct rvin_dev *vin)
> >> +{
> >> +	struct rvin_group *group;
> >> +
> >> +	if (rvin_group_data) {
> >> +		group = rvin_group_data;
> >> +		kref_get(&group->refcount);
> >> +		vin_dbg(vin, "%s: get group=%p\n", __func__, group);
> >> +		return group;
> >> +	}
> >> +
> >> +	group = kzalloc(sizeof(*group), GFP_KERNEL);
> >> +	if (!group)
> >> +		return NULL;
> >> +
> >> +	kref_init(&group->refcount);
> >> +	rvin_group_data = group;
> > 
> > Ouch. While I agree with the global mutex, a single global group variable
> > reminds me of the days when per-device data was happily stored in global
> > variables because, you know, we will never have more than one instance of
> > that device, right ? (Or, sometimes, because the driver author didn't
> > know what an instance was.)
> > 
> > Ideally we'd want a linked list of groups, and this function would either
> > retrieve the group that the VIN instance is part of, or allocate a new
> > one.
> 
> I agree that removing the global group variable would be for the better,
> and I was hoping to be able to use the device allocator API for this
> once it's ready. However for now on all supported hardware (I know of)
> there would only be one group for the whole system.
> 
> - On H3 and M3-W CSI20 is connected to all VINs.
> - On V3M there is only one CSI40 and it is connected to all VINs.
> 
> Is this satisfactory for you or do you believe it would be better to
> complete the device allocator first. Or implement parts of the ideas
> from that API locally to this driver? If I understand things correctly
> there are still obstacles blocking that API which are not trivial. And
> even if they are cleared it have no DT support AFIK so functionality to
> iterate over all nodes in a graph would be needed which in itself is no
> trivial task.

I don't think we need to wait for the API to be finalized. My dislike for 
global variables pushes me to ask for a local implementation based on a 
linked-list, but that would be a bit overkill given that all platforms will 
have a single group. I'm OK with your current implementation, but please add a 
FIXME comment that explains that the single global pointer should really be a 
linked list, and that it should eventually be replaced by usage of a global 
device allocator API.

> >> +	vin_dbg(vin, "%s: alloc group=%p\n", __func__, group);
> > 
> > Do you still need those two debug statements (and all of the other ones
> > below) ?
> 
> Wops, no they can be dropped.
> 
> >> +	return group;
> >> +}
> >> +
> >> +static int rvin_group_add_vin(struct rvin_dev *vin)
> >> +{
> >> +	int ret;
> >> +
> >> +	ret = rvin_group_read_id(vin, vin->dev->of_node);
> >> +	if (ret < 0)
> >> +		return ret;
> >> +
> >> +	mutex_lock(&vin->group->lock);
> >> +
> >> +	if (vin->group->vin[ret]) {
> >> +		mutex_unlock(&vin->group->lock);
> >> +		vin_err(vin, "VIN number %d already occupied\n", ret);
> >> +		return -EINVAL;
> > 
> > Can this happen ?
> 
> Sure, the values are read from DT so if the renesas,id property have the
> same value for more then one node. This is a incorrect DT of course but
> better to handle and warn for such a case I think?

Good point. How about phrasing the error as "Duplicate renesas,id %u" or even 
"Duplicate renesas,id property value %u" ? That would more clearly point to a 
DT error.

> >> +	}
> >> +
> >> +	vin->group->vin[ret] = vin;
> >> +
> >> +	mutex_unlock(&vin->group->lock);
> >> +
> >> +	vin_dbg(vin, "I'm VIN number %d", ret);
> >> +
> >> +	return 0;
> >> +}

[snip]

-- 
Regards,

Laurent Pinchart

Re: [PATCH v9 19/28] rcar-vin: use different v4l2 operations in media controller mode

[Niklas Söderlund]

Hi Laurent,

Thanks for your comments.

Apart from the issue with the input API Hans pointed which indicates I 
need to keep that around until it's fixed in the framework I agree with 
all your comments but one.

<snip>

On 2017-12-08 12:14:05 +0200, Laurent Pinchart wrote:
> > @@ -628,7 +628,8 @@ static int rvin_setup(struct rvin_dev *vin)
> >  		/* Default to TB */
> >  		vnmc = VNMC_IM_FULL;
> >  		/* Use BT if video standard can be read and is 60 Hz format */
> > -		if (!v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
> > +		if (!vin->info->use_mc &&
> > +		    !v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
> >  			if (std & V4L2_STD_525_60)
> >  				vnmc = VNMC_IM_FULL | VNMC_FOC;
> >  		}
> 
> I think the subdev should be queried in rcar-v4l2.c and the information cached 
> in the rvin_dev structure instead of queried here. Interactions with the 
> subdev should be minimized in rvin-dma.c. You can fix this in a separate patch 
> if you prefer.
> 

This can't be a cached value it needs to be read at stream on time. The 
input source could have changed format, e.g. the user may have 
disconnected a NTSC source and plugged in a PAL.

This is a shame as I otherwise agree with you that interactions with the 
subdevice should be kept at a minimum.

<snip>

-- 
Regards,
Niklas Söderlund

Re: [PATCH v9 19/28] rcar-vin: use different v4l2 operations in media controller mode

[Niklas Söderlund]

Hi Laurent,

Thanks for your comments.

Apart from the issue with the input API Hans pointed which indicates I 
need to keep that around until it's fixed in the framework I agree with 
all your comments but one.

<snip>

On 2017-12-08 12:14:05 +0200, Laurent Pinchart wrote:
> > @@ -628,7 +628,8 @@ static int rvin_setup(struct rvin_dev *vin)
> >  		/* Default to TB */
> >  		vnmc = VNMC_IM_FULL;
> >  		/* Use BT if video standard can be read and is 60 Hz format */
> > -		if (!v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
> > +		if (!vin->info->use_mc &&
> > +		    !v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
> >  			if (std & V4L2_STD_525_60)
> >  				vnmc = VNMC_IM_FULL | VNMC_FOC;
> >  		}
> 
> I think the subdev should be queried in rcar-v4l2.c and the information cached 
> in the rvin_dev structure instead of queried here. Interactions with the 
> subdev should be minimized in rvin-dma.c. You can fix this in a separate patch 
> if you prefer.
> 

This can't be a cached value it needs to be read at stream on time. The 
input source could have changed format, e.g. the user may have 
disconnected a NTSC source and plugged in a PAL.

This is a shame as I otherwise agree with you that interactions with the 
subdevice should be kept at a minimum.

<snip>

-- 
Regards,
Niklas S�derlund

Re: [PATCH v9 19/28] rcar-vin: use different v4l2 operations in media controller mode

[Laurent Pinchart]

Hi Niklas,

On Friday, 19 January 2018 02:46:03 EET Niklas Söderlund wrote:
> Hi Laurent,
> 
> Thanks for your comments.
> 
> Apart from the issue with the input API Hans pointed which indicates I
> need to keep that around until it's fixed in the framework I agree with
> all your comments but one.
> 
> <snip>
> 
> On 2017-12-08 12:14:05 +0200, Laurent Pinchart wrote:
> >> @@ -628,7 +628,8 @@ static int rvin_setup(struct rvin_dev *vin)
> >>  		/* Default to TB */
> >>  		vnmc = VNMC_IM_FULL;
> >>  		/* Use BT if video standard can be read and is 60 Hz format */
> >> -		if (!v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
> >> +		if (!vin->info->use_mc &&
> >> +		    !v4l2_subdev_call(vin_to_source(vin), video, g_std, &std)) {
> >>  			if (std & V4L2_STD_525_60)
> >>  				vnmc = VNMC_IM_FULL | VNMC_FOC;
> >>  		}
> > 
> > I think the subdev should be queried in rcar-v4l2.c and the information
> > cached in the rvin_dev structure instead of queried here. Interactions
> > with the subdev should be minimized in rvin-dma.c. You can fix this in a
> > separate patch if you prefer.
> 
> This can't be a cached value it needs to be read at stream on time. The
> input source could have changed format, e.g. the user may have
> disconnected a NTSC source and plugged in a PAL.

Please note that in that case the source subdev will not have changed its 
format yet, it only does so when the s_std operation is called.

> This is a shame as I otherwise agree with you that interactions with the
> subdevice should be kept at a minimum.
> 
> <snip>

-- 
Regards,

Laurent Pinchart