* [PATCH v3 00/23] media: atmel: atmel-isc: implement media controller
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
This series is the v3 of the series that attempts to support media controller
in the atmel ISC and XISC drivers.
This series also includes the csi2dc driver which was previously sent in a
separate series:
https://www.spinics.net/lists/linux-media/msg181042.html
https://www.spinics.net/lists/linux-media/msg181044.html
The driver now addresses comments received in latest v5 series from last year.
The series includes some minor changes and fixes that improve the isc common
code base, like removing the enum frameintervals VIDIOC, fixing bytesperline
for planar formats, etc.
Many thanks to folks from libcamera who helped a lot with understanding
how a media controller driver should behave.
Feedback is welcome !
Changes in v3:
- change in bindings, small fixes in csi2dc driver and conversion to mc
for the isc-base.
- removed some MAINTAINERS patches and used patterns in MAINTAINERS
Changes in v2:
- integrated many changes suggested by Jacopo in the review of the v1 series.
- add a few new patches
Eugen Hristev (23):
MAINTAINERS: add microchip csi2dc
dt-bindings: media: atmel: csi2dc: add bindings for microchip csi2dc
media: atmel: introduce microchip csi2dc driver
media: atmel: atmel-isc: split the clock code into separate source
file
media: atmel: atmel-isc: replace video device name with module name
media: atmel: atmel-sama7g5-isc: fix ispck leftover
media: atmel: atmel-isc-base: use streaming status when queueing
buffers
media: atmel: atmel-isc-base: remove frameintervals VIDIOC
media: atmel: atmel-isc-base: report frame sizes as full supported
range
media: atmel: atmel-isc-base: implement mbus_code support in enumfmt
media: atmel: atmel-isc-base: fix bytesperline value for planar
formats
media: atmel: atmel-isc: implement media controller
ARM: dts: at91: sama7g5: add nodes for video capture
ARM: configs: at91: sama7: add xisc and csi2dc
ARM: multi_v7_defconfig: add atmel video pipeline modules
media: atmel: atmel-sama5d2-isc: fix wrong mask in YUYV format check
media: atmel: atmel-isc-base: use mutex to lock awb workqueue from
streaming
media: atmel: atmel-isc-base: add wb debug messages
media: atmel: atmel-isc-base: clamp wb gain coefficients
media: atmel: atmel-sama7g5-isc: fix UYVY input format mbus_code typo
media: atmel: atmel-isc: add raw Bayer 8bit 10bit output formats
media: atmel: atmel-isc: compact the controller formats list
media: atmel: atmel-isc: change format propagation to subdev into only
verification
.../bindings/media/microchip,csi2dc.yaml | 197 +++++
MAINTAINERS | 14 +-
arch/arm/boot/dts/sama7g5.dtsi | 48 ++
arch/arm/configs/multi_v7_defconfig | 3 +
arch/arm/configs/sama7_defconfig | 2 +
drivers/media/platform/Makefile | 1 +
drivers/media/platform/atmel/Kconfig | 15 +
drivers/media/platform/atmel/Makefile | 4 +-
drivers/media/platform/atmel/atmel-isc-base.c | 790 ++++++-----------
drivers/media/platform/atmel/atmel-isc-clk.c | 311 +++++++
drivers/media/platform/atmel/atmel-isc-mc.c | 287 +++++++
drivers/media/platform/atmel/atmel-isc.h | 49 ++
.../media/platform/atmel/atmel-sama5d2-isc.c | 59 +-
.../media/platform/atmel/atmel-sama7g5-isc.c | 72 +-
.../media/platform/atmel/microchip-csi2dc.c | 806 ++++++++++++++++++
15 files changed, 2085 insertions(+), 573 deletions(-)
create mode 100644 Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
create mode 100644 drivers/media/platform/atmel/atmel-isc-clk.c
create mode 100644 drivers/media/platform/atmel/atmel-isc-mc.c
create mode 100644 drivers/media/platform/atmel/microchip-csi2dc.c
--
2.25.1
^ permalink raw reply [flat|nested] 94+ messages in thread
* [PATCH v3 00/23] media: atmel: atmel-isc: implement media controller
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
This series is the v3 of the series that attempts to support media controller
in the atmel ISC and XISC drivers.
This series also includes the csi2dc driver which was previously sent in a
separate series:
https://www.spinics.net/lists/linux-media/msg181042.html
https://www.spinics.net/lists/linux-media/msg181044.html
The driver now addresses comments received in latest v5 series from last year.
The series includes some minor changes and fixes that improve the isc common
code base, like removing the enum frameintervals VIDIOC, fixing bytesperline
for planar formats, etc.
Many thanks to folks from libcamera who helped a lot with understanding
how a media controller driver should behave.
Feedback is welcome !
Changes in v3:
- change in bindings, small fixes in csi2dc driver and conversion to mc
for the isc-base.
- removed some MAINTAINERS patches and used patterns in MAINTAINERS
Changes in v2:
- integrated many changes suggested by Jacopo in the review of the v1 series.
- add a few new patches
Eugen Hristev (23):
MAINTAINERS: add microchip csi2dc
dt-bindings: media: atmel: csi2dc: add bindings for microchip csi2dc
media: atmel: introduce microchip csi2dc driver
media: atmel: atmel-isc: split the clock code into separate source
file
media: atmel: atmel-isc: replace video device name with module name
media: atmel: atmel-sama7g5-isc: fix ispck leftover
media: atmel: atmel-isc-base: use streaming status when queueing
buffers
media: atmel: atmel-isc-base: remove frameintervals VIDIOC
media: atmel: atmel-isc-base: report frame sizes as full supported
range
media: atmel: atmel-isc-base: implement mbus_code support in enumfmt
media: atmel: atmel-isc-base: fix bytesperline value for planar
formats
media: atmel: atmel-isc: implement media controller
ARM: dts: at91: sama7g5: add nodes for video capture
ARM: configs: at91: sama7: add xisc and csi2dc
ARM: multi_v7_defconfig: add atmel video pipeline modules
media: atmel: atmel-sama5d2-isc: fix wrong mask in YUYV format check
media: atmel: atmel-isc-base: use mutex to lock awb workqueue from
streaming
media: atmel: atmel-isc-base: add wb debug messages
media: atmel: atmel-isc-base: clamp wb gain coefficients
media: atmel: atmel-sama7g5-isc: fix UYVY input format mbus_code typo
media: atmel: atmel-isc: add raw Bayer 8bit 10bit output formats
media: atmel: atmel-isc: compact the controller formats list
media: atmel: atmel-isc: change format propagation to subdev into only
verification
.../bindings/media/microchip,csi2dc.yaml | 197 +++++
MAINTAINERS | 14 +-
arch/arm/boot/dts/sama7g5.dtsi | 48 ++
arch/arm/configs/multi_v7_defconfig | 3 +
arch/arm/configs/sama7_defconfig | 2 +
drivers/media/platform/Makefile | 1 +
drivers/media/platform/atmel/Kconfig | 15 +
drivers/media/platform/atmel/Makefile | 4 +-
drivers/media/platform/atmel/atmel-isc-base.c | 790 ++++++-----------
drivers/media/platform/atmel/atmel-isc-clk.c | 311 +++++++
drivers/media/platform/atmel/atmel-isc-mc.c | 287 +++++++
drivers/media/platform/atmel/atmel-isc.h | 49 ++
.../media/platform/atmel/atmel-sama5d2-isc.c | 59 +-
.../media/platform/atmel/atmel-sama7g5-isc.c | 72 +-
.../media/platform/atmel/microchip-csi2dc.c | 806 ++++++++++++++++++
15 files changed, 2085 insertions(+), 573 deletions(-)
create mode 100644 Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
create mode 100644 drivers/media/platform/atmel/atmel-isc-clk.c
create mode 100644 drivers/media/platform/atmel/atmel-isc-mc.c
create mode 100644 drivers/media/platform/atmel/microchip-csi2dc.c
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* [PATCH v3 01/23] MAINTAINERS: add microchip csi2dc
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
Add Microchip CSI2DC driver in the list.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
MAINTAINERS | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index b2239eaff8d3..afb74460e5c1 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -12468,6 +12468,13 @@ L: alsa-devel@alsa-project.org (moderated for non-subscribers)
S: Supported
F: sound/soc/atmel
+MICROCHIP CSI2DC DRIVER
+M: Eugen Hristev <eugen.hristev@microchip.com>
+L: linux-media@vger.kernel.org
+S: Supported
+F: Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
+F: drivers/media/platform/atmel/microchip-csi2dc.c
+
MICROCHIP ECC DRIVER
M: Tudor Ambarus <tudor.ambarus@microchip.com>
L: linux-crypto@vger.kernel.org
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 01/23] MAINTAINERS: add microchip csi2dc
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
Add Microchip CSI2DC driver in the list.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
MAINTAINERS | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index b2239eaff8d3..afb74460e5c1 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -12468,6 +12468,13 @@ L: alsa-devel@alsa-project.org (moderated for non-subscribers)
S: Supported
F: sound/soc/atmel
+MICROCHIP CSI2DC DRIVER
+M: Eugen Hristev <eugen.hristev@microchip.com>
+L: linux-media@vger.kernel.org
+S: Supported
+F: Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
+F: drivers/media/platform/atmel/microchip-csi2dc.c
+
MICROCHIP ECC DRIVER
M: Tudor Ambarus <tudor.ambarus@microchip.com>
L: linux-crypto@vger.kernel.org
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 02/23] dt-bindings: media: atmel: csi2dc: add bindings for microchip csi2dc
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
Add bindings documentation for Microchip CSI2 Demultiplexer controller.
CSI2DC is a demultiplexer from Synopsys IDI interface specification to
parallel interface connection or direct memory access.
CSI2DC can also act as a parallel bypass from a parallel sensor to the
image sensor controller/interface.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
Changes in v3:
- Changed to port base as suggested by Rob
- Added properties from video-interfaces which are being used by the csi2dc
- added mention to the parallel port type as input
- did some rephrasing and rewording
Changes in v2:
- changed the endpoint/port references as suggested by Rob
Changes in this version :
- fixed 'sink' name to be actually source.
- added dma properties and example with dma
Previous change log:
Changes in v5:
- modified bindings as per Rob Herring review
Changes in v4:
- Removed property for inter-line-delay and for clock continuous/non-continuous
- Removed virtual channel by reg for second endpoint
Changes in v3:
- Removed some text from description, as it was explained in the schema
- fixed other things as per Rob's review
- moved some text inside the schema, like the clock description
Changes in v2:
- fixed warnings reported by dt_binding_check
.../bindings/media/microchip,csi2dc.yaml | 197 ++++++++++++++++++
1 file changed, 197 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
diff --git a/Documentation/devicetree/bindings/media/microchip,csi2dc.yaml b/Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
new file mode 100644
index 000000000000..e8544fb2d034
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
@@ -0,0 +1,197 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/microchip,csi2dc.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Microchip CSI2 Demux Controller (CSI2DC)
+
+maintainers:
+ - Eugen Hristev <eugen.hristev@microchip.com>
+
+description:
+ CSI2DC - Camera Serial Interface 2 Demux Controller
+
+ CSI2DC is a hardware block that receives incoming data from either from an
+ IDI interface or from a parallel bus interface.
+ It filters IDI packets based on their data type and virtual channel
+ identifier, then converts the byte stream to a pixel stream into a cross
+ clock domain towards a parallel interface that can be read by a sensor
+ controller.
+ IDI interface is Synopsys proprietary.
+ CSI2DC can act a simple bypass bridge if the incoming data is coming from
+ a parallel interface.
+
+ CSI2DC provides two pipes, one video pipe and one data pipe. Video pipe
+ is connected at the output to a sensor controller and the data pipe is
+ accessible as a DMA slave port to a DMA controller.
+
+ CSI2DC supports a single 'port' node as a sink port with either Synopsys
+ 32-bit IDI interface or a parallel interface.
+
+ CSI2DC supports one 'port' node as source port with parallel interface.
+ This is called video pipe.
+ This port has an 'endpoint' that can be connected to a sink port of another
+ controller (next in pipeline).
+
+ CSI2DC also supports direct access to the data through AHB, via DMA channel,
+ called data pipe.
+ For data pipe to be available, a dma controller and a dma channel must be
+ referenced.
+
+properties:
+ compatible:
+ const: microchip,sama7g5-csi2dc
+
+ reg:
+ maxItems: 1
+
+ clocks:
+ minItems: 2
+ maxItems: 2
+
+ clock-names:
+ description:
+ CSI2DC must have two clocks to function correctly. One clock is the
+ peripheral clock for the inside functionality of the hardware block.
+ This is named 'pclk'. The second clock must be the cross domain clock,
+ in which CSI2DC will perform clock crossing. This clock must be fed
+ by the next controller in pipeline, which usually is a sensor controller.
+ Normally this clock should be given by this sensor controller who
+ is also a clock source. This clock is named 'scck', sensor controller clock.
+ items:
+ - const: pclk
+ - const: scck
+
+ dmas:
+ maxItems: 1
+
+ dma-names:
+ const: rx
+
+ ports:
+ $ref: /schemas/graph.yaml#/properties/ports
+
+ properties:
+ port@0:
+ $ref: /schemas/graph.yaml#/$defs/port-base
+ description:
+ Input port node, single endpoint describing the input port.
+
+ properties:
+ endpoint:
+ $ref: video-interfaces.yaml#
+ unevaluatedProperties: false
+ description: Endpoint connected to input device
+
+ properties:
+ bus-type:
+ enum: [4, 5, 6]
+ default: 4
+
+ bus-width:
+ enum: [8, 9, 10, 11, 12, 13, 14]
+ default: 14
+
+ clock-noncontinuous:
+ type: boolean
+ description:
+ Presence of this boolean property decides whether clock is
+ continuous or noncontinuous.
+
+ remote-endpoint: true
+
+ port@1:
+ $ref: /schemas/graph.yaml#/$defs/port-base
+ description:
+ Output port node, single endpoint describing the output port.
+
+ properties:
+ endpoint:
+ unevaluatedProperties: false
+ $ref: video-interfaces.yaml#
+ description: Endpoint connected to output device
+
+ properties:
+ bus-type:
+ enum: [5, 6]
+ default: 5
+
+ bus-width:
+ enum: [8, 9, 10, 11, 12, 13, 14]
+ default: 14
+
+ remote-endpoint: true
+
+ required:
+ - port@0
+ - port@1
+
+additionalProperties: false
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - clock-names
+ - ports
+
+examples:
+ # Example for connecting to a parallel sensor controller block (video pipe)
+ # and the input is received from Synopsys IDI interface
+ - |
+ csi2dc@e1404000 {
+ compatible = "microchip,sama7g5-csi2dc";
+ reg = <0xe1404000 0x500>;
+ clocks = <&pclk>, <&scck>;
+ clock-names = "pclk", "scck";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ port@0 {
+ reg = <0>; /* must be 0, first child port */
+ csi2dc_in: endpoint { /* input from IDI interface */
+ bus-type = <4>; /* MIPI CSI2 D-PHY */
+ remote-endpoint = <&csi2host_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>; /* must be 1, second child port */
+ csi2dc_out: endpoint {
+ remote-endpoint = <&xisc_in>; /* output to sensor controller */
+ };
+ };
+ };
+ };
+
+ # Example for connecting to a DMA master as an AHB slave
+ # and the input is received from Synopsys IDI interface
+ - |
+ #include <dt-bindings/dma/at91.h>
+ csi2dc@e1404000 {
+ compatible = "microchip,sama7g5-csi2dc";
+ reg = <0xe1404000 0x500>;
+ clocks = <&pclk>, <&scck>;
+ clock-names = "pclk", "scck";
+ dmas = <&dma0 AT91_XDMAC_DT_PERID(34)>;
+ dma-names = "rx";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ port@0 {
+ reg = <0>; /* must be 0, first child port */
+ csi2dc_input: endpoint { /* input from IDI interface */
+ remote-endpoint = <&csi2host_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ };
+ };
+ };
+
+...
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 02/23] dt-bindings: media: atmel: csi2dc: add bindings for microchip csi2dc
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
Add bindings documentation for Microchip CSI2 Demultiplexer controller.
CSI2DC is a demultiplexer from Synopsys IDI interface specification to
parallel interface connection or direct memory access.
CSI2DC can also act as a parallel bypass from a parallel sensor to the
image sensor controller/interface.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
Changes in v3:
- Changed to port base as suggested by Rob
- Added properties from video-interfaces which are being used by the csi2dc
- added mention to the parallel port type as input
- did some rephrasing and rewording
Changes in v2:
- changed the endpoint/port references as suggested by Rob
Changes in this version :
- fixed 'sink' name to be actually source.
- added dma properties and example with dma
Previous change log:
Changes in v5:
- modified bindings as per Rob Herring review
Changes in v4:
- Removed property for inter-line-delay and for clock continuous/non-continuous
- Removed virtual channel by reg for second endpoint
Changes in v3:
- Removed some text from description, as it was explained in the schema
- fixed other things as per Rob's review
- moved some text inside the schema, like the clock description
Changes in v2:
- fixed warnings reported by dt_binding_check
.../bindings/media/microchip,csi2dc.yaml | 197 ++++++++++++++++++
1 file changed, 197 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
diff --git a/Documentation/devicetree/bindings/media/microchip,csi2dc.yaml b/Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
new file mode 100644
index 000000000000..e8544fb2d034
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
@@ -0,0 +1,197 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/microchip,csi2dc.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Microchip CSI2 Demux Controller (CSI2DC)
+
+maintainers:
+ - Eugen Hristev <eugen.hristev@microchip.com>
+
+description:
+ CSI2DC - Camera Serial Interface 2 Demux Controller
+
+ CSI2DC is a hardware block that receives incoming data from either from an
+ IDI interface or from a parallel bus interface.
+ It filters IDI packets based on their data type and virtual channel
+ identifier, then converts the byte stream to a pixel stream into a cross
+ clock domain towards a parallel interface that can be read by a sensor
+ controller.
+ IDI interface is Synopsys proprietary.
+ CSI2DC can act a simple bypass bridge if the incoming data is coming from
+ a parallel interface.
+
+ CSI2DC provides two pipes, one video pipe and one data pipe. Video pipe
+ is connected at the output to a sensor controller and the data pipe is
+ accessible as a DMA slave port to a DMA controller.
+
+ CSI2DC supports a single 'port' node as a sink port with either Synopsys
+ 32-bit IDI interface or a parallel interface.
+
+ CSI2DC supports one 'port' node as source port with parallel interface.
+ This is called video pipe.
+ This port has an 'endpoint' that can be connected to a sink port of another
+ controller (next in pipeline).
+
+ CSI2DC also supports direct access to the data through AHB, via DMA channel,
+ called data pipe.
+ For data pipe to be available, a dma controller and a dma channel must be
+ referenced.
+
+properties:
+ compatible:
+ const: microchip,sama7g5-csi2dc
+
+ reg:
+ maxItems: 1
+
+ clocks:
+ minItems: 2
+ maxItems: 2
+
+ clock-names:
+ description:
+ CSI2DC must have two clocks to function correctly. One clock is the
+ peripheral clock for the inside functionality of the hardware block.
+ This is named 'pclk'. The second clock must be the cross domain clock,
+ in which CSI2DC will perform clock crossing. This clock must be fed
+ by the next controller in pipeline, which usually is a sensor controller.
+ Normally this clock should be given by this sensor controller who
+ is also a clock source. This clock is named 'scck', sensor controller clock.
+ items:
+ - const: pclk
+ - const: scck
+
+ dmas:
+ maxItems: 1
+
+ dma-names:
+ const: rx
+
+ ports:
+ $ref: /schemas/graph.yaml#/properties/ports
+
+ properties:
+ port@0:
+ $ref: /schemas/graph.yaml#/$defs/port-base
+ description:
+ Input port node, single endpoint describing the input port.
+
+ properties:
+ endpoint:
+ $ref: video-interfaces.yaml#
+ unevaluatedProperties: false
+ description: Endpoint connected to input device
+
+ properties:
+ bus-type:
+ enum: [4, 5, 6]
+ default: 4
+
+ bus-width:
+ enum: [8, 9, 10, 11, 12, 13, 14]
+ default: 14
+
+ clock-noncontinuous:
+ type: boolean
+ description:
+ Presence of this boolean property decides whether clock is
+ continuous or noncontinuous.
+
+ remote-endpoint: true
+
+ port@1:
+ $ref: /schemas/graph.yaml#/$defs/port-base
+ description:
+ Output port node, single endpoint describing the output port.
+
+ properties:
+ endpoint:
+ unevaluatedProperties: false
+ $ref: video-interfaces.yaml#
+ description: Endpoint connected to output device
+
+ properties:
+ bus-type:
+ enum: [5, 6]
+ default: 5
+
+ bus-width:
+ enum: [8, 9, 10, 11, 12, 13, 14]
+ default: 14
+
+ remote-endpoint: true
+
+ required:
+ - port@0
+ - port@1
+
+additionalProperties: false
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - clock-names
+ - ports
+
+examples:
+ # Example for connecting to a parallel sensor controller block (video pipe)
+ # and the input is received from Synopsys IDI interface
+ - |
+ csi2dc@e1404000 {
+ compatible = "microchip,sama7g5-csi2dc";
+ reg = <0xe1404000 0x500>;
+ clocks = <&pclk>, <&scck>;
+ clock-names = "pclk", "scck";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ port@0 {
+ reg = <0>; /* must be 0, first child port */
+ csi2dc_in: endpoint { /* input from IDI interface */
+ bus-type = <4>; /* MIPI CSI2 D-PHY */
+ remote-endpoint = <&csi2host_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>; /* must be 1, second child port */
+ csi2dc_out: endpoint {
+ remote-endpoint = <&xisc_in>; /* output to sensor controller */
+ };
+ };
+ };
+ };
+
+ # Example for connecting to a DMA master as an AHB slave
+ # and the input is received from Synopsys IDI interface
+ - |
+ #include <dt-bindings/dma/at91.h>
+ csi2dc@e1404000 {
+ compatible = "microchip,sama7g5-csi2dc";
+ reg = <0xe1404000 0x500>;
+ clocks = <&pclk>, <&scck>;
+ clock-names = "pclk", "scck";
+ dmas = <&dma0 AT91_XDMAC_DT_PERID(34)>;
+ dma-names = "rx";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ port@0 {
+ reg = <0>; /* must be 0, first child port */
+ csi2dc_input: endpoint { /* input from IDI interface */
+ remote-endpoint = <&csi2host_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ };
+ };
+ };
+
+...
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 03/23] media: atmel: introduce microchip csi2dc driver
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
Microchip CSI2DC (CSI2 Demultiplexer Controller) is a misc bridge device
that converts a byte stream in IDI Synopsys format (coming from a CSI2HOST)
to a pixel stream that can be captured by a sensor controller.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
Changes in v3:
- on set format, disable setting the source sink.
- removed useless request to pm_request_idle
- fixed YUV DT
- added last frame debug info
- removed writing read only VPCOL and VPROW registers
- minor cosmetic changes
Changes in v2:
- implement try formats
- added parallel mode
- moved to fwlink endpoints
- many other minor corrections from Jacopo's review
Changes in this revision:
- addressed comments by Jacopo and Laurent as in this thread:
https://www.spinics.net/lists/linux-media/msg181044.html
Previous change log :
Changes in v5:
- only in bindings
Changes in v4:
- now using get_mbus_config ops to get data from the subdevice, like the
virtual channel id, and the clock type.
- now having possibility to select any of the RAW10 data modes
- at completion time, select which formats are also available in the subdevice,
and move to the dynamic list accordingly
- changed the pipeline integration, do not advertise subdev ready at probe time.
wait until completion is done, and then start a workqueue that will register
this device as a subdevice for the next element in pipeline.
- moved the s_power code into a different function called now csi2dc_power
that is called with CONFIG_PM functions. This is also called at completion,
to have the device ready in case CONFIG_PM is not selected on the platform.
- merged try_fmt into set_fmt
- driver cleanup, wrapped lines over 80 characters
Changes in v2:
- moved driver to platform/atmel
- fixed minor things as per Sakari's review
- still some things from v2 review are not yet addressed, to be followed up
drivers/media/platform/Makefile | 1 +
drivers/media/platform/atmel/Kconfig | 15 +
drivers/media/platform/atmel/Makefile | 1 +
.../media/platform/atmel/microchip-csi2dc.c | 806 ++++++++++++++++++
4 files changed, 823 insertions(+)
create mode 100644 drivers/media/platform/atmel/microchip-csi2dc.c
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 832357240e89..b18e5f704145 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -69,6 +69,7 @@ obj-$(CONFIG_VIDEO_RCAR_VIN) += rcar-vin/
obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel/
obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel/
obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel/
+obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += atmel/
obj-$(CONFIG_VIDEO_STM32_DCMI) += stm32/
diff --git a/drivers/media/platform/atmel/Kconfig b/drivers/media/platform/atmel/Kconfig
index dda2f27da317..f83bee373d82 100644
--- a/drivers/media/platform/atmel/Kconfig
+++ b/drivers/media/platform/atmel/Kconfig
@@ -40,3 +40,18 @@ config VIDEO_ATMEL_ISI
help
This module makes the ATMEL Image Sensor Interface available
as a v4l2 device.
+
+config VIDEO_MICROCHIP_CSI2DC
+ tristate "Microchip CSI2 Demux Controller"
+ depends on VIDEO_V4L2 && COMMON_CLK && OF
+ depends on ARCH_AT91 || COMPILE_TEST
+ select MEDIA_CONTROLLER
+ select VIDEO_V4L2_SUBDEV_API
+ select V4L2_FWNODE
+ help
+ CSI2 Demux Controller driver. CSI2DC is a helper chip
+ that converts IDI interface byte stream to a parallel pixel stream.
+ It supports various RAW formats as input.
+
+ To compile this driver as a module, choose M here: the
+ module will be called microchip-csi2dc.
diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
index 46d264ab7948..39f0a7eba702 100644
--- a/drivers/media/platform/atmel/Makefile
+++ b/drivers/media/platform/atmel/Makefile
@@ -6,3 +6,4 @@ obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o
obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel-isc.o
obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel-xisc.o
+obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += microchip-csi2dc.o
diff --git a/drivers/media/platform/atmel/microchip-csi2dc.c b/drivers/media/platform/atmel/microchip-csi2dc.c
new file mode 100644
index 000000000000..6bc549c28e05
--- /dev/null
+++ b/drivers/media/platform/atmel/microchip-csi2dc.c
@@ -0,0 +1,806 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Microchip CSI2 Demux Controller (CSI2DC) driver
+ *
+ * Copyright (C) 2018 Microchip Technology, Inc.
+ *
+ * Author: Eugen Hristev <eugen.hristev@microchip.com>
+ *
+ */
+
+#include <linux/clk.h>
+#include <linux/mod_devicetable.h>
+#include <linux/module.h>
+#include <linux/of_graph.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/videodev2.h>
+
+#include <media/v4l2-fwnode.h>
+#include <media/v4l2-subdev.h>
+
+/* Global configuration register */
+#define CSI2DC_GCFG 0x0
+
+/* MIPI sensor pixel clock is free running */
+#define CSI2DC_GCFG_MIPIFRN BIT(0)
+/* GPIO parallel interface selection */
+#define CSI2DC_GCFG_GPIOSEL BIT(1)
+/* Output waveform inter-line minimum delay */
+#define CSI2DC_GCFG_HLC(v) ((v) << 4)
+#define CSI2DC_GCFG_HLC_MASK GENMASK(7, 4)
+/* SAMA7G5 requires a HLC delay of 15 */
+#define SAMA7G5_HLC (15)
+
+/* Global control register */
+#define CSI2DC_GCTLR 0x04
+#define CSI2DC_GCTLR_SWRST BIT(0)
+
+/* Global status register */
+#define CSI2DC_GS 0x08
+
+/* SSP interrupt status register */
+#define CSI2DC_SSPIS 0x28
+/* Pipe update register */
+#define CSI2DC_PU 0xc0
+/* Video pipe attributes update */
+#define CSI2DC_PU_VP BIT(0)
+
+/* Pipe update status register */
+#define CSI2DC_PUS 0xc4
+
+/* Video pipeline Interrupt Status Register */
+#define CSI2DC_VPISR 0xf4
+
+/* Video pipeline enable register */
+#define CSI2DC_VPE 0xf8
+#define CSI2DC_VPE_ENABLE BIT(0)
+
+/* Video pipeline configuration register */
+#define CSI2DC_VPCFG 0xfc
+/* Data type */
+#define CSI2DC_VPCFG_DT(v) ((v) << 0)
+#define CSI2DC_VPCFG_DT_MASK GENMASK(5, 0)
+/* Virtual channel identifier */
+#define CSI2DC_VPCFG_VC(v) ((v) << 6)
+#define CSI2DC_VPCFG_VC_MASK GENMASK(7, 6)
+/* Decompression enable */
+#define CSI2DC_VPCFG_DE BIT(8)
+/* Decoder mode */
+#define CSI2DC_VPCFG_DM(v) ((v) << 9)
+#define CSI2DC_VPCFG_DM_DECODER8TO12 0
+/* Decoder predictor 2 selection */
+#define CSI2DC_VPCFG_DP2 BIT(12)
+/* Recommended memory storage */
+#define CSI2DC_VPCFG_RMS BIT(13)
+/* Post adjustment */
+#define CSI2DC_VPCFG_PA BIT(14)
+
+/* Video pipeline column register */
+#define CSI2DC_VPCOL 0x100
+/* Column number */
+#define CSI2DC_VPCOL_COL(v) ((v) << 0)
+#define CSI2DC_VPCOL_COL_MASK GENMASK(15, 0)
+
+/* Video pipeline row register */
+#define CSI2DC_VPROW 0x104
+/* Row number */
+#define CSI2DC_VPROW_ROW(v) ((v) << 0)
+#define CSI2DC_VPROW_ROW_MASK GENMASK(15, 0)
+
+/* Version register */
+#define CSI2DC_VERSION 0x1fc
+
+/* register read/write helpers */
+#define csi2dc_readl(st, reg) readl_relaxed((st)->base + (reg))
+#define csi2dc_writel(st, reg, val) writel_relaxed((val), \
+ (st)->base + (reg))
+
+/* supported RAW data types */
+#define CSI2DC_DT_RAW6 0x28
+#define CSI2DC_DT_RAW7 0x29
+#define CSI2DC_DT_RAW8 0x2a
+#define CSI2DC_DT_RAW10 0x2b
+#define CSI2DC_DT_RAW12 0x2c
+#define CSI2DC_DT_RAW14 0x2d
+/* YUV data types */
+#define CSI2DC_DT_YUV422_8B 0x1e
+
+/*
+ * struct csi2dc_format - CSI2DC format type struct
+ * @mbus_code: Media bus code for the format
+ * @dt: Data type constant for this format
+ */
+struct csi2dc_format {
+ u32 mbus_code;
+ u32 dt;
+};
+
+static const struct csi2dc_format csi2dc_formats[] = {
+ {
+ .mbus_code = MEDIA_BUS_FMT_SRGGB8_1X8,
+ .dt = CSI2DC_DT_RAW8,
+ }, {
+ .mbus_code = MEDIA_BUS_FMT_SBGGR8_1X8,
+ .dt = CSI2DC_DT_RAW8,
+ }, {
+ .mbus_code = MEDIA_BUS_FMT_SGRBG8_1X8,
+ .dt = CSI2DC_DT_RAW8,
+ }, {
+ .mbus_code = MEDIA_BUS_FMT_SGBRG8_1X8,
+ .dt = CSI2DC_DT_RAW8,
+ }, {
+ .mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10,
+ .dt = CSI2DC_DT_RAW10,
+ }, {
+ .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10,
+ .dt = CSI2DC_DT_RAW10,
+ }, {
+ .mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10,
+ .dt = CSI2DC_DT_RAW10,
+ }, {
+ .mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10,
+ .dt = CSI2DC_DT_RAW10,
+ }, {
+ .mbus_code = MEDIA_BUS_FMT_YUYV8_2X8,
+ .dt = CSI2DC_DT_YUV422_8B,
+ },
+};
+
+enum mipi_csi_pads {
+ CSI2DC_PAD_SINK = 0,
+ CSI2DC_PAD_SOURCE = 1,
+ CSI2DC_PADS_NUM = 2,
+};
+
+/*
+ * struct csi2dc_device - CSI2DC device driver data/config struct
+ * @base: Register map base address
+ * @csi2dc_sd: v4l2 subdevice for the csi2dc device
+ * This is the subdevice that the csi2dc device itself
+ * registers in v4l2 subsystem
+ * @dev: struct device for this csi2dc device
+ * @pclk: Peripheral clock reference
+ * Input clock that clocks the hardware block internal
+ * logic
+ * @scck: Sensor Controller clock reference
+ * Input clock that is used to generate the pixel clock
+ * @format: Current saved format used in g/s fmt
+ * @cur_fmt: Current state format
+ * @try_fmt: Try format that is being tried
+ * @pads: Media entity pads for the csi2dc subdevice
+ * @clk_gated: Whether the clock is gated or free running
+ * @video_pipe: Whether video pipeline is configured
+ * @parallel_mode: The underlying subdevice is connected on a parallel bus
+ * @vc: Current set virtual channel
+ * @notifier: Async notifier that is used to bound the underlying
+ * subdevice to the csi2dc subdevice
+ * @input_sd: Reference to the underlying subdevice bound to the
+ * csi2dc subdevice
+ * @remote_pad: Pad number of the underlying subdevice that is linked
+ * to the csi2dc subdevice sink pad.
+ */
+struct csi2dc_device {
+ void __iomem *base;
+ struct v4l2_subdev csi2dc_sd;
+ struct device *dev;
+ struct clk *pclk;
+ struct clk *scck;
+
+ struct v4l2_mbus_framefmt format;
+
+ const struct csi2dc_format *cur_fmt;
+ const struct csi2dc_format *try_fmt;
+
+ struct media_pad pads[CSI2DC_PADS_NUM];
+
+ bool clk_gated;
+ bool video_pipe;
+ bool parallel_mode;
+ u32 vc;
+
+ struct v4l2_async_notifier notifier;
+
+ struct v4l2_subdev *input_sd;
+
+ u32 remote_pad;
+};
+
+static inline struct csi2dc_device *
+csi2dc_sd_to_csi2dc_device(struct v4l2_subdev *csi2dc_sd)
+{
+ return container_of(csi2dc_sd, struct csi2dc_device, csi2dc_sd);
+}
+
+static int csi2dc_enum_mbus_code(struct v4l2_subdev *csi2dc_sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ if (code->index >= ARRAY_SIZE(csi2dc_formats))
+ return -EINVAL;
+
+ code->code = csi2dc_formats[code->index].mbus_code;
+
+ return 0;
+}
+
+static int csi2dc_get_fmt(struct v4l2_subdev *csi2dc_sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_format *format)
+{
+ struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
+ struct v4l2_mbus_framefmt *v4l2_try_fmt;
+
+ if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
+ v4l2_try_fmt = v4l2_subdev_get_try_format(csi2dc_sd, sd_state,
+ format->pad);
+ format->format = *v4l2_try_fmt;
+
+ return 0;
+ }
+
+ format->format = csi2dc->format;
+
+ return 0;
+}
+
+static int csi2dc_set_fmt(struct v4l2_subdev *csi2dc_sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_format *req_fmt)
+{
+ struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
+ const struct csi2dc_format *fmt, *try_fmt = NULL;
+ struct v4l2_mbus_framefmt *v4l2_try_fmt;
+ unsigned int i;
+
+ /*
+ * Setting the source pad is disabled.
+ * The same format is being propagated from the sink to source.
+ */
+ if (req_fmt->pad == CSI2DC_PAD_SOURCE)
+ return -EINVAL;
+
+ for (i = 0; i < ARRAY_SIZE(csi2dc_formats); i++) {
+ fmt = &csi2dc_formats[i];
+ if (req_fmt->format.code == fmt->mbus_code)
+ try_fmt = fmt;
+ fmt++;
+ }
+
+ /* in case we could not find the desired format, default to something */
+ if (!try_fmt) {
+ try_fmt = &csi2dc_formats[0];
+
+ dev_dbg(csi2dc->dev,
+ "CSI2DC unsupported format 0x%x, defaulting to 0x%x\n",
+ req_fmt->format.code, csi2dc_formats[0].mbus_code);
+ }
+
+ req_fmt->format.code = try_fmt->mbus_code;
+ req_fmt->format.colorspace = V4L2_COLORSPACE_SRGB;
+ req_fmt->format.field = V4L2_FIELD_NONE;
+
+ if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
+ v4l2_try_fmt = v4l2_subdev_get_try_format(csi2dc_sd, sd_state,
+ req_fmt->pad);
+ *v4l2_try_fmt = req_fmt->format;
+ /* Trying on the sink pad makes the source pad change too */
+ v4l2_try_fmt = v4l2_subdev_get_try_format(csi2dc_sd,
+ sd_state,
+ CSI2DC_PAD_SOURCE);
+ *v4l2_try_fmt = req_fmt->format;
+
+ /* if we are just trying, we are done */
+ return 0;
+ }
+
+ /* save the format for later requests */
+ csi2dc->format = req_fmt->format;
+
+ /* update config */
+ csi2dc->cur_fmt = try_fmt;
+
+ dev_dbg(csi2dc->dev, "new format set: 0x%x @%dx%d\n",
+ csi2dc->format.code, csi2dc->format.width,
+ csi2dc->format.height);
+
+ return 0;
+}
+
+static int csi2dc_power(struct csi2dc_device *csi2dc, int on)
+{
+ int ret = 0;
+
+ if (on) {
+ ret = clk_prepare_enable(csi2dc->pclk);
+ if (ret) {
+ dev_err(csi2dc->dev, "failed to enable pclk:%d\n", ret);
+ return ret;
+ }
+
+ ret = clk_prepare_enable(csi2dc->scck);
+ if (ret) {
+ dev_err(csi2dc->dev, "failed to enable scck:%d\n", ret);
+ clk_disable_unprepare(csi2dc->pclk);
+ return ret;
+ }
+
+ /* if powering up, deassert reset line */
+ csi2dc_writel(csi2dc, CSI2DC_GCTLR, CSI2DC_GCTLR_SWRST);
+ } else {
+ /* if powering down, assert reset line */
+ csi2dc_writel(csi2dc, CSI2DC_GCTLR, 0);
+
+ clk_disable_unprepare(csi2dc->scck);
+ clk_disable_unprepare(csi2dc->pclk);
+ }
+
+ return ret;
+}
+
+static int csi2dc_get_mbus_config(struct csi2dc_device *csi2dc)
+{
+ struct v4l2_mbus_config mbus_config = { 0 };
+ int ret;
+
+ ret = v4l2_subdev_call(csi2dc->input_sd, pad, get_mbus_config,
+ csi2dc->remote_pad, &mbus_config);
+ if (ret == -ENOIOCTLCMD) {
+ dev_dbg(csi2dc->dev,
+ "no remote mbus configuration available\n");
+ goto csi2dc_get_mbus_config_defaults;
+ }
+
+ if (ret) {
+ dev_err(csi2dc->dev,
+ "failed to get remote mbus configuration\n");
+ goto csi2dc_get_mbus_config_defaults;
+ }
+
+ if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_0)
+ csi2dc->vc = 0;
+ else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_1)
+ csi2dc->vc = 1;
+ else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_2)
+ csi2dc->vc = 2;
+ else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_3)
+ csi2dc->vc = 3;
+
+ dev_dbg(csi2dc->dev, "subdev sending on channel %d\n", csi2dc->vc);
+
+ csi2dc->clk_gated = mbus_config.flags &
+ V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK;
+
+ dev_dbg(csi2dc->dev, "mbus_config: %s clock\n",
+ csi2dc->clk_gated ? "gated" : "free running");
+
+ return 0;
+
+csi2dc_get_mbus_config_defaults:
+ csi2dc->vc = 0; /* Virtual ID 0 by default */
+
+ return 0;
+}
+
+static void csi2dc_vp_update(struct csi2dc_device *csi2dc)
+{
+ u32 vp, gcfg;
+
+ if (!csi2dc->video_pipe) {
+ dev_err(csi2dc->dev, "video pipeline unavailable\n");
+ return;
+ }
+
+ if (csi2dc->parallel_mode) {
+ /* In parallel mode, GPIO parallel interface must be selected */
+ gcfg = csi2dc_readl(csi2dc, CSI2DC_GCFG);
+ gcfg |= CSI2DC_GCFG_GPIOSEL;
+ csi2dc_writel(csi2dc, CSI2DC_GCFG, gcfg);
+ return;
+ }
+
+ /* serial video pipeline */
+
+ csi2dc_writel(csi2dc, CSI2DC_GCFG,
+ (SAMA7G5_HLC & CSI2DC_GCFG_HLC_MASK) |
+ (csi2dc->clk_gated ? 0 : CSI2DC_GCFG_MIPIFRN));
+
+ vp = CSI2DC_VPCFG_DT(csi2dc->cur_fmt->dt) & CSI2DC_VPCFG_DT_MASK;
+ vp |= CSI2DC_VPCFG_VC(csi2dc->vc) & CSI2DC_VPCFG_VC_MASK;
+ vp &= ~CSI2DC_VPCFG_DE;
+ vp |= CSI2DC_VPCFG_DM(CSI2DC_VPCFG_DM_DECODER8TO12);
+ vp &= ~CSI2DC_VPCFG_DP2;
+ vp &= ~CSI2DC_VPCFG_RMS;
+ vp |= CSI2DC_VPCFG_PA;
+
+ csi2dc_writel(csi2dc, CSI2DC_VPCFG, vp);
+ csi2dc_writel(csi2dc, CSI2DC_VPE, CSI2DC_VPE_ENABLE);
+ csi2dc_writel(csi2dc, CSI2DC_PU, CSI2DC_PU_VP);
+}
+
+static int csi2dc_s_stream(struct v4l2_subdev *csi2dc_sd, int enable)
+{
+ struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
+ int ret;
+
+ if (enable) {
+ ret = pm_runtime_resume_and_get(csi2dc->dev);
+ if (ret < 0)
+ return ret;
+
+ csi2dc_get_mbus_config(csi2dc);
+
+ csi2dc_vp_update(csi2dc);
+
+ return v4l2_subdev_call(csi2dc->input_sd, video, s_stream,
+ true);
+ }
+
+ dev_dbg(csi2dc->dev,
+ "Last frame received: VPCOLR = %u, VPROWR= %u, VPISR = %x\n",
+ csi2dc_readl(csi2dc, CSI2DC_VPCOL),
+ csi2dc_readl(csi2dc, CSI2DC_VPROW),
+ csi2dc_readl(csi2dc, CSI2DC_VPISR));
+
+ /* stop streaming scenario */
+ ret = v4l2_subdev_call(csi2dc->input_sd, video, s_stream, false);
+
+ pm_runtime_put_sync(csi2dc->dev);
+
+ return ret;
+}
+
+static int csi2dc_init_cfg(struct v4l2_subdev *csi2dc_sd,
+ struct v4l2_subdev_state *sd_state)
+{
+ struct v4l2_mbus_framefmt *v4l2_try_fmt =
+ v4l2_subdev_get_try_format(csi2dc_sd, sd_state, 0);
+
+ v4l2_try_fmt->height = 480;
+ v4l2_try_fmt->width = 640;
+ v4l2_try_fmt->code = csi2dc_formats[0].mbus_code;
+ v4l2_try_fmt->colorspace = V4L2_COLORSPACE_SRGB;
+ v4l2_try_fmt->field = V4L2_FIELD_NONE;
+ v4l2_try_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ v4l2_try_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
+ v4l2_try_fmt->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+
+ return 0;
+}
+
+static const struct v4l2_subdev_pad_ops csi2dc_pad_ops = {
+ .enum_mbus_code = csi2dc_enum_mbus_code,
+ .set_fmt = csi2dc_set_fmt,
+ .get_fmt = csi2dc_get_fmt,
+ .init_cfg = csi2dc_init_cfg,
+};
+
+static const struct v4l2_subdev_video_ops csi2dc_video_ops = {
+ .s_stream = csi2dc_s_stream,
+};
+
+static const struct v4l2_subdev_ops csi2dc_subdev_ops = {
+ .pad = &csi2dc_pad_ops,
+ .video = &csi2dc_video_ops,
+};
+
+static int csi2dc_async_bound(struct v4l2_async_notifier *notifier,
+ struct v4l2_subdev *subdev,
+ struct v4l2_async_subdev *asd)
+{
+ struct csi2dc_device *csi2dc = container_of(notifier,
+ struct csi2dc_device, notifier);
+ int pad;
+ int ret;
+
+ csi2dc->input_sd = subdev;
+
+ pad = media_entity_get_fwnode_pad(&subdev->entity, asd->match.fwnode,
+ MEDIA_PAD_FL_SOURCE);
+ if (pad < 0) {
+ dev_err(csi2dc->dev, "Failed to find pad for %s\n",
+ subdev->name);
+ return pad;
+ }
+
+ csi2dc->remote_pad = pad;
+
+ ret = media_create_pad_link(&csi2dc->input_sd->entity,
+ csi2dc->remote_pad,
+ &csi2dc->csi2dc_sd.entity, 0,
+ MEDIA_LNK_FL_ENABLED);
+ if (ret) {
+ dev_err(csi2dc->dev,
+ "Failed to create pad link: %s to %s\n",
+ csi2dc->input_sd->entity.name,
+ csi2dc->csi2dc_sd.entity.name);
+ return ret;
+ }
+
+ dev_dbg(csi2dc->dev, "link with %s pad: %d\n",
+ csi2dc->input_sd->name, csi2dc->remote_pad);
+
+ return ret;
+}
+
+static const struct v4l2_async_notifier_operations csi2dc_async_ops = {
+ .bound = csi2dc_async_bound,
+};
+
+static int csi2dc_prepare_notifier(struct csi2dc_device *csi2dc,
+ struct fwnode_handle *input_fwnode)
+{
+ struct v4l2_async_subdev *asd;
+ int ret = 0;
+
+ v4l2_async_nf_init(&csi2dc->notifier);
+
+ asd = v4l2_async_nf_add_fwnode_remote(&csi2dc->notifier,
+ input_fwnode,
+ struct v4l2_async_subdev);
+
+ fwnode_handle_put(input_fwnode);
+
+ if (IS_ERR(asd)) {
+ ret = PTR_ERR(asd);
+ dev_err(csi2dc->dev,
+ "failed to add async notifier for node %pOF: %d\n",
+ to_of_node(input_fwnode), ret);
+ v4l2_async_nf_cleanup(&csi2dc->notifier);
+ return ret;
+ }
+
+ csi2dc->notifier.ops = &csi2dc_async_ops;
+
+ ret = v4l2_async_subdev_nf_register(&csi2dc->csi2dc_sd,
+ &csi2dc->notifier);
+ if (ret) {
+ dev_err(csi2dc->dev, "fail to register async notifier: %d\n",
+ ret);
+ v4l2_async_nf_cleanup(&csi2dc->notifier);
+ }
+
+ return ret;
+}
+
+static int csi2dc_of_parse(struct csi2dc_device *csi2dc,
+ struct device_node *of_node)
+{
+ struct fwnode_handle *input_fwnode, *output_fwnode;
+ struct v4l2_fwnode_endpoint input_endpoint = { 0 },
+ output_endpoint = { 0 };
+ int ret;
+
+ input_fwnode = fwnode_graph_get_next_endpoint(of_fwnode_handle(of_node),
+ NULL);
+ if (!input_fwnode) {
+ dev_err(csi2dc->dev,
+ "missing port node at %pOF, input node is mandatory.\n",
+ of_node);
+ return -EINVAL;
+ }
+
+ ret = v4l2_fwnode_endpoint_parse(input_fwnode, &input_endpoint);
+ if (ret) {
+ dev_err(csi2dc->dev, "endpoint not defined at %pOF\n", of_node);
+ goto csi2dc_of_parse_err;
+ }
+
+ if (input_endpoint.bus_type == V4L2_MBUS_PARALLEL ||
+ input_endpoint.bus_type == V4L2_MBUS_BT656) {
+ csi2dc->parallel_mode = true;
+ dev_dbg(csi2dc->dev,
+ "subdevice connected on parallel interface\n");
+ }
+
+ if (input_endpoint.bus_type == V4L2_MBUS_CSI2_DPHY) {
+ csi2dc->clk_gated = input_endpoint.bus.mipi_csi2.flags &
+ V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK;
+ dev_dbg(csi2dc->dev,
+ "subdevice connected on serial interface\n");
+ dev_dbg(csi2dc->dev, "DT: %s clock\n",
+ csi2dc->clk_gated ? "gated" : "free running");
+ }
+
+ output_fwnode = fwnode_graph_get_next_endpoint
+ (of_fwnode_handle(of_node), input_fwnode);
+
+ if (output_fwnode)
+ ret = v4l2_fwnode_endpoint_parse(output_fwnode,
+ &output_endpoint);
+
+ fwnode_handle_put(output_fwnode);
+
+ if (!output_fwnode || ret) {
+ dev_info(csi2dc->dev,
+ "missing output node at %pOF, data pipe available only.\n",
+ of_node);
+ } else {
+ if (output_endpoint.bus_type != V4L2_MBUS_PARALLEL &&
+ output_endpoint.bus_type != V4L2_MBUS_BT656) {
+ dev_err(csi2dc->dev,
+ "output port must be parallel/bt656.\n");
+ ret = -EINVAL;
+ goto csi2dc_of_parse_err;
+ }
+
+ csi2dc->video_pipe = true;
+
+ dev_dbg(csi2dc->dev,
+ "block %pOF [%d.%d]->[%d.%d] video pipeline\n",
+ of_node, input_endpoint.base.port,
+ input_endpoint.base.id, output_endpoint.base.port,
+ output_endpoint.base.id);
+ }
+
+ /* prepare async notifier for subdevice completion */
+ return csi2dc_prepare_notifier(csi2dc, input_fwnode);
+
+csi2dc_of_parse_err:
+ fwnode_handle_put(input_fwnode);
+ return ret;
+}
+
+static void csi2dc_default_format(struct csi2dc_device *csi2dc)
+{
+ csi2dc->cur_fmt = &csi2dc_formats[0];
+
+ csi2dc->format.height = 480;
+ csi2dc->format.width = 640;
+ csi2dc->format.code = csi2dc_formats[0].mbus_code;
+ csi2dc->format.colorspace = V4L2_COLORSPACE_SRGB;
+ csi2dc->format.field = V4L2_FIELD_NONE;
+ csi2dc->format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ csi2dc->format.quantization = V4L2_QUANTIZATION_DEFAULT;
+ csi2dc->format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
+}
+
+static int csi2dc_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct csi2dc_device *csi2dc;
+ int ret = 0;
+ u32 ver;
+
+ csi2dc = devm_kzalloc(dev, sizeof(*csi2dc), GFP_KERNEL);
+ if (!csi2dc)
+ return -ENOMEM;
+
+ csi2dc->dev = dev;
+
+ csi2dc->base = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(csi2dc->base)) {
+ dev_err(dev, "base address not set\n");
+ return PTR_ERR(csi2dc->base);
+ }
+
+ csi2dc->pclk = devm_clk_get(dev, "pclk");
+ if (IS_ERR(csi2dc->pclk)) {
+ ret = PTR_ERR(csi2dc->pclk);
+ dev_err(dev, "failed to get pclk: %d\n", ret);
+ return ret;
+ }
+
+ csi2dc->scck = devm_clk_get(dev, "scck");
+ if (IS_ERR(csi2dc->scck)) {
+ ret = PTR_ERR(csi2dc->scck);
+ dev_err(dev, "failed to get scck: %d\n", ret);
+ return ret;
+ }
+
+ v4l2_subdev_init(&csi2dc->csi2dc_sd, &csi2dc_subdev_ops);
+
+ csi2dc->csi2dc_sd.owner = THIS_MODULE;
+ csi2dc->csi2dc_sd.dev = dev;
+ snprintf(csi2dc->csi2dc_sd.name, sizeof(csi2dc->csi2dc_sd.name),
+ "csi2dc");
+
+ csi2dc->csi2dc_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+ csi2dc->csi2dc_sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
+
+ platform_set_drvdata(pdev, csi2dc);
+
+ ret = csi2dc_of_parse(csi2dc, dev->of_node);
+ if (ret)
+ goto csi2dc_probe_cleanup_entity;
+
+ csi2dc->pads[CSI2DC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+ if (csi2dc->video_pipe)
+ csi2dc->pads[CSI2DC_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
+
+ ret = media_entity_pads_init(&csi2dc->csi2dc_sd.entity,
+ csi2dc->video_pipe ? CSI2DC_PADS_NUM : 1,
+ csi2dc->pads);
+ if (ret < 0) {
+ dev_err(dev, "media entity init failed\n");
+ goto csi2dc_probe_cleanup_notifier;
+ }
+
+ csi2dc_default_format(csi2dc);
+
+ /* turn power on to validate capabilities */
+ ret = csi2dc_power(csi2dc, true);
+ if (ret < 0)
+ goto csi2dc_probe_cleanup_notifier;
+
+ pm_runtime_set_active(dev);
+ pm_runtime_enable(dev);
+ ver = csi2dc_readl(csi2dc, CSI2DC_VERSION);
+
+ /*
+ * we must register the subdev after PM runtime has been requested,
+ * otherwise we might bound immediately and request pm_runtime_resume
+ * before runtime_enable.
+ */
+ ret = v4l2_async_register_subdev(&csi2dc->csi2dc_sd);
+ if (ret) {
+ dev_err(csi2dc->dev, "failed to register the subdevice\n");
+ goto csi2dc_probe_cleanup_notifier;
+ }
+
+ dev_info(dev, "Microchip CSI2DC version %x\n", ver);
+
+ return 0;
+
+csi2dc_probe_cleanup_notifier:
+ v4l2_async_nf_cleanup(&csi2dc->notifier);
+csi2dc_probe_cleanup_entity:
+ media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
+
+ return ret;
+}
+
+static int csi2dc_remove(struct platform_device *pdev)
+{
+ struct csi2dc_device *csi2dc = platform_get_drvdata(pdev);
+
+ pm_runtime_disable(&pdev->dev);
+
+ v4l2_async_unregister_subdev(&csi2dc->csi2dc_sd);
+ v4l2_async_nf_unregister(&csi2dc->notifier);
+ v4l2_async_nf_cleanup(&csi2dc->notifier);
+ media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
+
+ return 0;
+}
+
+static int __maybe_unused csi2dc_runtime_suspend(struct device *dev)
+{
+ struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
+
+ return csi2dc_power(csi2dc, false);
+}
+
+static int __maybe_unused csi2dc_runtime_resume(struct device *dev)
+{
+ struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
+
+ return csi2dc_power(csi2dc, true);
+}
+
+static const struct dev_pm_ops csi2dc_dev_pm_ops = {
+ SET_RUNTIME_PM_OPS(csi2dc_runtime_suspend, csi2dc_runtime_resume, NULL)
+};
+
+static const struct of_device_id csi2dc_of_match[] = {
+ { .compatible = "microchip,sama7g5-csi2dc" },
+ { }
+};
+
+MODULE_DEVICE_TABLE(of, csi2dc_of_match);
+
+static struct platform_driver csi2dc_driver = {
+ .probe = csi2dc_probe,
+ .remove = csi2dc_remove,
+ .driver = {
+ .name = "microchip-csi2dc",
+ .pm = &csi2dc_dev_pm_ops,
+ .of_match_table = of_match_ptr(csi2dc_of_match),
+ },
+};
+
+module_platform_driver(csi2dc_driver);
+
+MODULE_AUTHOR("Eugen Hristev <eugen.hristev@microchip.com>");
+MODULE_DESCRIPTION("Microchip CSI2 Demux Controller driver");
+MODULE_LICENSE("GPL v2");
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 03/23] media: atmel: introduce microchip csi2dc driver
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
Microchip CSI2DC (CSI2 Demultiplexer Controller) is a misc bridge device
that converts a byte stream in IDI Synopsys format (coming from a CSI2HOST)
to a pixel stream that can be captured by a sensor controller.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
Changes in v3:
- on set format, disable setting the source sink.
- removed useless request to pm_request_idle
- fixed YUV DT
- added last frame debug info
- removed writing read only VPCOL and VPROW registers
- minor cosmetic changes
Changes in v2:
- implement try formats
- added parallel mode
- moved to fwlink endpoints
- many other minor corrections from Jacopo's review
Changes in this revision:
- addressed comments by Jacopo and Laurent as in this thread:
https://www.spinics.net/lists/linux-media/msg181044.html
Previous change log :
Changes in v5:
- only in bindings
Changes in v4:
- now using get_mbus_config ops to get data from the subdevice, like the
virtual channel id, and the clock type.
- now having possibility to select any of the RAW10 data modes
- at completion time, select which formats are also available in the subdevice,
and move to the dynamic list accordingly
- changed the pipeline integration, do not advertise subdev ready at probe time.
wait until completion is done, and then start a workqueue that will register
this device as a subdevice for the next element in pipeline.
- moved the s_power code into a different function called now csi2dc_power
that is called with CONFIG_PM functions. This is also called at completion,
to have the device ready in case CONFIG_PM is not selected on the platform.
- merged try_fmt into set_fmt
- driver cleanup, wrapped lines over 80 characters
Changes in v2:
- moved driver to platform/atmel
- fixed minor things as per Sakari's review
- still some things from v2 review are not yet addressed, to be followed up
drivers/media/platform/Makefile | 1 +
drivers/media/platform/atmel/Kconfig | 15 +
drivers/media/platform/atmel/Makefile | 1 +
.../media/platform/atmel/microchip-csi2dc.c | 806 ++++++++++++++++++
4 files changed, 823 insertions(+)
create mode 100644 drivers/media/platform/atmel/microchip-csi2dc.c
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 832357240e89..b18e5f704145 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -69,6 +69,7 @@ obj-$(CONFIG_VIDEO_RCAR_VIN) += rcar-vin/
obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel/
obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel/
obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel/
+obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += atmel/
obj-$(CONFIG_VIDEO_STM32_DCMI) += stm32/
diff --git a/drivers/media/platform/atmel/Kconfig b/drivers/media/platform/atmel/Kconfig
index dda2f27da317..f83bee373d82 100644
--- a/drivers/media/platform/atmel/Kconfig
+++ b/drivers/media/platform/atmel/Kconfig
@@ -40,3 +40,18 @@ config VIDEO_ATMEL_ISI
help
This module makes the ATMEL Image Sensor Interface available
as a v4l2 device.
+
+config VIDEO_MICROCHIP_CSI2DC
+ tristate "Microchip CSI2 Demux Controller"
+ depends on VIDEO_V4L2 && COMMON_CLK && OF
+ depends on ARCH_AT91 || COMPILE_TEST
+ select MEDIA_CONTROLLER
+ select VIDEO_V4L2_SUBDEV_API
+ select V4L2_FWNODE
+ help
+ CSI2 Demux Controller driver. CSI2DC is a helper chip
+ that converts IDI interface byte stream to a parallel pixel stream.
+ It supports various RAW formats as input.
+
+ To compile this driver as a module, choose M here: the
+ module will be called microchip-csi2dc.
diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
index 46d264ab7948..39f0a7eba702 100644
--- a/drivers/media/platform/atmel/Makefile
+++ b/drivers/media/platform/atmel/Makefile
@@ -6,3 +6,4 @@ obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o
obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel-isc.o
obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel-xisc.o
+obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += microchip-csi2dc.o
diff --git a/drivers/media/platform/atmel/microchip-csi2dc.c b/drivers/media/platform/atmel/microchip-csi2dc.c
new file mode 100644
index 000000000000..6bc549c28e05
--- /dev/null
+++ b/drivers/media/platform/atmel/microchip-csi2dc.c
@@ -0,0 +1,806 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Microchip CSI2 Demux Controller (CSI2DC) driver
+ *
+ * Copyright (C) 2018 Microchip Technology, Inc.
+ *
+ * Author: Eugen Hristev <eugen.hristev@microchip.com>
+ *
+ */
+
+#include <linux/clk.h>
+#include <linux/mod_devicetable.h>
+#include <linux/module.h>
+#include <linux/of_graph.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/videodev2.h>
+
+#include <media/v4l2-fwnode.h>
+#include <media/v4l2-subdev.h>
+
+/* Global configuration register */
+#define CSI2DC_GCFG 0x0
+
+/* MIPI sensor pixel clock is free running */
+#define CSI2DC_GCFG_MIPIFRN BIT(0)
+/* GPIO parallel interface selection */
+#define CSI2DC_GCFG_GPIOSEL BIT(1)
+/* Output waveform inter-line minimum delay */
+#define CSI2DC_GCFG_HLC(v) ((v) << 4)
+#define CSI2DC_GCFG_HLC_MASK GENMASK(7, 4)
+/* SAMA7G5 requires a HLC delay of 15 */
+#define SAMA7G5_HLC (15)
+
+/* Global control register */
+#define CSI2DC_GCTLR 0x04
+#define CSI2DC_GCTLR_SWRST BIT(0)
+
+/* Global status register */
+#define CSI2DC_GS 0x08
+
+/* SSP interrupt status register */
+#define CSI2DC_SSPIS 0x28
+/* Pipe update register */
+#define CSI2DC_PU 0xc0
+/* Video pipe attributes update */
+#define CSI2DC_PU_VP BIT(0)
+
+/* Pipe update status register */
+#define CSI2DC_PUS 0xc4
+
+/* Video pipeline Interrupt Status Register */
+#define CSI2DC_VPISR 0xf4
+
+/* Video pipeline enable register */
+#define CSI2DC_VPE 0xf8
+#define CSI2DC_VPE_ENABLE BIT(0)
+
+/* Video pipeline configuration register */
+#define CSI2DC_VPCFG 0xfc
+/* Data type */
+#define CSI2DC_VPCFG_DT(v) ((v) << 0)
+#define CSI2DC_VPCFG_DT_MASK GENMASK(5, 0)
+/* Virtual channel identifier */
+#define CSI2DC_VPCFG_VC(v) ((v) << 6)
+#define CSI2DC_VPCFG_VC_MASK GENMASK(7, 6)
+/* Decompression enable */
+#define CSI2DC_VPCFG_DE BIT(8)
+/* Decoder mode */
+#define CSI2DC_VPCFG_DM(v) ((v) << 9)
+#define CSI2DC_VPCFG_DM_DECODER8TO12 0
+/* Decoder predictor 2 selection */
+#define CSI2DC_VPCFG_DP2 BIT(12)
+/* Recommended memory storage */
+#define CSI2DC_VPCFG_RMS BIT(13)
+/* Post adjustment */
+#define CSI2DC_VPCFG_PA BIT(14)
+
+/* Video pipeline column register */
+#define CSI2DC_VPCOL 0x100
+/* Column number */
+#define CSI2DC_VPCOL_COL(v) ((v) << 0)
+#define CSI2DC_VPCOL_COL_MASK GENMASK(15, 0)
+
+/* Video pipeline row register */
+#define CSI2DC_VPROW 0x104
+/* Row number */
+#define CSI2DC_VPROW_ROW(v) ((v) << 0)
+#define CSI2DC_VPROW_ROW_MASK GENMASK(15, 0)
+
+/* Version register */
+#define CSI2DC_VERSION 0x1fc
+
+/* register read/write helpers */
+#define csi2dc_readl(st, reg) readl_relaxed((st)->base + (reg))
+#define csi2dc_writel(st, reg, val) writel_relaxed((val), \
+ (st)->base + (reg))
+
+/* supported RAW data types */
+#define CSI2DC_DT_RAW6 0x28
+#define CSI2DC_DT_RAW7 0x29
+#define CSI2DC_DT_RAW8 0x2a
+#define CSI2DC_DT_RAW10 0x2b
+#define CSI2DC_DT_RAW12 0x2c
+#define CSI2DC_DT_RAW14 0x2d
+/* YUV data types */
+#define CSI2DC_DT_YUV422_8B 0x1e
+
+/*
+ * struct csi2dc_format - CSI2DC format type struct
+ * @mbus_code: Media bus code for the format
+ * @dt: Data type constant for this format
+ */
+struct csi2dc_format {
+ u32 mbus_code;
+ u32 dt;
+};
+
+static const struct csi2dc_format csi2dc_formats[] = {
+ {
+ .mbus_code = MEDIA_BUS_FMT_SRGGB8_1X8,
+ .dt = CSI2DC_DT_RAW8,
+ }, {
+ .mbus_code = MEDIA_BUS_FMT_SBGGR8_1X8,
+ .dt = CSI2DC_DT_RAW8,
+ }, {
+ .mbus_code = MEDIA_BUS_FMT_SGRBG8_1X8,
+ .dt = CSI2DC_DT_RAW8,
+ }, {
+ .mbus_code = MEDIA_BUS_FMT_SGBRG8_1X8,
+ .dt = CSI2DC_DT_RAW8,
+ }, {
+ .mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10,
+ .dt = CSI2DC_DT_RAW10,
+ }, {
+ .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10,
+ .dt = CSI2DC_DT_RAW10,
+ }, {
+ .mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10,
+ .dt = CSI2DC_DT_RAW10,
+ }, {
+ .mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10,
+ .dt = CSI2DC_DT_RAW10,
+ }, {
+ .mbus_code = MEDIA_BUS_FMT_YUYV8_2X8,
+ .dt = CSI2DC_DT_YUV422_8B,
+ },
+};
+
+enum mipi_csi_pads {
+ CSI2DC_PAD_SINK = 0,
+ CSI2DC_PAD_SOURCE = 1,
+ CSI2DC_PADS_NUM = 2,
+};
+
+/*
+ * struct csi2dc_device - CSI2DC device driver data/config struct
+ * @base: Register map base address
+ * @csi2dc_sd: v4l2 subdevice for the csi2dc device
+ * This is the subdevice that the csi2dc device itself
+ * registers in v4l2 subsystem
+ * @dev: struct device for this csi2dc device
+ * @pclk: Peripheral clock reference
+ * Input clock that clocks the hardware block internal
+ * logic
+ * @scck: Sensor Controller clock reference
+ * Input clock that is used to generate the pixel clock
+ * @format: Current saved format used in g/s fmt
+ * @cur_fmt: Current state format
+ * @try_fmt: Try format that is being tried
+ * @pads: Media entity pads for the csi2dc subdevice
+ * @clk_gated: Whether the clock is gated or free running
+ * @video_pipe: Whether video pipeline is configured
+ * @parallel_mode: The underlying subdevice is connected on a parallel bus
+ * @vc: Current set virtual channel
+ * @notifier: Async notifier that is used to bound the underlying
+ * subdevice to the csi2dc subdevice
+ * @input_sd: Reference to the underlying subdevice bound to the
+ * csi2dc subdevice
+ * @remote_pad: Pad number of the underlying subdevice that is linked
+ * to the csi2dc subdevice sink pad.
+ */
+struct csi2dc_device {
+ void __iomem *base;
+ struct v4l2_subdev csi2dc_sd;
+ struct device *dev;
+ struct clk *pclk;
+ struct clk *scck;
+
+ struct v4l2_mbus_framefmt format;
+
+ const struct csi2dc_format *cur_fmt;
+ const struct csi2dc_format *try_fmt;
+
+ struct media_pad pads[CSI2DC_PADS_NUM];
+
+ bool clk_gated;
+ bool video_pipe;
+ bool parallel_mode;
+ u32 vc;
+
+ struct v4l2_async_notifier notifier;
+
+ struct v4l2_subdev *input_sd;
+
+ u32 remote_pad;
+};
+
+static inline struct csi2dc_device *
+csi2dc_sd_to_csi2dc_device(struct v4l2_subdev *csi2dc_sd)
+{
+ return container_of(csi2dc_sd, struct csi2dc_device, csi2dc_sd);
+}
+
+static int csi2dc_enum_mbus_code(struct v4l2_subdev *csi2dc_sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ if (code->index >= ARRAY_SIZE(csi2dc_formats))
+ return -EINVAL;
+
+ code->code = csi2dc_formats[code->index].mbus_code;
+
+ return 0;
+}
+
+static int csi2dc_get_fmt(struct v4l2_subdev *csi2dc_sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_format *format)
+{
+ struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
+ struct v4l2_mbus_framefmt *v4l2_try_fmt;
+
+ if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
+ v4l2_try_fmt = v4l2_subdev_get_try_format(csi2dc_sd, sd_state,
+ format->pad);
+ format->format = *v4l2_try_fmt;
+
+ return 0;
+ }
+
+ format->format = csi2dc->format;
+
+ return 0;
+}
+
+static int csi2dc_set_fmt(struct v4l2_subdev *csi2dc_sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_format *req_fmt)
+{
+ struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
+ const struct csi2dc_format *fmt, *try_fmt = NULL;
+ struct v4l2_mbus_framefmt *v4l2_try_fmt;
+ unsigned int i;
+
+ /*
+ * Setting the source pad is disabled.
+ * The same format is being propagated from the sink to source.
+ */
+ if (req_fmt->pad == CSI2DC_PAD_SOURCE)
+ return -EINVAL;
+
+ for (i = 0; i < ARRAY_SIZE(csi2dc_formats); i++) {
+ fmt = &csi2dc_formats[i];
+ if (req_fmt->format.code == fmt->mbus_code)
+ try_fmt = fmt;
+ fmt++;
+ }
+
+ /* in case we could not find the desired format, default to something */
+ if (!try_fmt) {
+ try_fmt = &csi2dc_formats[0];
+
+ dev_dbg(csi2dc->dev,
+ "CSI2DC unsupported format 0x%x, defaulting to 0x%x\n",
+ req_fmt->format.code, csi2dc_formats[0].mbus_code);
+ }
+
+ req_fmt->format.code = try_fmt->mbus_code;
+ req_fmt->format.colorspace = V4L2_COLORSPACE_SRGB;
+ req_fmt->format.field = V4L2_FIELD_NONE;
+
+ if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
+ v4l2_try_fmt = v4l2_subdev_get_try_format(csi2dc_sd, sd_state,
+ req_fmt->pad);
+ *v4l2_try_fmt = req_fmt->format;
+ /* Trying on the sink pad makes the source pad change too */
+ v4l2_try_fmt = v4l2_subdev_get_try_format(csi2dc_sd,
+ sd_state,
+ CSI2DC_PAD_SOURCE);
+ *v4l2_try_fmt = req_fmt->format;
+
+ /* if we are just trying, we are done */
+ return 0;
+ }
+
+ /* save the format for later requests */
+ csi2dc->format = req_fmt->format;
+
+ /* update config */
+ csi2dc->cur_fmt = try_fmt;
+
+ dev_dbg(csi2dc->dev, "new format set: 0x%x @%dx%d\n",
+ csi2dc->format.code, csi2dc->format.width,
+ csi2dc->format.height);
+
+ return 0;
+}
+
+static int csi2dc_power(struct csi2dc_device *csi2dc, int on)
+{
+ int ret = 0;
+
+ if (on) {
+ ret = clk_prepare_enable(csi2dc->pclk);
+ if (ret) {
+ dev_err(csi2dc->dev, "failed to enable pclk:%d\n", ret);
+ return ret;
+ }
+
+ ret = clk_prepare_enable(csi2dc->scck);
+ if (ret) {
+ dev_err(csi2dc->dev, "failed to enable scck:%d\n", ret);
+ clk_disable_unprepare(csi2dc->pclk);
+ return ret;
+ }
+
+ /* if powering up, deassert reset line */
+ csi2dc_writel(csi2dc, CSI2DC_GCTLR, CSI2DC_GCTLR_SWRST);
+ } else {
+ /* if powering down, assert reset line */
+ csi2dc_writel(csi2dc, CSI2DC_GCTLR, 0);
+
+ clk_disable_unprepare(csi2dc->scck);
+ clk_disable_unprepare(csi2dc->pclk);
+ }
+
+ return ret;
+}
+
+static int csi2dc_get_mbus_config(struct csi2dc_device *csi2dc)
+{
+ struct v4l2_mbus_config mbus_config = { 0 };
+ int ret;
+
+ ret = v4l2_subdev_call(csi2dc->input_sd, pad, get_mbus_config,
+ csi2dc->remote_pad, &mbus_config);
+ if (ret == -ENOIOCTLCMD) {
+ dev_dbg(csi2dc->dev,
+ "no remote mbus configuration available\n");
+ goto csi2dc_get_mbus_config_defaults;
+ }
+
+ if (ret) {
+ dev_err(csi2dc->dev,
+ "failed to get remote mbus configuration\n");
+ goto csi2dc_get_mbus_config_defaults;
+ }
+
+ if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_0)
+ csi2dc->vc = 0;
+ else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_1)
+ csi2dc->vc = 1;
+ else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_2)
+ csi2dc->vc = 2;
+ else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_3)
+ csi2dc->vc = 3;
+
+ dev_dbg(csi2dc->dev, "subdev sending on channel %d\n", csi2dc->vc);
+
+ csi2dc->clk_gated = mbus_config.flags &
+ V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK;
+
+ dev_dbg(csi2dc->dev, "mbus_config: %s clock\n",
+ csi2dc->clk_gated ? "gated" : "free running");
+
+ return 0;
+
+csi2dc_get_mbus_config_defaults:
+ csi2dc->vc = 0; /* Virtual ID 0 by default */
+
+ return 0;
+}
+
+static void csi2dc_vp_update(struct csi2dc_device *csi2dc)
+{
+ u32 vp, gcfg;
+
+ if (!csi2dc->video_pipe) {
+ dev_err(csi2dc->dev, "video pipeline unavailable\n");
+ return;
+ }
+
+ if (csi2dc->parallel_mode) {
+ /* In parallel mode, GPIO parallel interface must be selected */
+ gcfg = csi2dc_readl(csi2dc, CSI2DC_GCFG);
+ gcfg |= CSI2DC_GCFG_GPIOSEL;
+ csi2dc_writel(csi2dc, CSI2DC_GCFG, gcfg);
+ return;
+ }
+
+ /* serial video pipeline */
+
+ csi2dc_writel(csi2dc, CSI2DC_GCFG,
+ (SAMA7G5_HLC & CSI2DC_GCFG_HLC_MASK) |
+ (csi2dc->clk_gated ? 0 : CSI2DC_GCFG_MIPIFRN));
+
+ vp = CSI2DC_VPCFG_DT(csi2dc->cur_fmt->dt) & CSI2DC_VPCFG_DT_MASK;
+ vp |= CSI2DC_VPCFG_VC(csi2dc->vc) & CSI2DC_VPCFG_VC_MASK;
+ vp &= ~CSI2DC_VPCFG_DE;
+ vp |= CSI2DC_VPCFG_DM(CSI2DC_VPCFG_DM_DECODER8TO12);
+ vp &= ~CSI2DC_VPCFG_DP2;
+ vp &= ~CSI2DC_VPCFG_RMS;
+ vp |= CSI2DC_VPCFG_PA;
+
+ csi2dc_writel(csi2dc, CSI2DC_VPCFG, vp);
+ csi2dc_writel(csi2dc, CSI2DC_VPE, CSI2DC_VPE_ENABLE);
+ csi2dc_writel(csi2dc, CSI2DC_PU, CSI2DC_PU_VP);
+}
+
+static int csi2dc_s_stream(struct v4l2_subdev *csi2dc_sd, int enable)
+{
+ struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
+ int ret;
+
+ if (enable) {
+ ret = pm_runtime_resume_and_get(csi2dc->dev);
+ if (ret < 0)
+ return ret;
+
+ csi2dc_get_mbus_config(csi2dc);
+
+ csi2dc_vp_update(csi2dc);
+
+ return v4l2_subdev_call(csi2dc->input_sd, video, s_stream,
+ true);
+ }
+
+ dev_dbg(csi2dc->dev,
+ "Last frame received: VPCOLR = %u, VPROWR= %u, VPISR = %x\n",
+ csi2dc_readl(csi2dc, CSI2DC_VPCOL),
+ csi2dc_readl(csi2dc, CSI2DC_VPROW),
+ csi2dc_readl(csi2dc, CSI2DC_VPISR));
+
+ /* stop streaming scenario */
+ ret = v4l2_subdev_call(csi2dc->input_sd, video, s_stream, false);
+
+ pm_runtime_put_sync(csi2dc->dev);
+
+ return ret;
+}
+
+static int csi2dc_init_cfg(struct v4l2_subdev *csi2dc_sd,
+ struct v4l2_subdev_state *sd_state)
+{
+ struct v4l2_mbus_framefmt *v4l2_try_fmt =
+ v4l2_subdev_get_try_format(csi2dc_sd, sd_state, 0);
+
+ v4l2_try_fmt->height = 480;
+ v4l2_try_fmt->width = 640;
+ v4l2_try_fmt->code = csi2dc_formats[0].mbus_code;
+ v4l2_try_fmt->colorspace = V4L2_COLORSPACE_SRGB;
+ v4l2_try_fmt->field = V4L2_FIELD_NONE;
+ v4l2_try_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ v4l2_try_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
+ v4l2_try_fmt->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+
+ return 0;
+}
+
+static const struct v4l2_subdev_pad_ops csi2dc_pad_ops = {
+ .enum_mbus_code = csi2dc_enum_mbus_code,
+ .set_fmt = csi2dc_set_fmt,
+ .get_fmt = csi2dc_get_fmt,
+ .init_cfg = csi2dc_init_cfg,
+};
+
+static const struct v4l2_subdev_video_ops csi2dc_video_ops = {
+ .s_stream = csi2dc_s_stream,
+};
+
+static const struct v4l2_subdev_ops csi2dc_subdev_ops = {
+ .pad = &csi2dc_pad_ops,
+ .video = &csi2dc_video_ops,
+};
+
+static int csi2dc_async_bound(struct v4l2_async_notifier *notifier,
+ struct v4l2_subdev *subdev,
+ struct v4l2_async_subdev *asd)
+{
+ struct csi2dc_device *csi2dc = container_of(notifier,
+ struct csi2dc_device, notifier);
+ int pad;
+ int ret;
+
+ csi2dc->input_sd = subdev;
+
+ pad = media_entity_get_fwnode_pad(&subdev->entity, asd->match.fwnode,
+ MEDIA_PAD_FL_SOURCE);
+ if (pad < 0) {
+ dev_err(csi2dc->dev, "Failed to find pad for %s\n",
+ subdev->name);
+ return pad;
+ }
+
+ csi2dc->remote_pad = pad;
+
+ ret = media_create_pad_link(&csi2dc->input_sd->entity,
+ csi2dc->remote_pad,
+ &csi2dc->csi2dc_sd.entity, 0,
+ MEDIA_LNK_FL_ENABLED);
+ if (ret) {
+ dev_err(csi2dc->dev,
+ "Failed to create pad link: %s to %s\n",
+ csi2dc->input_sd->entity.name,
+ csi2dc->csi2dc_sd.entity.name);
+ return ret;
+ }
+
+ dev_dbg(csi2dc->dev, "link with %s pad: %d\n",
+ csi2dc->input_sd->name, csi2dc->remote_pad);
+
+ return ret;
+}
+
+static const struct v4l2_async_notifier_operations csi2dc_async_ops = {
+ .bound = csi2dc_async_bound,
+};
+
+static int csi2dc_prepare_notifier(struct csi2dc_device *csi2dc,
+ struct fwnode_handle *input_fwnode)
+{
+ struct v4l2_async_subdev *asd;
+ int ret = 0;
+
+ v4l2_async_nf_init(&csi2dc->notifier);
+
+ asd = v4l2_async_nf_add_fwnode_remote(&csi2dc->notifier,
+ input_fwnode,
+ struct v4l2_async_subdev);
+
+ fwnode_handle_put(input_fwnode);
+
+ if (IS_ERR(asd)) {
+ ret = PTR_ERR(asd);
+ dev_err(csi2dc->dev,
+ "failed to add async notifier for node %pOF: %d\n",
+ to_of_node(input_fwnode), ret);
+ v4l2_async_nf_cleanup(&csi2dc->notifier);
+ return ret;
+ }
+
+ csi2dc->notifier.ops = &csi2dc_async_ops;
+
+ ret = v4l2_async_subdev_nf_register(&csi2dc->csi2dc_sd,
+ &csi2dc->notifier);
+ if (ret) {
+ dev_err(csi2dc->dev, "fail to register async notifier: %d\n",
+ ret);
+ v4l2_async_nf_cleanup(&csi2dc->notifier);
+ }
+
+ return ret;
+}
+
+static int csi2dc_of_parse(struct csi2dc_device *csi2dc,
+ struct device_node *of_node)
+{
+ struct fwnode_handle *input_fwnode, *output_fwnode;
+ struct v4l2_fwnode_endpoint input_endpoint = { 0 },
+ output_endpoint = { 0 };
+ int ret;
+
+ input_fwnode = fwnode_graph_get_next_endpoint(of_fwnode_handle(of_node),
+ NULL);
+ if (!input_fwnode) {
+ dev_err(csi2dc->dev,
+ "missing port node at %pOF, input node is mandatory.\n",
+ of_node);
+ return -EINVAL;
+ }
+
+ ret = v4l2_fwnode_endpoint_parse(input_fwnode, &input_endpoint);
+ if (ret) {
+ dev_err(csi2dc->dev, "endpoint not defined at %pOF\n", of_node);
+ goto csi2dc_of_parse_err;
+ }
+
+ if (input_endpoint.bus_type == V4L2_MBUS_PARALLEL ||
+ input_endpoint.bus_type == V4L2_MBUS_BT656) {
+ csi2dc->parallel_mode = true;
+ dev_dbg(csi2dc->dev,
+ "subdevice connected on parallel interface\n");
+ }
+
+ if (input_endpoint.bus_type == V4L2_MBUS_CSI2_DPHY) {
+ csi2dc->clk_gated = input_endpoint.bus.mipi_csi2.flags &
+ V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK;
+ dev_dbg(csi2dc->dev,
+ "subdevice connected on serial interface\n");
+ dev_dbg(csi2dc->dev, "DT: %s clock\n",
+ csi2dc->clk_gated ? "gated" : "free running");
+ }
+
+ output_fwnode = fwnode_graph_get_next_endpoint
+ (of_fwnode_handle(of_node), input_fwnode);
+
+ if (output_fwnode)
+ ret = v4l2_fwnode_endpoint_parse(output_fwnode,
+ &output_endpoint);
+
+ fwnode_handle_put(output_fwnode);
+
+ if (!output_fwnode || ret) {
+ dev_info(csi2dc->dev,
+ "missing output node at %pOF, data pipe available only.\n",
+ of_node);
+ } else {
+ if (output_endpoint.bus_type != V4L2_MBUS_PARALLEL &&
+ output_endpoint.bus_type != V4L2_MBUS_BT656) {
+ dev_err(csi2dc->dev,
+ "output port must be parallel/bt656.\n");
+ ret = -EINVAL;
+ goto csi2dc_of_parse_err;
+ }
+
+ csi2dc->video_pipe = true;
+
+ dev_dbg(csi2dc->dev,
+ "block %pOF [%d.%d]->[%d.%d] video pipeline\n",
+ of_node, input_endpoint.base.port,
+ input_endpoint.base.id, output_endpoint.base.port,
+ output_endpoint.base.id);
+ }
+
+ /* prepare async notifier for subdevice completion */
+ return csi2dc_prepare_notifier(csi2dc, input_fwnode);
+
+csi2dc_of_parse_err:
+ fwnode_handle_put(input_fwnode);
+ return ret;
+}
+
+static void csi2dc_default_format(struct csi2dc_device *csi2dc)
+{
+ csi2dc->cur_fmt = &csi2dc_formats[0];
+
+ csi2dc->format.height = 480;
+ csi2dc->format.width = 640;
+ csi2dc->format.code = csi2dc_formats[0].mbus_code;
+ csi2dc->format.colorspace = V4L2_COLORSPACE_SRGB;
+ csi2dc->format.field = V4L2_FIELD_NONE;
+ csi2dc->format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ csi2dc->format.quantization = V4L2_QUANTIZATION_DEFAULT;
+ csi2dc->format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
+}
+
+static int csi2dc_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct csi2dc_device *csi2dc;
+ int ret = 0;
+ u32 ver;
+
+ csi2dc = devm_kzalloc(dev, sizeof(*csi2dc), GFP_KERNEL);
+ if (!csi2dc)
+ return -ENOMEM;
+
+ csi2dc->dev = dev;
+
+ csi2dc->base = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(csi2dc->base)) {
+ dev_err(dev, "base address not set\n");
+ return PTR_ERR(csi2dc->base);
+ }
+
+ csi2dc->pclk = devm_clk_get(dev, "pclk");
+ if (IS_ERR(csi2dc->pclk)) {
+ ret = PTR_ERR(csi2dc->pclk);
+ dev_err(dev, "failed to get pclk: %d\n", ret);
+ return ret;
+ }
+
+ csi2dc->scck = devm_clk_get(dev, "scck");
+ if (IS_ERR(csi2dc->scck)) {
+ ret = PTR_ERR(csi2dc->scck);
+ dev_err(dev, "failed to get scck: %d\n", ret);
+ return ret;
+ }
+
+ v4l2_subdev_init(&csi2dc->csi2dc_sd, &csi2dc_subdev_ops);
+
+ csi2dc->csi2dc_sd.owner = THIS_MODULE;
+ csi2dc->csi2dc_sd.dev = dev;
+ snprintf(csi2dc->csi2dc_sd.name, sizeof(csi2dc->csi2dc_sd.name),
+ "csi2dc");
+
+ csi2dc->csi2dc_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+ csi2dc->csi2dc_sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
+
+ platform_set_drvdata(pdev, csi2dc);
+
+ ret = csi2dc_of_parse(csi2dc, dev->of_node);
+ if (ret)
+ goto csi2dc_probe_cleanup_entity;
+
+ csi2dc->pads[CSI2DC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+ if (csi2dc->video_pipe)
+ csi2dc->pads[CSI2DC_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
+
+ ret = media_entity_pads_init(&csi2dc->csi2dc_sd.entity,
+ csi2dc->video_pipe ? CSI2DC_PADS_NUM : 1,
+ csi2dc->pads);
+ if (ret < 0) {
+ dev_err(dev, "media entity init failed\n");
+ goto csi2dc_probe_cleanup_notifier;
+ }
+
+ csi2dc_default_format(csi2dc);
+
+ /* turn power on to validate capabilities */
+ ret = csi2dc_power(csi2dc, true);
+ if (ret < 0)
+ goto csi2dc_probe_cleanup_notifier;
+
+ pm_runtime_set_active(dev);
+ pm_runtime_enable(dev);
+ ver = csi2dc_readl(csi2dc, CSI2DC_VERSION);
+
+ /*
+ * we must register the subdev after PM runtime has been requested,
+ * otherwise we might bound immediately and request pm_runtime_resume
+ * before runtime_enable.
+ */
+ ret = v4l2_async_register_subdev(&csi2dc->csi2dc_sd);
+ if (ret) {
+ dev_err(csi2dc->dev, "failed to register the subdevice\n");
+ goto csi2dc_probe_cleanup_notifier;
+ }
+
+ dev_info(dev, "Microchip CSI2DC version %x\n", ver);
+
+ return 0;
+
+csi2dc_probe_cleanup_notifier:
+ v4l2_async_nf_cleanup(&csi2dc->notifier);
+csi2dc_probe_cleanup_entity:
+ media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
+
+ return ret;
+}
+
+static int csi2dc_remove(struct platform_device *pdev)
+{
+ struct csi2dc_device *csi2dc = platform_get_drvdata(pdev);
+
+ pm_runtime_disable(&pdev->dev);
+
+ v4l2_async_unregister_subdev(&csi2dc->csi2dc_sd);
+ v4l2_async_nf_unregister(&csi2dc->notifier);
+ v4l2_async_nf_cleanup(&csi2dc->notifier);
+ media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
+
+ return 0;
+}
+
+static int __maybe_unused csi2dc_runtime_suspend(struct device *dev)
+{
+ struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
+
+ return csi2dc_power(csi2dc, false);
+}
+
+static int __maybe_unused csi2dc_runtime_resume(struct device *dev)
+{
+ struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
+
+ return csi2dc_power(csi2dc, true);
+}
+
+static const struct dev_pm_ops csi2dc_dev_pm_ops = {
+ SET_RUNTIME_PM_OPS(csi2dc_runtime_suspend, csi2dc_runtime_resume, NULL)
+};
+
+static const struct of_device_id csi2dc_of_match[] = {
+ { .compatible = "microchip,sama7g5-csi2dc" },
+ { }
+};
+
+MODULE_DEVICE_TABLE(of, csi2dc_of_match);
+
+static struct platform_driver csi2dc_driver = {
+ .probe = csi2dc_probe,
+ .remove = csi2dc_remove,
+ .driver = {
+ .name = "microchip-csi2dc",
+ .pm = &csi2dc_dev_pm_ops,
+ .of_match_table = of_match_ptr(csi2dc_of_match),
+ },
+};
+
+module_platform_driver(csi2dc_driver);
+
+MODULE_AUTHOR("Eugen Hristev <eugen.hristev@microchip.com>");
+MODULE_DESCRIPTION("Microchip CSI2 Demux Controller driver");
+MODULE_LICENSE("GPL v2");
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 04/23] media: atmel: atmel-isc: split the clock code into separate source file
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
The atmel-isc-base is getting crowded. Split the clock functions into
atmel-isc-clk.c.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
Changes in v3:
- squash with maintainers, add patterns in maintainers
MAINTAINERS | 7 +-
drivers/media/platform/atmel/Makefile | 3 +-
drivers/media/platform/atmel/atmel-isc-base.c | 295 -----------------
drivers/media/platform/atmel/atmel-isc-clk.c | 311 ++++++++++++++++++
drivers/media/platform/atmel/atmel-isc.h | 7 +
5 files changed, 322 insertions(+), 301 deletions(-)
create mode 100644 drivers/media/platform/atmel/atmel-isc-clk.c
diff --git a/MAINTAINERS b/MAINTAINERS
index afb74460e5c1..6aa274129751 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -12500,11 +12500,8 @@ L: linux-media@vger.kernel.org
S: Supported
F: Documentation/devicetree/bindings/media/atmel,isc.yaml
F: Documentation/devicetree/bindings/media/microchip,xisc.yaml
-F: drivers/media/platform/atmel/atmel-isc-base.c
-F: drivers/media/platform/atmel/atmel-isc-regs.h
-F: drivers/media/platform/atmel/atmel-isc.h
-F: drivers/media/platform/atmel/atmel-sama5d2-isc.c
-F: drivers/media/platform/atmel/atmel-sama7g5-isc.c
+F: drivers/media/platform/atmel/atmel-isc*
+F: drivers/media/platform/atmel/atmel-sama*-isc*
F: include/linux/atmel-isc-media.h
MICROCHIP ISI DRIVER
diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
index 39f0a7eba702..794e8f739287 100644
--- a/drivers/media/platform/atmel/Makefile
+++ b/drivers/media/platform/atmel/Makefile
@@ -1,9 +1,10 @@
# SPDX-License-Identifier: GPL-2.0-only
atmel-isc-objs = atmel-sama5d2-isc.o
atmel-xisc-objs = atmel-sama7g5-isc.o
+atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o
obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
-obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o
+obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-common.o
obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel-isc.o
obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel-xisc.o
obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += microchip-csi2dc.o
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index 660cd0ab6749..58f468d32613 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -8,10 +8,6 @@
* Author: Eugen Hristev <eugen.hristev@microchip.com>
*
*/
-
-#include <linux/clk.h>
-#include <linux/clkdev.h>
-#include <linux/clk-provider.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/math64.h>
@@ -100,297 +96,6 @@ static inline void isc_reset_awb_ctrls(struct isc_device *isc)
}
}
-static int isc_wait_clk_stable(struct clk_hw *hw)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
- struct regmap *regmap = isc_clk->regmap;
- unsigned long timeout = jiffies + usecs_to_jiffies(1000);
- unsigned int status;
-
- while (time_before(jiffies, timeout)) {
- regmap_read(regmap, ISC_CLKSR, &status);
- if (!(status & ISC_CLKSR_SIP))
- return 0;
-
- usleep_range(10, 250);
- }
-
- return -ETIMEDOUT;
-}
-
-static int isc_clk_prepare(struct clk_hw *hw)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
- int ret;
-
- ret = pm_runtime_resume_and_get(isc_clk->dev);
- if (ret < 0)
- return ret;
-
- return isc_wait_clk_stable(hw);
-}
-
-static void isc_clk_unprepare(struct clk_hw *hw)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
-
- isc_wait_clk_stable(hw);
-
- pm_runtime_put_sync(isc_clk->dev);
-}
-
-static int isc_clk_enable(struct clk_hw *hw)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
- u32 id = isc_clk->id;
- struct regmap *regmap = isc_clk->regmap;
- unsigned long flags;
- unsigned int status;
-
- dev_dbg(isc_clk->dev, "ISC CLK: %s, id = %d, div = %d, parent id = %d\n",
- __func__, id, isc_clk->div, isc_clk->parent_id);
-
- spin_lock_irqsave(&isc_clk->lock, flags);
- regmap_update_bits(regmap, ISC_CLKCFG,
- ISC_CLKCFG_DIV_MASK(id) | ISC_CLKCFG_SEL_MASK(id),
- (isc_clk->div << ISC_CLKCFG_DIV_SHIFT(id)) |
- (isc_clk->parent_id << ISC_CLKCFG_SEL_SHIFT(id)));
-
- regmap_write(regmap, ISC_CLKEN, ISC_CLK(id));
- spin_unlock_irqrestore(&isc_clk->lock, flags);
-
- regmap_read(regmap, ISC_CLKSR, &status);
- if (status & ISC_CLK(id))
- return 0;
- else
- return -EINVAL;
-}
-
-static void isc_clk_disable(struct clk_hw *hw)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
- u32 id = isc_clk->id;
- unsigned long flags;
-
- spin_lock_irqsave(&isc_clk->lock, flags);
- regmap_write(isc_clk->regmap, ISC_CLKDIS, ISC_CLK(id));
- spin_unlock_irqrestore(&isc_clk->lock, flags);
-}
-
-static int isc_clk_is_enabled(struct clk_hw *hw)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
- u32 status;
- int ret;
-
- ret = pm_runtime_resume_and_get(isc_clk->dev);
- if (ret < 0)
- return 0;
-
- regmap_read(isc_clk->regmap, ISC_CLKSR, &status);
-
- pm_runtime_put_sync(isc_clk->dev);
-
- return status & ISC_CLK(isc_clk->id) ? 1 : 0;
-}
-
-static unsigned long
-isc_clk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
-
- return DIV_ROUND_CLOSEST(parent_rate, isc_clk->div + 1);
-}
-
-static int isc_clk_determine_rate(struct clk_hw *hw,
- struct clk_rate_request *req)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
- long best_rate = -EINVAL;
- int best_diff = -1;
- unsigned int i, div;
-
- for (i = 0; i < clk_hw_get_num_parents(hw); i++) {
- struct clk_hw *parent;
- unsigned long parent_rate;
-
- parent = clk_hw_get_parent_by_index(hw, i);
- if (!parent)
- continue;
-
- parent_rate = clk_hw_get_rate(parent);
- if (!parent_rate)
- continue;
-
- for (div = 1; div < ISC_CLK_MAX_DIV + 2; div++) {
- unsigned long rate;
- int diff;
-
- rate = DIV_ROUND_CLOSEST(parent_rate, div);
- diff = abs(req->rate - rate);
-
- if (best_diff < 0 || best_diff > diff) {
- best_rate = rate;
- best_diff = diff;
- req->best_parent_rate = parent_rate;
- req->best_parent_hw = parent;
- }
-
- if (!best_diff || rate < req->rate)
- break;
- }
-
- if (!best_diff)
- break;
- }
-
- dev_dbg(isc_clk->dev,
- "ISC CLK: %s, best_rate = %ld, parent clk: %s @ %ld\n",
- __func__, best_rate,
- __clk_get_name((req->best_parent_hw)->clk),
- req->best_parent_rate);
-
- if (best_rate < 0)
- return best_rate;
-
- req->rate = best_rate;
-
- return 0;
-}
-
-static int isc_clk_set_parent(struct clk_hw *hw, u8 index)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
-
- if (index >= clk_hw_get_num_parents(hw))
- return -EINVAL;
-
- isc_clk->parent_id = index;
-
- return 0;
-}
-
-static u8 isc_clk_get_parent(struct clk_hw *hw)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
-
- return isc_clk->parent_id;
-}
-
-static int isc_clk_set_rate(struct clk_hw *hw,
- unsigned long rate,
- unsigned long parent_rate)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
- u32 div;
-
- if (!rate)
- return -EINVAL;
-
- div = DIV_ROUND_CLOSEST(parent_rate, rate);
- if (div > (ISC_CLK_MAX_DIV + 1) || !div)
- return -EINVAL;
-
- isc_clk->div = div - 1;
-
- return 0;
-}
-
-static const struct clk_ops isc_clk_ops = {
- .prepare = isc_clk_prepare,
- .unprepare = isc_clk_unprepare,
- .enable = isc_clk_enable,
- .disable = isc_clk_disable,
- .is_enabled = isc_clk_is_enabled,
- .recalc_rate = isc_clk_recalc_rate,
- .determine_rate = isc_clk_determine_rate,
- .set_parent = isc_clk_set_parent,
- .get_parent = isc_clk_get_parent,
- .set_rate = isc_clk_set_rate,
-};
-
-static int isc_clk_register(struct isc_device *isc, unsigned int id)
-{
- struct regmap *regmap = isc->regmap;
- struct device_node *np = isc->dev->of_node;
- struct isc_clk *isc_clk;
- struct clk_init_data init;
- const char *clk_name = np->name;
- const char *parent_names[3];
- int num_parents;
-
- if (id == ISC_ISPCK && !isc->ispck_required)
- return 0;
-
- num_parents = of_clk_get_parent_count(np);
- if (num_parents < 1 || num_parents > 3)
- return -EINVAL;
-
- if (num_parents > 2 && id == ISC_ISPCK)
- num_parents = 2;
-
- of_clk_parent_fill(np, parent_names, num_parents);
-
- if (id == ISC_MCK)
- of_property_read_string(np, "clock-output-names", &clk_name);
- else
- clk_name = "isc-ispck";
-
- init.parent_names = parent_names;
- init.num_parents = num_parents;
- init.name = clk_name;
- init.ops = &isc_clk_ops;
- init.flags = CLK_SET_RATE_GATE | CLK_SET_PARENT_GATE;
-
- isc_clk = &isc->isc_clks[id];
- isc_clk->hw.init = &init;
- isc_clk->regmap = regmap;
- isc_clk->id = id;
- isc_clk->dev = isc->dev;
- spin_lock_init(&isc_clk->lock);
-
- isc_clk->clk = clk_register(isc->dev, &isc_clk->hw);
- if (IS_ERR(isc_clk->clk)) {
- dev_err(isc->dev, "%s: clock register fail\n", clk_name);
- return PTR_ERR(isc_clk->clk);
- } else if (id == ISC_MCK)
- of_clk_add_provider(np, of_clk_src_simple_get, isc_clk->clk);
-
- return 0;
-}
-
-int isc_clk_init(struct isc_device *isc)
-{
- unsigned int i;
- int ret;
-
- for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++)
- isc->isc_clks[i].clk = ERR_PTR(-EINVAL);
-
- for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
- ret = isc_clk_register(isc, i);
- if (ret)
- return ret;
- }
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(isc_clk_init);
-
-void isc_clk_cleanup(struct isc_device *isc)
-{
- unsigned int i;
-
- of_clk_del_provider(isc->dev->of_node);
-
- for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
- struct isc_clk *isc_clk = &isc->isc_clks[i];
-
- if (!IS_ERR(isc_clk->clk))
- clk_unregister(isc_clk->clk);
- }
-}
-EXPORT_SYMBOL_GPL(isc_clk_cleanup);
static int isc_queue_setup(struct vb2_queue *vq,
unsigned int *nbuffers, unsigned int *nplanes,
diff --git a/drivers/media/platform/atmel/atmel-isc-clk.c b/drivers/media/platform/atmel/atmel-isc-clk.c
new file mode 100644
index 000000000000..2059fe376b00
--- /dev/null
+++ b/drivers/media/platform/atmel/atmel-isc-clk.c
@@ -0,0 +1,311 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Microchip Image Sensor Controller (ISC) common clock driver setup
+ *
+ * Copyright (C) 2016 Microchip Technology, Inc.
+ *
+ * Author: Songjun Wu
+ * Author: Eugen Hristev <eugen.hristev@microchip.com>
+ *
+ */
+#include <linux/clk.h>
+#include <linux/clkdev.h>
+#include <linux/clk-provider.h>
+#include <linux/pm_runtime.h>
+#include <linux/regmap.h>
+
+#include "atmel-isc-regs.h"
+#include "atmel-isc.h"
+
+static int isc_wait_clk_stable(struct clk_hw *hw)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+ struct regmap *regmap = isc_clk->regmap;
+ unsigned long timeout = jiffies + usecs_to_jiffies(1000);
+ unsigned int status;
+
+ while (time_before(jiffies, timeout)) {
+ regmap_read(regmap, ISC_CLKSR, &status);
+ if (!(status & ISC_CLKSR_SIP))
+ return 0;
+
+ usleep_range(10, 250);
+ }
+
+ return -ETIMEDOUT;
+}
+
+static int isc_clk_prepare(struct clk_hw *hw)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+ int ret;
+
+ ret = pm_runtime_resume_and_get(isc_clk->dev);
+ if (ret < 0)
+ return ret;
+
+ return isc_wait_clk_stable(hw);
+}
+
+static void isc_clk_unprepare(struct clk_hw *hw)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+
+ isc_wait_clk_stable(hw);
+
+ pm_runtime_put_sync(isc_clk->dev);
+}
+
+static int isc_clk_enable(struct clk_hw *hw)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+ u32 id = isc_clk->id;
+ struct regmap *regmap = isc_clk->regmap;
+ unsigned long flags;
+ unsigned int status;
+
+ dev_dbg(isc_clk->dev, "ISC CLK: %s, id = %d, div = %d, parent id = %d\n",
+ __func__, id, isc_clk->div, isc_clk->parent_id);
+
+ spin_lock_irqsave(&isc_clk->lock, flags);
+ regmap_update_bits(regmap, ISC_CLKCFG,
+ ISC_CLKCFG_DIV_MASK(id) | ISC_CLKCFG_SEL_MASK(id),
+ (isc_clk->div << ISC_CLKCFG_DIV_SHIFT(id)) |
+ (isc_clk->parent_id << ISC_CLKCFG_SEL_SHIFT(id)));
+
+ regmap_write(regmap, ISC_CLKEN, ISC_CLK(id));
+ spin_unlock_irqrestore(&isc_clk->lock, flags);
+
+ regmap_read(regmap, ISC_CLKSR, &status);
+ if (status & ISC_CLK(id))
+ return 0;
+ else
+ return -EINVAL;
+}
+
+static void isc_clk_disable(struct clk_hw *hw)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+ u32 id = isc_clk->id;
+ unsigned long flags;
+
+ spin_lock_irqsave(&isc_clk->lock, flags);
+ regmap_write(isc_clk->regmap, ISC_CLKDIS, ISC_CLK(id));
+ spin_unlock_irqrestore(&isc_clk->lock, flags);
+}
+
+static int isc_clk_is_enabled(struct clk_hw *hw)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+ u32 status;
+ int ret;
+
+ ret = pm_runtime_resume_and_get(isc_clk->dev);
+ if (ret < 0)
+ return 0;
+
+ regmap_read(isc_clk->regmap, ISC_CLKSR, &status);
+
+ pm_runtime_put_sync(isc_clk->dev);
+
+ return status & ISC_CLK(isc_clk->id) ? 1 : 0;
+}
+
+static unsigned long
+isc_clk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+
+ return DIV_ROUND_CLOSEST(parent_rate, isc_clk->div + 1);
+}
+
+static int isc_clk_determine_rate(struct clk_hw *hw,
+ struct clk_rate_request *req)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+ long best_rate = -EINVAL;
+ int best_diff = -1;
+ unsigned int i, div;
+
+ for (i = 0; i < clk_hw_get_num_parents(hw); i++) {
+ struct clk_hw *parent;
+ unsigned long parent_rate;
+
+ parent = clk_hw_get_parent_by_index(hw, i);
+ if (!parent)
+ continue;
+
+ parent_rate = clk_hw_get_rate(parent);
+ if (!parent_rate)
+ continue;
+
+ for (div = 1; div < ISC_CLK_MAX_DIV + 2; div++) {
+ unsigned long rate;
+ int diff;
+
+ rate = DIV_ROUND_CLOSEST(parent_rate, div);
+ diff = abs(req->rate - rate);
+
+ if (best_diff < 0 || best_diff > diff) {
+ best_rate = rate;
+ best_diff = diff;
+ req->best_parent_rate = parent_rate;
+ req->best_parent_hw = parent;
+ }
+
+ if (!best_diff || rate < req->rate)
+ break;
+ }
+
+ if (!best_diff)
+ break;
+ }
+
+ dev_dbg(isc_clk->dev,
+ "ISC CLK: %s, best_rate = %ld, parent clk: %s @ %ld\n",
+ __func__, best_rate,
+ __clk_get_name((req->best_parent_hw)->clk),
+ req->best_parent_rate);
+
+ if (best_rate < 0)
+ return best_rate;
+
+ req->rate = best_rate;
+
+ return 0;
+}
+
+static int isc_clk_set_parent(struct clk_hw *hw, u8 index)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+
+ if (index >= clk_hw_get_num_parents(hw))
+ return -EINVAL;
+
+ isc_clk->parent_id = index;
+
+ return 0;
+}
+
+static u8 isc_clk_get_parent(struct clk_hw *hw)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+
+ return isc_clk->parent_id;
+}
+
+static int isc_clk_set_rate(struct clk_hw *hw,
+ unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+ u32 div;
+
+ if (!rate)
+ return -EINVAL;
+
+ div = DIV_ROUND_CLOSEST(parent_rate, rate);
+ if (div > (ISC_CLK_MAX_DIV + 1) || !div)
+ return -EINVAL;
+
+ isc_clk->div = div - 1;
+
+ return 0;
+}
+
+static const struct clk_ops isc_clk_ops = {
+ .prepare = isc_clk_prepare,
+ .unprepare = isc_clk_unprepare,
+ .enable = isc_clk_enable,
+ .disable = isc_clk_disable,
+ .is_enabled = isc_clk_is_enabled,
+ .recalc_rate = isc_clk_recalc_rate,
+ .determine_rate = isc_clk_determine_rate,
+ .set_parent = isc_clk_set_parent,
+ .get_parent = isc_clk_get_parent,
+ .set_rate = isc_clk_set_rate,
+};
+
+static int isc_clk_register(struct isc_device *isc, unsigned int id)
+{
+ struct regmap *regmap = isc->regmap;
+ struct device_node *np = isc->dev->of_node;
+ struct isc_clk *isc_clk;
+ struct clk_init_data init;
+ const char *clk_name = np->name;
+ const char *parent_names[3];
+ int num_parents;
+
+ if (id == ISC_ISPCK && !isc->ispck_required)
+ return 0;
+
+ num_parents = of_clk_get_parent_count(np);
+ if (num_parents < 1 || num_parents > 3)
+ return -EINVAL;
+
+ if (num_parents > 2 && id == ISC_ISPCK)
+ num_parents = 2;
+
+ of_clk_parent_fill(np, parent_names, num_parents);
+
+ if (id == ISC_MCK)
+ of_property_read_string(np, "clock-output-names", &clk_name);
+ else
+ clk_name = "isc-ispck";
+
+ init.parent_names = parent_names;
+ init.num_parents = num_parents;
+ init.name = clk_name;
+ init.ops = &isc_clk_ops;
+ init.flags = CLK_SET_RATE_GATE | CLK_SET_PARENT_GATE;
+
+ isc_clk = &isc->isc_clks[id];
+ isc_clk->hw.init = &init;
+ isc_clk->regmap = regmap;
+ isc_clk->id = id;
+ isc_clk->dev = isc->dev;
+ spin_lock_init(&isc_clk->lock);
+
+ isc_clk->clk = clk_register(isc->dev, &isc_clk->hw);
+ if (IS_ERR(isc_clk->clk)) {
+ dev_err(isc->dev, "%s: clock register fail\n", clk_name);
+ return PTR_ERR(isc_clk->clk);
+ } else if (id == ISC_MCK) {
+ of_clk_add_provider(np, of_clk_src_simple_get, isc_clk->clk);
+ }
+
+ return 0;
+}
+
+int isc_clk_init(struct isc_device *isc)
+{
+ unsigned int i;
+ int ret;
+
+ for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++)
+ isc->isc_clks[i].clk = ERR_PTR(-EINVAL);
+
+ for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
+ ret = isc_clk_register(isc, i);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(isc_clk_init);
+
+void isc_clk_cleanup(struct isc_device *isc)
+{
+ unsigned int i;
+
+ of_clk_del_provider(isc->dev->of_node);
+
+ for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
+ struct isc_clk *isc_clk = &isc->isc_clks[i];
+
+ if (!IS_ERR(isc_clk->clk))
+ clk_unregister(isc_clk->clk);
+ }
+}
+EXPORT_SYMBOL_GPL(isc_clk_cleanup);
diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
index 2bfcb135ef13..32448ccfc636 100644
--- a/drivers/media/platform/atmel/atmel-isc.h
+++ b/drivers/media/platform/atmel/atmel-isc.h
@@ -10,6 +10,13 @@
*/
#ifndef _ATMEL_ISC_H_
+#include <linux/clk-provider.h>
+#include <linux/platform_device.h>
+
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/videobuf2-dma-contig.h>
+
#define ISC_CLK_MAX_DIV 255
enum isc_clk_id {
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 04/23] media: atmel: atmel-isc: split the clock code into separate source file
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
The atmel-isc-base is getting crowded. Split the clock functions into
atmel-isc-clk.c.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
Changes in v3:
- squash with maintainers, add patterns in maintainers
MAINTAINERS | 7 +-
drivers/media/platform/atmel/Makefile | 3 +-
drivers/media/platform/atmel/atmel-isc-base.c | 295 -----------------
drivers/media/platform/atmel/atmel-isc-clk.c | 311 ++++++++++++++++++
drivers/media/platform/atmel/atmel-isc.h | 7 +
5 files changed, 322 insertions(+), 301 deletions(-)
create mode 100644 drivers/media/platform/atmel/atmel-isc-clk.c
diff --git a/MAINTAINERS b/MAINTAINERS
index afb74460e5c1..6aa274129751 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -12500,11 +12500,8 @@ L: linux-media@vger.kernel.org
S: Supported
F: Documentation/devicetree/bindings/media/atmel,isc.yaml
F: Documentation/devicetree/bindings/media/microchip,xisc.yaml
-F: drivers/media/platform/atmel/atmel-isc-base.c
-F: drivers/media/platform/atmel/atmel-isc-regs.h
-F: drivers/media/platform/atmel/atmel-isc.h
-F: drivers/media/platform/atmel/atmel-sama5d2-isc.c
-F: drivers/media/platform/atmel/atmel-sama7g5-isc.c
+F: drivers/media/platform/atmel/atmel-isc*
+F: drivers/media/platform/atmel/atmel-sama*-isc*
F: include/linux/atmel-isc-media.h
MICROCHIP ISI DRIVER
diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
index 39f0a7eba702..794e8f739287 100644
--- a/drivers/media/platform/atmel/Makefile
+++ b/drivers/media/platform/atmel/Makefile
@@ -1,9 +1,10 @@
# SPDX-License-Identifier: GPL-2.0-only
atmel-isc-objs = atmel-sama5d2-isc.o
atmel-xisc-objs = atmel-sama7g5-isc.o
+atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o
obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
-obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o
+obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-common.o
obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel-isc.o
obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel-xisc.o
obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += microchip-csi2dc.o
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index 660cd0ab6749..58f468d32613 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -8,10 +8,6 @@
* Author: Eugen Hristev <eugen.hristev@microchip.com>
*
*/
-
-#include <linux/clk.h>
-#include <linux/clkdev.h>
-#include <linux/clk-provider.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/math64.h>
@@ -100,297 +96,6 @@ static inline void isc_reset_awb_ctrls(struct isc_device *isc)
}
}
-static int isc_wait_clk_stable(struct clk_hw *hw)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
- struct regmap *regmap = isc_clk->regmap;
- unsigned long timeout = jiffies + usecs_to_jiffies(1000);
- unsigned int status;
-
- while (time_before(jiffies, timeout)) {
- regmap_read(regmap, ISC_CLKSR, &status);
- if (!(status & ISC_CLKSR_SIP))
- return 0;
-
- usleep_range(10, 250);
- }
-
- return -ETIMEDOUT;
-}
-
-static int isc_clk_prepare(struct clk_hw *hw)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
- int ret;
-
- ret = pm_runtime_resume_and_get(isc_clk->dev);
- if (ret < 0)
- return ret;
-
- return isc_wait_clk_stable(hw);
-}
-
-static void isc_clk_unprepare(struct clk_hw *hw)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
-
- isc_wait_clk_stable(hw);
-
- pm_runtime_put_sync(isc_clk->dev);
-}
-
-static int isc_clk_enable(struct clk_hw *hw)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
- u32 id = isc_clk->id;
- struct regmap *regmap = isc_clk->regmap;
- unsigned long flags;
- unsigned int status;
-
- dev_dbg(isc_clk->dev, "ISC CLK: %s, id = %d, div = %d, parent id = %d\n",
- __func__, id, isc_clk->div, isc_clk->parent_id);
-
- spin_lock_irqsave(&isc_clk->lock, flags);
- regmap_update_bits(regmap, ISC_CLKCFG,
- ISC_CLKCFG_DIV_MASK(id) | ISC_CLKCFG_SEL_MASK(id),
- (isc_clk->div << ISC_CLKCFG_DIV_SHIFT(id)) |
- (isc_clk->parent_id << ISC_CLKCFG_SEL_SHIFT(id)));
-
- regmap_write(regmap, ISC_CLKEN, ISC_CLK(id));
- spin_unlock_irqrestore(&isc_clk->lock, flags);
-
- regmap_read(regmap, ISC_CLKSR, &status);
- if (status & ISC_CLK(id))
- return 0;
- else
- return -EINVAL;
-}
-
-static void isc_clk_disable(struct clk_hw *hw)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
- u32 id = isc_clk->id;
- unsigned long flags;
-
- spin_lock_irqsave(&isc_clk->lock, flags);
- regmap_write(isc_clk->regmap, ISC_CLKDIS, ISC_CLK(id));
- spin_unlock_irqrestore(&isc_clk->lock, flags);
-}
-
-static int isc_clk_is_enabled(struct clk_hw *hw)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
- u32 status;
- int ret;
-
- ret = pm_runtime_resume_and_get(isc_clk->dev);
- if (ret < 0)
- return 0;
-
- regmap_read(isc_clk->regmap, ISC_CLKSR, &status);
-
- pm_runtime_put_sync(isc_clk->dev);
-
- return status & ISC_CLK(isc_clk->id) ? 1 : 0;
-}
-
-static unsigned long
-isc_clk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
-
- return DIV_ROUND_CLOSEST(parent_rate, isc_clk->div + 1);
-}
-
-static int isc_clk_determine_rate(struct clk_hw *hw,
- struct clk_rate_request *req)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
- long best_rate = -EINVAL;
- int best_diff = -1;
- unsigned int i, div;
-
- for (i = 0; i < clk_hw_get_num_parents(hw); i++) {
- struct clk_hw *parent;
- unsigned long parent_rate;
-
- parent = clk_hw_get_parent_by_index(hw, i);
- if (!parent)
- continue;
-
- parent_rate = clk_hw_get_rate(parent);
- if (!parent_rate)
- continue;
-
- for (div = 1; div < ISC_CLK_MAX_DIV + 2; div++) {
- unsigned long rate;
- int diff;
-
- rate = DIV_ROUND_CLOSEST(parent_rate, div);
- diff = abs(req->rate - rate);
-
- if (best_diff < 0 || best_diff > diff) {
- best_rate = rate;
- best_diff = diff;
- req->best_parent_rate = parent_rate;
- req->best_parent_hw = parent;
- }
-
- if (!best_diff || rate < req->rate)
- break;
- }
-
- if (!best_diff)
- break;
- }
-
- dev_dbg(isc_clk->dev,
- "ISC CLK: %s, best_rate = %ld, parent clk: %s @ %ld\n",
- __func__, best_rate,
- __clk_get_name((req->best_parent_hw)->clk),
- req->best_parent_rate);
-
- if (best_rate < 0)
- return best_rate;
-
- req->rate = best_rate;
-
- return 0;
-}
-
-static int isc_clk_set_parent(struct clk_hw *hw, u8 index)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
-
- if (index >= clk_hw_get_num_parents(hw))
- return -EINVAL;
-
- isc_clk->parent_id = index;
-
- return 0;
-}
-
-static u8 isc_clk_get_parent(struct clk_hw *hw)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
-
- return isc_clk->parent_id;
-}
-
-static int isc_clk_set_rate(struct clk_hw *hw,
- unsigned long rate,
- unsigned long parent_rate)
-{
- struct isc_clk *isc_clk = to_isc_clk(hw);
- u32 div;
-
- if (!rate)
- return -EINVAL;
-
- div = DIV_ROUND_CLOSEST(parent_rate, rate);
- if (div > (ISC_CLK_MAX_DIV + 1) || !div)
- return -EINVAL;
-
- isc_clk->div = div - 1;
-
- return 0;
-}
-
-static const struct clk_ops isc_clk_ops = {
- .prepare = isc_clk_prepare,
- .unprepare = isc_clk_unprepare,
- .enable = isc_clk_enable,
- .disable = isc_clk_disable,
- .is_enabled = isc_clk_is_enabled,
- .recalc_rate = isc_clk_recalc_rate,
- .determine_rate = isc_clk_determine_rate,
- .set_parent = isc_clk_set_parent,
- .get_parent = isc_clk_get_parent,
- .set_rate = isc_clk_set_rate,
-};
-
-static int isc_clk_register(struct isc_device *isc, unsigned int id)
-{
- struct regmap *regmap = isc->regmap;
- struct device_node *np = isc->dev->of_node;
- struct isc_clk *isc_clk;
- struct clk_init_data init;
- const char *clk_name = np->name;
- const char *parent_names[3];
- int num_parents;
-
- if (id == ISC_ISPCK && !isc->ispck_required)
- return 0;
-
- num_parents = of_clk_get_parent_count(np);
- if (num_parents < 1 || num_parents > 3)
- return -EINVAL;
-
- if (num_parents > 2 && id == ISC_ISPCK)
- num_parents = 2;
-
- of_clk_parent_fill(np, parent_names, num_parents);
-
- if (id == ISC_MCK)
- of_property_read_string(np, "clock-output-names", &clk_name);
- else
- clk_name = "isc-ispck";
-
- init.parent_names = parent_names;
- init.num_parents = num_parents;
- init.name = clk_name;
- init.ops = &isc_clk_ops;
- init.flags = CLK_SET_RATE_GATE | CLK_SET_PARENT_GATE;
-
- isc_clk = &isc->isc_clks[id];
- isc_clk->hw.init = &init;
- isc_clk->regmap = regmap;
- isc_clk->id = id;
- isc_clk->dev = isc->dev;
- spin_lock_init(&isc_clk->lock);
-
- isc_clk->clk = clk_register(isc->dev, &isc_clk->hw);
- if (IS_ERR(isc_clk->clk)) {
- dev_err(isc->dev, "%s: clock register fail\n", clk_name);
- return PTR_ERR(isc_clk->clk);
- } else if (id == ISC_MCK)
- of_clk_add_provider(np, of_clk_src_simple_get, isc_clk->clk);
-
- return 0;
-}
-
-int isc_clk_init(struct isc_device *isc)
-{
- unsigned int i;
- int ret;
-
- for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++)
- isc->isc_clks[i].clk = ERR_PTR(-EINVAL);
-
- for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
- ret = isc_clk_register(isc, i);
- if (ret)
- return ret;
- }
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(isc_clk_init);
-
-void isc_clk_cleanup(struct isc_device *isc)
-{
- unsigned int i;
-
- of_clk_del_provider(isc->dev->of_node);
-
- for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
- struct isc_clk *isc_clk = &isc->isc_clks[i];
-
- if (!IS_ERR(isc_clk->clk))
- clk_unregister(isc_clk->clk);
- }
-}
-EXPORT_SYMBOL_GPL(isc_clk_cleanup);
static int isc_queue_setup(struct vb2_queue *vq,
unsigned int *nbuffers, unsigned int *nplanes,
diff --git a/drivers/media/platform/atmel/atmel-isc-clk.c b/drivers/media/platform/atmel/atmel-isc-clk.c
new file mode 100644
index 000000000000..2059fe376b00
--- /dev/null
+++ b/drivers/media/platform/atmel/atmel-isc-clk.c
@@ -0,0 +1,311 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Microchip Image Sensor Controller (ISC) common clock driver setup
+ *
+ * Copyright (C) 2016 Microchip Technology, Inc.
+ *
+ * Author: Songjun Wu
+ * Author: Eugen Hristev <eugen.hristev@microchip.com>
+ *
+ */
+#include <linux/clk.h>
+#include <linux/clkdev.h>
+#include <linux/clk-provider.h>
+#include <linux/pm_runtime.h>
+#include <linux/regmap.h>
+
+#include "atmel-isc-regs.h"
+#include "atmel-isc.h"
+
+static int isc_wait_clk_stable(struct clk_hw *hw)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+ struct regmap *regmap = isc_clk->regmap;
+ unsigned long timeout = jiffies + usecs_to_jiffies(1000);
+ unsigned int status;
+
+ while (time_before(jiffies, timeout)) {
+ regmap_read(regmap, ISC_CLKSR, &status);
+ if (!(status & ISC_CLKSR_SIP))
+ return 0;
+
+ usleep_range(10, 250);
+ }
+
+ return -ETIMEDOUT;
+}
+
+static int isc_clk_prepare(struct clk_hw *hw)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+ int ret;
+
+ ret = pm_runtime_resume_and_get(isc_clk->dev);
+ if (ret < 0)
+ return ret;
+
+ return isc_wait_clk_stable(hw);
+}
+
+static void isc_clk_unprepare(struct clk_hw *hw)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+
+ isc_wait_clk_stable(hw);
+
+ pm_runtime_put_sync(isc_clk->dev);
+}
+
+static int isc_clk_enable(struct clk_hw *hw)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+ u32 id = isc_clk->id;
+ struct regmap *regmap = isc_clk->regmap;
+ unsigned long flags;
+ unsigned int status;
+
+ dev_dbg(isc_clk->dev, "ISC CLK: %s, id = %d, div = %d, parent id = %d\n",
+ __func__, id, isc_clk->div, isc_clk->parent_id);
+
+ spin_lock_irqsave(&isc_clk->lock, flags);
+ regmap_update_bits(regmap, ISC_CLKCFG,
+ ISC_CLKCFG_DIV_MASK(id) | ISC_CLKCFG_SEL_MASK(id),
+ (isc_clk->div << ISC_CLKCFG_DIV_SHIFT(id)) |
+ (isc_clk->parent_id << ISC_CLKCFG_SEL_SHIFT(id)));
+
+ regmap_write(regmap, ISC_CLKEN, ISC_CLK(id));
+ spin_unlock_irqrestore(&isc_clk->lock, flags);
+
+ regmap_read(regmap, ISC_CLKSR, &status);
+ if (status & ISC_CLK(id))
+ return 0;
+ else
+ return -EINVAL;
+}
+
+static void isc_clk_disable(struct clk_hw *hw)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+ u32 id = isc_clk->id;
+ unsigned long flags;
+
+ spin_lock_irqsave(&isc_clk->lock, flags);
+ regmap_write(isc_clk->regmap, ISC_CLKDIS, ISC_CLK(id));
+ spin_unlock_irqrestore(&isc_clk->lock, flags);
+}
+
+static int isc_clk_is_enabled(struct clk_hw *hw)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+ u32 status;
+ int ret;
+
+ ret = pm_runtime_resume_and_get(isc_clk->dev);
+ if (ret < 0)
+ return 0;
+
+ regmap_read(isc_clk->regmap, ISC_CLKSR, &status);
+
+ pm_runtime_put_sync(isc_clk->dev);
+
+ return status & ISC_CLK(isc_clk->id) ? 1 : 0;
+}
+
+static unsigned long
+isc_clk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+
+ return DIV_ROUND_CLOSEST(parent_rate, isc_clk->div + 1);
+}
+
+static int isc_clk_determine_rate(struct clk_hw *hw,
+ struct clk_rate_request *req)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+ long best_rate = -EINVAL;
+ int best_diff = -1;
+ unsigned int i, div;
+
+ for (i = 0; i < clk_hw_get_num_parents(hw); i++) {
+ struct clk_hw *parent;
+ unsigned long parent_rate;
+
+ parent = clk_hw_get_parent_by_index(hw, i);
+ if (!parent)
+ continue;
+
+ parent_rate = clk_hw_get_rate(parent);
+ if (!parent_rate)
+ continue;
+
+ for (div = 1; div < ISC_CLK_MAX_DIV + 2; div++) {
+ unsigned long rate;
+ int diff;
+
+ rate = DIV_ROUND_CLOSEST(parent_rate, div);
+ diff = abs(req->rate - rate);
+
+ if (best_diff < 0 || best_diff > diff) {
+ best_rate = rate;
+ best_diff = diff;
+ req->best_parent_rate = parent_rate;
+ req->best_parent_hw = parent;
+ }
+
+ if (!best_diff || rate < req->rate)
+ break;
+ }
+
+ if (!best_diff)
+ break;
+ }
+
+ dev_dbg(isc_clk->dev,
+ "ISC CLK: %s, best_rate = %ld, parent clk: %s @ %ld\n",
+ __func__, best_rate,
+ __clk_get_name((req->best_parent_hw)->clk),
+ req->best_parent_rate);
+
+ if (best_rate < 0)
+ return best_rate;
+
+ req->rate = best_rate;
+
+ return 0;
+}
+
+static int isc_clk_set_parent(struct clk_hw *hw, u8 index)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+
+ if (index >= clk_hw_get_num_parents(hw))
+ return -EINVAL;
+
+ isc_clk->parent_id = index;
+
+ return 0;
+}
+
+static u8 isc_clk_get_parent(struct clk_hw *hw)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+
+ return isc_clk->parent_id;
+}
+
+static int isc_clk_set_rate(struct clk_hw *hw,
+ unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct isc_clk *isc_clk = to_isc_clk(hw);
+ u32 div;
+
+ if (!rate)
+ return -EINVAL;
+
+ div = DIV_ROUND_CLOSEST(parent_rate, rate);
+ if (div > (ISC_CLK_MAX_DIV + 1) || !div)
+ return -EINVAL;
+
+ isc_clk->div = div - 1;
+
+ return 0;
+}
+
+static const struct clk_ops isc_clk_ops = {
+ .prepare = isc_clk_prepare,
+ .unprepare = isc_clk_unprepare,
+ .enable = isc_clk_enable,
+ .disable = isc_clk_disable,
+ .is_enabled = isc_clk_is_enabled,
+ .recalc_rate = isc_clk_recalc_rate,
+ .determine_rate = isc_clk_determine_rate,
+ .set_parent = isc_clk_set_parent,
+ .get_parent = isc_clk_get_parent,
+ .set_rate = isc_clk_set_rate,
+};
+
+static int isc_clk_register(struct isc_device *isc, unsigned int id)
+{
+ struct regmap *regmap = isc->regmap;
+ struct device_node *np = isc->dev->of_node;
+ struct isc_clk *isc_clk;
+ struct clk_init_data init;
+ const char *clk_name = np->name;
+ const char *parent_names[3];
+ int num_parents;
+
+ if (id == ISC_ISPCK && !isc->ispck_required)
+ return 0;
+
+ num_parents = of_clk_get_parent_count(np);
+ if (num_parents < 1 || num_parents > 3)
+ return -EINVAL;
+
+ if (num_parents > 2 && id == ISC_ISPCK)
+ num_parents = 2;
+
+ of_clk_parent_fill(np, parent_names, num_parents);
+
+ if (id == ISC_MCK)
+ of_property_read_string(np, "clock-output-names", &clk_name);
+ else
+ clk_name = "isc-ispck";
+
+ init.parent_names = parent_names;
+ init.num_parents = num_parents;
+ init.name = clk_name;
+ init.ops = &isc_clk_ops;
+ init.flags = CLK_SET_RATE_GATE | CLK_SET_PARENT_GATE;
+
+ isc_clk = &isc->isc_clks[id];
+ isc_clk->hw.init = &init;
+ isc_clk->regmap = regmap;
+ isc_clk->id = id;
+ isc_clk->dev = isc->dev;
+ spin_lock_init(&isc_clk->lock);
+
+ isc_clk->clk = clk_register(isc->dev, &isc_clk->hw);
+ if (IS_ERR(isc_clk->clk)) {
+ dev_err(isc->dev, "%s: clock register fail\n", clk_name);
+ return PTR_ERR(isc_clk->clk);
+ } else if (id == ISC_MCK) {
+ of_clk_add_provider(np, of_clk_src_simple_get, isc_clk->clk);
+ }
+
+ return 0;
+}
+
+int isc_clk_init(struct isc_device *isc)
+{
+ unsigned int i;
+ int ret;
+
+ for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++)
+ isc->isc_clks[i].clk = ERR_PTR(-EINVAL);
+
+ for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
+ ret = isc_clk_register(isc, i);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(isc_clk_init);
+
+void isc_clk_cleanup(struct isc_device *isc)
+{
+ unsigned int i;
+
+ of_clk_del_provider(isc->dev->of_node);
+
+ for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
+ struct isc_clk *isc_clk = &isc->isc_clks[i];
+
+ if (!IS_ERR(isc_clk->clk))
+ clk_unregister(isc_clk->clk);
+ }
+}
+EXPORT_SYMBOL_GPL(isc_clk_cleanup);
diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
index 2bfcb135ef13..32448ccfc636 100644
--- a/drivers/media/platform/atmel/atmel-isc.h
+++ b/drivers/media/platform/atmel/atmel-isc.h
@@ -10,6 +10,13 @@
*/
#ifndef _ATMEL_ISC_H_
+#include <linux/clk-provider.h>
+#include <linux/platform_device.h>
+
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/videobuf2-dma-contig.h>
+
#define ISC_CLK_MAX_DIV 255
enum isc_clk_id {
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 05/23] media: atmel: atmel-isc: replace video device name with module name
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
To have consistency with future media controller development,
replace the video device name with KBUILD_MODNAME.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
drivers/media/platform/atmel/atmel-isc-base.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index 58f468d32613..26a6090f056c 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -1886,7 +1886,7 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
}
/* Register video device */
- strscpy(vdev->name, "microchip-isc", sizeof(vdev->name));
+ strscpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name));
vdev->release = video_device_release_empty;
vdev->fops = &isc_fops;
vdev->ioctl_ops = &isc_ioctl_ops;
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 05/23] media: atmel: atmel-isc: replace video device name with module name
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
To have consistency with future media controller development,
replace the video device name with KBUILD_MODNAME.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
drivers/media/platform/atmel/atmel-isc-base.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index 58f468d32613..26a6090f056c 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -1886,7 +1886,7 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
}
/* Register video device */
- strscpy(vdev->name, "microchip-isc", sizeof(vdev->name));
+ strscpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name));
vdev->release = video_device_release_empty;
vdev->fops = &isc_fops;
vdev->ioctl_ops = &isc_ioctl_ops;
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 06/23] media: atmel: atmel-sama7g5-isc: fix ispck leftover
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
The ispck is not used for sama7g5 variant of the ISC.
Calls to ispck have to be removed also from module insert/removal.
Fixes: d7f26849ed7c ("media: atmel: fix the ispck initialization")
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
---
drivers/media/platform/atmel/atmel-sama7g5-isc.c | 6 ------
1 file changed, 6 deletions(-)
diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
index 5d1c76f680f3..2b1082295c13 100644
--- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
@@ -556,7 +556,6 @@ static int microchip_xisc_remove(struct platform_device *pdev)
v4l2_device_unregister(&isc->v4l2_dev);
- clk_disable_unprepare(isc->ispck);
clk_disable_unprepare(isc->hclock);
isc_clk_cleanup(isc);
@@ -568,7 +567,6 @@ static int __maybe_unused xisc_runtime_suspend(struct device *dev)
{
struct isc_device *isc = dev_get_drvdata(dev);
- clk_disable_unprepare(isc->ispck);
clk_disable_unprepare(isc->hclock);
return 0;
@@ -583,10 +581,6 @@ static int __maybe_unused xisc_runtime_resume(struct device *dev)
if (ret)
return ret;
- ret = clk_prepare_enable(isc->ispck);
- if (ret)
- clk_disable_unprepare(isc->hclock);
-
return ret;
}
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 06/23] media: atmel: atmel-sama7g5-isc: fix ispck leftover
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
The ispck is not used for sama7g5 variant of the ISC.
Calls to ispck have to be removed also from module insert/removal.
Fixes: d7f26849ed7c ("media: atmel: fix the ispck initialization")
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
---
drivers/media/platform/atmel/atmel-sama7g5-isc.c | 6 ------
1 file changed, 6 deletions(-)
diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
index 5d1c76f680f3..2b1082295c13 100644
--- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
@@ -556,7 +556,6 @@ static int microchip_xisc_remove(struct platform_device *pdev)
v4l2_device_unregister(&isc->v4l2_dev);
- clk_disable_unprepare(isc->ispck);
clk_disable_unprepare(isc->hclock);
isc_clk_cleanup(isc);
@@ -568,7 +567,6 @@ static int __maybe_unused xisc_runtime_suspend(struct device *dev)
{
struct isc_device *isc = dev_get_drvdata(dev);
- clk_disable_unprepare(isc->ispck);
clk_disable_unprepare(isc->hclock);
return 0;
@@ -583,10 +581,6 @@ static int __maybe_unused xisc_runtime_resume(struct device *dev)
if (ret)
return ret;
- ret = clk_prepare_enable(isc->ispck);
- if (ret)
- clk_disable_unprepare(isc->hclock);
-
return ret;
}
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 07/23] media: atmel: atmel-isc-base: use streaming status when queueing buffers
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
During experiments with libcamera, it looks like vb2_is_streaming returns
true before our start streaming is called.
Order of operations is streamon -> queue -> start_streaming
ISC would have started the DMA immediately when a buffer is being added
to the vbqueue if the queue is streaming.
It is more safe to start the DMA after the start streaming of the driver is
called.
Thus, even if vb2queue is streaming, add the buffer to the dma queue of the
driver instead of actually starting the DMA process, if the start streaming
has not been called yet.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
drivers/media/platform/atmel/atmel-isc-base.c | 17 ++++++++++-------
1 file changed, 10 insertions(+), 7 deletions(-)
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index 26a6090f056c..e6c9071c04f0 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -441,12 +441,14 @@ static void isc_buffer_queue(struct vb2_buffer *vb)
unsigned long flags;
spin_lock_irqsave(&isc->dma_queue_lock, flags);
- if (!isc->cur_frm && list_empty(&isc->dma_queue) &&
- vb2_is_streaming(vb->vb2_queue)) {
+
+ if (!isc->cur_frm && list_empty(&isc->dma_queue) && !isc->stop) {
isc->cur_frm = buf;
isc_start_dma(isc);
- } else
+ } else {
list_add_tail(&buf->list, &isc->dma_queue);
+ }
+
spin_unlock_irqrestore(&isc->dma_queue_lock, flags);
}
@@ -1014,7 +1016,7 @@ static int isc_s_fmt_vid_cap(struct file *file, void *priv,
{
struct isc_device *isc = video_drvdata(file);
- if (vb2_is_streaming(&isc->vb2_vidq))
+ if (!isc->stop)
return -EBUSY;
return isc_set_fmt(isc, f);
@@ -1536,7 +1538,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
isc_update_awb_ctrls(isc);
- if (vb2_is_streaming(&isc->vb2_vidq)) {
+ if (!isc->stop) {
/*
* If we are streaming, we can update profile to
* have the new settings in place.
@@ -1552,8 +1554,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
}
/* if we have autowhitebalance on, start histogram procedure */
- if (ctrls->awb == ISC_WB_AUTO &&
- vb2_is_streaming(&isc->vb2_vidq) &&
+ if (ctrls->awb == ISC_WB_AUTO && !isc->stop &&
ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
isc_set_histogram(isc, true);
@@ -1829,6 +1830,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
struct vb2_queue *q = &isc->vb2_vidq;
int ret = 0;
+ isc->stop = true;
+
INIT_WORK(&isc->awb_work, isc_awb_work);
ret = v4l2_device_register_subdev_nodes(&isc->v4l2_dev);
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 07/23] media: atmel: atmel-isc-base: use streaming status when queueing buffers
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
During experiments with libcamera, it looks like vb2_is_streaming returns
true before our start streaming is called.
Order of operations is streamon -> queue -> start_streaming
ISC would have started the DMA immediately when a buffer is being added
to the vbqueue if the queue is streaming.
It is more safe to start the DMA after the start streaming of the driver is
called.
Thus, even if vb2queue is streaming, add the buffer to the dma queue of the
driver instead of actually starting the DMA process, if the start streaming
has not been called yet.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
drivers/media/platform/atmel/atmel-isc-base.c | 17 ++++++++++-------
1 file changed, 10 insertions(+), 7 deletions(-)
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index 26a6090f056c..e6c9071c04f0 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -441,12 +441,14 @@ static void isc_buffer_queue(struct vb2_buffer *vb)
unsigned long flags;
spin_lock_irqsave(&isc->dma_queue_lock, flags);
- if (!isc->cur_frm && list_empty(&isc->dma_queue) &&
- vb2_is_streaming(vb->vb2_queue)) {
+
+ if (!isc->cur_frm && list_empty(&isc->dma_queue) && !isc->stop) {
isc->cur_frm = buf;
isc_start_dma(isc);
- } else
+ } else {
list_add_tail(&buf->list, &isc->dma_queue);
+ }
+
spin_unlock_irqrestore(&isc->dma_queue_lock, flags);
}
@@ -1014,7 +1016,7 @@ static int isc_s_fmt_vid_cap(struct file *file, void *priv,
{
struct isc_device *isc = video_drvdata(file);
- if (vb2_is_streaming(&isc->vb2_vidq))
+ if (!isc->stop)
return -EBUSY;
return isc_set_fmt(isc, f);
@@ -1536,7 +1538,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
isc_update_awb_ctrls(isc);
- if (vb2_is_streaming(&isc->vb2_vidq)) {
+ if (!isc->stop) {
/*
* If we are streaming, we can update profile to
* have the new settings in place.
@@ -1552,8 +1554,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
}
/* if we have autowhitebalance on, start histogram procedure */
- if (ctrls->awb == ISC_WB_AUTO &&
- vb2_is_streaming(&isc->vb2_vidq) &&
+ if (ctrls->awb == ISC_WB_AUTO && !isc->stop &&
ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
isc_set_histogram(isc, true);
@@ -1829,6 +1830,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
struct vb2_queue *q = &isc->vb2_vidq;
int ret = 0;
+ isc->stop = true;
+
INIT_WORK(&isc->awb_work, isc_awb_work);
ret = v4l2_device_register_subdev_nodes(&isc->v4l2_dev);
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 08/23] media: atmel: atmel-isc-base: remove frameintervals VIDIOC
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
VIDIOC_ENUM_FRAMEINTERVALS is not recommended for a top video driver.
The frame rate is defined by the sensor subdevice, thus it can be queried
directly by anyone interested in the frame intervals.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
---
drivers/media/platform/atmel/atmel-isc-base.c | 37 -------------------
1 file changed, 37 deletions(-)
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index e6c9071c04f0..3389671fbfb9 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -1107,42 +1107,6 @@ static int isc_enum_framesizes(struct file *file, void *fh,
return 0;
}
-static int isc_enum_frameintervals(struct file *file, void *fh,
- struct v4l2_frmivalenum *fival)
-{
- struct isc_device *isc = video_drvdata(file);
- struct v4l2_subdev_frame_interval_enum fie = {
- .code = isc->config.sd_format->mbus_code,
- .index = fival->index,
- .width = fival->width,
- .height = fival->height,
- .which = V4L2_SUBDEV_FORMAT_ACTIVE,
- };
- int ret = -EINVAL;
- unsigned int i;
-
- for (i = 0; i < isc->num_user_formats; i++)
- if (isc->user_formats[i]->fourcc == fival->pixel_format)
- ret = 0;
-
- for (i = 0; i < isc->controller_formats_size; i++)
- if (isc->controller_formats[i].fourcc == fival->pixel_format)
- ret = 0;
-
- if (ret)
- return ret;
-
- ret = v4l2_subdev_call(isc->current_subdev->sd, pad,
- enum_frame_interval, NULL, &fie);
- if (ret)
- return ret;
-
- fival->type = V4L2_FRMIVAL_TYPE_DISCRETE;
- fival->discrete = fie.interval;
-
- return 0;
-}
-
static const struct v4l2_ioctl_ops isc_ioctl_ops = {
.vidioc_querycap = isc_querycap,
.vidioc_enum_fmt_vid_cap = isc_enum_fmt_vid_cap,
@@ -1167,7 +1131,6 @@ static const struct v4l2_ioctl_ops isc_ioctl_ops = {
.vidioc_g_parm = isc_g_parm,
.vidioc_s_parm = isc_s_parm,
.vidioc_enum_framesizes = isc_enum_framesizes,
- .vidioc_enum_frameintervals = isc_enum_frameintervals,
.vidioc_log_status = v4l2_ctrl_log_status,
.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 08/23] media: atmel: atmel-isc-base: remove frameintervals VIDIOC
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
VIDIOC_ENUM_FRAMEINTERVALS is not recommended for a top video driver.
The frame rate is defined by the sensor subdevice, thus it can be queried
directly by anyone interested in the frame intervals.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
---
drivers/media/platform/atmel/atmel-isc-base.c | 37 -------------------
1 file changed, 37 deletions(-)
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index e6c9071c04f0..3389671fbfb9 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -1107,42 +1107,6 @@ static int isc_enum_framesizes(struct file *file, void *fh,
return 0;
}
-static int isc_enum_frameintervals(struct file *file, void *fh,
- struct v4l2_frmivalenum *fival)
-{
- struct isc_device *isc = video_drvdata(file);
- struct v4l2_subdev_frame_interval_enum fie = {
- .code = isc->config.sd_format->mbus_code,
- .index = fival->index,
- .width = fival->width,
- .height = fival->height,
- .which = V4L2_SUBDEV_FORMAT_ACTIVE,
- };
- int ret = -EINVAL;
- unsigned int i;
-
- for (i = 0; i < isc->num_user_formats; i++)
- if (isc->user_formats[i]->fourcc == fival->pixel_format)
- ret = 0;
-
- for (i = 0; i < isc->controller_formats_size; i++)
- if (isc->controller_formats[i].fourcc == fival->pixel_format)
- ret = 0;
-
- if (ret)
- return ret;
-
- ret = v4l2_subdev_call(isc->current_subdev->sd, pad,
- enum_frame_interval, NULL, &fie);
- if (ret)
- return ret;
-
- fival->type = V4L2_FRMIVAL_TYPE_DISCRETE;
- fival->discrete = fie.interval;
-
- return 0;
-}
-
static const struct v4l2_ioctl_ops isc_ioctl_ops = {
.vidioc_querycap = isc_querycap,
.vidioc_enum_fmt_vid_cap = isc_enum_fmt_vid_cap,
@@ -1167,7 +1131,6 @@ static const struct v4l2_ioctl_ops isc_ioctl_ops = {
.vidioc_g_parm = isc_g_parm,
.vidioc_s_parm = isc_s_parm,
.vidioc_enum_framesizes = isc_enum_framesizes,
- .vidioc_enum_frameintervals = isc_enum_frameintervals,
.vidioc_log_status = v4l2_ctrl_log_status,
.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 09/23] media: atmel: atmel-isc-base: report frame sizes as full supported range
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
The ISC supports a full broad range of frame sizes.
Until now, the subdevice was queried for possible frame sizes and these
were reported to the user space.
However, the ISC should not care about which frame sizes the subdev supports,
as long as this frame size is supported.
Thus, report a continuous range from smallest frame size up to the max
resolution.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
---
drivers/media/platform/atmel/atmel-isc-base.c | 22 +++++++++----------
1 file changed, 10 insertions(+), 12 deletions(-)
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index 3389671fbfb9..bf638d201b29 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -1076,14 +1076,12 @@ static int isc_enum_framesizes(struct file *file, void *fh,
struct v4l2_frmsizeenum *fsize)
{
struct isc_device *isc = video_drvdata(file);
- struct v4l2_subdev_frame_size_enum fse = {
- .code = isc->config.sd_format->mbus_code,
- .index = fsize->index,
- .which = V4L2_SUBDEV_FORMAT_ACTIVE,
- };
int ret = -EINVAL;
int i;
+ if (fsize->index)
+ return -EINVAL;
+
for (i = 0; i < isc->num_user_formats; i++)
if (isc->user_formats[i]->fourcc == fsize->pixel_format)
ret = 0;
@@ -1095,14 +1093,14 @@ static int isc_enum_framesizes(struct file *file, void *fh,
if (ret)
return ret;
- ret = v4l2_subdev_call(isc->current_subdev->sd, pad, enum_frame_size,
- NULL, &fse);
- if (ret)
- return ret;
+ fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
- fsize->type = V4L2_FRMSIZE_TYPE_DISCRETE;
- fsize->discrete.width = fse.max_width;
- fsize->discrete.height = fse.max_height;
+ fsize->stepwise.min_width = 16;
+ fsize->stepwise.max_width = isc->max_width;
+ fsize->stepwise.min_height = 16;
+ fsize->stepwise.max_height = isc->max_height;
+ fsize->stepwise.step_width = 1;
+ fsize->stepwise.step_height = 1;
return 0;
}
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 09/23] media: atmel: atmel-isc-base: report frame sizes as full supported range
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
The ISC supports a full broad range of frame sizes.
Until now, the subdevice was queried for possible frame sizes and these
were reported to the user space.
However, the ISC should not care about which frame sizes the subdev supports,
as long as this frame size is supported.
Thus, report a continuous range from smallest frame size up to the max
resolution.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
---
drivers/media/platform/atmel/atmel-isc-base.c | 22 +++++++++----------
1 file changed, 10 insertions(+), 12 deletions(-)
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index 3389671fbfb9..bf638d201b29 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -1076,14 +1076,12 @@ static int isc_enum_framesizes(struct file *file, void *fh,
struct v4l2_frmsizeenum *fsize)
{
struct isc_device *isc = video_drvdata(file);
- struct v4l2_subdev_frame_size_enum fse = {
- .code = isc->config.sd_format->mbus_code,
- .index = fsize->index,
- .which = V4L2_SUBDEV_FORMAT_ACTIVE,
- };
int ret = -EINVAL;
int i;
+ if (fsize->index)
+ return -EINVAL;
+
for (i = 0; i < isc->num_user_formats; i++)
if (isc->user_formats[i]->fourcc == fsize->pixel_format)
ret = 0;
@@ -1095,14 +1093,14 @@ static int isc_enum_framesizes(struct file *file, void *fh,
if (ret)
return ret;
- ret = v4l2_subdev_call(isc->current_subdev->sd, pad, enum_frame_size,
- NULL, &fse);
- if (ret)
- return ret;
+ fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
- fsize->type = V4L2_FRMSIZE_TYPE_DISCRETE;
- fsize->discrete.width = fse.max_width;
- fsize->discrete.height = fse.max_height;
+ fsize->stepwise.min_width = 16;
+ fsize->stepwise.max_width = isc->max_width;
+ fsize->stepwise.min_height = 16;
+ fsize->stepwise.max_height = isc->max_height;
+ fsize->stepwise.step_width = 1;
+ fsize->stepwise.step_height = 1;
return 0;
}
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 10/23] media: atmel: atmel-isc-base: implement mbus_code support in enumfmt
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
If enumfmt is called with an mbus_code, the enumfmt handler should only
return the formats that are supported for this mbus_code.
To make it more easy to understand the formats, changed the report order
to report first the native formats, and after that the formats that the ISC
can convert to.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
drivers/media/platform/atmel/atmel-isc-base.c | 51 ++++++++++++++++---
1 file changed, 43 insertions(+), 8 deletions(-)
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index bf638d201b29..2cb8446ff90c 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -498,21 +498,56 @@ static int isc_enum_fmt_vid_cap(struct file *file, void *priv,
u32 index = f->index;
u32 i, supported_index;
- if (index < isc->controller_formats_size) {
- f->pixelformat = isc->controller_formats[index].fourcc;
- return 0;
+ supported_index = 0;
+
+ for (i = 0; i < isc->formats_list_size; i++) {
+ if (!isc->formats_list[i].sd_support)
+ continue;
+ /*
+ * If specific mbus_code is requested, provide only
+ * supported formats with this mbus code
+ */
+ if (f->mbus_code && f->mbus_code !=
+ isc->formats_list[i].mbus_code)
+ continue;
+ if (supported_index == index) {
+ f->pixelformat = isc->formats_list[i].fourcc;
+ return 0;
+ }
+ supported_index++;
}
- index -= isc->controller_formats_size;
+ /*
+ * If the sensor does not support this mbus_code whatsoever,
+ * there is no reason to advertise any of our output formats
+ */
+ if (supported_index == 0)
+ return -EINVAL;
+
+ /*
+ * If the sensor uses a format that is not raw, then we cannot
+ * convert it to any of the formats that we usually can with a
+ * RAW sensor. Thus, do not advertise them.
+ */
+ if (!isc->config.sd_format ||
+ !ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
+ return -EINVAL;
+ /*
+ * Iterate again through the formats that we can convert to.
+ * However, to avoid duplicates, skip the formats that
+ * the sensor already supports directly
+ */
+ index -= supported_index;
supported_index = 0;
- for (i = 0; i < isc->formats_list_size; i++) {
- if (!ISC_IS_FORMAT_RAW(isc->formats_list[i].mbus_code) ||
- !isc->formats_list[i].sd_support)
+ for (i = 0; i < isc->controller_formats_size; i++) {
+ /* if this format is already supported by sensor, skip it */
+ if (find_format_by_fourcc(isc, isc->controller_formats[i].fourcc))
continue;
if (supported_index == index) {
- f->pixelformat = isc->formats_list[i].fourcc;
+ f->pixelformat =
+ isc->controller_formats[i].fourcc;
return 0;
}
supported_index++;
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 10/23] media: atmel: atmel-isc-base: implement mbus_code support in enumfmt
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
If enumfmt is called with an mbus_code, the enumfmt handler should only
return the formats that are supported for this mbus_code.
To make it more easy to understand the formats, changed the report order
to report first the native formats, and after that the formats that the ISC
can convert to.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
drivers/media/platform/atmel/atmel-isc-base.c | 51 ++++++++++++++++---
1 file changed, 43 insertions(+), 8 deletions(-)
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index bf638d201b29..2cb8446ff90c 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -498,21 +498,56 @@ static int isc_enum_fmt_vid_cap(struct file *file, void *priv,
u32 index = f->index;
u32 i, supported_index;
- if (index < isc->controller_formats_size) {
- f->pixelformat = isc->controller_formats[index].fourcc;
- return 0;
+ supported_index = 0;
+
+ for (i = 0; i < isc->formats_list_size; i++) {
+ if (!isc->formats_list[i].sd_support)
+ continue;
+ /*
+ * If specific mbus_code is requested, provide only
+ * supported formats with this mbus code
+ */
+ if (f->mbus_code && f->mbus_code !=
+ isc->formats_list[i].mbus_code)
+ continue;
+ if (supported_index == index) {
+ f->pixelformat = isc->formats_list[i].fourcc;
+ return 0;
+ }
+ supported_index++;
}
- index -= isc->controller_formats_size;
+ /*
+ * If the sensor does not support this mbus_code whatsoever,
+ * there is no reason to advertise any of our output formats
+ */
+ if (supported_index == 0)
+ return -EINVAL;
+
+ /*
+ * If the sensor uses a format that is not raw, then we cannot
+ * convert it to any of the formats that we usually can with a
+ * RAW sensor. Thus, do not advertise them.
+ */
+ if (!isc->config.sd_format ||
+ !ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
+ return -EINVAL;
+ /*
+ * Iterate again through the formats that we can convert to.
+ * However, to avoid duplicates, skip the formats that
+ * the sensor already supports directly
+ */
+ index -= supported_index;
supported_index = 0;
- for (i = 0; i < isc->formats_list_size; i++) {
- if (!ISC_IS_FORMAT_RAW(isc->formats_list[i].mbus_code) ||
- !isc->formats_list[i].sd_support)
+ for (i = 0; i < isc->controller_formats_size; i++) {
+ /* if this format is already supported by sensor, skip it */
+ if (find_format_by_fourcc(isc, isc->controller_formats[i].fourcc))
continue;
if (supported_index == index) {
- f->pixelformat = isc->formats_list[i].fourcc;
+ f->pixelformat =
+ isc->controller_formats[i].fourcc;
return 0;
}
supported_index++;
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 11/23] media: atmel: atmel-isc-base: fix bytesperline value for planar formats
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
The bytesperline field of the pixfmt should be only for the first plane
in case of planar formats like YUV420 or YUV422.
The bytesperline is used by the driver to compute the framesize.
We have to report a different bpp (bytes per pixel) to v4l2 in bytesperline
than the actual bpp.
For example for YUV420, the real bpp is 12, but the first plane has only
8 bpp. Thus we report a bytesperline 8*width instead of 12*width.
However, for real framezise we have to compute 12*width*height.
Hence added a new variable to hold this information and to correctly
compute the frame size.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
---
drivers/media/platform/atmel/atmel-isc-base.c | 19 +++++++++++++++++--
drivers/media/platform/atmel/atmel-isc.h | 4 ++++
2 files changed, 21 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index 2cb8446ff90c..d0542b97a391 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -654,6 +654,7 @@ static int isc_try_configure_rlp_dma(struct isc_device *isc, bool direct_dump)
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED8;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 8;
+ isc->try_config.bpp_v4l2 = 8;
break;
case V4L2_PIX_FMT_SBGGR10:
case V4L2_PIX_FMT_SGBRG10:
@@ -663,6 +664,7 @@ static int isc_try_configure_rlp_dma(struct isc_device *isc, bool direct_dump)
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 16;
+ isc->try_config.bpp_v4l2 = 16;
break;
case V4L2_PIX_FMT_SBGGR12:
case V4L2_PIX_FMT_SGBRG12:
@@ -672,24 +674,28 @@ static int isc_try_configure_rlp_dma(struct isc_device *isc, bool direct_dump)
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 16;
+ isc->try_config.bpp_v4l2 = 16;
break;
case V4L2_PIX_FMT_RGB565:
isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_RGB565;
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 16;
+ isc->try_config.bpp_v4l2 = 16;
break;
case V4L2_PIX_FMT_ARGB444:
isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_ARGB444;
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 16;
+ isc->try_config.bpp_v4l2 = 16;
break;
case V4L2_PIX_FMT_ARGB555:
isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_ARGB555;
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 16;
+ isc->try_config.bpp_v4l2 = 16;
break;
case V4L2_PIX_FMT_ABGR32:
case V4L2_PIX_FMT_XBGR32:
@@ -697,42 +703,49 @@ static int isc_try_configure_rlp_dma(struct isc_device *isc, bool direct_dump)
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED32;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 32;
+ isc->try_config.bpp_v4l2 = 32;
break;
case V4L2_PIX_FMT_YUV420:
isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_YYCC;
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_YC420P;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PLANAR;
isc->try_config.bpp = 12;
+ isc->try_config.bpp_v4l2 = 8; /* only first plane */
break;
case V4L2_PIX_FMT_YUV422P:
isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_YYCC;
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_YC422P;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PLANAR;
isc->try_config.bpp = 16;
+ isc->try_config.bpp_v4l2 = 8; /* only first plane */
break;
case V4L2_PIX_FMT_YUYV:
isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_YCYC | ISC_RLP_CFG_YMODE_YUYV;
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED32;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 16;
+ isc->try_config.bpp_v4l2 = 16;
break;
case V4L2_PIX_FMT_UYVY:
isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_YCYC | ISC_RLP_CFG_YMODE_UYVY;
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED32;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 16;
+ isc->try_config.bpp_v4l2 = 16;
break;
case V4L2_PIX_FMT_VYUY:
isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_YCYC | ISC_RLP_CFG_YMODE_VYUY;
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED32;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 16;
+ isc->try_config.bpp_v4l2 = 16;
break;
case V4L2_PIX_FMT_GREY:
isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_DATY8;
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED8;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 8;
+ isc->try_config.bpp_v4l2 = 8;
break;
case V4L2_PIX_FMT_Y16:
isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_DATY10 | ISC_RLP_CFG_LSH;
@@ -742,6 +755,7 @@ static int isc_try_configure_rlp_dma(struct isc_device *isc, bool direct_dump)
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 16;
+ isc->try_config.bpp_v4l2 = 16;
break;
default:
return -EINVAL;
@@ -990,8 +1004,9 @@ static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f,
pixfmt->height = isc->max_height;
pixfmt->field = V4L2_FIELD_NONE;
- pixfmt->bytesperline = (pixfmt->width * isc->try_config.bpp) >> 3;
- pixfmt->sizeimage = pixfmt->bytesperline * pixfmt->height;
+ pixfmt->bytesperline = (pixfmt->width * isc->try_config.bpp_v4l2) >> 3;
+ pixfmt->sizeimage = ((pixfmt->width * isc->try_config.bpp) >> 3) *
+ pixfmt->height;
if (code)
*code = mbus_code;
diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
index 32448ccfc636..07fa6dbf8460 100644
--- a/drivers/media/platform/atmel/atmel-isc.h
+++ b/drivers/media/platform/atmel/atmel-isc.h
@@ -102,6 +102,9 @@ struct isc_format {
configuration.
* @fourcc: Fourcc code for this format.
* @bpp: Bytes per pixel in the current format.
+ * @bpp_v4l2: Bytes per pixel in the current format, for v4l2.
+ This differs from 'bpp' in the sense that in planar
+ formats, it refers only to the first plane.
* @rlp_cfg_mode: Configuration of the RLP (rounding, limiting packaging)
* @dcfg_imode: Configuration of the input of the DMA module
* @dctrl_dview: Configuration of the output of the DMA module
@@ -112,6 +115,7 @@ struct fmt_config {
u32 fourcc;
u8 bpp;
+ u8 bpp_v4l2;
u32 rlp_cfg_mode;
u32 dcfg_imode;
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 11/23] media: atmel: atmel-isc-base: fix bytesperline value for planar formats
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
The bytesperline field of the pixfmt should be only for the first plane
in case of planar formats like YUV420 or YUV422.
The bytesperline is used by the driver to compute the framesize.
We have to report a different bpp (bytes per pixel) to v4l2 in bytesperline
than the actual bpp.
For example for YUV420, the real bpp is 12, but the first plane has only
8 bpp. Thus we report a bytesperline 8*width instead of 12*width.
However, for real framezise we have to compute 12*width*height.
Hence added a new variable to hold this information and to correctly
compute the frame size.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
---
drivers/media/platform/atmel/atmel-isc-base.c | 19 +++++++++++++++++--
drivers/media/platform/atmel/atmel-isc.h | 4 ++++
2 files changed, 21 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index 2cb8446ff90c..d0542b97a391 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -654,6 +654,7 @@ static int isc_try_configure_rlp_dma(struct isc_device *isc, bool direct_dump)
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED8;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 8;
+ isc->try_config.bpp_v4l2 = 8;
break;
case V4L2_PIX_FMT_SBGGR10:
case V4L2_PIX_FMT_SGBRG10:
@@ -663,6 +664,7 @@ static int isc_try_configure_rlp_dma(struct isc_device *isc, bool direct_dump)
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 16;
+ isc->try_config.bpp_v4l2 = 16;
break;
case V4L2_PIX_FMT_SBGGR12:
case V4L2_PIX_FMT_SGBRG12:
@@ -672,24 +674,28 @@ static int isc_try_configure_rlp_dma(struct isc_device *isc, bool direct_dump)
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 16;
+ isc->try_config.bpp_v4l2 = 16;
break;
case V4L2_PIX_FMT_RGB565:
isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_RGB565;
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 16;
+ isc->try_config.bpp_v4l2 = 16;
break;
case V4L2_PIX_FMT_ARGB444:
isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_ARGB444;
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 16;
+ isc->try_config.bpp_v4l2 = 16;
break;
case V4L2_PIX_FMT_ARGB555:
isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_ARGB555;
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 16;
+ isc->try_config.bpp_v4l2 = 16;
break;
case V4L2_PIX_FMT_ABGR32:
case V4L2_PIX_FMT_XBGR32:
@@ -697,42 +703,49 @@ static int isc_try_configure_rlp_dma(struct isc_device *isc, bool direct_dump)
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED32;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 32;
+ isc->try_config.bpp_v4l2 = 32;
break;
case V4L2_PIX_FMT_YUV420:
isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_YYCC;
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_YC420P;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PLANAR;
isc->try_config.bpp = 12;
+ isc->try_config.bpp_v4l2 = 8; /* only first plane */
break;
case V4L2_PIX_FMT_YUV422P:
isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_YYCC;
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_YC422P;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PLANAR;
isc->try_config.bpp = 16;
+ isc->try_config.bpp_v4l2 = 8; /* only first plane */
break;
case V4L2_PIX_FMT_YUYV:
isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_YCYC | ISC_RLP_CFG_YMODE_YUYV;
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED32;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 16;
+ isc->try_config.bpp_v4l2 = 16;
break;
case V4L2_PIX_FMT_UYVY:
isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_YCYC | ISC_RLP_CFG_YMODE_UYVY;
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED32;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 16;
+ isc->try_config.bpp_v4l2 = 16;
break;
case V4L2_PIX_FMT_VYUY:
isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_YCYC | ISC_RLP_CFG_YMODE_VYUY;
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED32;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 16;
+ isc->try_config.bpp_v4l2 = 16;
break;
case V4L2_PIX_FMT_GREY:
isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_DATY8;
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED8;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 8;
+ isc->try_config.bpp_v4l2 = 8;
break;
case V4L2_PIX_FMT_Y16:
isc->try_config.rlp_cfg_mode = ISC_RLP_CFG_MODE_DATY10 | ISC_RLP_CFG_LSH;
@@ -742,6 +755,7 @@ static int isc_try_configure_rlp_dma(struct isc_device *isc, bool direct_dump)
isc->try_config.dcfg_imode = ISC_DCFG_IMODE_PACKED16;
isc->try_config.dctrl_dview = ISC_DCTRL_DVIEW_PACKED;
isc->try_config.bpp = 16;
+ isc->try_config.bpp_v4l2 = 16;
break;
default:
return -EINVAL;
@@ -990,8 +1004,9 @@ static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f,
pixfmt->height = isc->max_height;
pixfmt->field = V4L2_FIELD_NONE;
- pixfmt->bytesperline = (pixfmt->width * isc->try_config.bpp) >> 3;
- pixfmt->sizeimage = pixfmt->bytesperline * pixfmt->height;
+ pixfmt->bytesperline = (pixfmt->width * isc->try_config.bpp_v4l2) >> 3;
+ pixfmt->sizeimage = ((pixfmt->width * isc->try_config.bpp) >> 3) *
+ pixfmt->height;
if (code)
*code = mbus_code;
diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
index 32448ccfc636..07fa6dbf8460 100644
--- a/drivers/media/platform/atmel/atmel-isc.h
+++ b/drivers/media/platform/atmel/atmel-isc.h
@@ -102,6 +102,9 @@ struct isc_format {
configuration.
* @fourcc: Fourcc code for this format.
* @bpp: Bytes per pixel in the current format.
+ * @bpp_v4l2: Bytes per pixel in the current format, for v4l2.
+ This differs from 'bpp' in the sense that in planar
+ formats, it refers only to the first plane.
* @rlp_cfg_mode: Configuration of the RLP (rounding, limiting packaging)
* @dcfg_imode: Configuration of the input of the DMA module
* @dctrl_dview: Configuration of the output of the DMA module
@@ -112,6 +115,7 @@ struct fmt_config {
u32 fourcc;
u8 bpp;
+ u8 bpp_v4l2;
u32 rlp_cfg_mode;
u32 dcfg_imode;
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 12/23] media: atmel: atmel-isc: implement media controller
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
Implement the support for media-controller.
This means that the capabilities of the driver have changed and now
it also advertises the IO_MC .
The driver will register it's media device, and add the video entity to this
media device. The subdevices are registered to the same media device.
The ISC will have a base entity which is auto-detected as atmel_isc_base.
It will also register a subdevice that allows cropping of the incoming frame
to the maximum frame size supported by the ISC.
The ISC will create a link between the subdevice that is asynchronously
registered and the atmel_isc_scaler entity.
Then, the atmel_isc_scaler and atmel_isc_base are connected through another
link.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
Changes in v2:
- implement try formats
drivers/media/platform/atmel/Makefile | 2 +-
drivers/media/platform/atmel/atmel-isc-base.c | 27 +-
drivers/media/platform/atmel/atmel-isc-mc.c | 287 ++++++++++++++++++
drivers/media/platform/atmel/atmel-isc.h | 36 +++
.../media/platform/atmel/atmel-sama5d2-isc.c | 14 +-
.../media/platform/atmel/atmel-sama7g5-isc.c | 12 +-
6 files changed, 370 insertions(+), 8 deletions(-)
create mode 100644 drivers/media/platform/atmel/atmel-isc-mc.c
diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
index 794e8f739287..aa7a18b428f6 100644
--- a/drivers/media/platform/atmel/Makefile
+++ b/drivers/media/platform/atmel/Makefile
@@ -1,7 +1,7 @@
# SPDX-License-Identifier: GPL-2.0-only
atmel-isc-objs = atmel-sama5d2-isc.o
atmel-xisc-objs = atmel-sama7g5-isc.o
-atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o
+atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o atmel-isc-mc.o
obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-common.o
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index d0542b97a391..b0c3ed21f372 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -1726,6 +1726,7 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
struct isc_device, v4l2_dev);
struct isc_subdev_entity *subdev_entity =
container_of(notifier, struct isc_subdev_entity, notifier);
+ int pad;
if (video_is_registered(&isc->video_dev)) {
v4l2_err(&isc->v4l2_dev, "only supports one sub-device.\n");
@@ -1734,6 +1735,16 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
subdev_entity->sd = subdev;
+ pad = media_entity_get_fwnode_pad(&subdev->entity, asd->match.fwnode,
+ MEDIA_PAD_FL_SOURCE);
+ if (pad < 0) {
+ v4l2_err(&isc->v4l2_dev, "failed to find pad for %s\n",
+ subdev->name);
+ return pad;
+ }
+
+ isc->remote_pad = pad;
+
return 0;
}
@@ -1748,8 +1759,8 @@ static void isc_async_unbind(struct v4l2_async_notifier *notifier,
v4l2_ctrl_handler_free(&isc->ctrls.handler);
}
-static struct isc_format *find_format_by_code(struct isc_device *isc,
- unsigned int code, int *index)
+struct isc_format *isc_find_format_by_code(struct isc_device *isc,
+ unsigned int code, int *index)
{
struct isc_format *fmt = &isc->formats_list[0];
unsigned int i;
@@ -1765,6 +1776,7 @@ static struct isc_format *find_format_by_code(struct isc_device *isc,
return NULL;
}
+EXPORT_SYMBOL_GPL(isc_find_format_by_code);
static int isc_formats_init(struct isc_device *isc)
{
@@ -1781,7 +1793,7 @@ static int isc_formats_init(struct isc_device *isc)
NULL, &mbus_code)) {
mbus_code.index++;
- fmt = find_format_by_code(isc, mbus_code.code, &i);
+ fmt = isc_find_format_by_code(isc, mbus_code.code, &i);
if (!fmt) {
v4l2_warn(&isc->v4l2_dev, "Mbus code %x not supported\n",
mbus_code.code);
@@ -1909,7 +1921,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
vdev->queue = q;
vdev->lock = &isc->lock;
vdev->ctrl_handler = &isc->ctrls.handler;
- vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE;
+ vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE |
+ V4L2_CAP_IO_MC;
video_set_drvdata(vdev, isc);
ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
@@ -1919,8 +1932,14 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
goto isc_async_complete_err;
}
+ ret = isc_mc_register(isc);
+ if (ret < 0)
+ goto isc_async_complete_unregister_device;
return 0;
+isc_async_complete_unregister_device:
+ video_unregister_device(vdev);
+
isc_async_complete_err:
mutex_destroy(&isc->lock);
return ret;
diff --git a/drivers/media/platform/atmel/atmel-isc-mc.c b/drivers/media/platform/atmel/atmel-isc-mc.c
new file mode 100644
index 000000000000..2aea81d77b2f
--- /dev/null
+++ b/drivers/media/platform/atmel/atmel-isc-mc.c
@@ -0,0 +1,287 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Microchip Image Sensor Controller (ISC) Media Controller support
+ *
+ * Copyright (C) 2021 Microchip Technology, Inc.
+ *
+ * Author: Eugen Hristev <eugen.hristev@microchip.com>
+ *
+ */
+
+#include <media/v4l2-device.h>
+#include <media/v4l2-subdev.h>
+
+#include "atmel-isc-regs.h"
+#include "atmel-isc.h"
+
+static const struct media_device_ops isc_media_ops = {
+};
+
+static int isc_scaler_get_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_format *format)
+{
+ struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
+ struct v4l2_mbus_framefmt *v4l2_try_fmt;
+
+ if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
+ v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state,
+ format->pad);
+ format->format = *v4l2_try_fmt;
+
+ return 0;
+ }
+
+ format->format = isc->scaler_format;
+
+ return 0;
+}
+
+static int isc_scaler_set_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_format *req_fmt)
+{
+ struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
+ struct v4l2_mbus_framefmt *v4l2_try_fmt;
+ struct isc_format *fmt;
+ unsigned int i;
+
+ if (req_fmt->pad == ISC_SCALER_PAD_SOURCE)
+ v4l_bound_align_image
+ (&req_fmt->format.width, 16, isc->max_width, 0,
+ &req_fmt->format.height, 16, isc->max_height, 0, 0);
+ else
+ v4l_bound_align_image
+ (&req_fmt->format.width, 16, 10000, 0,
+ &req_fmt->format.height, 16, 10000, 0, 0);
+
+ req_fmt->format.colorspace = V4L2_COLORSPACE_SRGB;
+ req_fmt->format.field = V4L2_FIELD_NONE;
+ req_fmt->format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ req_fmt->format.quantization = V4L2_QUANTIZATION_DEFAULT;
+ req_fmt->format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
+
+ fmt = isc_find_format_by_code(isc, req_fmt->format.code, &i);
+
+ if (!fmt)
+ fmt = &isc->formats_list[0];
+
+ req_fmt->format.code = fmt->mbus_code;
+
+ if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
+ v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state,
+ req_fmt->pad);
+ *v4l2_try_fmt = req_fmt->format;
+ /* Trying on the pad sink makes the source sink change too */
+ if (req_fmt->pad == ISC_SCALER_PAD_SINK) {
+ v4l2_try_fmt =
+ v4l2_subdev_get_try_format(sd, sd_state,
+ ISC_SCALER_PAD_SOURCE);
+ *v4l2_try_fmt = req_fmt->format;
+
+ v4l_bound_align_image(&v4l2_try_fmt->width,
+ 16, isc->max_width, 0,
+ &v4l2_try_fmt->height,
+ 16, isc->max_height, 0, 0);
+ }
+ /* if we are just trying, we are done */
+ return 0;
+ }
+
+ isc->scaler_format = req_fmt->format;
+
+ return 0;
+}
+
+static int isc_scaler_enum_mbus_code(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
+ int supported_index = 0;
+ int i;
+
+ for (i = 0; i < isc->formats_list_size; i++) {
+ if (!isc->formats_list[i].sd_support)
+ continue;
+ if (supported_index == code->index) {
+ code->code = isc->formats_list[i].mbus_code;
+ return 0;
+ }
+ supported_index++;
+ }
+
+ return -EINVAL;
+}
+
+static int isc_scaler_g_sel(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_selection *sel)
+{
+ struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
+
+ if (sel->pad == ISC_SCALER_PAD_SOURCE)
+ return -EINVAL;
+
+ if (sel->target != V4L2_SEL_TGT_CROP_BOUNDS &&
+ sel->target != V4L2_SEL_TGT_CROP)
+ return -EINVAL;
+
+ sel->r.height = isc->max_height;
+ sel->r.width = isc->max_width;
+
+ sel->r.left = 0;
+ sel->r.top = 0;
+
+ return 0;
+}
+
+static int isc_scaler_init_cfg(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state)
+{
+ struct v4l2_mbus_framefmt *v4l2_try_fmt =
+ v4l2_subdev_get_try_format(sd, sd_state, 0);
+ struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
+
+ *v4l2_try_fmt = isc->scaler_format;
+
+ return 0;
+}
+
+static const struct v4l2_subdev_pad_ops isc_scaler_pad_ops = {
+ .enum_mbus_code = isc_scaler_enum_mbus_code,
+ .set_fmt = isc_scaler_set_fmt,
+ .get_fmt = isc_scaler_get_fmt,
+ .get_selection = isc_scaler_g_sel,
+ .init_cfg = isc_scaler_init_cfg,
+};
+
+static const struct v4l2_subdev_ops xisc_scaler_subdev_ops = {
+ .pad = &isc_scaler_pad_ops,
+};
+
+static int isc_init_own_sd(struct isc_device *isc)
+{
+ int ret;
+
+ v4l2_subdev_init(&isc->scaler_sd, &xisc_scaler_subdev_ops);
+
+ isc->scaler_sd.owner = THIS_MODULE;
+ isc->scaler_sd.dev = isc->dev;
+ snprintf(isc->scaler_sd.name, sizeof(isc->scaler_sd.name),
+ "atmel_isc_scaler");
+
+ isc->scaler_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+ isc->scaler_sd.entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
+ isc->scaler_pads[ISC_SCALER_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+ isc->scaler_pads[ISC_SCALER_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
+
+ isc->scaler_format.height = isc->max_height;
+ isc->scaler_format.width = isc->max_width;
+ isc->scaler_format.code = isc->formats_list[0].mbus_code;
+ isc->scaler_format.colorspace = V4L2_COLORSPACE_SRGB;
+ isc->scaler_format.field = V4L2_FIELD_NONE;
+ isc->scaler_format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ isc->scaler_format.quantization = V4L2_QUANTIZATION_DEFAULT;
+ isc->scaler_format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
+
+ ret = media_entity_pads_init(&isc->scaler_sd.entity,
+ ISC_SCALER_PADS_NUM,
+ isc->scaler_pads);
+ if (ret < 0) {
+ dev_err(isc->dev, "scaler sd media entity init failed\n");
+ return ret;
+ }
+ ret = v4l2_device_register_subdev(&isc->v4l2_dev, &isc->scaler_sd);
+ if (ret < 0) {
+ dev_err(isc->dev, "scaler sd failed to register subdev\n");
+ return ret;
+ }
+
+ return ret;
+}
+
+int isc_mc_init(struct isc_device *isc, u32 ver)
+{
+ const struct of_device_id *match;
+ int ret;
+
+ isc->video_dev.entity.function = MEDIA_ENT_F_IO_V4L;
+ isc->video_dev.entity.flags = MEDIA_ENT_FL_DEFAULT;
+ isc->pads[ISC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+
+ ret = media_entity_pads_init(&isc->video_dev.entity, ISC_PADS_NUM,
+ isc->pads);
+ if (ret < 0) {
+ dev_err(isc->dev, "media entity init failed\n");
+ return ret;
+ }
+
+ isc->mdev.dev = isc->dev;
+ isc->mdev.ops = &isc_media_ops;
+
+ match = of_match_node(isc->dev->driver->of_match_table,
+ isc->dev->of_node);
+
+ strscpy(isc->mdev.driver_name, KBUILD_MODNAME,
+ sizeof(isc->mdev.driver_name));
+ strscpy(isc->mdev.model, match->compatible, sizeof(isc->mdev.model));
+ snprintf(isc->mdev.bus_info, sizeof(isc->mdev.bus_info), "platform:%s",
+ isc->v4l2_dev.name);
+ isc->mdev.hw_revision = ver;
+
+ media_device_init(&isc->mdev);
+
+ isc->v4l2_dev.mdev = &isc->mdev;
+
+ return isc_init_own_sd(isc);
+}
+EXPORT_SYMBOL_GPL(isc_mc_init);
+
+int isc_mc_register(struct isc_device *isc)
+{
+ int ret;
+
+ ret = media_create_pad_link(&isc->current_subdev->sd->entity,
+ isc->remote_pad, &isc->scaler_sd.entity,
+ ISC_SCALER_PAD_SINK,
+ MEDIA_LNK_FL_ENABLED |
+ MEDIA_LNK_FL_IMMUTABLE);
+
+ if (ret < 0) {
+ v4l2_err(&isc->v4l2_dev,
+ "Failed to create pad link: %s to %s\n",
+ isc->current_subdev->sd->entity.name,
+ isc->scaler_sd.entity.name);
+ return ret;
+ }
+
+ dev_dbg(isc->dev, "link with %s pad: %d\n",
+ isc->current_subdev->sd->name, isc->remote_pad);
+
+ ret = media_create_pad_link(&isc->scaler_sd.entity,
+ ISC_SCALER_PAD_SOURCE,
+ &isc->video_dev.entity, ISC_PAD_SINK,
+ MEDIA_LNK_FL_ENABLED |
+ MEDIA_LNK_FL_IMMUTABLE);
+
+ if (ret < 0) {
+ v4l2_err(&isc->v4l2_dev,
+ "Failed to create pad link: %s to %s\n",
+ isc->scaler_sd.entity.name,
+ isc->video_dev.entity.name);
+ return ret;
+ }
+
+ dev_dbg(isc->dev, "link with %s pad: %d\n", isc->scaler_sd.name,
+ ISC_SCALER_PAD_SOURCE);
+
+ return media_device_register(&isc->mdev);
+}
+EXPORT_SYMBOL_GPL(isc_mc_register);
+
+void isc_mc_cleanup(struct isc_device *isc)
+{
+ media_entity_cleanup(&isc->video_dev.entity);
+}
+EXPORT_SYMBOL_GPL(isc_mc_cleanup);
diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
index 07fa6dbf8460..0b6370d7775f 100644
--- a/drivers/media/platform/atmel/atmel-isc.h
+++ b/drivers/media/platform/atmel/atmel-isc.h
@@ -183,6 +183,17 @@ struct isc_reg_offsets {
u32 his_entry;
};
+enum isc_mc_pads {
+ ISC_PAD_SINK = 0,
+ ISC_PADS_NUM = 1,
+};
+
+enum isc_scaler_pads {
+ ISC_SCALER_PAD_SINK = 0,
+ ISC_SCALER_PAD_SOURCE = 1,
+ ISC_SCALER_PADS_NUM = 2,
+};
+
/*
* struct isc_device - ISC device driver data/config struct
* @regmap: Register map
@@ -258,6 +269,12 @@ struct isc_reg_offsets {
* be used as an input to the controller
* @controller_formats_size: size of controller_formats array
* @formats_list_size: size of formats_list array
+ * @pads: media controller pads for isc video entity
+ * @mdev: media device that is registered by the isc
+ * @remote_pad: remote pad on the connected subdevice
+ * @scaler_sd: subdevice for the scaler that isc registers
+ * @scaler_pads: media controller pads for the scaler subdevice
+ * @scaler_format: current format for the scaler subdevice
*/
struct isc_device {
struct regmap *regmap;
@@ -346,6 +363,19 @@ struct isc_device {
struct isc_format *formats_list;
u32 controller_formats_size;
u32 formats_list_size;
+
+ struct {
+ struct media_pad pads[ISC_PADS_NUM];
+ struct media_device mdev;
+
+ u32 remote_pad;
+ };
+
+ struct {
+ struct v4l2_subdev scaler_sd;
+ struct media_pad scaler_pads[ISC_SCALER_PADS_NUM];
+ struct v4l2_mbus_framefmt scaler_format;
+ };
};
extern const struct regmap_config isc_regmap_config;
@@ -357,4 +387,10 @@ int isc_clk_init(struct isc_device *isc);
void isc_subdev_cleanup(struct isc_device *isc);
void isc_clk_cleanup(struct isc_device *isc);
+int isc_mc_init(struct isc_device *isc, u32 ver);
+int isc_mc_register(struct isc_device *isc);
+void isc_mc_cleanup(struct isc_device *isc);
+
+struct isc_format *isc_find_format_by_code(struct isc_device *isc,
+ unsigned int code, int *index);
#endif
diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
index 1b2063cce0f7..a3e4c32a66c4 100644
--- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
@@ -529,6 +529,12 @@ static int atmel_isc_probe(struct platform_device *pdev)
break;
}
+ regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
+
+ ret = isc_mc_init(isc, ver);
+ if (ret < 0)
+ goto isc_probe_mc_init_err;
+
pm_runtime_set_active(dev);
pm_runtime_enable(dev);
pm_request_idle(dev);
@@ -538,7 +544,7 @@ static int atmel_isc_probe(struct platform_device *pdev)
ret = clk_prepare_enable(isc->ispck);
if (ret) {
dev_err(dev, "failed to enable ispck: %d\n", ret);
- goto cleanup_subdev;
+ goto isc_probe_mc_init_err;
}
/* ispck should be greater or equal to hclock */
@@ -548,7 +554,6 @@ static int atmel_isc_probe(struct platform_device *pdev)
goto unprepare_clk;
}
- regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
dev_info(dev, "Microchip ISC version %x\n", ver);
return 0;
@@ -556,6 +561,9 @@ static int atmel_isc_probe(struct platform_device *pdev)
unprepare_clk:
clk_disable_unprepare(isc->ispck);
+isc_probe_mc_init_err:
+ isc_mc_cleanup(isc);
+
cleanup_subdev:
isc_subdev_cleanup(isc);
@@ -576,6 +584,8 @@ static int atmel_isc_remove(struct platform_device *pdev)
pm_runtime_disable(&pdev->dev);
+ isc_mc_cleanup(isc);
+
isc_subdev_cleanup(isc);
v4l2_device_unregister(&isc->v4l2_dev);
diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
index 2b1082295c13..d2ac80b1ccc3 100644
--- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
@@ -523,15 +523,23 @@ static int microchip_xisc_probe(struct platform_device *pdev)
break;
}
+ regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
+
+ ret = isc_mc_init(isc, ver);
+ if (ret < 0)
+ goto isc_probe_mc_init_err;
+
pm_runtime_set_active(dev);
pm_runtime_enable(dev);
pm_request_idle(dev);
- regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
dev_info(dev, "Microchip XISC version %x\n", ver);
return 0;
+isc_probe_mc_init_err:
+ isc_mc_cleanup(isc);
+
cleanup_subdev:
isc_subdev_cleanup(isc);
@@ -552,6 +560,8 @@ static int microchip_xisc_remove(struct platform_device *pdev)
pm_runtime_disable(&pdev->dev);
+ isc_mc_cleanup(isc);
+
isc_subdev_cleanup(isc);
v4l2_device_unregister(&isc->v4l2_dev);
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 12/23] media: atmel: atmel-isc: implement media controller
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
Implement the support for media-controller.
This means that the capabilities of the driver have changed and now
it also advertises the IO_MC .
The driver will register it's media device, and add the video entity to this
media device. The subdevices are registered to the same media device.
The ISC will have a base entity which is auto-detected as atmel_isc_base.
It will also register a subdevice that allows cropping of the incoming frame
to the maximum frame size supported by the ISC.
The ISC will create a link between the subdevice that is asynchronously
registered and the atmel_isc_scaler entity.
Then, the atmel_isc_scaler and atmel_isc_base are connected through another
link.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
Changes in v2:
- implement try formats
drivers/media/platform/atmel/Makefile | 2 +-
drivers/media/platform/atmel/atmel-isc-base.c | 27 +-
drivers/media/platform/atmel/atmel-isc-mc.c | 287 ++++++++++++++++++
drivers/media/platform/atmel/atmel-isc.h | 36 +++
.../media/platform/atmel/atmel-sama5d2-isc.c | 14 +-
.../media/platform/atmel/atmel-sama7g5-isc.c | 12 +-
6 files changed, 370 insertions(+), 8 deletions(-)
create mode 100644 drivers/media/platform/atmel/atmel-isc-mc.c
diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
index 794e8f739287..aa7a18b428f6 100644
--- a/drivers/media/platform/atmel/Makefile
+++ b/drivers/media/platform/atmel/Makefile
@@ -1,7 +1,7 @@
# SPDX-License-Identifier: GPL-2.0-only
atmel-isc-objs = atmel-sama5d2-isc.o
atmel-xisc-objs = atmel-sama7g5-isc.o
-atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o
+atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o atmel-isc-mc.o
obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-common.o
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index d0542b97a391..b0c3ed21f372 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -1726,6 +1726,7 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
struct isc_device, v4l2_dev);
struct isc_subdev_entity *subdev_entity =
container_of(notifier, struct isc_subdev_entity, notifier);
+ int pad;
if (video_is_registered(&isc->video_dev)) {
v4l2_err(&isc->v4l2_dev, "only supports one sub-device.\n");
@@ -1734,6 +1735,16 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
subdev_entity->sd = subdev;
+ pad = media_entity_get_fwnode_pad(&subdev->entity, asd->match.fwnode,
+ MEDIA_PAD_FL_SOURCE);
+ if (pad < 0) {
+ v4l2_err(&isc->v4l2_dev, "failed to find pad for %s\n",
+ subdev->name);
+ return pad;
+ }
+
+ isc->remote_pad = pad;
+
return 0;
}
@@ -1748,8 +1759,8 @@ static void isc_async_unbind(struct v4l2_async_notifier *notifier,
v4l2_ctrl_handler_free(&isc->ctrls.handler);
}
-static struct isc_format *find_format_by_code(struct isc_device *isc,
- unsigned int code, int *index)
+struct isc_format *isc_find_format_by_code(struct isc_device *isc,
+ unsigned int code, int *index)
{
struct isc_format *fmt = &isc->formats_list[0];
unsigned int i;
@@ -1765,6 +1776,7 @@ static struct isc_format *find_format_by_code(struct isc_device *isc,
return NULL;
}
+EXPORT_SYMBOL_GPL(isc_find_format_by_code);
static int isc_formats_init(struct isc_device *isc)
{
@@ -1781,7 +1793,7 @@ static int isc_formats_init(struct isc_device *isc)
NULL, &mbus_code)) {
mbus_code.index++;
- fmt = find_format_by_code(isc, mbus_code.code, &i);
+ fmt = isc_find_format_by_code(isc, mbus_code.code, &i);
if (!fmt) {
v4l2_warn(&isc->v4l2_dev, "Mbus code %x not supported\n",
mbus_code.code);
@@ -1909,7 +1921,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
vdev->queue = q;
vdev->lock = &isc->lock;
vdev->ctrl_handler = &isc->ctrls.handler;
- vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE;
+ vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE |
+ V4L2_CAP_IO_MC;
video_set_drvdata(vdev, isc);
ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
@@ -1919,8 +1932,14 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
goto isc_async_complete_err;
}
+ ret = isc_mc_register(isc);
+ if (ret < 0)
+ goto isc_async_complete_unregister_device;
return 0;
+isc_async_complete_unregister_device:
+ video_unregister_device(vdev);
+
isc_async_complete_err:
mutex_destroy(&isc->lock);
return ret;
diff --git a/drivers/media/platform/atmel/atmel-isc-mc.c b/drivers/media/platform/atmel/atmel-isc-mc.c
new file mode 100644
index 000000000000..2aea81d77b2f
--- /dev/null
+++ b/drivers/media/platform/atmel/atmel-isc-mc.c
@@ -0,0 +1,287 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Microchip Image Sensor Controller (ISC) Media Controller support
+ *
+ * Copyright (C) 2021 Microchip Technology, Inc.
+ *
+ * Author: Eugen Hristev <eugen.hristev@microchip.com>
+ *
+ */
+
+#include <media/v4l2-device.h>
+#include <media/v4l2-subdev.h>
+
+#include "atmel-isc-regs.h"
+#include "atmel-isc.h"
+
+static const struct media_device_ops isc_media_ops = {
+};
+
+static int isc_scaler_get_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_format *format)
+{
+ struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
+ struct v4l2_mbus_framefmt *v4l2_try_fmt;
+
+ if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
+ v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state,
+ format->pad);
+ format->format = *v4l2_try_fmt;
+
+ return 0;
+ }
+
+ format->format = isc->scaler_format;
+
+ return 0;
+}
+
+static int isc_scaler_set_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_format *req_fmt)
+{
+ struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
+ struct v4l2_mbus_framefmt *v4l2_try_fmt;
+ struct isc_format *fmt;
+ unsigned int i;
+
+ if (req_fmt->pad == ISC_SCALER_PAD_SOURCE)
+ v4l_bound_align_image
+ (&req_fmt->format.width, 16, isc->max_width, 0,
+ &req_fmt->format.height, 16, isc->max_height, 0, 0);
+ else
+ v4l_bound_align_image
+ (&req_fmt->format.width, 16, 10000, 0,
+ &req_fmt->format.height, 16, 10000, 0, 0);
+
+ req_fmt->format.colorspace = V4L2_COLORSPACE_SRGB;
+ req_fmt->format.field = V4L2_FIELD_NONE;
+ req_fmt->format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ req_fmt->format.quantization = V4L2_QUANTIZATION_DEFAULT;
+ req_fmt->format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
+
+ fmt = isc_find_format_by_code(isc, req_fmt->format.code, &i);
+
+ if (!fmt)
+ fmt = &isc->formats_list[0];
+
+ req_fmt->format.code = fmt->mbus_code;
+
+ if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
+ v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state,
+ req_fmt->pad);
+ *v4l2_try_fmt = req_fmt->format;
+ /* Trying on the pad sink makes the source sink change too */
+ if (req_fmt->pad == ISC_SCALER_PAD_SINK) {
+ v4l2_try_fmt =
+ v4l2_subdev_get_try_format(sd, sd_state,
+ ISC_SCALER_PAD_SOURCE);
+ *v4l2_try_fmt = req_fmt->format;
+
+ v4l_bound_align_image(&v4l2_try_fmt->width,
+ 16, isc->max_width, 0,
+ &v4l2_try_fmt->height,
+ 16, isc->max_height, 0, 0);
+ }
+ /* if we are just trying, we are done */
+ return 0;
+ }
+
+ isc->scaler_format = req_fmt->format;
+
+ return 0;
+}
+
+static int isc_scaler_enum_mbus_code(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
+ int supported_index = 0;
+ int i;
+
+ for (i = 0; i < isc->formats_list_size; i++) {
+ if (!isc->formats_list[i].sd_support)
+ continue;
+ if (supported_index == code->index) {
+ code->code = isc->formats_list[i].mbus_code;
+ return 0;
+ }
+ supported_index++;
+ }
+
+ return -EINVAL;
+}
+
+static int isc_scaler_g_sel(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_selection *sel)
+{
+ struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
+
+ if (sel->pad == ISC_SCALER_PAD_SOURCE)
+ return -EINVAL;
+
+ if (sel->target != V4L2_SEL_TGT_CROP_BOUNDS &&
+ sel->target != V4L2_SEL_TGT_CROP)
+ return -EINVAL;
+
+ sel->r.height = isc->max_height;
+ sel->r.width = isc->max_width;
+
+ sel->r.left = 0;
+ sel->r.top = 0;
+
+ return 0;
+}
+
+static int isc_scaler_init_cfg(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state)
+{
+ struct v4l2_mbus_framefmt *v4l2_try_fmt =
+ v4l2_subdev_get_try_format(sd, sd_state, 0);
+ struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
+
+ *v4l2_try_fmt = isc->scaler_format;
+
+ return 0;
+}
+
+static const struct v4l2_subdev_pad_ops isc_scaler_pad_ops = {
+ .enum_mbus_code = isc_scaler_enum_mbus_code,
+ .set_fmt = isc_scaler_set_fmt,
+ .get_fmt = isc_scaler_get_fmt,
+ .get_selection = isc_scaler_g_sel,
+ .init_cfg = isc_scaler_init_cfg,
+};
+
+static const struct v4l2_subdev_ops xisc_scaler_subdev_ops = {
+ .pad = &isc_scaler_pad_ops,
+};
+
+static int isc_init_own_sd(struct isc_device *isc)
+{
+ int ret;
+
+ v4l2_subdev_init(&isc->scaler_sd, &xisc_scaler_subdev_ops);
+
+ isc->scaler_sd.owner = THIS_MODULE;
+ isc->scaler_sd.dev = isc->dev;
+ snprintf(isc->scaler_sd.name, sizeof(isc->scaler_sd.name),
+ "atmel_isc_scaler");
+
+ isc->scaler_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+ isc->scaler_sd.entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
+ isc->scaler_pads[ISC_SCALER_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+ isc->scaler_pads[ISC_SCALER_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
+
+ isc->scaler_format.height = isc->max_height;
+ isc->scaler_format.width = isc->max_width;
+ isc->scaler_format.code = isc->formats_list[0].mbus_code;
+ isc->scaler_format.colorspace = V4L2_COLORSPACE_SRGB;
+ isc->scaler_format.field = V4L2_FIELD_NONE;
+ isc->scaler_format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ isc->scaler_format.quantization = V4L2_QUANTIZATION_DEFAULT;
+ isc->scaler_format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
+
+ ret = media_entity_pads_init(&isc->scaler_sd.entity,
+ ISC_SCALER_PADS_NUM,
+ isc->scaler_pads);
+ if (ret < 0) {
+ dev_err(isc->dev, "scaler sd media entity init failed\n");
+ return ret;
+ }
+ ret = v4l2_device_register_subdev(&isc->v4l2_dev, &isc->scaler_sd);
+ if (ret < 0) {
+ dev_err(isc->dev, "scaler sd failed to register subdev\n");
+ return ret;
+ }
+
+ return ret;
+}
+
+int isc_mc_init(struct isc_device *isc, u32 ver)
+{
+ const struct of_device_id *match;
+ int ret;
+
+ isc->video_dev.entity.function = MEDIA_ENT_F_IO_V4L;
+ isc->video_dev.entity.flags = MEDIA_ENT_FL_DEFAULT;
+ isc->pads[ISC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+
+ ret = media_entity_pads_init(&isc->video_dev.entity, ISC_PADS_NUM,
+ isc->pads);
+ if (ret < 0) {
+ dev_err(isc->dev, "media entity init failed\n");
+ return ret;
+ }
+
+ isc->mdev.dev = isc->dev;
+ isc->mdev.ops = &isc_media_ops;
+
+ match = of_match_node(isc->dev->driver->of_match_table,
+ isc->dev->of_node);
+
+ strscpy(isc->mdev.driver_name, KBUILD_MODNAME,
+ sizeof(isc->mdev.driver_name));
+ strscpy(isc->mdev.model, match->compatible, sizeof(isc->mdev.model));
+ snprintf(isc->mdev.bus_info, sizeof(isc->mdev.bus_info), "platform:%s",
+ isc->v4l2_dev.name);
+ isc->mdev.hw_revision = ver;
+
+ media_device_init(&isc->mdev);
+
+ isc->v4l2_dev.mdev = &isc->mdev;
+
+ return isc_init_own_sd(isc);
+}
+EXPORT_SYMBOL_GPL(isc_mc_init);
+
+int isc_mc_register(struct isc_device *isc)
+{
+ int ret;
+
+ ret = media_create_pad_link(&isc->current_subdev->sd->entity,
+ isc->remote_pad, &isc->scaler_sd.entity,
+ ISC_SCALER_PAD_SINK,
+ MEDIA_LNK_FL_ENABLED |
+ MEDIA_LNK_FL_IMMUTABLE);
+
+ if (ret < 0) {
+ v4l2_err(&isc->v4l2_dev,
+ "Failed to create pad link: %s to %s\n",
+ isc->current_subdev->sd->entity.name,
+ isc->scaler_sd.entity.name);
+ return ret;
+ }
+
+ dev_dbg(isc->dev, "link with %s pad: %d\n",
+ isc->current_subdev->sd->name, isc->remote_pad);
+
+ ret = media_create_pad_link(&isc->scaler_sd.entity,
+ ISC_SCALER_PAD_SOURCE,
+ &isc->video_dev.entity, ISC_PAD_SINK,
+ MEDIA_LNK_FL_ENABLED |
+ MEDIA_LNK_FL_IMMUTABLE);
+
+ if (ret < 0) {
+ v4l2_err(&isc->v4l2_dev,
+ "Failed to create pad link: %s to %s\n",
+ isc->scaler_sd.entity.name,
+ isc->video_dev.entity.name);
+ return ret;
+ }
+
+ dev_dbg(isc->dev, "link with %s pad: %d\n", isc->scaler_sd.name,
+ ISC_SCALER_PAD_SOURCE);
+
+ return media_device_register(&isc->mdev);
+}
+EXPORT_SYMBOL_GPL(isc_mc_register);
+
+void isc_mc_cleanup(struct isc_device *isc)
+{
+ media_entity_cleanup(&isc->video_dev.entity);
+}
+EXPORT_SYMBOL_GPL(isc_mc_cleanup);
diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
index 07fa6dbf8460..0b6370d7775f 100644
--- a/drivers/media/platform/atmel/atmel-isc.h
+++ b/drivers/media/platform/atmel/atmel-isc.h
@@ -183,6 +183,17 @@ struct isc_reg_offsets {
u32 his_entry;
};
+enum isc_mc_pads {
+ ISC_PAD_SINK = 0,
+ ISC_PADS_NUM = 1,
+};
+
+enum isc_scaler_pads {
+ ISC_SCALER_PAD_SINK = 0,
+ ISC_SCALER_PAD_SOURCE = 1,
+ ISC_SCALER_PADS_NUM = 2,
+};
+
/*
* struct isc_device - ISC device driver data/config struct
* @regmap: Register map
@@ -258,6 +269,12 @@ struct isc_reg_offsets {
* be used as an input to the controller
* @controller_formats_size: size of controller_formats array
* @formats_list_size: size of formats_list array
+ * @pads: media controller pads for isc video entity
+ * @mdev: media device that is registered by the isc
+ * @remote_pad: remote pad on the connected subdevice
+ * @scaler_sd: subdevice for the scaler that isc registers
+ * @scaler_pads: media controller pads for the scaler subdevice
+ * @scaler_format: current format for the scaler subdevice
*/
struct isc_device {
struct regmap *regmap;
@@ -346,6 +363,19 @@ struct isc_device {
struct isc_format *formats_list;
u32 controller_formats_size;
u32 formats_list_size;
+
+ struct {
+ struct media_pad pads[ISC_PADS_NUM];
+ struct media_device mdev;
+
+ u32 remote_pad;
+ };
+
+ struct {
+ struct v4l2_subdev scaler_sd;
+ struct media_pad scaler_pads[ISC_SCALER_PADS_NUM];
+ struct v4l2_mbus_framefmt scaler_format;
+ };
};
extern const struct regmap_config isc_regmap_config;
@@ -357,4 +387,10 @@ int isc_clk_init(struct isc_device *isc);
void isc_subdev_cleanup(struct isc_device *isc);
void isc_clk_cleanup(struct isc_device *isc);
+int isc_mc_init(struct isc_device *isc, u32 ver);
+int isc_mc_register(struct isc_device *isc);
+void isc_mc_cleanup(struct isc_device *isc);
+
+struct isc_format *isc_find_format_by_code(struct isc_device *isc,
+ unsigned int code, int *index);
#endif
diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
index 1b2063cce0f7..a3e4c32a66c4 100644
--- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
@@ -529,6 +529,12 @@ static int atmel_isc_probe(struct platform_device *pdev)
break;
}
+ regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
+
+ ret = isc_mc_init(isc, ver);
+ if (ret < 0)
+ goto isc_probe_mc_init_err;
+
pm_runtime_set_active(dev);
pm_runtime_enable(dev);
pm_request_idle(dev);
@@ -538,7 +544,7 @@ static int atmel_isc_probe(struct platform_device *pdev)
ret = clk_prepare_enable(isc->ispck);
if (ret) {
dev_err(dev, "failed to enable ispck: %d\n", ret);
- goto cleanup_subdev;
+ goto isc_probe_mc_init_err;
}
/* ispck should be greater or equal to hclock */
@@ -548,7 +554,6 @@ static int atmel_isc_probe(struct platform_device *pdev)
goto unprepare_clk;
}
- regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
dev_info(dev, "Microchip ISC version %x\n", ver);
return 0;
@@ -556,6 +561,9 @@ static int atmel_isc_probe(struct platform_device *pdev)
unprepare_clk:
clk_disable_unprepare(isc->ispck);
+isc_probe_mc_init_err:
+ isc_mc_cleanup(isc);
+
cleanup_subdev:
isc_subdev_cleanup(isc);
@@ -576,6 +584,8 @@ static int atmel_isc_remove(struct platform_device *pdev)
pm_runtime_disable(&pdev->dev);
+ isc_mc_cleanup(isc);
+
isc_subdev_cleanup(isc);
v4l2_device_unregister(&isc->v4l2_dev);
diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
index 2b1082295c13..d2ac80b1ccc3 100644
--- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
@@ -523,15 +523,23 @@ static int microchip_xisc_probe(struct platform_device *pdev)
break;
}
+ regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
+
+ ret = isc_mc_init(isc, ver);
+ if (ret < 0)
+ goto isc_probe_mc_init_err;
+
pm_runtime_set_active(dev);
pm_runtime_enable(dev);
pm_request_idle(dev);
- regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
dev_info(dev, "Microchip XISC version %x\n", ver);
return 0;
+isc_probe_mc_init_err:
+ isc_mc_cleanup(isc);
+
cleanup_subdev:
isc_subdev_cleanup(isc);
@@ -552,6 +560,8 @@ static int microchip_xisc_remove(struct platform_device *pdev)
pm_runtime_disable(&pdev->dev);
+ isc_mc_cleanup(isc);
+
isc_subdev_cleanup(isc);
v4l2_device_unregister(&isc->v4l2_dev);
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 13/23] ARM: dts: at91: sama7g5: add nodes for video capture
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
Add node for the XISC (eXtended Image Sensor Controller) and CSI2DC
(csi2 demux controller).
These nodes represent the top level of the video capture hardware pipeline
and are directly connected in hardware.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
Changes in v3:
- change bus width for endpoints to the default 14
arch/arm/boot/dts/sama7g5.dtsi | 48 ++++++++++++++++++++++++++++++++++
1 file changed, 48 insertions(+)
diff --git a/arch/arm/boot/dts/sama7g5.dtsi b/arch/arm/boot/dts/sama7g5.dtsi
index 7039311bf678..4d0a93541d5f 100644
--- a/arch/arm/boot/dts/sama7g5.dtsi
+++ b/arch/arm/boot/dts/sama7g5.dtsi
@@ -236,6 +236,54 @@ sdmmc2: mmc@e120c000 {
status = "disabled";
};
+ csi2dc: csi2dc@e1404000 {
+ compatible = "microchip,sama7g5-csi2dc";
+ reg = <0xe1404000 0x500>;
+ clocks = <&pmc PMC_TYPE_PERIPHERAL 34>, <&xisc>;
+ clock-names = "pclk", "scck";
+ assigned-clocks = <&xisc>;
+ assigned-clock-rates = <266000000>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ port@0 {
+ reg = <0>;
+ csi2dc_in: endpoint {
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ csi2dc_out: endpoint {
+ bus-width = <14>;
+ hsync-active = <1>;
+ vsync-active = <1>;
+ remote-endpoint = <&xisc_in>;
+ };
+ };
+ };
+ };
+
+ xisc: xisc@e1408000 {
+ compatible = "microchip,sama7g5-isc";
+ reg = <0xe1408000 0x2000>;
+ interrupts = <GIC_SPI 56 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&pmc PMC_TYPE_PERIPHERAL 56>;
+ clock-names = "hclock";
+ #clock-cells = <0>;
+ clock-output-names = "isc-mck";
+
+ port {
+ xisc_in: endpoint {
+ bus-width = <14>;
+ hsync-active = <1>;
+ vsync-active = <1>;
+ remote-endpoint = <&csi2dc_out>;
+ };
+ };
+ };
+
pwm: pwm@e1604000 {
compatible = "microchip,sama7g5-pwm", "atmel,sama5d2-pwm";
reg = <0xe1604000 0x4000>;
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 13/23] ARM: dts: at91: sama7g5: add nodes for video capture
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
Add node for the XISC (eXtended Image Sensor Controller) and CSI2DC
(csi2 demux controller).
These nodes represent the top level of the video capture hardware pipeline
and are directly connected in hardware.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
Changes in v3:
- change bus width for endpoints to the default 14
arch/arm/boot/dts/sama7g5.dtsi | 48 ++++++++++++++++++++++++++++++++++
1 file changed, 48 insertions(+)
diff --git a/arch/arm/boot/dts/sama7g5.dtsi b/arch/arm/boot/dts/sama7g5.dtsi
index 7039311bf678..4d0a93541d5f 100644
--- a/arch/arm/boot/dts/sama7g5.dtsi
+++ b/arch/arm/boot/dts/sama7g5.dtsi
@@ -236,6 +236,54 @@ sdmmc2: mmc@e120c000 {
status = "disabled";
};
+ csi2dc: csi2dc@e1404000 {
+ compatible = "microchip,sama7g5-csi2dc";
+ reg = <0xe1404000 0x500>;
+ clocks = <&pmc PMC_TYPE_PERIPHERAL 34>, <&xisc>;
+ clock-names = "pclk", "scck";
+ assigned-clocks = <&xisc>;
+ assigned-clock-rates = <266000000>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ port@0 {
+ reg = <0>;
+ csi2dc_in: endpoint {
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+ csi2dc_out: endpoint {
+ bus-width = <14>;
+ hsync-active = <1>;
+ vsync-active = <1>;
+ remote-endpoint = <&xisc_in>;
+ };
+ };
+ };
+ };
+
+ xisc: xisc@e1408000 {
+ compatible = "microchip,sama7g5-isc";
+ reg = <0xe1408000 0x2000>;
+ interrupts = <GIC_SPI 56 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&pmc PMC_TYPE_PERIPHERAL 56>;
+ clock-names = "hclock";
+ #clock-cells = <0>;
+ clock-output-names = "isc-mck";
+
+ port {
+ xisc_in: endpoint {
+ bus-width = <14>;
+ hsync-active = <1>;
+ vsync-active = <1>;
+ remote-endpoint = <&csi2dc_out>;
+ };
+ };
+ };
+
pwm: pwm@e1604000 {
compatible = "microchip,sama7g5-pwm", "atmel,sama5d2-pwm";
reg = <0xe1604000 0x4000>;
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 14/23] ARM: configs: at91: sama7: add xisc and csi2dc
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
Enable XISC and CSI2DC drivers.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
arch/arm/configs/sama7_defconfig | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/arm/configs/sama7_defconfig b/arch/arm/configs/sama7_defconfig
index 938aae4bd80b..15978f2ab4ea 100644
--- a/arch/arm/configs/sama7_defconfig
+++ b/arch/arm/configs/sama7_defconfig
@@ -126,6 +126,8 @@ CONFIG_MEDIA_SUPPORT_FILTER=y
CONFIG_MEDIA_CAMERA_SUPPORT=y
CONFIG_MEDIA_PLATFORM_SUPPORT=y
CONFIG_V4L_PLATFORM_DRIVERS=y
+CONFIG_VIDEO_ATMEL_XISC=y
+CONFIG_VIDEO_MICROCHIP_CSI2DC=y
CONFIG_VIDEO_IMX219=m
CONFIG_VIDEO_IMX274=m
CONFIG_VIDEO_OV5647=m
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 14/23] ARM: configs: at91: sama7: add xisc and csi2dc
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
Enable XISC and CSI2DC drivers.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
arch/arm/configs/sama7_defconfig | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/arm/configs/sama7_defconfig b/arch/arm/configs/sama7_defconfig
index 938aae4bd80b..15978f2ab4ea 100644
--- a/arch/arm/configs/sama7_defconfig
+++ b/arch/arm/configs/sama7_defconfig
@@ -126,6 +126,8 @@ CONFIG_MEDIA_SUPPORT_FILTER=y
CONFIG_MEDIA_CAMERA_SUPPORT=y
CONFIG_MEDIA_PLATFORM_SUPPORT=y
CONFIG_V4L_PLATFORM_DRIVERS=y
+CONFIG_VIDEO_ATMEL_XISC=y
+CONFIG_VIDEO_MICROCHIP_CSI2DC=y
CONFIG_VIDEO_IMX219=m
CONFIG_VIDEO_IMX274=m
CONFIG_VIDEO_OV5647=m
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 15/23] ARM: multi_v7_defconfig: add atmel video pipeline modules
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
Add drivers for the atmel video capture pipeline: atmel isc, xisc and
microchip csi2dc.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
arch/arm/configs/multi_v7_defconfig | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/arm/configs/multi_v7_defconfig b/arch/arm/configs/multi_v7_defconfig
index c951aeed2138..92b7749a6df8 100644
--- a/arch/arm/configs/multi_v7_defconfig
+++ b/arch/arm/configs/multi_v7_defconfig
@@ -638,7 +638,10 @@ CONFIG_VIDEO_S5P_MIPI_CSIS=m
CONFIG_VIDEO_EXYNOS_FIMC_LITE=m
CONFIG_VIDEO_EXYNOS4_FIMC_IS=m
CONFIG_VIDEO_RCAR_VIN=m
+CONFIG_VIDEO_ATMEL_ISC=m
+CONFIG_VIDEO_ATMEL_XISC=m
CONFIG_VIDEO_ATMEL_ISI=m
+CONFIG_VIDEO_MICROCHIP_CSI2DC=m
CONFIG_V4L_MEM2MEM_DRIVERS=y
CONFIG_VIDEO_SAMSUNG_S5P_JPEG=m
CONFIG_VIDEO_SAMSUNG_S5P_MFC=m
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 15/23] ARM: multi_v7_defconfig: add atmel video pipeline modules
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
Add drivers for the atmel video capture pipeline: atmel isc, xisc and
microchip csi2dc.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
arch/arm/configs/multi_v7_defconfig | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/arm/configs/multi_v7_defconfig b/arch/arm/configs/multi_v7_defconfig
index c951aeed2138..92b7749a6df8 100644
--- a/arch/arm/configs/multi_v7_defconfig
+++ b/arch/arm/configs/multi_v7_defconfig
@@ -638,7 +638,10 @@ CONFIG_VIDEO_S5P_MIPI_CSIS=m
CONFIG_VIDEO_EXYNOS_FIMC_LITE=m
CONFIG_VIDEO_EXYNOS4_FIMC_IS=m
CONFIG_VIDEO_RCAR_VIN=m
+CONFIG_VIDEO_ATMEL_ISC=m
+CONFIG_VIDEO_ATMEL_XISC=m
CONFIG_VIDEO_ATMEL_ISI=m
+CONFIG_VIDEO_MICROCHIP_CSI2DC=m
CONFIG_V4L_MEM2MEM_DRIVERS=y
CONFIG_VIDEO_SAMSUNG_S5P_JPEG=m
CONFIG_VIDEO_SAMSUNG_S5P_MFC=m
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 16/23] media: atmel: atmel-sama5d2-isc: fix wrong mask in YUYV format check
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
While this does not happen in production, this check should be done
versus the mask, as checking with the YCYC value may not include
some bits that may be set.
Is it correct and safe to check the whole mask.
Fixes: 123aaf816b95 ("media: atmel: atmel-sama5d2-isc: fix YUYV format")
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
drivers/media/platform/atmel/atmel-sama5d2-isc.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
index a3e4c32a66c4..a8d4ba60d3ac 100644
--- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
@@ -267,7 +267,7 @@ static void isc_sama5d2_config_rlp(struct isc_device *isc)
* Thus, if the YCYC mode is selected, replace it with the
* sama5d2-compliant mode which is YYCC .
*/
- if ((rlp_mode & ISC_RLP_CFG_MODE_YCYC) == ISC_RLP_CFG_MODE_YCYC) {
+ if ((rlp_mode & ISC_RLP_CFG_MODE_MASK) == ISC_RLP_CFG_MODE_YCYC) {
rlp_mode &= ~ISC_RLP_CFG_MODE_MASK;
rlp_mode |= ISC_RLP_CFG_MODE_YYCC;
}
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 16/23] media: atmel: atmel-sama5d2-isc: fix wrong mask in YUYV format check
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
While this does not happen in production, this check should be done
versus the mask, as checking with the YCYC value may not include
some bits that may be set.
Is it correct and safe to check the whole mask.
Fixes: 123aaf816b95 ("media: atmel: atmel-sama5d2-isc: fix YUYV format")
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
drivers/media/platform/atmel/atmel-sama5d2-isc.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
index a3e4c32a66c4..a8d4ba60d3ac 100644
--- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
@@ -267,7 +267,7 @@ static void isc_sama5d2_config_rlp(struct isc_device *isc)
* Thus, if the YCYC mode is selected, replace it with the
* sama5d2-compliant mode which is YYCC .
*/
- if ((rlp_mode & ISC_RLP_CFG_MODE_YCYC) == ISC_RLP_CFG_MODE_YCYC) {
+ if ((rlp_mode & ISC_RLP_CFG_MODE_MASK) == ISC_RLP_CFG_MODE_YCYC) {
rlp_mode &= ~ISC_RLP_CFG_MODE_MASK;
rlp_mode |= ISC_RLP_CFG_MODE_YYCC;
}
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 17/23] media: atmel: atmel-isc-base: use mutex to lock awb workqueue from streaming
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
The AWB workqueue runs in a kernel thread and needs to be synchronized
w.r.t. the streaming status.
It is possible that streaming is stopped while the AWB workq is running.
In this case it is likely that the check for isc->stop is done at one point
in time, but the AWB computations are done later, including a call to
isc_update_profile, which requires streaming to be started.
Thus , isc_update_profile will fail if during this operation sequence the
streaming was stopped.
To solve this issue, a mutex is added, that will serialize the awb work and
streaming stopping, with the mention that either streaming is stopped
completely including termination of the last frame is done, and after that
the AWB work can check stream status and stop; either first AWB work is
completed and after that the streaming can stop correctly.
The awb spin lock cannot be used since this spinlock is taken in the same
context and using it in the stop streaming will result in a recursion BUG.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
drivers/media/platform/atmel/atmel-isc-base.c | 31 ++++++++++++++++---
drivers/media/platform/atmel/atmel-isc.h | 1 +
2 files changed, 28 insertions(+), 4 deletions(-)
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index b0c3ed21f372..53cac1aac0fd 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -401,6 +401,7 @@ static void isc_stop_streaming(struct vb2_queue *vq)
struct isc_buffer *buf;
int ret;
+ mutex_lock(&isc->awb_mutex);
v4l2_ctrl_activate(isc->do_wb_ctrl, false);
isc->stop = true;
@@ -410,6 +411,8 @@ static void isc_stop_streaming(struct vb2_queue *vq)
v4l2_err(&isc->v4l2_dev,
"Timeout waiting for end of the capture\n");
+ mutex_unlock(&isc->awb_mutex);
+
/* Disable DMA interrupt */
regmap_write(isc->regmap, ISC_INTDIS, ISC_INT_DDONE);
@@ -1416,10 +1419,6 @@ static void isc_awb_work(struct work_struct *w)
u32 min, max;
int ret;
- /* streaming is not active anymore */
- if (isc->stop)
- return;
-
if (ctrls->hist_stat != HIST_ENABLED)
return;
@@ -1470,7 +1469,24 @@ static void isc_awb_work(struct work_struct *w)
}
regmap_write(regmap, ISC_HIS_CFG + isc->offsets.his,
hist_id | baysel | ISC_HIS_CFG_RAR);
+
+ /*
+ * We have to make sure the streaming has not stopped meanwhile.
+ * ISC requires a frame to clock the internal profile update.
+ * To avoid issues, lock the sequence with a mutex
+ */
+ mutex_lock(&isc->awb_mutex);
+
+ /* streaming is not active anymore */
+ if (isc->stop) {
+ mutex_unlock(&isc->awb_mutex);
+ return;
+ };
+
isc_update_profile(isc);
+
+ mutex_unlock(&isc->awb_mutex);
+
/* if awb has been disabled, we don't need to start another histogram */
if (ctrls->awb)
regmap_write(regmap, ISC_CTRLEN, ISC_CTRL_HISREQ);
@@ -1549,6 +1565,8 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
isc_update_awb_ctrls(isc);
+ mutex_lock(&isc->awb_mutex);
+
if (!isc->stop) {
/*
* If we are streaming, we can update profile to
@@ -1563,6 +1581,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
*/
v4l2_ctrl_activate(isc->do_wb_ctrl, false);
}
+ mutex_unlock(&isc->awb_mutex);
/* if we have autowhitebalance on, start histogram procedure */
if (ctrls->awb == ISC_WB_AUTO && !isc->stop &&
@@ -1754,6 +1773,7 @@ static void isc_async_unbind(struct v4l2_async_notifier *notifier,
{
struct isc_device *isc = container_of(notifier->v4l2_dev,
struct isc_device, v4l2_dev);
+ mutex_destroy(&isc->awb_mutex);
cancel_work_sync(&isc->awb_work);
video_unregister_device(&isc->video_dev);
v4l2_ctrl_handler_free(&isc->ctrls.handler);
@@ -1866,6 +1886,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
isc->current_subdev = container_of(notifier,
struct isc_subdev_entity, notifier);
mutex_init(&isc->lock);
+ mutex_init(&isc->awb_mutex);
+
init_completion(&isc->comp);
/* Initialize videobuf2 queue */
@@ -1941,6 +1963,7 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
video_unregister_device(vdev);
isc_async_complete_err:
+ mutex_destroy(&isc->awb_mutex);
mutex_destroy(&isc->lock);
return ret;
}
diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
index 0b6370d7775f..c2cb805faff3 100644
--- a/drivers/media/platform/atmel/atmel-isc.h
+++ b/drivers/media/platform/atmel/atmel-isc.h
@@ -307,6 +307,7 @@ struct isc_device {
struct work_struct awb_work;
struct mutex lock; /* serialize access to file operations */
+ struct mutex awb_mutex; /* serialize access to streaming status from awb work queue */
spinlock_t awb_lock; /* serialize access to DMA buffers from awb work queue */
struct regmap_field *pipeline[ISC_PIPE_LINE_NODE_NUM];
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 17/23] media: atmel: atmel-isc-base: use mutex to lock awb workqueue from streaming
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
The AWB workqueue runs in a kernel thread and needs to be synchronized
w.r.t. the streaming status.
It is possible that streaming is stopped while the AWB workq is running.
In this case it is likely that the check for isc->stop is done at one point
in time, but the AWB computations are done later, including a call to
isc_update_profile, which requires streaming to be started.
Thus , isc_update_profile will fail if during this operation sequence the
streaming was stopped.
To solve this issue, a mutex is added, that will serialize the awb work and
streaming stopping, with the mention that either streaming is stopped
completely including termination of the last frame is done, and after that
the AWB work can check stream status and stop; either first AWB work is
completed and after that the streaming can stop correctly.
The awb spin lock cannot be used since this spinlock is taken in the same
context and using it in the stop streaming will result in a recursion BUG.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
drivers/media/platform/atmel/atmel-isc-base.c | 31 ++++++++++++++++---
drivers/media/platform/atmel/atmel-isc.h | 1 +
2 files changed, 28 insertions(+), 4 deletions(-)
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index b0c3ed21f372..53cac1aac0fd 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -401,6 +401,7 @@ static void isc_stop_streaming(struct vb2_queue *vq)
struct isc_buffer *buf;
int ret;
+ mutex_lock(&isc->awb_mutex);
v4l2_ctrl_activate(isc->do_wb_ctrl, false);
isc->stop = true;
@@ -410,6 +411,8 @@ static void isc_stop_streaming(struct vb2_queue *vq)
v4l2_err(&isc->v4l2_dev,
"Timeout waiting for end of the capture\n");
+ mutex_unlock(&isc->awb_mutex);
+
/* Disable DMA interrupt */
regmap_write(isc->regmap, ISC_INTDIS, ISC_INT_DDONE);
@@ -1416,10 +1419,6 @@ static void isc_awb_work(struct work_struct *w)
u32 min, max;
int ret;
- /* streaming is not active anymore */
- if (isc->stop)
- return;
-
if (ctrls->hist_stat != HIST_ENABLED)
return;
@@ -1470,7 +1469,24 @@ static void isc_awb_work(struct work_struct *w)
}
regmap_write(regmap, ISC_HIS_CFG + isc->offsets.his,
hist_id | baysel | ISC_HIS_CFG_RAR);
+
+ /*
+ * We have to make sure the streaming has not stopped meanwhile.
+ * ISC requires a frame to clock the internal profile update.
+ * To avoid issues, lock the sequence with a mutex
+ */
+ mutex_lock(&isc->awb_mutex);
+
+ /* streaming is not active anymore */
+ if (isc->stop) {
+ mutex_unlock(&isc->awb_mutex);
+ return;
+ };
+
isc_update_profile(isc);
+
+ mutex_unlock(&isc->awb_mutex);
+
/* if awb has been disabled, we don't need to start another histogram */
if (ctrls->awb)
regmap_write(regmap, ISC_CTRLEN, ISC_CTRL_HISREQ);
@@ -1549,6 +1565,8 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
isc_update_awb_ctrls(isc);
+ mutex_lock(&isc->awb_mutex);
+
if (!isc->stop) {
/*
* If we are streaming, we can update profile to
@@ -1563,6 +1581,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
*/
v4l2_ctrl_activate(isc->do_wb_ctrl, false);
}
+ mutex_unlock(&isc->awb_mutex);
/* if we have autowhitebalance on, start histogram procedure */
if (ctrls->awb == ISC_WB_AUTO && !isc->stop &&
@@ -1754,6 +1773,7 @@ static void isc_async_unbind(struct v4l2_async_notifier *notifier,
{
struct isc_device *isc = container_of(notifier->v4l2_dev,
struct isc_device, v4l2_dev);
+ mutex_destroy(&isc->awb_mutex);
cancel_work_sync(&isc->awb_work);
video_unregister_device(&isc->video_dev);
v4l2_ctrl_handler_free(&isc->ctrls.handler);
@@ -1866,6 +1886,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
isc->current_subdev = container_of(notifier,
struct isc_subdev_entity, notifier);
mutex_init(&isc->lock);
+ mutex_init(&isc->awb_mutex);
+
init_completion(&isc->comp);
/* Initialize videobuf2 queue */
@@ -1941,6 +1963,7 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
video_unregister_device(vdev);
isc_async_complete_err:
+ mutex_destroy(&isc->awb_mutex);
mutex_destroy(&isc->lock);
return ret;
}
diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
index 0b6370d7775f..c2cb805faff3 100644
--- a/drivers/media/platform/atmel/atmel-isc.h
+++ b/drivers/media/platform/atmel/atmel-isc.h
@@ -307,6 +307,7 @@ struct isc_device {
struct work_struct awb_work;
struct mutex lock; /* serialize access to file operations */
+ struct mutex awb_mutex; /* serialize access to streaming status from awb work queue */
spinlock_t awb_lock; /* serialize access to DMA buffers from awb work queue */
struct regmap_field *pipeline[ISC_PIPE_LINE_NODE_NUM];
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 18/23] media: atmel: atmel-isc-base: add wb debug messages
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
Add debug messages that make it easier to debug white balance algorithm.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
drivers/media/platform/atmel/atmel-isc-base.c | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index 53cac1aac0fd..f1f1019f9d82 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -1329,10 +1329,15 @@ static void isc_hist_count(struct isc_device *isc, u32 *min, u32 *max)
if (!*min)
*min = 1;
+
+ v4l2_dbg(1, debug, &isc->v4l2_dev,
+ "isc wb: hist_id %u, hist_count %u",
+ ctrls->hist_id, *hist_count);
}
static void isc_wb_update(struct isc_ctrls *ctrls)
{
+ struct isc_device *isc = container_of(ctrls, struct isc_device, ctrls);
u32 *hist_count = &ctrls->hist_count[0];
u32 c, offset[4];
u64 avg = 0;
@@ -1349,6 +1354,9 @@ static void isc_wb_update(struct isc_ctrls *ctrls)
(u64)hist_count[ISC_HIS_CFG_MODE_GB];
avg >>= 1;
+ v4l2_dbg(1, debug, &isc->v4l2_dev,
+ "isc wb: green components average %llu\n", avg);
+
/* Green histogram is null, nothing to do */
if (!avg)
return;
@@ -1401,9 +1409,15 @@ static void isc_wb_update(struct isc_ctrls *ctrls)
else
gw_gain[c] = 1 << 9;
+ v4l2_dbg(1, debug, &isc->v4l2_dev,
+ "isc wb: component %d, s_gain %u, gw_gain %u\n",
+ c, s_gain[c], gw_gain[c]);
/* multiply both gains and adjust for decimals */
ctrls->gain[c] = s_gain[c] * gw_gain[c];
ctrls->gain[c] >>= 9;
+ v4l2_dbg(1, debug, &isc->v4l2_dev,
+ "isc wb: component %d, final gain %u\n",
+ c, ctrls->gain[c]);
}
}
@@ -1423,6 +1437,10 @@ static void isc_awb_work(struct work_struct *w)
return;
isc_hist_count(isc, &min, &max);
+
+ v4l2_dbg(1, debug, &isc->v4l2_dev,
+ "isc wb mode %d: hist min %u , max %u\n", hist_id, min, max);
+
ctrls->hist_minmax[hist_id][HIST_MIN_INDEX] = min;
ctrls->hist_minmax[hist_id][HIST_MAX_INDEX] = max;
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 18/23] media: atmel: atmel-isc-base: add wb debug messages
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
Add debug messages that make it easier to debug white balance algorithm.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
drivers/media/platform/atmel/atmel-isc-base.c | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index 53cac1aac0fd..f1f1019f9d82 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -1329,10 +1329,15 @@ static void isc_hist_count(struct isc_device *isc, u32 *min, u32 *max)
if (!*min)
*min = 1;
+
+ v4l2_dbg(1, debug, &isc->v4l2_dev,
+ "isc wb: hist_id %u, hist_count %u",
+ ctrls->hist_id, *hist_count);
}
static void isc_wb_update(struct isc_ctrls *ctrls)
{
+ struct isc_device *isc = container_of(ctrls, struct isc_device, ctrls);
u32 *hist_count = &ctrls->hist_count[0];
u32 c, offset[4];
u64 avg = 0;
@@ -1349,6 +1354,9 @@ static void isc_wb_update(struct isc_ctrls *ctrls)
(u64)hist_count[ISC_HIS_CFG_MODE_GB];
avg >>= 1;
+ v4l2_dbg(1, debug, &isc->v4l2_dev,
+ "isc wb: green components average %llu\n", avg);
+
/* Green histogram is null, nothing to do */
if (!avg)
return;
@@ -1401,9 +1409,15 @@ static void isc_wb_update(struct isc_ctrls *ctrls)
else
gw_gain[c] = 1 << 9;
+ v4l2_dbg(1, debug, &isc->v4l2_dev,
+ "isc wb: component %d, s_gain %u, gw_gain %u\n",
+ c, s_gain[c], gw_gain[c]);
/* multiply both gains and adjust for decimals */
ctrls->gain[c] = s_gain[c] * gw_gain[c];
ctrls->gain[c] >>= 9;
+ v4l2_dbg(1, debug, &isc->v4l2_dev,
+ "isc wb: component %d, final gain %u\n",
+ c, ctrls->gain[c]);
}
}
@@ -1423,6 +1437,10 @@ static void isc_awb_work(struct work_struct *w)
return;
isc_hist_count(isc, &min, &max);
+
+ v4l2_dbg(1, debug, &isc->v4l2_dev,
+ "isc wb mode %d: hist min %u , max %u\n", hist_id, min, max);
+
ctrls->hist_minmax[hist_id][HIST_MIN_INDEX] = min;
ctrls->hist_minmax[hist_id][HIST_MAX_INDEX] = max;
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 19/23] media: atmel: atmel-isc-base: clamp wb gain coefficients
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
White balance computed gains can overflow above the 13 bits hardware
coefficient that can be used, in some specific scenarios like a subexposure
from the sensor when the image is mostly black.
In this case the computed gain has to be clamped to the maximum value
allowed by the hardware.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
drivers/media/platform/atmel/atmel-isc-base.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index f1f1019f9d82..31c8e3029eee 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -1415,6 +1415,10 @@ static void isc_wb_update(struct isc_ctrls *ctrls)
/* multiply both gains and adjust for decimals */
ctrls->gain[c] = s_gain[c] * gw_gain[c];
ctrls->gain[c] >>= 9;
+
+ /* make sure we are not out of range */
+ ctrls->gain[c] = clamp_val(ctrls->gain[c], 0, GENMASK(12, 0));
+
v4l2_dbg(1, debug, &isc->v4l2_dev,
"isc wb: component %d, final gain %u\n",
c, ctrls->gain[c]);
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 19/23] media: atmel: atmel-isc-base: clamp wb gain coefficients
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
White balance computed gains can overflow above the 13 bits hardware
coefficient that can be used, in some specific scenarios like a subexposure
from the sensor when the image is mostly black.
In this case the computed gain has to be clamped to the maximum value
allowed by the hardware.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
drivers/media/platform/atmel/atmel-isc-base.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index f1f1019f9d82..31c8e3029eee 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -1415,6 +1415,10 @@ static void isc_wb_update(struct isc_ctrls *ctrls)
/* multiply both gains and adjust for decimals */
ctrls->gain[c] = s_gain[c] * gw_gain[c];
ctrls->gain[c] >>= 9;
+
+ /* make sure we are not out of range */
+ ctrls->gain[c] = clamp_val(ctrls->gain[c], 0, GENMASK(12, 0));
+
v4l2_dbg(1, debug, &isc->v4l2_dev,
"isc wb: component %d, final gain %u\n",
c, ctrls->gain[c]);
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 20/23] media: atmel: atmel-sama7g5-isc: fix UYVY input format mbus_code typo
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
Fix the mbus code for UYVY input format type to the right color rotation.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
drivers/media/platform/atmel/atmel-sama7g5-isc.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
index d2ac80b1ccc3..38721bd902e2 100644
--- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
@@ -188,7 +188,7 @@ static struct isc_format sama7g5_formats_list[] = {
},
{
.fourcc = V4L2_PIX_FMT_UYVY,
- .mbus_code = MEDIA_BUS_FMT_YUYV8_2X8,
+ .mbus_code = MEDIA_BUS_FMT_UYVY8_2X8,
.pfe_cfg0_bps = ISC_PFE_CFG0_BPS_EIGHT,
},
{
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 20/23] media: atmel: atmel-sama7g5-isc: fix UYVY input format mbus_code typo
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
Fix the mbus code for UYVY input format type to the right color rotation.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
drivers/media/platform/atmel/atmel-sama7g5-isc.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
index d2ac80b1ccc3..38721bd902e2 100644
--- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
@@ -188,7 +188,7 @@ static struct isc_format sama7g5_formats_list[] = {
},
{
.fourcc = V4L2_PIX_FMT_UYVY,
- .mbus_code = MEDIA_BUS_FMT_YUYV8_2X8,
+ .mbus_code = MEDIA_BUS_FMT_UYVY8_2X8,
.pfe_cfg0_bps = ISC_PFE_CFG0_BPS_EIGHT,
},
{
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 21/23] media: atmel: atmel-isc: add raw Bayer 8bit 10bit output formats
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
The ISC can dump the 8 bit and 10 bit raw bayer formats directly to
the memory.
Thus, add them to the supported output format list.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
.../media/platform/atmel/atmel-sama5d2-isc.c | 24 +++++++++++++++++++
.../media/platform/atmel/atmel-sama7g5-isc.c | 24 +++++++++++++++++++
2 files changed, 48 insertions(+)
diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
index a8d4ba60d3ac..025c3e8a7e95 100644
--- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
@@ -88,6 +88,30 @@ static const struct isc_format sama5d2_controller_formats[] = {
{
.fourcc = V4L2_PIX_FMT_Y10,
},
+ {
+ .fourcc = V4L2_PIX_FMT_SBGGR8,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SGBRG8,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SGRBG8,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SRGGB8,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SBGGR10,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SGBRG10,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SGRBG10,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SRGGB10,
+ },
};
/* This is a list of formats that the ISC can receive as *input* */
diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
index 38721bd902e2..9dc75eed0098 100644
--- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
@@ -100,6 +100,30 @@ static const struct isc_format sama7g5_controller_formats[] = {
{
.fourcc = V4L2_PIX_FMT_Y16,
},
+ {
+ .fourcc = V4L2_PIX_FMT_SBGGR8,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SGBRG8,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SGRBG8,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SRGGB8,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SBGGR10,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SGBRG10,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SGRBG10,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SRGGB10,
+ },
};
/* This is a list of formats that the ISC can receive as *input* */
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 21/23] media: atmel: atmel-isc: add raw Bayer 8bit 10bit output formats
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
The ISC can dump the 8 bit and 10 bit raw bayer formats directly to
the memory.
Thus, add them to the supported output format list.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
.../media/platform/atmel/atmel-sama5d2-isc.c | 24 +++++++++++++++++++
.../media/platform/atmel/atmel-sama7g5-isc.c | 24 +++++++++++++++++++
2 files changed, 48 insertions(+)
diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
index a8d4ba60d3ac..025c3e8a7e95 100644
--- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
@@ -88,6 +88,30 @@ static const struct isc_format sama5d2_controller_formats[] = {
{
.fourcc = V4L2_PIX_FMT_Y10,
},
+ {
+ .fourcc = V4L2_PIX_FMT_SBGGR8,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SGBRG8,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SGRBG8,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SRGGB8,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SBGGR10,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SGBRG10,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SGRBG10,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SRGGB10,
+ },
};
/* This is a list of formats that the ISC can receive as *input* */
diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
index 38721bd902e2..9dc75eed0098 100644
--- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
@@ -100,6 +100,30 @@ static const struct isc_format sama7g5_controller_formats[] = {
{
.fourcc = V4L2_PIX_FMT_Y16,
},
+ {
+ .fourcc = V4L2_PIX_FMT_SBGGR8,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SGBRG8,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SGRBG8,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SRGGB8,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SBGGR10,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SGBRG10,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SGRBG10,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_SRGGB10,
+ },
};
/* This is a list of formats that the ISC can receive as *input* */
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 22/23] media: atmel: atmel-isc: compact the controller formats list
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
Compact the list array to be more readable.
No other changes, only cosmetic.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
.../media/platform/atmel/atmel-sama5d2-isc.c | 51 ++++++----------
.../media/platform/atmel/atmel-sama7g5-isc.c | 60 +++++++------------
2 files changed, 37 insertions(+), 74 deletions(-)
diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
index 025c3e8a7e95..d96ee3373889 100644
--- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
@@ -60,56 +60,39 @@
static const struct isc_format sama5d2_controller_formats[] = {
{
.fourcc = V4L2_PIX_FMT_ARGB444,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_ARGB555,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_RGB565,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_ABGR32,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_XBGR32,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_YUV420,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_YUYV,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_YUV422P,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_GREY,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_Y10,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SBGGR8,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SGBRG8,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SGRBG8,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SRGGB8,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SBGGR10,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SGBRG10,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SGRBG10,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SRGGB10,
},
};
diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
index 9dc75eed0098..e07ae188c15f 100644
--- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
@@ -63,65 +63,45 @@
static const struct isc_format sama7g5_controller_formats[] = {
{
.fourcc = V4L2_PIX_FMT_ARGB444,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_ARGB555,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_RGB565,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_ABGR32,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_XBGR32,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_YUV420,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_UYVY,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_VYUY,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_YUYV,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_YUV422P,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_GREY,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_Y10,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_Y16,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SBGGR8,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SGBRG8,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SGRBG8,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SRGGB8,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SBGGR10,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SGBRG10,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SGRBG10,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SRGGB10,
},
};
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 22/23] media: atmel: atmel-isc: compact the controller formats list
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
Compact the list array to be more readable.
No other changes, only cosmetic.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
.../media/platform/atmel/atmel-sama5d2-isc.c | 51 ++++++----------
.../media/platform/atmel/atmel-sama7g5-isc.c | 60 +++++++------------
2 files changed, 37 insertions(+), 74 deletions(-)
diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
index 025c3e8a7e95..d96ee3373889 100644
--- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
@@ -60,56 +60,39 @@
static const struct isc_format sama5d2_controller_formats[] = {
{
.fourcc = V4L2_PIX_FMT_ARGB444,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_ARGB555,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_RGB565,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_ABGR32,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_XBGR32,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_YUV420,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_YUYV,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_YUV422P,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_GREY,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_Y10,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SBGGR8,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SGBRG8,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SGRBG8,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SRGGB8,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SBGGR10,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SGBRG10,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SGRBG10,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SRGGB10,
},
};
diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
index 9dc75eed0098..e07ae188c15f 100644
--- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
+++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
@@ -63,65 +63,45 @@
static const struct isc_format sama7g5_controller_formats[] = {
{
.fourcc = V4L2_PIX_FMT_ARGB444,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_ARGB555,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_RGB565,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_ABGR32,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_XBGR32,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_YUV420,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_UYVY,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_VYUY,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_YUYV,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_YUV422P,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_GREY,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_Y10,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_Y16,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SBGGR8,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SGBRG8,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SGRBG8,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SRGGB8,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SBGGR10,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SGBRG10,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SGRBG10,
- },
- {
+ }, {
.fourcc = V4L2_PIX_FMT_SRGGB10,
},
};
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 23/23] media: atmel: atmel-isc: change format propagation to subdev into only verification
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-13 13:49 ` Eugen Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre, Eugen Hristev
As a top MC video driver, the atmel-isc should not propagate the format to the
subdevice.
It should rather check at streamon() time if the subdev is properly configured
with a compatible format.
Removed the whole format finding logic, and reworked the format verification
at streamon time, such that the ISC will return an error if the subdevice
is not properly configured.
With this being done, the module parameter 'sensor_prefered' makes no sense
anymore. The ISC should not decide which format the sensor is using. The
ISC should only cope with the situation and inform userspace if the streaming
is possible in the current configuration.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
Changes in v3:
- clamp to maximum resolution once the frame size from the subdev is found
drivers/media/platform/atmel/atmel-isc-base.c | 271 ++++++++----------
drivers/media/platform/atmel/atmel-isc.h | 1 +
2 files changed, 126 insertions(+), 146 deletions(-)
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index 31c8e3029eee..00c8c9588a78 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -36,11 +36,6 @@ static unsigned int debug;
module_param(debug, int, 0644);
MODULE_PARM_DESC(debug, "debug level (0-2)");
-static unsigned int sensor_preferred = 1;
-module_param(sensor_preferred, uint, 0644);
-MODULE_PARM_DESC(sensor_preferred,
- "Sensor is preferred to output the specified format (1-on 0-off), default 1");
-
#define ISC_IS_FORMAT_RAW(mbus_code) \
(((mbus_code) & 0xf000) == 0x3000)
@@ -532,7 +527,7 @@ static int isc_enum_fmt_vid_cap(struct file *file, void *priv,
* convert it to any of the formats that we usually can with a
* RAW sensor. Thus, do not advertise them.
*/
- if (!isc->config.sd_format ||
+ if (isc->config.sd_format &&
!ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
return -EINVAL;
@@ -621,20 +616,30 @@ static int isc_try_validate_formats(struct isc_device *isc)
break;
default:
/* any other different formats are not supported */
+ v4l2_err(&isc->v4l2_dev, "Requested unsupported format.\n");
ret = -EINVAL;
}
v4l2_dbg(1, debug, &isc->v4l2_dev,
"Format validation, requested rgb=%u, yuv=%u, grey=%u, bayer=%u\n",
rgb, yuv, grey, bayer);
- /* we cannot output RAW if we do not receive RAW */
- if ((bayer) && !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code))
+ if ((bayer) &&
+ !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code)) {
+ v4l2_err(&isc->v4l2_dev, "Cannot output RAW if we do not receive RAW.\n");
return -EINVAL;
+ }
- /* we cannot output GREY if we do not receive RAW/GREY */
if (grey && !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code) &&
- !ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code))
+ !ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) {
+ v4l2_err(&isc->v4l2_dev, "Cannot output GREY if we do not receive RAW/GREY.\n");
return -EINVAL;
+ }
+
+ if ((rgb || bayer || yuv) &&
+ ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) {
+ v4l2_err(&isc->v4l2_dev, "Cannot convert GREY to another format.\n");
+ return -EINVAL;
+ }
return ret;
}
@@ -862,7 +867,7 @@ static void isc_try_fse(struct isc_device *isc,
* If we do not know yet which format the subdev is using, we cannot
* do anything.
*/
- if (!isc->try_config.sd_format)
+ if (!isc->config.sd_format)
return;
fse.code = isc->try_config.sd_format->mbus_code;
@@ -883,180 +888,141 @@ static void isc_try_fse(struct isc_device *isc,
}
}
-static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f,
- u32 *code)
+static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f)
{
- int i;
- struct isc_format *sd_fmt = NULL, *direct_fmt = NULL;
struct v4l2_pix_format *pixfmt = &f->fmt.pix;
- struct v4l2_subdev_pad_config pad_cfg = {};
- struct v4l2_subdev_state pad_state = {
- .pads = &pad_cfg
- };
- struct v4l2_subdev_format format = {
- .which = V4L2_SUBDEV_FORMAT_TRY,
- };
- u32 mbus_code;
- int ret;
- bool rlp_dma_direct_dump = false;
+ unsigned int i;
if (f->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
return -EINVAL;
- /* Step 1: find a RAW format that is supported */
- for (i = 0; i < isc->num_user_formats; i++) {
- if (ISC_IS_FORMAT_RAW(isc->user_formats[i]->mbus_code)) {
- sd_fmt = isc->user_formats[i];
+ isc->try_config.fourcc = isc->user_formats[0]->fourcc;
+
+ /* find if the format requested is supported */
+ for (i = 0; i < isc->controller_formats_size; i++)
+ if (isc->controller_formats[i].fourcc == pixfmt->pixelformat) {
+ isc->try_config.fourcc = pixfmt->pixelformat;
break;
}
- }
- /* Step 2: We can continue with this RAW format, or we can look
- * for better: maybe sensor supports directly what we need.
- */
- direct_fmt = find_format_by_fourcc(isc, pixfmt->pixelformat);
-
- /* Step 3: We have both. We decide given the module parameter which
- * one to use.
- */
- if (direct_fmt && sd_fmt && sensor_preferred)
- sd_fmt = direct_fmt;
-
- /* Step 4: we do not have RAW but we have a direct format. Use it. */
- if (direct_fmt && !sd_fmt)
- sd_fmt = direct_fmt;
-
- /* Step 5: if we are using a direct format, we need to package
- * everything as 8 bit data and just dump it
- */
- if (sd_fmt == direct_fmt)
- rlp_dma_direct_dump = true;
-
- /* Step 6: We have no format. This can happen if the userspace
- * requests some weird/invalid format.
- * In this case, default to whatever we have
- */
- if (!sd_fmt && !direct_fmt) {
- sd_fmt = isc->user_formats[isc->num_user_formats - 1];
- v4l2_dbg(1, debug, &isc->v4l2_dev,
- "Sensor not supporting %.4s, using %.4s\n",
- (char *)&pixfmt->pixelformat, (char *)&sd_fmt->fourcc);
- }
-
- if (!sd_fmt) {
- ret = -EINVAL;
- goto isc_try_fmt_err;
- }
-
- /* Step 7: Print out what we decided for debugging */
- v4l2_dbg(1, debug, &isc->v4l2_dev,
- "Preferring to have sensor using format %.4s\n",
- (char *)&sd_fmt->fourcc);
-
- /* Step 8: at this moment we decided which format the subdev will use */
- isc->try_config.sd_format = sd_fmt;
-
- /* Limit to Atmel ISC hardware capabilities */
- if (pixfmt->width > isc->max_width)
- pixfmt->width = isc->max_width;
- if (pixfmt->height > isc->max_height)
- pixfmt->height = isc->max_height;
-
- /*
- * The mbus format is the one the subdev outputs.
- * The pixels will be transferred in this format Sensor -> ISC
- */
- mbus_code = sd_fmt->mbus_code;
-
- /*
- * Validate formats. If the required format is not OK, default to raw.
- */
-
- isc->try_config.fourcc = pixfmt->pixelformat;
-
- if (isc_try_validate_formats(isc)) {
- pixfmt->pixelformat = isc->try_config.fourcc = sd_fmt->fourcc;
- /* Re-try to validate the new format */
- ret = isc_try_validate_formats(isc);
- if (ret)
- goto isc_try_fmt_err;
- }
-
- ret = isc_try_configure_rlp_dma(isc, rlp_dma_direct_dump);
- if (ret)
- goto isc_try_fmt_err;
-
- ret = isc_try_configure_pipeline(isc);
- if (ret)
- goto isc_try_fmt_err;
- /* Obtain frame sizes if possible to have crop requirements ready */
- isc_try_fse(isc, &pad_state);
-
- v4l2_fill_mbus_format(&format.format, pixfmt, mbus_code);
- ret = v4l2_subdev_call(isc->current_subdev->sd, pad, set_fmt,
- &pad_state, &format);
- if (ret < 0)
- goto isc_try_fmt_subdev_err;
+ /* If we did not find the requested format, we will fallback here */
+ pixfmt->pixelformat = isc->try_config.fourcc;
+ pixfmt->colorspace = V4L2_COLORSPACE_SRGB;
+ pixfmt->field = V4L2_FIELD_NONE;
- v4l2_fill_pix_format(pixfmt, &format.format);
+ isc_try_configure_rlp_dma(isc, false);
/* Limit to Atmel ISC hardware capabilities */
- if (pixfmt->width > isc->max_width)
- pixfmt->width = isc->max_width;
- if (pixfmt->height > isc->max_height)
- pixfmt->height = isc->max_height;
+ v4l_bound_align_image(&pixfmt->width, 16, isc->max_width, 0,
+ &pixfmt->height, 16, isc->max_height, 0, 0);
pixfmt->field = V4L2_FIELD_NONE;
pixfmt->bytesperline = (pixfmt->width * isc->try_config.bpp_v4l2) >> 3;
pixfmt->sizeimage = ((pixfmt->width * isc->try_config.bpp) >> 3) *
pixfmt->height;
- if (code)
- *code = mbus_code;
+ isc->try_fmt = *f;
return 0;
+}
-isc_try_fmt_err:
- v4l2_err(&isc->v4l2_dev, "Could not find any possible format for a working pipeline\n");
-isc_try_fmt_subdev_err:
- memset(&isc->try_config, 0, sizeof(isc->try_config));
+static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
+{
+ isc_try_fmt(isc, f);
- return ret;
+ /* make the try configuration active */
+ isc->config = isc->try_config;
+ isc->fmt = isc->try_fmt;
+
+ v4l2_dbg(1, debug, &isc->v4l2_dev, "ISC set_fmt to %.4s @%dx%d\n",
+ (char *)&f->fmt.pix.pixelformat,
+ f->fmt.pix.width, f->fmt.pix.height);
+
+ return 0;
}
-static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
+static int isc_validate(struct isc_device *isc)
{
+ int ret;
+ int i;
+ struct isc_format *sd_fmt = NULL;
+ struct v4l2_pix_format *pixfmt = &isc->fmt.fmt.pix;
struct v4l2_subdev_format format = {
.which = V4L2_SUBDEV_FORMAT_ACTIVE,
+ .pad = isc->remote_pad,
+ };
+ struct v4l2_subdev_pad_config pad_cfg = {};
+ struct v4l2_subdev_state pad_state = {
+ .pads = &pad_cfg,
};
- u32 mbus_code = 0;
- int ret;
- ret = isc_try_fmt(isc, f, &mbus_code);
+ /* Get current format from subdev */
+ ret = v4l2_subdev_call(isc->current_subdev->sd, pad, get_fmt, NULL,
+ &format);
if (ret)
return ret;
- v4l2_fill_mbus_format(&format.format, &f->fmt.pix, mbus_code);
- ret = v4l2_subdev_call(isc->current_subdev->sd, pad,
- set_fmt, NULL, &format);
- if (ret < 0)
- return ret;
+ /* Identify the subdev's format configuration */
+ for (i = 0; i < isc->num_user_formats; i++)
+ if (isc->user_formats[i]->mbus_code == format.format.code) {
+ sd_fmt = isc->user_formats[i];
+ break;
+ }
+
+ /* Check if the format is not supported */
+ if (!sd_fmt) {
+ v4l2_err(&isc->v4l2_dev,
+ "Current subdevice is streaming a media bus code that is not supported 0x%x\n",
+ format.format.code);
+ return -EPIPE;
+ }
+
+ /* At this moment we know which format the subdev will use */
+ isc->try_config.sd_format = sd_fmt;
+
+ /* If the sensor is not RAW, we can only do a direct dump */
+ if (!ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code))
+ isc_try_configure_rlp_dma(isc, true);
/* Limit to Atmel ISC hardware capabilities */
- if (f->fmt.pix.width > isc->max_width)
- f->fmt.pix.width = isc->max_width;
- if (f->fmt.pix.height > isc->max_height)
- f->fmt.pix.height = isc->max_height;
+ v4l_bound_align_image(&format.format.width, 16, isc->max_width, 0,
+ &format.format.height, 16, isc->max_height, 0, 0);
- isc->fmt = *f;
+ /* Check if the frame size is the same. Otherwise we may overflow */
+ if (pixfmt->height != format.format.height ||
+ pixfmt->width != format.format.width) {
+ v4l2_err(&isc->v4l2_dev,
+ "ISC not configured with the proper frame size: %dx%d\n",
+ format.format.width, format.format.height);
+ return -EPIPE;
+ }
+ v4l2_dbg(1, debug, &isc->v4l2_dev,
+ "Identified subdev using format %.4s with %dx%d %d bpp\n",
+ (char *)&sd_fmt->fourcc, pixfmt->width, pixfmt->height,
+ isc->try_config.bpp);
+
+ /* Reset and restart AWB if the subdevice changed the format */
if (isc->try_config.sd_format && isc->config.sd_format &&
isc->try_config.sd_format != isc->config.sd_format) {
isc->ctrls.hist_stat = HIST_INIT;
isc_reset_awb_ctrls(isc);
isc_update_v4l2_ctrls(isc);
}
- /* make the try configuration active */
+
+ /* Validate formats */
+ ret = isc_try_validate_formats(isc);
+ if (ret)
+ return ret;
+
+ /* Obtain frame sizes if possible to have crop requirements ready */
+ isc_try_fse(isc, &pad_state);
+
+ /* Configure ISC pipeline for the config */
+ ret = isc_try_configure_pipeline(isc);
+ if (ret)
+ return ret;
+
isc->config = isc->try_config;
v4l2_dbg(1, debug, &isc->v4l2_dev, "New ISC configuration in place\n");
@@ -1064,6 +1030,19 @@ static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
return 0;
}
+static int isc_streamon(struct file *file, void *priv, enum v4l2_buf_type bt)
+{
+ struct isc_device *isc = video_drvdata(file);
+ int ret;
+
+ ret = isc_validate(isc);
+
+ if (ret)
+ return ret;
+
+ return vb2_ioctl_streamon(file, priv, bt);
+}
+
static int isc_s_fmt_vid_cap(struct file *file, void *priv,
struct v4l2_format *f)
{
@@ -1080,7 +1059,7 @@ static int isc_try_fmt_vid_cap(struct file *file, void *priv,
{
struct isc_device *isc = video_drvdata(file);
- return isc_try_fmt(isc, f, NULL);
+ return isc_try_fmt(isc, f);
}
static int isc_enum_input(struct file *file, void *priv,
@@ -1176,7 +1155,7 @@ static const struct v4l2_ioctl_ops isc_ioctl_ops = {
.vidioc_dqbuf = vb2_ioctl_dqbuf,
.vidioc_create_bufs = vb2_ioctl_create_bufs,
.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
- .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamon = isc_streamon,
.vidioc_streamoff = vb2_ioctl_streamoff,
.vidioc_g_parm = isc_g_parm,
@@ -1879,7 +1858,7 @@ static int isc_set_default_fmt(struct isc_device *isc)
};
int ret;
- ret = isc_try_fmt(isc, &f, NULL);
+ ret = isc_try_fmt(isc, &f);
if (ret)
return ret;
diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
index c2cb805faff3..7081698adddd 100644
--- a/drivers/media/platform/atmel/atmel-isc.h
+++ b/drivers/media/platform/atmel/atmel-isc.h
@@ -297,6 +297,7 @@ struct isc_device {
struct completion comp;
struct v4l2_format fmt;
+ struct v4l2_format try_fmt;
struct isc_format **user_formats;
unsigned int num_user_formats;
--
2.25.1
^ permalink raw reply related [flat|nested] 94+ messages in thread
* [PATCH v3 23/23] media: atmel: atmel-isc: change format propagation to subdev into only verification
@ 2021-12-13 13:49 ` Eugen Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen Hristev @ 2021-12-13 13:49 UTC (permalink / raw)
To: linux-media, robh+dt, jacopo, laurent.pinchart, sakari.ailus
Cc: devicetree, Eugen Hristev, linux-kernel, linux-arm-kernel
As a top MC video driver, the atmel-isc should not propagate the format to the
subdevice.
It should rather check at streamon() time if the subdev is properly configured
with a compatible format.
Removed the whole format finding logic, and reworked the format verification
at streamon time, such that the ISC will return an error if the subdevice
is not properly configured.
With this being done, the module parameter 'sensor_prefered' makes no sense
anymore. The ISC should not decide which format the sensor is using. The
ISC should only cope with the situation and inform userspace if the streaming
is possible in the current configuration.
Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
---
Changes in v3:
- clamp to maximum resolution once the frame size from the subdev is found
drivers/media/platform/atmel/atmel-isc-base.c | 271 ++++++++----------
drivers/media/platform/atmel/atmel-isc.h | 1 +
2 files changed, 126 insertions(+), 146 deletions(-)
diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
index 31c8e3029eee..00c8c9588a78 100644
--- a/drivers/media/platform/atmel/atmel-isc-base.c
+++ b/drivers/media/platform/atmel/atmel-isc-base.c
@@ -36,11 +36,6 @@ static unsigned int debug;
module_param(debug, int, 0644);
MODULE_PARM_DESC(debug, "debug level (0-2)");
-static unsigned int sensor_preferred = 1;
-module_param(sensor_preferred, uint, 0644);
-MODULE_PARM_DESC(sensor_preferred,
- "Sensor is preferred to output the specified format (1-on 0-off), default 1");
-
#define ISC_IS_FORMAT_RAW(mbus_code) \
(((mbus_code) & 0xf000) == 0x3000)
@@ -532,7 +527,7 @@ static int isc_enum_fmt_vid_cap(struct file *file, void *priv,
* convert it to any of the formats that we usually can with a
* RAW sensor. Thus, do not advertise them.
*/
- if (!isc->config.sd_format ||
+ if (isc->config.sd_format &&
!ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
return -EINVAL;
@@ -621,20 +616,30 @@ static int isc_try_validate_formats(struct isc_device *isc)
break;
default:
/* any other different formats are not supported */
+ v4l2_err(&isc->v4l2_dev, "Requested unsupported format.\n");
ret = -EINVAL;
}
v4l2_dbg(1, debug, &isc->v4l2_dev,
"Format validation, requested rgb=%u, yuv=%u, grey=%u, bayer=%u\n",
rgb, yuv, grey, bayer);
- /* we cannot output RAW if we do not receive RAW */
- if ((bayer) && !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code))
+ if ((bayer) &&
+ !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code)) {
+ v4l2_err(&isc->v4l2_dev, "Cannot output RAW if we do not receive RAW.\n");
return -EINVAL;
+ }
- /* we cannot output GREY if we do not receive RAW/GREY */
if (grey && !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code) &&
- !ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code))
+ !ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) {
+ v4l2_err(&isc->v4l2_dev, "Cannot output GREY if we do not receive RAW/GREY.\n");
return -EINVAL;
+ }
+
+ if ((rgb || bayer || yuv) &&
+ ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) {
+ v4l2_err(&isc->v4l2_dev, "Cannot convert GREY to another format.\n");
+ return -EINVAL;
+ }
return ret;
}
@@ -862,7 +867,7 @@ static void isc_try_fse(struct isc_device *isc,
* If we do not know yet which format the subdev is using, we cannot
* do anything.
*/
- if (!isc->try_config.sd_format)
+ if (!isc->config.sd_format)
return;
fse.code = isc->try_config.sd_format->mbus_code;
@@ -883,180 +888,141 @@ static void isc_try_fse(struct isc_device *isc,
}
}
-static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f,
- u32 *code)
+static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f)
{
- int i;
- struct isc_format *sd_fmt = NULL, *direct_fmt = NULL;
struct v4l2_pix_format *pixfmt = &f->fmt.pix;
- struct v4l2_subdev_pad_config pad_cfg = {};
- struct v4l2_subdev_state pad_state = {
- .pads = &pad_cfg
- };
- struct v4l2_subdev_format format = {
- .which = V4L2_SUBDEV_FORMAT_TRY,
- };
- u32 mbus_code;
- int ret;
- bool rlp_dma_direct_dump = false;
+ unsigned int i;
if (f->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
return -EINVAL;
- /* Step 1: find a RAW format that is supported */
- for (i = 0; i < isc->num_user_formats; i++) {
- if (ISC_IS_FORMAT_RAW(isc->user_formats[i]->mbus_code)) {
- sd_fmt = isc->user_formats[i];
+ isc->try_config.fourcc = isc->user_formats[0]->fourcc;
+
+ /* find if the format requested is supported */
+ for (i = 0; i < isc->controller_formats_size; i++)
+ if (isc->controller_formats[i].fourcc == pixfmt->pixelformat) {
+ isc->try_config.fourcc = pixfmt->pixelformat;
break;
}
- }
- /* Step 2: We can continue with this RAW format, or we can look
- * for better: maybe sensor supports directly what we need.
- */
- direct_fmt = find_format_by_fourcc(isc, pixfmt->pixelformat);
-
- /* Step 3: We have both. We decide given the module parameter which
- * one to use.
- */
- if (direct_fmt && sd_fmt && sensor_preferred)
- sd_fmt = direct_fmt;
-
- /* Step 4: we do not have RAW but we have a direct format. Use it. */
- if (direct_fmt && !sd_fmt)
- sd_fmt = direct_fmt;
-
- /* Step 5: if we are using a direct format, we need to package
- * everything as 8 bit data and just dump it
- */
- if (sd_fmt == direct_fmt)
- rlp_dma_direct_dump = true;
-
- /* Step 6: We have no format. This can happen if the userspace
- * requests some weird/invalid format.
- * In this case, default to whatever we have
- */
- if (!sd_fmt && !direct_fmt) {
- sd_fmt = isc->user_formats[isc->num_user_formats - 1];
- v4l2_dbg(1, debug, &isc->v4l2_dev,
- "Sensor not supporting %.4s, using %.4s\n",
- (char *)&pixfmt->pixelformat, (char *)&sd_fmt->fourcc);
- }
-
- if (!sd_fmt) {
- ret = -EINVAL;
- goto isc_try_fmt_err;
- }
-
- /* Step 7: Print out what we decided for debugging */
- v4l2_dbg(1, debug, &isc->v4l2_dev,
- "Preferring to have sensor using format %.4s\n",
- (char *)&sd_fmt->fourcc);
-
- /* Step 8: at this moment we decided which format the subdev will use */
- isc->try_config.sd_format = sd_fmt;
-
- /* Limit to Atmel ISC hardware capabilities */
- if (pixfmt->width > isc->max_width)
- pixfmt->width = isc->max_width;
- if (pixfmt->height > isc->max_height)
- pixfmt->height = isc->max_height;
-
- /*
- * The mbus format is the one the subdev outputs.
- * The pixels will be transferred in this format Sensor -> ISC
- */
- mbus_code = sd_fmt->mbus_code;
-
- /*
- * Validate formats. If the required format is not OK, default to raw.
- */
-
- isc->try_config.fourcc = pixfmt->pixelformat;
-
- if (isc_try_validate_formats(isc)) {
- pixfmt->pixelformat = isc->try_config.fourcc = sd_fmt->fourcc;
- /* Re-try to validate the new format */
- ret = isc_try_validate_formats(isc);
- if (ret)
- goto isc_try_fmt_err;
- }
-
- ret = isc_try_configure_rlp_dma(isc, rlp_dma_direct_dump);
- if (ret)
- goto isc_try_fmt_err;
-
- ret = isc_try_configure_pipeline(isc);
- if (ret)
- goto isc_try_fmt_err;
- /* Obtain frame sizes if possible to have crop requirements ready */
- isc_try_fse(isc, &pad_state);
-
- v4l2_fill_mbus_format(&format.format, pixfmt, mbus_code);
- ret = v4l2_subdev_call(isc->current_subdev->sd, pad, set_fmt,
- &pad_state, &format);
- if (ret < 0)
- goto isc_try_fmt_subdev_err;
+ /* If we did not find the requested format, we will fallback here */
+ pixfmt->pixelformat = isc->try_config.fourcc;
+ pixfmt->colorspace = V4L2_COLORSPACE_SRGB;
+ pixfmt->field = V4L2_FIELD_NONE;
- v4l2_fill_pix_format(pixfmt, &format.format);
+ isc_try_configure_rlp_dma(isc, false);
/* Limit to Atmel ISC hardware capabilities */
- if (pixfmt->width > isc->max_width)
- pixfmt->width = isc->max_width;
- if (pixfmt->height > isc->max_height)
- pixfmt->height = isc->max_height;
+ v4l_bound_align_image(&pixfmt->width, 16, isc->max_width, 0,
+ &pixfmt->height, 16, isc->max_height, 0, 0);
pixfmt->field = V4L2_FIELD_NONE;
pixfmt->bytesperline = (pixfmt->width * isc->try_config.bpp_v4l2) >> 3;
pixfmt->sizeimage = ((pixfmt->width * isc->try_config.bpp) >> 3) *
pixfmt->height;
- if (code)
- *code = mbus_code;
+ isc->try_fmt = *f;
return 0;
+}
-isc_try_fmt_err:
- v4l2_err(&isc->v4l2_dev, "Could not find any possible format for a working pipeline\n");
-isc_try_fmt_subdev_err:
- memset(&isc->try_config, 0, sizeof(isc->try_config));
+static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
+{
+ isc_try_fmt(isc, f);
- return ret;
+ /* make the try configuration active */
+ isc->config = isc->try_config;
+ isc->fmt = isc->try_fmt;
+
+ v4l2_dbg(1, debug, &isc->v4l2_dev, "ISC set_fmt to %.4s @%dx%d\n",
+ (char *)&f->fmt.pix.pixelformat,
+ f->fmt.pix.width, f->fmt.pix.height);
+
+ return 0;
}
-static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
+static int isc_validate(struct isc_device *isc)
{
+ int ret;
+ int i;
+ struct isc_format *sd_fmt = NULL;
+ struct v4l2_pix_format *pixfmt = &isc->fmt.fmt.pix;
struct v4l2_subdev_format format = {
.which = V4L2_SUBDEV_FORMAT_ACTIVE,
+ .pad = isc->remote_pad,
+ };
+ struct v4l2_subdev_pad_config pad_cfg = {};
+ struct v4l2_subdev_state pad_state = {
+ .pads = &pad_cfg,
};
- u32 mbus_code = 0;
- int ret;
- ret = isc_try_fmt(isc, f, &mbus_code);
+ /* Get current format from subdev */
+ ret = v4l2_subdev_call(isc->current_subdev->sd, pad, get_fmt, NULL,
+ &format);
if (ret)
return ret;
- v4l2_fill_mbus_format(&format.format, &f->fmt.pix, mbus_code);
- ret = v4l2_subdev_call(isc->current_subdev->sd, pad,
- set_fmt, NULL, &format);
- if (ret < 0)
- return ret;
+ /* Identify the subdev's format configuration */
+ for (i = 0; i < isc->num_user_formats; i++)
+ if (isc->user_formats[i]->mbus_code == format.format.code) {
+ sd_fmt = isc->user_formats[i];
+ break;
+ }
+
+ /* Check if the format is not supported */
+ if (!sd_fmt) {
+ v4l2_err(&isc->v4l2_dev,
+ "Current subdevice is streaming a media bus code that is not supported 0x%x\n",
+ format.format.code);
+ return -EPIPE;
+ }
+
+ /* At this moment we know which format the subdev will use */
+ isc->try_config.sd_format = sd_fmt;
+
+ /* If the sensor is not RAW, we can only do a direct dump */
+ if (!ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code))
+ isc_try_configure_rlp_dma(isc, true);
/* Limit to Atmel ISC hardware capabilities */
- if (f->fmt.pix.width > isc->max_width)
- f->fmt.pix.width = isc->max_width;
- if (f->fmt.pix.height > isc->max_height)
- f->fmt.pix.height = isc->max_height;
+ v4l_bound_align_image(&format.format.width, 16, isc->max_width, 0,
+ &format.format.height, 16, isc->max_height, 0, 0);
- isc->fmt = *f;
+ /* Check if the frame size is the same. Otherwise we may overflow */
+ if (pixfmt->height != format.format.height ||
+ pixfmt->width != format.format.width) {
+ v4l2_err(&isc->v4l2_dev,
+ "ISC not configured with the proper frame size: %dx%d\n",
+ format.format.width, format.format.height);
+ return -EPIPE;
+ }
+ v4l2_dbg(1, debug, &isc->v4l2_dev,
+ "Identified subdev using format %.4s with %dx%d %d bpp\n",
+ (char *)&sd_fmt->fourcc, pixfmt->width, pixfmt->height,
+ isc->try_config.bpp);
+
+ /* Reset and restart AWB if the subdevice changed the format */
if (isc->try_config.sd_format && isc->config.sd_format &&
isc->try_config.sd_format != isc->config.sd_format) {
isc->ctrls.hist_stat = HIST_INIT;
isc_reset_awb_ctrls(isc);
isc_update_v4l2_ctrls(isc);
}
- /* make the try configuration active */
+
+ /* Validate formats */
+ ret = isc_try_validate_formats(isc);
+ if (ret)
+ return ret;
+
+ /* Obtain frame sizes if possible to have crop requirements ready */
+ isc_try_fse(isc, &pad_state);
+
+ /* Configure ISC pipeline for the config */
+ ret = isc_try_configure_pipeline(isc);
+ if (ret)
+ return ret;
+
isc->config = isc->try_config;
v4l2_dbg(1, debug, &isc->v4l2_dev, "New ISC configuration in place\n");
@@ -1064,6 +1030,19 @@ static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
return 0;
}
+static int isc_streamon(struct file *file, void *priv, enum v4l2_buf_type bt)
+{
+ struct isc_device *isc = video_drvdata(file);
+ int ret;
+
+ ret = isc_validate(isc);
+
+ if (ret)
+ return ret;
+
+ return vb2_ioctl_streamon(file, priv, bt);
+}
+
static int isc_s_fmt_vid_cap(struct file *file, void *priv,
struct v4l2_format *f)
{
@@ -1080,7 +1059,7 @@ static int isc_try_fmt_vid_cap(struct file *file, void *priv,
{
struct isc_device *isc = video_drvdata(file);
- return isc_try_fmt(isc, f, NULL);
+ return isc_try_fmt(isc, f);
}
static int isc_enum_input(struct file *file, void *priv,
@@ -1176,7 +1155,7 @@ static const struct v4l2_ioctl_ops isc_ioctl_ops = {
.vidioc_dqbuf = vb2_ioctl_dqbuf,
.vidioc_create_bufs = vb2_ioctl_create_bufs,
.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
- .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamon = isc_streamon,
.vidioc_streamoff = vb2_ioctl_streamoff,
.vidioc_g_parm = isc_g_parm,
@@ -1879,7 +1858,7 @@ static int isc_set_default_fmt(struct isc_device *isc)
};
int ret;
- ret = isc_try_fmt(isc, &f, NULL);
+ ret = isc_try_fmt(isc, &f);
if (ret)
return ret;
diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
index c2cb805faff3..7081698adddd 100644
--- a/drivers/media/platform/atmel/atmel-isc.h
+++ b/drivers/media/platform/atmel/atmel-isc.h
@@ -297,6 +297,7 @@ struct isc_device {
struct completion comp;
struct v4l2_format fmt;
+ struct v4l2_format try_fmt;
struct isc_format **user_formats;
unsigned int num_user_formats;
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 94+ messages in thread
* Re: [PATCH v3 02/23] dt-bindings: media: atmel: csi2dc: add bindings for microchip csi2dc
2021-12-13 13:49 ` Eugen Hristev
@ 2021-12-15 20:16 ` Rob Herring
-1 siblings, 0 replies; 94+ messages in thread
From: Rob Herring @ 2021-12-15 20:16 UTC (permalink / raw)
To: Eugen Hristev
Cc: jacopo, laurent.pinchart, linux-media, linux-kernel,
sakari.ailus, devicetree, nicolas.ferre, robh+dt,
linux-arm-kernel
On Mon, 13 Dec 2021 15:49:19 +0200, Eugen Hristev wrote:
> Add bindings documentation for Microchip CSI2 Demultiplexer controller.
>
> CSI2DC is a demultiplexer from Synopsys IDI interface specification to
> parallel interface connection or direct memory access.
> CSI2DC can also act as a parallel bypass from a parallel sensor to the
> image sensor controller/interface.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> ---
> Changes in v3:
> - Changed to port base as suggested by Rob
> - Added properties from video-interfaces which are being used by the csi2dc
> - added mention to the parallel port type as input
> - did some rephrasing and rewording
>
> Changes in v2:
> - changed the endpoint/port references as suggested by Rob
>
> Changes in this version :
> - fixed 'sink' name to be actually source.
> - added dma properties and example with dma
>
> Previous change log:
> Changes in v5:
> - modified bindings as per Rob Herring review
>
> Changes in v4:
> - Removed property for inter-line-delay and for clock continuous/non-continuous
> - Removed virtual channel by reg for second endpoint
>
> Changes in v3:
> - Removed some text from description, as it was explained in the schema
> - fixed other things as per Rob's review
> - moved some text inside the schema, like the clock description
>
> Changes in v2:
> - fixed warnings reported by dt_binding_check
>
>
> .../bindings/media/microchip,csi2dc.yaml | 197 ++++++++++++++++++
> 1 file changed, 197 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
>
Reviewed-by: Rob Herring <robh@kernel.org>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 02/23] dt-bindings: media: atmel: csi2dc: add bindings for microchip csi2dc
@ 2021-12-15 20:16 ` Rob Herring
0 siblings, 0 replies; 94+ messages in thread
From: Rob Herring @ 2021-12-15 20:16 UTC (permalink / raw)
To: Eugen Hristev
Cc: devicetree, jacopo, linux-kernel, robh+dt, sakari.ailus,
laurent.pinchart, linux-arm-kernel, linux-media
On Mon, 13 Dec 2021 15:49:19 +0200, Eugen Hristev wrote:
> Add bindings documentation for Microchip CSI2 Demultiplexer controller.
>
> CSI2DC is a demultiplexer from Synopsys IDI interface specification to
> parallel interface connection or direct memory access.
> CSI2DC can also act as a parallel bypass from a parallel sensor to the
> image sensor controller/interface.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> ---
> Changes in v3:
> - Changed to port base as suggested by Rob
> - Added properties from video-interfaces which are being used by the csi2dc
> - added mention to the parallel port type as input
> - did some rephrasing and rewording
>
> Changes in v2:
> - changed the endpoint/port references as suggested by Rob
>
> Changes in this version :
> - fixed 'sink' name to be actually source.
> - added dma properties and example with dma
>
> Previous change log:
> Changes in v5:
> - modified bindings as per Rob Herring review
>
> Changes in v4:
> - Removed property for inter-line-delay and for clock continuous/non-continuous
> - Removed virtual channel by reg for second endpoint
>
> Changes in v3:
> - Removed some text from description, as it was explained in the schema
> - fixed other things as per Rob's review
> - moved some text inside the schema, like the clock description
>
> Changes in v2:
> - fixed warnings reported by dt_binding_check
>
>
> .../bindings/media/microchip,csi2dc.yaml | 197 ++++++++++++++++++
> 1 file changed, 197 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
>
Reviewed-by: Rob Herring <robh@kernel.org>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 03/23] media: atmel: introduce microchip csi2dc driver
2021-12-13 13:49 ` Eugen Hristev
@ 2022-01-11 15:43 ` Jacopo Mondi
-1 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-11 15:43 UTC (permalink / raw)
To: Eugen Hristev
Cc: linux-media, robh+dt, laurent.pinchart, sakari.ailus, devicetree,
linux-kernel, linux-arm-kernel, nicolas.ferre
Hi Eugen
On Mon, Dec 13, 2021 at 03:49:20PM +0200, Eugen Hristev wrote:
> Microchip CSI2DC (CSI2 Demultiplexer Controller) is a misc bridge device
> that converts a byte stream in IDI Synopsys format (coming from a CSI2HOST)
> to a pixel stream that can be captured by a sensor controller.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
Thanks for addressing my comments on previous versions
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Thanks
j
> ---
> Changes in v3:
> - on set format, disable setting the source sink.
> - removed useless request to pm_request_idle
> - fixed YUV DT
> - added last frame debug info
> - removed writing read only VPCOL and VPROW registers
> - minor cosmetic changes
>
> Changes in v2:
> - implement try formats
> - added parallel mode
> - moved to fwlink endpoints
> - many other minor corrections from Jacopo's review
>
> Changes in this revision:
> - addressed comments by Jacopo and Laurent as in this thread:
> https://www.spinics.net/lists/linux-media/msg181044.html
>
> Previous change log :
> Changes in v5:
> - only in bindings
>
> Changes in v4:
> - now using get_mbus_config ops to get data from the subdevice, like the
> virtual channel id, and the clock type.
> - now having possibility to select any of the RAW10 data modes
> - at completion time, select which formats are also available in the subdevice,
> and move to the dynamic list accordingly
> - changed the pipeline integration, do not advertise subdev ready at probe time.
> wait until completion is done, and then start a workqueue that will register
> this device as a subdevice for the next element in pipeline.
> - moved the s_power code into a different function called now csi2dc_power
> that is called with CONFIG_PM functions. This is also called at completion,
> to have the device ready in case CONFIG_PM is not selected on the platform.
> - merged try_fmt into set_fmt
> - driver cleanup, wrapped lines over 80 characters
>
> Changes in v2:
> - moved driver to platform/atmel
> - fixed minor things as per Sakari's review
> - still some things from v2 review are not yet addressed, to be followed up
>
>
>
> drivers/media/platform/Makefile | 1 +
> drivers/media/platform/atmel/Kconfig | 15 +
> drivers/media/platform/atmel/Makefile | 1 +
> .../media/platform/atmel/microchip-csi2dc.c | 806 ++++++++++++++++++
> 4 files changed, 823 insertions(+)
> create mode 100644 drivers/media/platform/atmel/microchip-csi2dc.c
>
> diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> index 832357240e89..b18e5f704145 100644
> --- a/drivers/media/platform/Makefile
> +++ b/drivers/media/platform/Makefile
> @@ -69,6 +69,7 @@ obj-$(CONFIG_VIDEO_RCAR_VIN) += rcar-vin/
> obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel/
> obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel/
> obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel/
> +obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += atmel/
>
> obj-$(CONFIG_VIDEO_STM32_DCMI) += stm32/
>
> diff --git a/drivers/media/platform/atmel/Kconfig b/drivers/media/platform/atmel/Kconfig
> index dda2f27da317..f83bee373d82 100644
> --- a/drivers/media/platform/atmel/Kconfig
> +++ b/drivers/media/platform/atmel/Kconfig
> @@ -40,3 +40,18 @@ config VIDEO_ATMEL_ISI
> help
> This module makes the ATMEL Image Sensor Interface available
> as a v4l2 device.
> +
> +config VIDEO_MICROCHIP_CSI2DC
> + tristate "Microchip CSI2 Demux Controller"
> + depends on VIDEO_V4L2 && COMMON_CLK && OF
> + depends on ARCH_AT91 || COMPILE_TEST
> + select MEDIA_CONTROLLER
> + select VIDEO_V4L2_SUBDEV_API
> + select V4L2_FWNODE
> + help
> + CSI2 Demux Controller driver. CSI2DC is a helper chip
> + that converts IDI interface byte stream to a parallel pixel stream.
> + It supports various RAW formats as input.
> +
> + To compile this driver as a module, choose M here: the
> + module will be called microchip-csi2dc.
> diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
> index 46d264ab7948..39f0a7eba702 100644
> --- a/drivers/media/platform/atmel/Makefile
> +++ b/drivers/media/platform/atmel/Makefile
> @@ -6,3 +6,4 @@ obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
> obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o
> obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel-isc.o
> obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel-xisc.o
> +obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += microchip-csi2dc.o
> diff --git a/drivers/media/platform/atmel/microchip-csi2dc.c b/drivers/media/platform/atmel/microchip-csi2dc.c
> new file mode 100644
> index 000000000000..6bc549c28e05
> --- /dev/null
> +++ b/drivers/media/platform/atmel/microchip-csi2dc.c
> @@ -0,0 +1,806 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Microchip CSI2 Demux Controller (CSI2DC) driver
> + *
> + * Copyright (C) 2018 Microchip Technology, Inc.
> + *
> + * Author: Eugen Hristev <eugen.hristev@microchip.com>
> + *
> + */
> +
> +#include <linux/clk.h>
> +#include <linux/mod_devicetable.h>
> +#include <linux/module.h>
> +#include <linux/of_graph.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/videodev2.h>
> +
> +#include <media/v4l2-fwnode.h>
> +#include <media/v4l2-subdev.h>
> +
> +/* Global configuration register */
> +#define CSI2DC_GCFG 0x0
> +
> +/* MIPI sensor pixel clock is free running */
> +#define CSI2DC_GCFG_MIPIFRN BIT(0)
> +/* GPIO parallel interface selection */
> +#define CSI2DC_GCFG_GPIOSEL BIT(1)
> +/* Output waveform inter-line minimum delay */
> +#define CSI2DC_GCFG_HLC(v) ((v) << 4)
> +#define CSI2DC_GCFG_HLC_MASK GENMASK(7, 4)
> +/* SAMA7G5 requires a HLC delay of 15 */
> +#define SAMA7G5_HLC (15)
> +
> +/* Global control register */
> +#define CSI2DC_GCTLR 0x04
> +#define CSI2DC_GCTLR_SWRST BIT(0)
> +
> +/* Global status register */
> +#define CSI2DC_GS 0x08
> +
> +/* SSP interrupt status register */
> +#define CSI2DC_SSPIS 0x28
> +/* Pipe update register */
> +#define CSI2DC_PU 0xc0
> +/* Video pipe attributes update */
> +#define CSI2DC_PU_VP BIT(0)
> +
> +/* Pipe update status register */
> +#define CSI2DC_PUS 0xc4
> +
> +/* Video pipeline Interrupt Status Register */
> +#define CSI2DC_VPISR 0xf4
> +
> +/* Video pipeline enable register */
> +#define CSI2DC_VPE 0xf8
> +#define CSI2DC_VPE_ENABLE BIT(0)
> +
> +/* Video pipeline configuration register */
> +#define CSI2DC_VPCFG 0xfc
> +/* Data type */
> +#define CSI2DC_VPCFG_DT(v) ((v) << 0)
> +#define CSI2DC_VPCFG_DT_MASK GENMASK(5, 0)
> +/* Virtual channel identifier */
> +#define CSI2DC_VPCFG_VC(v) ((v) << 6)
> +#define CSI2DC_VPCFG_VC_MASK GENMASK(7, 6)
> +/* Decompression enable */
> +#define CSI2DC_VPCFG_DE BIT(8)
> +/* Decoder mode */
> +#define CSI2DC_VPCFG_DM(v) ((v) << 9)
> +#define CSI2DC_VPCFG_DM_DECODER8TO12 0
> +/* Decoder predictor 2 selection */
> +#define CSI2DC_VPCFG_DP2 BIT(12)
> +/* Recommended memory storage */
> +#define CSI2DC_VPCFG_RMS BIT(13)
> +/* Post adjustment */
> +#define CSI2DC_VPCFG_PA BIT(14)
> +
> +/* Video pipeline column register */
> +#define CSI2DC_VPCOL 0x100
> +/* Column number */
> +#define CSI2DC_VPCOL_COL(v) ((v) << 0)
> +#define CSI2DC_VPCOL_COL_MASK GENMASK(15, 0)
> +
> +/* Video pipeline row register */
> +#define CSI2DC_VPROW 0x104
> +/* Row number */
> +#define CSI2DC_VPROW_ROW(v) ((v) << 0)
> +#define CSI2DC_VPROW_ROW_MASK GENMASK(15, 0)
> +
> +/* Version register */
> +#define CSI2DC_VERSION 0x1fc
> +
> +/* register read/write helpers */
> +#define csi2dc_readl(st, reg) readl_relaxed((st)->base + (reg))
> +#define csi2dc_writel(st, reg, val) writel_relaxed((val), \
> + (st)->base + (reg))
> +
> +/* supported RAW data types */
> +#define CSI2DC_DT_RAW6 0x28
> +#define CSI2DC_DT_RAW7 0x29
> +#define CSI2DC_DT_RAW8 0x2a
> +#define CSI2DC_DT_RAW10 0x2b
> +#define CSI2DC_DT_RAW12 0x2c
> +#define CSI2DC_DT_RAW14 0x2d
> +/* YUV data types */
> +#define CSI2DC_DT_YUV422_8B 0x1e
> +
> +/*
> + * struct csi2dc_format - CSI2DC format type struct
> + * @mbus_code: Media bus code for the format
> + * @dt: Data type constant for this format
> + */
> +struct csi2dc_format {
> + u32 mbus_code;
> + u32 dt;
> +};
> +
> +static const struct csi2dc_format csi2dc_formats[] = {
> + {
> + .mbus_code = MEDIA_BUS_FMT_SRGGB8_1X8,
> + .dt = CSI2DC_DT_RAW8,
> + }, {
> + .mbus_code = MEDIA_BUS_FMT_SBGGR8_1X8,
> + .dt = CSI2DC_DT_RAW8,
> + }, {
> + .mbus_code = MEDIA_BUS_FMT_SGRBG8_1X8,
> + .dt = CSI2DC_DT_RAW8,
> + }, {
> + .mbus_code = MEDIA_BUS_FMT_SGBRG8_1X8,
> + .dt = CSI2DC_DT_RAW8,
> + }, {
> + .mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10,
> + .dt = CSI2DC_DT_RAW10,
> + }, {
> + .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10,
> + .dt = CSI2DC_DT_RAW10,
> + }, {
> + .mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10,
> + .dt = CSI2DC_DT_RAW10,
> + }, {
> + .mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10,
> + .dt = CSI2DC_DT_RAW10,
> + }, {
> + .mbus_code = MEDIA_BUS_FMT_YUYV8_2X8,
> + .dt = CSI2DC_DT_YUV422_8B,
> + },
> +};
> +
> +enum mipi_csi_pads {
> + CSI2DC_PAD_SINK = 0,
> + CSI2DC_PAD_SOURCE = 1,
> + CSI2DC_PADS_NUM = 2,
> +};
> +
> +/*
> + * struct csi2dc_device - CSI2DC device driver data/config struct
> + * @base: Register map base address
> + * @csi2dc_sd: v4l2 subdevice for the csi2dc device
> + * This is the subdevice that the csi2dc device itself
> + * registers in v4l2 subsystem
> + * @dev: struct device for this csi2dc device
> + * @pclk: Peripheral clock reference
> + * Input clock that clocks the hardware block internal
> + * logic
> + * @scck: Sensor Controller clock reference
> + * Input clock that is used to generate the pixel clock
> + * @format: Current saved format used in g/s fmt
> + * @cur_fmt: Current state format
> + * @try_fmt: Try format that is being tried
> + * @pads: Media entity pads for the csi2dc subdevice
> + * @clk_gated: Whether the clock is gated or free running
> + * @video_pipe: Whether video pipeline is configured
> + * @parallel_mode: The underlying subdevice is connected on a parallel bus
> + * @vc: Current set virtual channel
> + * @notifier: Async notifier that is used to bound the underlying
> + * subdevice to the csi2dc subdevice
> + * @input_sd: Reference to the underlying subdevice bound to the
> + * csi2dc subdevice
> + * @remote_pad: Pad number of the underlying subdevice that is linked
> + * to the csi2dc subdevice sink pad.
> + */
> +struct csi2dc_device {
> + void __iomem *base;
> + struct v4l2_subdev csi2dc_sd;
> + struct device *dev;
> + struct clk *pclk;
> + struct clk *scck;
> +
> + struct v4l2_mbus_framefmt format;
> +
> + const struct csi2dc_format *cur_fmt;
> + const struct csi2dc_format *try_fmt;
> +
> + struct media_pad pads[CSI2DC_PADS_NUM];
> +
> + bool clk_gated;
> + bool video_pipe;
> + bool parallel_mode;
> + u32 vc;
> +
> + struct v4l2_async_notifier notifier;
> +
> + struct v4l2_subdev *input_sd;
> +
> + u32 remote_pad;
> +};
> +
> +static inline struct csi2dc_device *
> +csi2dc_sd_to_csi2dc_device(struct v4l2_subdev *csi2dc_sd)
> +{
> + return container_of(csi2dc_sd, struct csi2dc_device, csi2dc_sd);
> +}
> +
> +static int csi2dc_enum_mbus_code(struct v4l2_subdev *csi2dc_sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_mbus_code_enum *code)
> +{
> + if (code->index >= ARRAY_SIZE(csi2dc_formats))
> + return -EINVAL;
> +
> + code->code = csi2dc_formats[code->index].mbus_code;
> +
> + return 0;
> +}
> +
> +static int csi2dc_get_fmt(struct v4l2_subdev *csi2dc_sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_format *format)
> +{
> + struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
> + struct v4l2_mbus_framefmt *v4l2_try_fmt;
> +
> + if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
> + v4l2_try_fmt = v4l2_subdev_get_try_format(csi2dc_sd, sd_state,
> + format->pad);
> + format->format = *v4l2_try_fmt;
> +
> + return 0;
> + }
> +
> + format->format = csi2dc->format;
> +
> + return 0;
> +}
> +
> +static int csi2dc_set_fmt(struct v4l2_subdev *csi2dc_sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_format *req_fmt)
> +{
> + struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
> + const struct csi2dc_format *fmt, *try_fmt = NULL;
> + struct v4l2_mbus_framefmt *v4l2_try_fmt;
> + unsigned int i;
> +
> + /*
> + * Setting the source pad is disabled.
> + * The same format is being propagated from the sink to source.
> + */
> + if (req_fmt->pad == CSI2DC_PAD_SOURCE)
> + return -EINVAL;
> +
> + for (i = 0; i < ARRAY_SIZE(csi2dc_formats); i++) {
> + fmt = &csi2dc_formats[i];
> + if (req_fmt->format.code == fmt->mbus_code)
> + try_fmt = fmt;
> + fmt++;
> + }
> +
> + /* in case we could not find the desired format, default to something */
> + if (!try_fmt) {
> + try_fmt = &csi2dc_formats[0];
> +
> + dev_dbg(csi2dc->dev,
> + "CSI2DC unsupported format 0x%x, defaulting to 0x%x\n",
> + req_fmt->format.code, csi2dc_formats[0].mbus_code);
> + }
> +
> + req_fmt->format.code = try_fmt->mbus_code;
> + req_fmt->format.colorspace = V4L2_COLORSPACE_SRGB;
> + req_fmt->format.field = V4L2_FIELD_NONE;
> +
> + if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> + v4l2_try_fmt = v4l2_subdev_get_try_format(csi2dc_sd, sd_state,
> + req_fmt->pad);
> + *v4l2_try_fmt = req_fmt->format;
> + /* Trying on the sink pad makes the source pad change too */
> + v4l2_try_fmt = v4l2_subdev_get_try_format(csi2dc_sd,
> + sd_state,
> + CSI2DC_PAD_SOURCE);
> + *v4l2_try_fmt = req_fmt->format;
> +
> + /* if we are just trying, we are done */
> + return 0;
> + }
> +
> + /* save the format for later requests */
> + csi2dc->format = req_fmt->format;
> +
> + /* update config */
> + csi2dc->cur_fmt = try_fmt;
> +
> + dev_dbg(csi2dc->dev, "new format set: 0x%x @%dx%d\n",
> + csi2dc->format.code, csi2dc->format.width,
> + csi2dc->format.height);
> +
> + return 0;
> +}
> +
> +static int csi2dc_power(struct csi2dc_device *csi2dc, int on)
> +{
> + int ret = 0;
> +
> + if (on) {
> + ret = clk_prepare_enable(csi2dc->pclk);
> + if (ret) {
> + dev_err(csi2dc->dev, "failed to enable pclk:%d\n", ret);
> + return ret;
> + }
> +
> + ret = clk_prepare_enable(csi2dc->scck);
> + if (ret) {
> + dev_err(csi2dc->dev, "failed to enable scck:%d\n", ret);
> + clk_disable_unprepare(csi2dc->pclk);
> + return ret;
> + }
> +
> + /* if powering up, deassert reset line */
> + csi2dc_writel(csi2dc, CSI2DC_GCTLR, CSI2DC_GCTLR_SWRST);
> + } else {
> + /* if powering down, assert reset line */
> + csi2dc_writel(csi2dc, CSI2DC_GCTLR, 0);
> +
> + clk_disable_unprepare(csi2dc->scck);
> + clk_disable_unprepare(csi2dc->pclk);
> + }
> +
> + return ret;
> +}
> +
> +static int csi2dc_get_mbus_config(struct csi2dc_device *csi2dc)
> +{
> + struct v4l2_mbus_config mbus_config = { 0 };
> + int ret;
> +
> + ret = v4l2_subdev_call(csi2dc->input_sd, pad, get_mbus_config,
> + csi2dc->remote_pad, &mbus_config);
> + if (ret == -ENOIOCTLCMD) {
> + dev_dbg(csi2dc->dev,
> + "no remote mbus configuration available\n");
> + goto csi2dc_get_mbus_config_defaults;
> + }
> +
> + if (ret) {
> + dev_err(csi2dc->dev,
> + "failed to get remote mbus configuration\n");
> + goto csi2dc_get_mbus_config_defaults;
> + }
> +
> + if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_0)
> + csi2dc->vc = 0;
> + else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_1)
> + csi2dc->vc = 1;
> + else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_2)
> + csi2dc->vc = 2;
> + else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_3)
> + csi2dc->vc = 3;
> +
> + dev_dbg(csi2dc->dev, "subdev sending on channel %d\n", csi2dc->vc);
> +
> + csi2dc->clk_gated = mbus_config.flags &
> + V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK;
> +
> + dev_dbg(csi2dc->dev, "mbus_config: %s clock\n",
> + csi2dc->clk_gated ? "gated" : "free running");
> +
> + return 0;
> +
> +csi2dc_get_mbus_config_defaults:
> + csi2dc->vc = 0; /* Virtual ID 0 by default */
> +
> + return 0;
> +}
> +
> +static void csi2dc_vp_update(struct csi2dc_device *csi2dc)
> +{
> + u32 vp, gcfg;
> +
> + if (!csi2dc->video_pipe) {
> + dev_err(csi2dc->dev, "video pipeline unavailable\n");
> + return;
> + }
> +
> + if (csi2dc->parallel_mode) {
> + /* In parallel mode, GPIO parallel interface must be selected */
> + gcfg = csi2dc_readl(csi2dc, CSI2DC_GCFG);
> + gcfg |= CSI2DC_GCFG_GPIOSEL;
> + csi2dc_writel(csi2dc, CSI2DC_GCFG, gcfg);
> + return;
> + }
> +
> + /* serial video pipeline */
> +
> + csi2dc_writel(csi2dc, CSI2DC_GCFG,
> + (SAMA7G5_HLC & CSI2DC_GCFG_HLC_MASK) |
> + (csi2dc->clk_gated ? 0 : CSI2DC_GCFG_MIPIFRN));
> +
> + vp = CSI2DC_VPCFG_DT(csi2dc->cur_fmt->dt) & CSI2DC_VPCFG_DT_MASK;
> + vp |= CSI2DC_VPCFG_VC(csi2dc->vc) & CSI2DC_VPCFG_VC_MASK;
> + vp &= ~CSI2DC_VPCFG_DE;
> + vp |= CSI2DC_VPCFG_DM(CSI2DC_VPCFG_DM_DECODER8TO12);
> + vp &= ~CSI2DC_VPCFG_DP2;
> + vp &= ~CSI2DC_VPCFG_RMS;
> + vp |= CSI2DC_VPCFG_PA;
> +
> + csi2dc_writel(csi2dc, CSI2DC_VPCFG, vp);
> + csi2dc_writel(csi2dc, CSI2DC_VPE, CSI2DC_VPE_ENABLE);
> + csi2dc_writel(csi2dc, CSI2DC_PU, CSI2DC_PU_VP);
> +}
> +
> +static int csi2dc_s_stream(struct v4l2_subdev *csi2dc_sd, int enable)
> +{
> + struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
> + int ret;
> +
> + if (enable) {
> + ret = pm_runtime_resume_and_get(csi2dc->dev);
> + if (ret < 0)
> + return ret;
> +
> + csi2dc_get_mbus_config(csi2dc);
> +
> + csi2dc_vp_update(csi2dc);
> +
> + return v4l2_subdev_call(csi2dc->input_sd, video, s_stream,
> + true);
> + }
> +
> + dev_dbg(csi2dc->dev,
> + "Last frame received: VPCOLR = %u, VPROWR= %u, VPISR = %x\n",
> + csi2dc_readl(csi2dc, CSI2DC_VPCOL),
> + csi2dc_readl(csi2dc, CSI2DC_VPROW),
> + csi2dc_readl(csi2dc, CSI2DC_VPISR));
> +
> + /* stop streaming scenario */
> + ret = v4l2_subdev_call(csi2dc->input_sd, video, s_stream, false);
> +
> + pm_runtime_put_sync(csi2dc->dev);
> +
> + return ret;
> +}
> +
> +static int csi2dc_init_cfg(struct v4l2_subdev *csi2dc_sd,
> + struct v4l2_subdev_state *sd_state)
> +{
> + struct v4l2_mbus_framefmt *v4l2_try_fmt =
> + v4l2_subdev_get_try_format(csi2dc_sd, sd_state, 0);
> +
> + v4l2_try_fmt->height = 480;
> + v4l2_try_fmt->width = 640;
> + v4l2_try_fmt->code = csi2dc_formats[0].mbus_code;
> + v4l2_try_fmt->colorspace = V4L2_COLORSPACE_SRGB;
> + v4l2_try_fmt->field = V4L2_FIELD_NONE;
> + v4l2_try_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> + v4l2_try_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
> + v4l2_try_fmt->xfer_func = V4L2_XFER_FUNC_DEFAULT;
> +
> + return 0;
> +}
> +
> +static const struct v4l2_subdev_pad_ops csi2dc_pad_ops = {
> + .enum_mbus_code = csi2dc_enum_mbus_code,
> + .set_fmt = csi2dc_set_fmt,
> + .get_fmt = csi2dc_get_fmt,
> + .init_cfg = csi2dc_init_cfg,
> +};
> +
> +static const struct v4l2_subdev_video_ops csi2dc_video_ops = {
> + .s_stream = csi2dc_s_stream,
> +};
> +
> +static const struct v4l2_subdev_ops csi2dc_subdev_ops = {
> + .pad = &csi2dc_pad_ops,
> + .video = &csi2dc_video_ops,
> +};
> +
> +static int csi2dc_async_bound(struct v4l2_async_notifier *notifier,
> + struct v4l2_subdev *subdev,
> + struct v4l2_async_subdev *asd)
> +{
> + struct csi2dc_device *csi2dc = container_of(notifier,
> + struct csi2dc_device, notifier);
> + int pad;
> + int ret;
> +
> + csi2dc->input_sd = subdev;
> +
> + pad = media_entity_get_fwnode_pad(&subdev->entity, asd->match.fwnode,
> + MEDIA_PAD_FL_SOURCE);
> + if (pad < 0) {
> + dev_err(csi2dc->dev, "Failed to find pad for %s\n",
> + subdev->name);
> + return pad;
> + }
> +
> + csi2dc->remote_pad = pad;
> +
> + ret = media_create_pad_link(&csi2dc->input_sd->entity,
> + csi2dc->remote_pad,
> + &csi2dc->csi2dc_sd.entity, 0,
> + MEDIA_LNK_FL_ENABLED);
> + if (ret) {
> + dev_err(csi2dc->dev,
> + "Failed to create pad link: %s to %s\n",
> + csi2dc->input_sd->entity.name,
> + csi2dc->csi2dc_sd.entity.name);
> + return ret;
> + }
> +
> + dev_dbg(csi2dc->dev, "link with %s pad: %d\n",
> + csi2dc->input_sd->name, csi2dc->remote_pad);
> +
> + return ret;
> +}
> +
> +static const struct v4l2_async_notifier_operations csi2dc_async_ops = {
> + .bound = csi2dc_async_bound,
> +};
> +
> +static int csi2dc_prepare_notifier(struct csi2dc_device *csi2dc,
> + struct fwnode_handle *input_fwnode)
> +{
> + struct v4l2_async_subdev *asd;
> + int ret = 0;
> +
> + v4l2_async_nf_init(&csi2dc->notifier);
> +
> + asd = v4l2_async_nf_add_fwnode_remote(&csi2dc->notifier,
> + input_fwnode,
> + struct v4l2_async_subdev);
> +
> + fwnode_handle_put(input_fwnode);
> +
> + if (IS_ERR(asd)) {
> + ret = PTR_ERR(asd);
> + dev_err(csi2dc->dev,
> + "failed to add async notifier for node %pOF: %d\n",
> + to_of_node(input_fwnode), ret);
> + v4l2_async_nf_cleanup(&csi2dc->notifier);
> + return ret;
> + }
> +
> + csi2dc->notifier.ops = &csi2dc_async_ops;
> +
> + ret = v4l2_async_subdev_nf_register(&csi2dc->csi2dc_sd,
> + &csi2dc->notifier);
> + if (ret) {
> + dev_err(csi2dc->dev, "fail to register async notifier: %d\n",
> + ret);
> + v4l2_async_nf_cleanup(&csi2dc->notifier);
> + }
> +
> + return ret;
> +}
> +
> +static int csi2dc_of_parse(struct csi2dc_device *csi2dc,
> + struct device_node *of_node)
> +{
> + struct fwnode_handle *input_fwnode, *output_fwnode;
> + struct v4l2_fwnode_endpoint input_endpoint = { 0 },
> + output_endpoint = { 0 };
> + int ret;
> +
> + input_fwnode = fwnode_graph_get_next_endpoint(of_fwnode_handle(of_node),
> + NULL);
> + if (!input_fwnode) {
> + dev_err(csi2dc->dev,
> + "missing port node at %pOF, input node is mandatory.\n",
> + of_node);
> + return -EINVAL;
> + }
> +
> + ret = v4l2_fwnode_endpoint_parse(input_fwnode, &input_endpoint);
> + if (ret) {
> + dev_err(csi2dc->dev, "endpoint not defined at %pOF\n", of_node);
> + goto csi2dc_of_parse_err;
> + }
> +
> + if (input_endpoint.bus_type == V4L2_MBUS_PARALLEL ||
> + input_endpoint.bus_type == V4L2_MBUS_BT656) {
> + csi2dc->parallel_mode = true;
> + dev_dbg(csi2dc->dev,
> + "subdevice connected on parallel interface\n");
> + }
> +
> + if (input_endpoint.bus_type == V4L2_MBUS_CSI2_DPHY) {
> + csi2dc->clk_gated = input_endpoint.bus.mipi_csi2.flags &
> + V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK;
> + dev_dbg(csi2dc->dev,
> + "subdevice connected on serial interface\n");
> + dev_dbg(csi2dc->dev, "DT: %s clock\n",
> + csi2dc->clk_gated ? "gated" : "free running");
> + }
> +
> + output_fwnode = fwnode_graph_get_next_endpoint
> + (of_fwnode_handle(of_node), input_fwnode);
> +
> + if (output_fwnode)
> + ret = v4l2_fwnode_endpoint_parse(output_fwnode,
> + &output_endpoint);
> +
> + fwnode_handle_put(output_fwnode);
> +
> + if (!output_fwnode || ret) {
> + dev_info(csi2dc->dev,
> + "missing output node at %pOF, data pipe available only.\n",
> + of_node);
> + } else {
> + if (output_endpoint.bus_type != V4L2_MBUS_PARALLEL &&
> + output_endpoint.bus_type != V4L2_MBUS_BT656) {
> + dev_err(csi2dc->dev,
> + "output port must be parallel/bt656.\n");
> + ret = -EINVAL;
> + goto csi2dc_of_parse_err;
> + }
> +
> + csi2dc->video_pipe = true;
> +
> + dev_dbg(csi2dc->dev,
> + "block %pOF [%d.%d]->[%d.%d] video pipeline\n",
> + of_node, input_endpoint.base.port,
> + input_endpoint.base.id, output_endpoint.base.port,
> + output_endpoint.base.id);
> + }
> +
> + /* prepare async notifier for subdevice completion */
> + return csi2dc_prepare_notifier(csi2dc, input_fwnode);
> +
> +csi2dc_of_parse_err:
> + fwnode_handle_put(input_fwnode);
> + return ret;
> +}
> +
> +static void csi2dc_default_format(struct csi2dc_device *csi2dc)
> +{
> + csi2dc->cur_fmt = &csi2dc_formats[0];
> +
> + csi2dc->format.height = 480;
> + csi2dc->format.width = 640;
> + csi2dc->format.code = csi2dc_formats[0].mbus_code;
> + csi2dc->format.colorspace = V4L2_COLORSPACE_SRGB;
> + csi2dc->format.field = V4L2_FIELD_NONE;
> + csi2dc->format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> + csi2dc->format.quantization = V4L2_QUANTIZATION_DEFAULT;
> + csi2dc->format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
> +}
> +
> +static int csi2dc_probe(struct platform_device *pdev)
> +{
> + struct device *dev = &pdev->dev;
> + struct csi2dc_device *csi2dc;
> + int ret = 0;
> + u32 ver;
> +
> + csi2dc = devm_kzalloc(dev, sizeof(*csi2dc), GFP_KERNEL);
> + if (!csi2dc)
> + return -ENOMEM;
> +
> + csi2dc->dev = dev;
> +
> + csi2dc->base = devm_platform_ioremap_resource(pdev, 0);
> + if (IS_ERR(csi2dc->base)) {
> + dev_err(dev, "base address not set\n");
> + return PTR_ERR(csi2dc->base);
> + }
> +
> + csi2dc->pclk = devm_clk_get(dev, "pclk");
> + if (IS_ERR(csi2dc->pclk)) {
> + ret = PTR_ERR(csi2dc->pclk);
> + dev_err(dev, "failed to get pclk: %d\n", ret);
> + return ret;
> + }
> +
> + csi2dc->scck = devm_clk_get(dev, "scck");
> + if (IS_ERR(csi2dc->scck)) {
> + ret = PTR_ERR(csi2dc->scck);
> + dev_err(dev, "failed to get scck: %d\n", ret);
> + return ret;
> + }
> +
> + v4l2_subdev_init(&csi2dc->csi2dc_sd, &csi2dc_subdev_ops);
> +
> + csi2dc->csi2dc_sd.owner = THIS_MODULE;
> + csi2dc->csi2dc_sd.dev = dev;
> + snprintf(csi2dc->csi2dc_sd.name, sizeof(csi2dc->csi2dc_sd.name),
> + "csi2dc");
> +
> + csi2dc->csi2dc_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> + csi2dc->csi2dc_sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> +
> + platform_set_drvdata(pdev, csi2dc);
> +
> + ret = csi2dc_of_parse(csi2dc, dev->of_node);
> + if (ret)
> + goto csi2dc_probe_cleanup_entity;
> +
> + csi2dc->pads[CSI2DC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> + if (csi2dc->video_pipe)
> + csi2dc->pads[CSI2DC_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
> +
> + ret = media_entity_pads_init(&csi2dc->csi2dc_sd.entity,
> + csi2dc->video_pipe ? CSI2DC_PADS_NUM : 1,
> + csi2dc->pads);
> + if (ret < 0) {
> + dev_err(dev, "media entity init failed\n");
> + goto csi2dc_probe_cleanup_notifier;
> + }
> +
> + csi2dc_default_format(csi2dc);
> +
> + /* turn power on to validate capabilities */
> + ret = csi2dc_power(csi2dc, true);
> + if (ret < 0)
> + goto csi2dc_probe_cleanup_notifier;
> +
> + pm_runtime_set_active(dev);
> + pm_runtime_enable(dev);
> + ver = csi2dc_readl(csi2dc, CSI2DC_VERSION);
> +
> + /*
> + * we must register the subdev after PM runtime has been requested,
> + * otherwise we might bound immediately and request pm_runtime_resume
> + * before runtime_enable.
> + */
> + ret = v4l2_async_register_subdev(&csi2dc->csi2dc_sd);
> + if (ret) {
> + dev_err(csi2dc->dev, "failed to register the subdevice\n");
> + goto csi2dc_probe_cleanup_notifier;
> + }
> +
> + dev_info(dev, "Microchip CSI2DC version %x\n", ver);
> +
> + return 0;
> +
> +csi2dc_probe_cleanup_notifier:
> + v4l2_async_nf_cleanup(&csi2dc->notifier);
> +csi2dc_probe_cleanup_entity:
> + media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
> +
> + return ret;
> +}
> +
> +static int csi2dc_remove(struct platform_device *pdev)
> +{
> + struct csi2dc_device *csi2dc = platform_get_drvdata(pdev);
> +
> + pm_runtime_disable(&pdev->dev);
> +
> + v4l2_async_unregister_subdev(&csi2dc->csi2dc_sd);
> + v4l2_async_nf_unregister(&csi2dc->notifier);
> + v4l2_async_nf_cleanup(&csi2dc->notifier);
> + media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
> +
> + return 0;
> +}
> +
> +static int __maybe_unused csi2dc_runtime_suspend(struct device *dev)
> +{
> + struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
> +
> + return csi2dc_power(csi2dc, false);
> +}
> +
> +static int __maybe_unused csi2dc_runtime_resume(struct device *dev)
> +{
> + struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
> +
> + return csi2dc_power(csi2dc, true);
> +}
> +
> +static const struct dev_pm_ops csi2dc_dev_pm_ops = {
> + SET_RUNTIME_PM_OPS(csi2dc_runtime_suspend, csi2dc_runtime_resume, NULL)
> +};
> +
> +static const struct of_device_id csi2dc_of_match[] = {
> + { .compatible = "microchip,sama7g5-csi2dc" },
> + { }
> +};
> +
> +MODULE_DEVICE_TABLE(of, csi2dc_of_match);
> +
> +static struct platform_driver csi2dc_driver = {
> + .probe = csi2dc_probe,
> + .remove = csi2dc_remove,
> + .driver = {
> + .name = "microchip-csi2dc",
> + .pm = &csi2dc_dev_pm_ops,
> + .of_match_table = of_match_ptr(csi2dc_of_match),
> + },
> +};
> +
> +module_platform_driver(csi2dc_driver);
> +
> +MODULE_AUTHOR("Eugen Hristev <eugen.hristev@microchip.com>");
> +MODULE_DESCRIPTION("Microchip CSI2 Demux Controller driver");
> +MODULE_LICENSE("GPL v2");
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 03/23] media: atmel: introduce microchip csi2dc driver
@ 2022-01-11 15:43 ` Jacopo Mondi
0 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-11 15:43 UTC (permalink / raw)
To: Eugen Hristev
Cc: devicetree, linux-kernel, robh+dt, sakari.ailus,
laurent.pinchart, linux-arm-kernel, linux-media
Hi Eugen
On Mon, Dec 13, 2021 at 03:49:20PM +0200, Eugen Hristev wrote:
> Microchip CSI2DC (CSI2 Demultiplexer Controller) is a misc bridge device
> that converts a byte stream in IDI Synopsys format (coming from a CSI2HOST)
> to a pixel stream that can be captured by a sensor controller.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
Thanks for addressing my comments on previous versions
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Thanks
j
> ---
> Changes in v3:
> - on set format, disable setting the source sink.
> - removed useless request to pm_request_idle
> - fixed YUV DT
> - added last frame debug info
> - removed writing read only VPCOL and VPROW registers
> - minor cosmetic changes
>
> Changes in v2:
> - implement try formats
> - added parallel mode
> - moved to fwlink endpoints
> - many other minor corrections from Jacopo's review
>
> Changes in this revision:
> - addressed comments by Jacopo and Laurent as in this thread:
> https://www.spinics.net/lists/linux-media/msg181044.html
>
> Previous change log :
> Changes in v5:
> - only in bindings
>
> Changes in v4:
> - now using get_mbus_config ops to get data from the subdevice, like the
> virtual channel id, and the clock type.
> - now having possibility to select any of the RAW10 data modes
> - at completion time, select which formats are also available in the subdevice,
> and move to the dynamic list accordingly
> - changed the pipeline integration, do not advertise subdev ready at probe time.
> wait until completion is done, and then start a workqueue that will register
> this device as a subdevice for the next element in pipeline.
> - moved the s_power code into a different function called now csi2dc_power
> that is called with CONFIG_PM functions. This is also called at completion,
> to have the device ready in case CONFIG_PM is not selected on the platform.
> - merged try_fmt into set_fmt
> - driver cleanup, wrapped lines over 80 characters
>
> Changes in v2:
> - moved driver to platform/atmel
> - fixed minor things as per Sakari's review
> - still some things from v2 review are not yet addressed, to be followed up
>
>
>
> drivers/media/platform/Makefile | 1 +
> drivers/media/platform/atmel/Kconfig | 15 +
> drivers/media/platform/atmel/Makefile | 1 +
> .../media/platform/atmel/microchip-csi2dc.c | 806 ++++++++++++++++++
> 4 files changed, 823 insertions(+)
> create mode 100644 drivers/media/platform/atmel/microchip-csi2dc.c
>
> diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> index 832357240e89..b18e5f704145 100644
> --- a/drivers/media/platform/Makefile
> +++ b/drivers/media/platform/Makefile
> @@ -69,6 +69,7 @@ obj-$(CONFIG_VIDEO_RCAR_VIN) += rcar-vin/
> obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel/
> obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel/
> obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel/
> +obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += atmel/
>
> obj-$(CONFIG_VIDEO_STM32_DCMI) += stm32/
>
> diff --git a/drivers/media/platform/atmel/Kconfig b/drivers/media/platform/atmel/Kconfig
> index dda2f27da317..f83bee373d82 100644
> --- a/drivers/media/platform/atmel/Kconfig
> +++ b/drivers/media/platform/atmel/Kconfig
> @@ -40,3 +40,18 @@ config VIDEO_ATMEL_ISI
> help
> This module makes the ATMEL Image Sensor Interface available
> as a v4l2 device.
> +
> +config VIDEO_MICROCHIP_CSI2DC
> + tristate "Microchip CSI2 Demux Controller"
> + depends on VIDEO_V4L2 && COMMON_CLK && OF
> + depends on ARCH_AT91 || COMPILE_TEST
> + select MEDIA_CONTROLLER
> + select VIDEO_V4L2_SUBDEV_API
> + select V4L2_FWNODE
> + help
> + CSI2 Demux Controller driver. CSI2DC is a helper chip
> + that converts IDI interface byte stream to a parallel pixel stream.
> + It supports various RAW formats as input.
> +
> + To compile this driver as a module, choose M here: the
> + module will be called microchip-csi2dc.
> diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
> index 46d264ab7948..39f0a7eba702 100644
> --- a/drivers/media/platform/atmel/Makefile
> +++ b/drivers/media/platform/atmel/Makefile
> @@ -6,3 +6,4 @@ obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
> obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o
> obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel-isc.o
> obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel-xisc.o
> +obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += microchip-csi2dc.o
> diff --git a/drivers/media/platform/atmel/microchip-csi2dc.c b/drivers/media/platform/atmel/microchip-csi2dc.c
> new file mode 100644
> index 000000000000..6bc549c28e05
> --- /dev/null
> +++ b/drivers/media/platform/atmel/microchip-csi2dc.c
> @@ -0,0 +1,806 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Microchip CSI2 Demux Controller (CSI2DC) driver
> + *
> + * Copyright (C) 2018 Microchip Technology, Inc.
> + *
> + * Author: Eugen Hristev <eugen.hristev@microchip.com>
> + *
> + */
> +
> +#include <linux/clk.h>
> +#include <linux/mod_devicetable.h>
> +#include <linux/module.h>
> +#include <linux/of_graph.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/videodev2.h>
> +
> +#include <media/v4l2-fwnode.h>
> +#include <media/v4l2-subdev.h>
> +
> +/* Global configuration register */
> +#define CSI2DC_GCFG 0x0
> +
> +/* MIPI sensor pixel clock is free running */
> +#define CSI2DC_GCFG_MIPIFRN BIT(0)
> +/* GPIO parallel interface selection */
> +#define CSI2DC_GCFG_GPIOSEL BIT(1)
> +/* Output waveform inter-line minimum delay */
> +#define CSI2DC_GCFG_HLC(v) ((v) << 4)
> +#define CSI2DC_GCFG_HLC_MASK GENMASK(7, 4)
> +/* SAMA7G5 requires a HLC delay of 15 */
> +#define SAMA7G5_HLC (15)
> +
> +/* Global control register */
> +#define CSI2DC_GCTLR 0x04
> +#define CSI2DC_GCTLR_SWRST BIT(0)
> +
> +/* Global status register */
> +#define CSI2DC_GS 0x08
> +
> +/* SSP interrupt status register */
> +#define CSI2DC_SSPIS 0x28
> +/* Pipe update register */
> +#define CSI2DC_PU 0xc0
> +/* Video pipe attributes update */
> +#define CSI2DC_PU_VP BIT(0)
> +
> +/* Pipe update status register */
> +#define CSI2DC_PUS 0xc4
> +
> +/* Video pipeline Interrupt Status Register */
> +#define CSI2DC_VPISR 0xf4
> +
> +/* Video pipeline enable register */
> +#define CSI2DC_VPE 0xf8
> +#define CSI2DC_VPE_ENABLE BIT(0)
> +
> +/* Video pipeline configuration register */
> +#define CSI2DC_VPCFG 0xfc
> +/* Data type */
> +#define CSI2DC_VPCFG_DT(v) ((v) << 0)
> +#define CSI2DC_VPCFG_DT_MASK GENMASK(5, 0)
> +/* Virtual channel identifier */
> +#define CSI2DC_VPCFG_VC(v) ((v) << 6)
> +#define CSI2DC_VPCFG_VC_MASK GENMASK(7, 6)
> +/* Decompression enable */
> +#define CSI2DC_VPCFG_DE BIT(8)
> +/* Decoder mode */
> +#define CSI2DC_VPCFG_DM(v) ((v) << 9)
> +#define CSI2DC_VPCFG_DM_DECODER8TO12 0
> +/* Decoder predictor 2 selection */
> +#define CSI2DC_VPCFG_DP2 BIT(12)
> +/* Recommended memory storage */
> +#define CSI2DC_VPCFG_RMS BIT(13)
> +/* Post adjustment */
> +#define CSI2DC_VPCFG_PA BIT(14)
> +
> +/* Video pipeline column register */
> +#define CSI2DC_VPCOL 0x100
> +/* Column number */
> +#define CSI2DC_VPCOL_COL(v) ((v) << 0)
> +#define CSI2DC_VPCOL_COL_MASK GENMASK(15, 0)
> +
> +/* Video pipeline row register */
> +#define CSI2DC_VPROW 0x104
> +/* Row number */
> +#define CSI2DC_VPROW_ROW(v) ((v) << 0)
> +#define CSI2DC_VPROW_ROW_MASK GENMASK(15, 0)
> +
> +/* Version register */
> +#define CSI2DC_VERSION 0x1fc
> +
> +/* register read/write helpers */
> +#define csi2dc_readl(st, reg) readl_relaxed((st)->base + (reg))
> +#define csi2dc_writel(st, reg, val) writel_relaxed((val), \
> + (st)->base + (reg))
> +
> +/* supported RAW data types */
> +#define CSI2DC_DT_RAW6 0x28
> +#define CSI2DC_DT_RAW7 0x29
> +#define CSI2DC_DT_RAW8 0x2a
> +#define CSI2DC_DT_RAW10 0x2b
> +#define CSI2DC_DT_RAW12 0x2c
> +#define CSI2DC_DT_RAW14 0x2d
> +/* YUV data types */
> +#define CSI2DC_DT_YUV422_8B 0x1e
> +
> +/*
> + * struct csi2dc_format - CSI2DC format type struct
> + * @mbus_code: Media bus code for the format
> + * @dt: Data type constant for this format
> + */
> +struct csi2dc_format {
> + u32 mbus_code;
> + u32 dt;
> +};
> +
> +static const struct csi2dc_format csi2dc_formats[] = {
> + {
> + .mbus_code = MEDIA_BUS_FMT_SRGGB8_1X8,
> + .dt = CSI2DC_DT_RAW8,
> + }, {
> + .mbus_code = MEDIA_BUS_FMT_SBGGR8_1X8,
> + .dt = CSI2DC_DT_RAW8,
> + }, {
> + .mbus_code = MEDIA_BUS_FMT_SGRBG8_1X8,
> + .dt = CSI2DC_DT_RAW8,
> + }, {
> + .mbus_code = MEDIA_BUS_FMT_SGBRG8_1X8,
> + .dt = CSI2DC_DT_RAW8,
> + }, {
> + .mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10,
> + .dt = CSI2DC_DT_RAW10,
> + }, {
> + .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10,
> + .dt = CSI2DC_DT_RAW10,
> + }, {
> + .mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10,
> + .dt = CSI2DC_DT_RAW10,
> + }, {
> + .mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10,
> + .dt = CSI2DC_DT_RAW10,
> + }, {
> + .mbus_code = MEDIA_BUS_FMT_YUYV8_2X8,
> + .dt = CSI2DC_DT_YUV422_8B,
> + },
> +};
> +
> +enum mipi_csi_pads {
> + CSI2DC_PAD_SINK = 0,
> + CSI2DC_PAD_SOURCE = 1,
> + CSI2DC_PADS_NUM = 2,
> +};
> +
> +/*
> + * struct csi2dc_device - CSI2DC device driver data/config struct
> + * @base: Register map base address
> + * @csi2dc_sd: v4l2 subdevice for the csi2dc device
> + * This is the subdevice that the csi2dc device itself
> + * registers in v4l2 subsystem
> + * @dev: struct device for this csi2dc device
> + * @pclk: Peripheral clock reference
> + * Input clock that clocks the hardware block internal
> + * logic
> + * @scck: Sensor Controller clock reference
> + * Input clock that is used to generate the pixel clock
> + * @format: Current saved format used in g/s fmt
> + * @cur_fmt: Current state format
> + * @try_fmt: Try format that is being tried
> + * @pads: Media entity pads for the csi2dc subdevice
> + * @clk_gated: Whether the clock is gated or free running
> + * @video_pipe: Whether video pipeline is configured
> + * @parallel_mode: The underlying subdevice is connected on a parallel bus
> + * @vc: Current set virtual channel
> + * @notifier: Async notifier that is used to bound the underlying
> + * subdevice to the csi2dc subdevice
> + * @input_sd: Reference to the underlying subdevice bound to the
> + * csi2dc subdevice
> + * @remote_pad: Pad number of the underlying subdevice that is linked
> + * to the csi2dc subdevice sink pad.
> + */
> +struct csi2dc_device {
> + void __iomem *base;
> + struct v4l2_subdev csi2dc_sd;
> + struct device *dev;
> + struct clk *pclk;
> + struct clk *scck;
> +
> + struct v4l2_mbus_framefmt format;
> +
> + const struct csi2dc_format *cur_fmt;
> + const struct csi2dc_format *try_fmt;
> +
> + struct media_pad pads[CSI2DC_PADS_NUM];
> +
> + bool clk_gated;
> + bool video_pipe;
> + bool parallel_mode;
> + u32 vc;
> +
> + struct v4l2_async_notifier notifier;
> +
> + struct v4l2_subdev *input_sd;
> +
> + u32 remote_pad;
> +};
> +
> +static inline struct csi2dc_device *
> +csi2dc_sd_to_csi2dc_device(struct v4l2_subdev *csi2dc_sd)
> +{
> + return container_of(csi2dc_sd, struct csi2dc_device, csi2dc_sd);
> +}
> +
> +static int csi2dc_enum_mbus_code(struct v4l2_subdev *csi2dc_sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_mbus_code_enum *code)
> +{
> + if (code->index >= ARRAY_SIZE(csi2dc_formats))
> + return -EINVAL;
> +
> + code->code = csi2dc_formats[code->index].mbus_code;
> +
> + return 0;
> +}
> +
> +static int csi2dc_get_fmt(struct v4l2_subdev *csi2dc_sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_format *format)
> +{
> + struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
> + struct v4l2_mbus_framefmt *v4l2_try_fmt;
> +
> + if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
> + v4l2_try_fmt = v4l2_subdev_get_try_format(csi2dc_sd, sd_state,
> + format->pad);
> + format->format = *v4l2_try_fmt;
> +
> + return 0;
> + }
> +
> + format->format = csi2dc->format;
> +
> + return 0;
> +}
> +
> +static int csi2dc_set_fmt(struct v4l2_subdev *csi2dc_sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_format *req_fmt)
> +{
> + struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
> + const struct csi2dc_format *fmt, *try_fmt = NULL;
> + struct v4l2_mbus_framefmt *v4l2_try_fmt;
> + unsigned int i;
> +
> + /*
> + * Setting the source pad is disabled.
> + * The same format is being propagated from the sink to source.
> + */
> + if (req_fmt->pad == CSI2DC_PAD_SOURCE)
> + return -EINVAL;
> +
> + for (i = 0; i < ARRAY_SIZE(csi2dc_formats); i++) {
> + fmt = &csi2dc_formats[i];
> + if (req_fmt->format.code == fmt->mbus_code)
> + try_fmt = fmt;
> + fmt++;
> + }
> +
> + /* in case we could not find the desired format, default to something */
> + if (!try_fmt) {
> + try_fmt = &csi2dc_formats[0];
> +
> + dev_dbg(csi2dc->dev,
> + "CSI2DC unsupported format 0x%x, defaulting to 0x%x\n",
> + req_fmt->format.code, csi2dc_formats[0].mbus_code);
> + }
> +
> + req_fmt->format.code = try_fmt->mbus_code;
> + req_fmt->format.colorspace = V4L2_COLORSPACE_SRGB;
> + req_fmt->format.field = V4L2_FIELD_NONE;
> +
> + if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> + v4l2_try_fmt = v4l2_subdev_get_try_format(csi2dc_sd, sd_state,
> + req_fmt->pad);
> + *v4l2_try_fmt = req_fmt->format;
> + /* Trying on the sink pad makes the source pad change too */
> + v4l2_try_fmt = v4l2_subdev_get_try_format(csi2dc_sd,
> + sd_state,
> + CSI2DC_PAD_SOURCE);
> + *v4l2_try_fmt = req_fmt->format;
> +
> + /* if we are just trying, we are done */
> + return 0;
> + }
> +
> + /* save the format for later requests */
> + csi2dc->format = req_fmt->format;
> +
> + /* update config */
> + csi2dc->cur_fmt = try_fmt;
> +
> + dev_dbg(csi2dc->dev, "new format set: 0x%x @%dx%d\n",
> + csi2dc->format.code, csi2dc->format.width,
> + csi2dc->format.height);
> +
> + return 0;
> +}
> +
> +static int csi2dc_power(struct csi2dc_device *csi2dc, int on)
> +{
> + int ret = 0;
> +
> + if (on) {
> + ret = clk_prepare_enable(csi2dc->pclk);
> + if (ret) {
> + dev_err(csi2dc->dev, "failed to enable pclk:%d\n", ret);
> + return ret;
> + }
> +
> + ret = clk_prepare_enable(csi2dc->scck);
> + if (ret) {
> + dev_err(csi2dc->dev, "failed to enable scck:%d\n", ret);
> + clk_disable_unprepare(csi2dc->pclk);
> + return ret;
> + }
> +
> + /* if powering up, deassert reset line */
> + csi2dc_writel(csi2dc, CSI2DC_GCTLR, CSI2DC_GCTLR_SWRST);
> + } else {
> + /* if powering down, assert reset line */
> + csi2dc_writel(csi2dc, CSI2DC_GCTLR, 0);
> +
> + clk_disable_unprepare(csi2dc->scck);
> + clk_disable_unprepare(csi2dc->pclk);
> + }
> +
> + return ret;
> +}
> +
> +static int csi2dc_get_mbus_config(struct csi2dc_device *csi2dc)
> +{
> + struct v4l2_mbus_config mbus_config = { 0 };
> + int ret;
> +
> + ret = v4l2_subdev_call(csi2dc->input_sd, pad, get_mbus_config,
> + csi2dc->remote_pad, &mbus_config);
> + if (ret == -ENOIOCTLCMD) {
> + dev_dbg(csi2dc->dev,
> + "no remote mbus configuration available\n");
> + goto csi2dc_get_mbus_config_defaults;
> + }
> +
> + if (ret) {
> + dev_err(csi2dc->dev,
> + "failed to get remote mbus configuration\n");
> + goto csi2dc_get_mbus_config_defaults;
> + }
> +
> + if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_0)
> + csi2dc->vc = 0;
> + else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_1)
> + csi2dc->vc = 1;
> + else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_2)
> + csi2dc->vc = 2;
> + else if (mbus_config.flags & V4L2_MBUS_CSI2_CHANNEL_3)
> + csi2dc->vc = 3;
> +
> + dev_dbg(csi2dc->dev, "subdev sending on channel %d\n", csi2dc->vc);
> +
> + csi2dc->clk_gated = mbus_config.flags &
> + V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK;
> +
> + dev_dbg(csi2dc->dev, "mbus_config: %s clock\n",
> + csi2dc->clk_gated ? "gated" : "free running");
> +
> + return 0;
> +
> +csi2dc_get_mbus_config_defaults:
> + csi2dc->vc = 0; /* Virtual ID 0 by default */
> +
> + return 0;
> +}
> +
> +static void csi2dc_vp_update(struct csi2dc_device *csi2dc)
> +{
> + u32 vp, gcfg;
> +
> + if (!csi2dc->video_pipe) {
> + dev_err(csi2dc->dev, "video pipeline unavailable\n");
> + return;
> + }
> +
> + if (csi2dc->parallel_mode) {
> + /* In parallel mode, GPIO parallel interface must be selected */
> + gcfg = csi2dc_readl(csi2dc, CSI2DC_GCFG);
> + gcfg |= CSI2DC_GCFG_GPIOSEL;
> + csi2dc_writel(csi2dc, CSI2DC_GCFG, gcfg);
> + return;
> + }
> +
> + /* serial video pipeline */
> +
> + csi2dc_writel(csi2dc, CSI2DC_GCFG,
> + (SAMA7G5_HLC & CSI2DC_GCFG_HLC_MASK) |
> + (csi2dc->clk_gated ? 0 : CSI2DC_GCFG_MIPIFRN));
> +
> + vp = CSI2DC_VPCFG_DT(csi2dc->cur_fmt->dt) & CSI2DC_VPCFG_DT_MASK;
> + vp |= CSI2DC_VPCFG_VC(csi2dc->vc) & CSI2DC_VPCFG_VC_MASK;
> + vp &= ~CSI2DC_VPCFG_DE;
> + vp |= CSI2DC_VPCFG_DM(CSI2DC_VPCFG_DM_DECODER8TO12);
> + vp &= ~CSI2DC_VPCFG_DP2;
> + vp &= ~CSI2DC_VPCFG_RMS;
> + vp |= CSI2DC_VPCFG_PA;
> +
> + csi2dc_writel(csi2dc, CSI2DC_VPCFG, vp);
> + csi2dc_writel(csi2dc, CSI2DC_VPE, CSI2DC_VPE_ENABLE);
> + csi2dc_writel(csi2dc, CSI2DC_PU, CSI2DC_PU_VP);
> +}
> +
> +static int csi2dc_s_stream(struct v4l2_subdev *csi2dc_sd, int enable)
> +{
> + struct csi2dc_device *csi2dc = csi2dc_sd_to_csi2dc_device(csi2dc_sd);
> + int ret;
> +
> + if (enable) {
> + ret = pm_runtime_resume_and_get(csi2dc->dev);
> + if (ret < 0)
> + return ret;
> +
> + csi2dc_get_mbus_config(csi2dc);
> +
> + csi2dc_vp_update(csi2dc);
> +
> + return v4l2_subdev_call(csi2dc->input_sd, video, s_stream,
> + true);
> + }
> +
> + dev_dbg(csi2dc->dev,
> + "Last frame received: VPCOLR = %u, VPROWR= %u, VPISR = %x\n",
> + csi2dc_readl(csi2dc, CSI2DC_VPCOL),
> + csi2dc_readl(csi2dc, CSI2DC_VPROW),
> + csi2dc_readl(csi2dc, CSI2DC_VPISR));
> +
> + /* stop streaming scenario */
> + ret = v4l2_subdev_call(csi2dc->input_sd, video, s_stream, false);
> +
> + pm_runtime_put_sync(csi2dc->dev);
> +
> + return ret;
> +}
> +
> +static int csi2dc_init_cfg(struct v4l2_subdev *csi2dc_sd,
> + struct v4l2_subdev_state *sd_state)
> +{
> + struct v4l2_mbus_framefmt *v4l2_try_fmt =
> + v4l2_subdev_get_try_format(csi2dc_sd, sd_state, 0);
> +
> + v4l2_try_fmt->height = 480;
> + v4l2_try_fmt->width = 640;
> + v4l2_try_fmt->code = csi2dc_formats[0].mbus_code;
> + v4l2_try_fmt->colorspace = V4L2_COLORSPACE_SRGB;
> + v4l2_try_fmt->field = V4L2_FIELD_NONE;
> + v4l2_try_fmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> + v4l2_try_fmt->quantization = V4L2_QUANTIZATION_DEFAULT;
> + v4l2_try_fmt->xfer_func = V4L2_XFER_FUNC_DEFAULT;
> +
> + return 0;
> +}
> +
> +static const struct v4l2_subdev_pad_ops csi2dc_pad_ops = {
> + .enum_mbus_code = csi2dc_enum_mbus_code,
> + .set_fmt = csi2dc_set_fmt,
> + .get_fmt = csi2dc_get_fmt,
> + .init_cfg = csi2dc_init_cfg,
> +};
> +
> +static const struct v4l2_subdev_video_ops csi2dc_video_ops = {
> + .s_stream = csi2dc_s_stream,
> +};
> +
> +static const struct v4l2_subdev_ops csi2dc_subdev_ops = {
> + .pad = &csi2dc_pad_ops,
> + .video = &csi2dc_video_ops,
> +};
> +
> +static int csi2dc_async_bound(struct v4l2_async_notifier *notifier,
> + struct v4l2_subdev *subdev,
> + struct v4l2_async_subdev *asd)
> +{
> + struct csi2dc_device *csi2dc = container_of(notifier,
> + struct csi2dc_device, notifier);
> + int pad;
> + int ret;
> +
> + csi2dc->input_sd = subdev;
> +
> + pad = media_entity_get_fwnode_pad(&subdev->entity, asd->match.fwnode,
> + MEDIA_PAD_FL_SOURCE);
> + if (pad < 0) {
> + dev_err(csi2dc->dev, "Failed to find pad for %s\n",
> + subdev->name);
> + return pad;
> + }
> +
> + csi2dc->remote_pad = pad;
> +
> + ret = media_create_pad_link(&csi2dc->input_sd->entity,
> + csi2dc->remote_pad,
> + &csi2dc->csi2dc_sd.entity, 0,
> + MEDIA_LNK_FL_ENABLED);
> + if (ret) {
> + dev_err(csi2dc->dev,
> + "Failed to create pad link: %s to %s\n",
> + csi2dc->input_sd->entity.name,
> + csi2dc->csi2dc_sd.entity.name);
> + return ret;
> + }
> +
> + dev_dbg(csi2dc->dev, "link with %s pad: %d\n",
> + csi2dc->input_sd->name, csi2dc->remote_pad);
> +
> + return ret;
> +}
> +
> +static const struct v4l2_async_notifier_operations csi2dc_async_ops = {
> + .bound = csi2dc_async_bound,
> +};
> +
> +static int csi2dc_prepare_notifier(struct csi2dc_device *csi2dc,
> + struct fwnode_handle *input_fwnode)
> +{
> + struct v4l2_async_subdev *asd;
> + int ret = 0;
> +
> + v4l2_async_nf_init(&csi2dc->notifier);
> +
> + asd = v4l2_async_nf_add_fwnode_remote(&csi2dc->notifier,
> + input_fwnode,
> + struct v4l2_async_subdev);
> +
> + fwnode_handle_put(input_fwnode);
> +
> + if (IS_ERR(asd)) {
> + ret = PTR_ERR(asd);
> + dev_err(csi2dc->dev,
> + "failed to add async notifier for node %pOF: %d\n",
> + to_of_node(input_fwnode), ret);
> + v4l2_async_nf_cleanup(&csi2dc->notifier);
> + return ret;
> + }
> +
> + csi2dc->notifier.ops = &csi2dc_async_ops;
> +
> + ret = v4l2_async_subdev_nf_register(&csi2dc->csi2dc_sd,
> + &csi2dc->notifier);
> + if (ret) {
> + dev_err(csi2dc->dev, "fail to register async notifier: %d\n",
> + ret);
> + v4l2_async_nf_cleanup(&csi2dc->notifier);
> + }
> +
> + return ret;
> +}
> +
> +static int csi2dc_of_parse(struct csi2dc_device *csi2dc,
> + struct device_node *of_node)
> +{
> + struct fwnode_handle *input_fwnode, *output_fwnode;
> + struct v4l2_fwnode_endpoint input_endpoint = { 0 },
> + output_endpoint = { 0 };
> + int ret;
> +
> + input_fwnode = fwnode_graph_get_next_endpoint(of_fwnode_handle(of_node),
> + NULL);
> + if (!input_fwnode) {
> + dev_err(csi2dc->dev,
> + "missing port node at %pOF, input node is mandatory.\n",
> + of_node);
> + return -EINVAL;
> + }
> +
> + ret = v4l2_fwnode_endpoint_parse(input_fwnode, &input_endpoint);
> + if (ret) {
> + dev_err(csi2dc->dev, "endpoint not defined at %pOF\n", of_node);
> + goto csi2dc_of_parse_err;
> + }
> +
> + if (input_endpoint.bus_type == V4L2_MBUS_PARALLEL ||
> + input_endpoint.bus_type == V4L2_MBUS_BT656) {
> + csi2dc->parallel_mode = true;
> + dev_dbg(csi2dc->dev,
> + "subdevice connected on parallel interface\n");
> + }
> +
> + if (input_endpoint.bus_type == V4L2_MBUS_CSI2_DPHY) {
> + csi2dc->clk_gated = input_endpoint.bus.mipi_csi2.flags &
> + V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK;
> + dev_dbg(csi2dc->dev,
> + "subdevice connected on serial interface\n");
> + dev_dbg(csi2dc->dev, "DT: %s clock\n",
> + csi2dc->clk_gated ? "gated" : "free running");
> + }
> +
> + output_fwnode = fwnode_graph_get_next_endpoint
> + (of_fwnode_handle(of_node), input_fwnode);
> +
> + if (output_fwnode)
> + ret = v4l2_fwnode_endpoint_parse(output_fwnode,
> + &output_endpoint);
> +
> + fwnode_handle_put(output_fwnode);
> +
> + if (!output_fwnode || ret) {
> + dev_info(csi2dc->dev,
> + "missing output node at %pOF, data pipe available only.\n",
> + of_node);
> + } else {
> + if (output_endpoint.bus_type != V4L2_MBUS_PARALLEL &&
> + output_endpoint.bus_type != V4L2_MBUS_BT656) {
> + dev_err(csi2dc->dev,
> + "output port must be parallel/bt656.\n");
> + ret = -EINVAL;
> + goto csi2dc_of_parse_err;
> + }
> +
> + csi2dc->video_pipe = true;
> +
> + dev_dbg(csi2dc->dev,
> + "block %pOF [%d.%d]->[%d.%d] video pipeline\n",
> + of_node, input_endpoint.base.port,
> + input_endpoint.base.id, output_endpoint.base.port,
> + output_endpoint.base.id);
> + }
> +
> + /* prepare async notifier for subdevice completion */
> + return csi2dc_prepare_notifier(csi2dc, input_fwnode);
> +
> +csi2dc_of_parse_err:
> + fwnode_handle_put(input_fwnode);
> + return ret;
> +}
> +
> +static void csi2dc_default_format(struct csi2dc_device *csi2dc)
> +{
> + csi2dc->cur_fmt = &csi2dc_formats[0];
> +
> + csi2dc->format.height = 480;
> + csi2dc->format.width = 640;
> + csi2dc->format.code = csi2dc_formats[0].mbus_code;
> + csi2dc->format.colorspace = V4L2_COLORSPACE_SRGB;
> + csi2dc->format.field = V4L2_FIELD_NONE;
> + csi2dc->format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> + csi2dc->format.quantization = V4L2_QUANTIZATION_DEFAULT;
> + csi2dc->format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
> +}
> +
> +static int csi2dc_probe(struct platform_device *pdev)
> +{
> + struct device *dev = &pdev->dev;
> + struct csi2dc_device *csi2dc;
> + int ret = 0;
> + u32 ver;
> +
> + csi2dc = devm_kzalloc(dev, sizeof(*csi2dc), GFP_KERNEL);
> + if (!csi2dc)
> + return -ENOMEM;
> +
> + csi2dc->dev = dev;
> +
> + csi2dc->base = devm_platform_ioremap_resource(pdev, 0);
> + if (IS_ERR(csi2dc->base)) {
> + dev_err(dev, "base address not set\n");
> + return PTR_ERR(csi2dc->base);
> + }
> +
> + csi2dc->pclk = devm_clk_get(dev, "pclk");
> + if (IS_ERR(csi2dc->pclk)) {
> + ret = PTR_ERR(csi2dc->pclk);
> + dev_err(dev, "failed to get pclk: %d\n", ret);
> + return ret;
> + }
> +
> + csi2dc->scck = devm_clk_get(dev, "scck");
> + if (IS_ERR(csi2dc->scck)) {
> + ret = PTR_ERR(csi2dc->scck);
> + dev_err(dev, "failed to get scck: %d\n", ret);
> + return ret;
> + }
> +
> + v4l2_subdev_init(&csi2dc->csi2dc_sd, &csi2dc_subdev_ops);
> +
> + csi2dc->csi2dc_sd.owner = THIS_MODULE;
> + csi2dc->csi2dc_sd.dev = dev;
> + snprintf(csi2dc->csi2dc_sd.name, sizeof(csi2dc->csi2dc_sd.name),
> + "csi2dc");
> +
> + csi2dc->csi2dc_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> + csi2dc->csi2dc_sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> +
> + platform_set_drvdata(pdev, csi2dc);
> +
> + ret = csi2dc_of_parse(csi2dc, dev->of_node);
> + if (ret)
> + goto csi2dc_probe_cleanup_entity;
> +
> + csi2dc->pads[CSI2DC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> + if (csi2dc->video_pipe)
> + csi2dc->pads[CSI2DC_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
> +
> + ret = media_entity_pads_init(&csi2dc->csi2dc_sd.entity,
> + csi2dc->video_pipe ? CSI2DC_PADS_NUM : 1,
> + csi2dc->pads);
> + if (ret < 0) {
> + dev_err(dev, "media entity init failed\n");
> + goto csi2dc_probe_cleanup_notifier;
> + }
> +
> + csi2dc_default_format(csi2dc);
> +
> + /* turn power on to validate capabilities */
> + ret = csi2dc_power(csi2dc, true);
> + if (ret < 0)
> + goto csi2dc_probe_cleanup_notifier;
> +
> + pm_runtime_set_active(dev);
> + pm_runtime_enable(dev);
> + ver = csi2dc_readl(csi2dc, CSI2DC_VERSION);
> +
> + /*
> + * we must register the subdev after PM runtime has been requested,
> + * otherwise we might bound immediately and request pm_runtime_resume
> + * before runtime_enable.
> + */
> + ret = v4l2_async_register_subdev(&csi2dc->csi2dc_sd);
> + if (ret) {
> + dev_err(csi2dc->dev, "failed to register the subdevice\n");
> + goto csi2dc_probe_cleanup_notifier;
> + }
> +
> + dev_info(dev, "Microchip CSI2DC version %x\n", ver);
> +
> + return 0;
> +
> +csi2dc_probe_cleanup_notifier:
> + v4l2_async_nf_cleanup(&csi2dc->notifier);
> +csi2dc_probe_cleanup_entity:
> + media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
> +
> + return ret;
> +}
> +
> +static int csi2dc_remove(struct platform_device *pdev)
> +{
> + struct csi2dc_device *csi2dc = platform_get_drvdata(pdev);
> +
> + pm_runtime_disable(&pdev->dev);
> +
> + v4l2_async_unregister_subdev(&csi2dc->csi2dc_sd);
> + v4l2_async_nf_unregister(&csi2dc->notifier);
> + v4l2_async_nf_cleanup(&csi2dc->notifier);
> + media_entity_cleanup(&csi2dc->csi2dc_sd.entity);
> +
> + return 0;
> +}
> +
> +static int __maybe_unused csi2dc_runtime_suspend(struct device *dev)
> +{
> + struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
> +
> + return csi2dc_power(csi2dc, false);
> +}
> +
> +static int __maybe_unused csi2dc_runtime_resume(struct device *dev)
> +{
> + struct csi2dc_device *csi2dc = dev_get_drvdata(dev);
> +
> + return csi2dc_power(csi2dc, true);
> +}
> +
> +static const struct dev_pm_ops csi2dc_dev_pm_ops = {
> + SET_RUNTIME_PM_OPS(csi2dc_runtime_suspend, csi2dc_runtime_resume, NULL)
> +};
> +
> +static const struct of_device_id csi2dc_of_match[] = {
> + { .compatible = "microchip,sama7g5-csi2dc" },
> + { }
> +};
> +
> +MODULE_DEVICE_TABLE(of, csi2dc_of_match);
> +
> +static struct platform_driver csi2dc_driver = {
> + .probe = csi2dc_probe,
> + .remove = csi2dc_remove,
> + .driver = {
> + .name = "microchip-csi2dc",
> + .pm = &csi2dc_dev_pm_ops,
> + .of_match_table = of_match_ptr(csi2dc_of_match),
> + },
> +};
> +
> +module_platform_driver(csi2dc_driver);
> +
> +MODULE_AUTHOR("Eugen Hristev <eugen.hristev@microchip.com>");
> +MODULE_DESCRIPTION("Microchip CSI2 Demux Controller driver");
> +MODULE_LICENSE("GPL v2");
> --
> 2.25.1
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 04/23] media: atmel: atmel-isc: split the clock code into separate source file
2021-12-13 13:49 ` Eugen Hristev
@ 2022-01-11 15:43 ` Jacopo Mondi
-1 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-11 15:43 UTC (permalink / raw)
To: Eugen Hristev
Cc: linux-media, robh+dt, laurent.pinchart, sakari.ailus, devicetree,
linux-kernel, linux-arm-kernel, nicolas.ferre
Hi Eugen
On Mon, Dec 13, 2021 at 03:49:21PM +0200, Eugen Hristev wrote:
> The atmel-isc-base is getting crowded. Split the clock functions into
> atmel-isc-clk.c.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Thanks
j
> ---
> Changes in v3:
> - squash with maintainers, add patterns in maintainers
>
> MAINTAINERS | 7 +-
> drivers/media/platform/atmel/Makefile | 3 +-
> drivers/media/platform/atmel/atmel-isc-base.c | 295 -----------------
> drivers/media/platform/atmel/atmel-isc-clk.c | 311 ++++++++++++++++++
> drivers/media/platform/atmel/atmel-isc.h | 7 +
> 5 files changed, 322 insertions(+), 301 deletions(-)
> create mode 100644 drivers/media/platform/atmel/atmel-isc-clk.c
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index afb74460e5c1..6aa274129751 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -12500,11 +12500,8 @@ L: linux-media@vger.kernel.org
> S: Supported
> F: Documentation/devicetree/bindings/media/atmel,isc.yaml
> F: Documentation/devicetree/bindings/media/microchip,xisc.yaml
> -F: drivers/media/platform/atmel/atmel-isc-base.c
> -F: drivers/media/platform/atmel/atmel-isc-regs.h
> -F: drivers/media/platform/atmel/atmel-isc.h
> -F: drivers/media/platform/atmel/atmel-sama5d2-isc.c
> -F: drivers/media/platform/atmel/atmel-sama7g5-isc.c
> +F: drivers/media/platform/atmel/atmel-isc*
> +F: drivers/media/platform/atmel/atmel-sama*-isc*
> F: include/linux/atmel-isc-media.h
>
> MICROCHIP ISI DRIVER
> diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
> index 39f0a7eba702..794e8f739287 100644
> --- a/drivers/media/platform/atmel/Makefile
> +++ b/drivers/media/platform/atmel/Makefile
> @@ -1,9 +1,10 @@
> # SPDX-License-Identifier: GPL-2.0-only
> atmel-isc-objs = atmel-sama5d2-isc.o
> atmel-xisc-objs = atmel-sama7g5-isc.o
> +atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o
>
> obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
> -obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o
> +obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-common.o
> obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel-isc.o
> obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel-xisc.o
> obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += microchip-csi2dc.o
> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> index 660cd0ab6749..58f468d32613 100644
> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> @@ -8,10 +8,6 @@
> * Author: Eugen Hristev <eugen.hristev@microchip.com>
> *
> */
> -
> -#include <linux/clk.h>
> -#include <linux/clkdev.h>
> -#include <linux/clk-provider.h>
> #include <linux/delay.h>
> #include <linux/interrupt.h>
> #include <linux/math64.h>
> @@ -100,297 +96,6 @@ static inline void isc_reset_awb_ctrls(struct isc_device *isc)
> }
> }
>
> -static int isc_wait_clk_stable(struct clk_hw *hw)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> - struct regmap *regmap = isc_clk->regmap;
> - unsigned long timeout = jiffies + usecs_to_jiffies(1000);
> - unsigned int status;
> -
> - while (time_before(jiffies, timeout)) {
> - regmap_read(regmap, ISC_CLKSR, &status);
> - if (!(status & ISC_CLKSR_SIP))
> - return 0;
> -
> - usleep_range(10, 250);
> - }
> -
> - return -ETIMEDOUT;
> -}
> -
> -static int isc_clk_prepare(struct clk_hw *hw)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> - int ret;
> -
> - ret = pm_runtime_resume_and_get(isc_clk->dev);
> - if (ret < 0)
> - return ret;
> -
> - return isc_wait_clk_stable(hw);
> -}
> -
> -static void isc_clk_unprepare(struct clk_hw *hw)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> -
> - isc_wait_clk_stable(hw);
> -
> - pm_runtime_put_sync(isc_clk->dev);
> -}
> -
> -static int isc_clk_enable(struct clk_hw *hw)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> - u32 id = isc_clk->id;
> - struct regmap *regmap = isc_clk->regmap;
> - unsigned long flags;
> - unsigned int status;
> -
> - dev_dbg(isc_clk->dev, "ISC CLK: %s, id = %d, div = %d, parent id = %d\n",
> - __func__, id, isc_clk->div, isc_clk->parent_id);
> -
> - spin_lock_irqsave(&isc_clk->lock, flags);
> - regmap_update_bits(regmap, ISC_CLKCFG,
> - ISC_CLKCFG_DIV_MASK(id) | ISC_CLKCFG_SEL_MASK(id),
> - (isc_clk->div << ISC_CLKCFG_DIV_SHIFT(id)) |
> - (isc_clk->parent_id << ISC_CLKCFG_SEL_SHIFT(id)));
> -
> - regmap_write(regmap, ISC_CLKEN, ISC_CLK(id));
> - spin_unlock_irqrestore(&isc_clk->lock, flags);
> -
> - regmap_read(regmap, ISC_CLKSR, &status);
> - if (status & ISC_CLK(id))
> - return 0;
> - else
> - return -EINVAL;
> -}
> -
> -static void isc_clk_disable(struct clk_hw *hw)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> - u32 id = isc_clk->id;
> - unsigned long flags;
> -
> - spin_lock_irqsave(&isc_clk->lock, flags);
> - regmap_write(isc_clk->regmap, ISC_CLKDIS, ISC_CLK(id));
> - spin_unlock_irqrestore(&isc_clk->lock, flags);
> -}
> -
> -static int isc_clk_is_enabled(struct clk_hw *hw)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> - u32 status;
> - int ret;
> -
> - ret = pm_runtime_resume_and_get(isc_clk->dev);
> - if (ret < 0)
> - return 0;
> -
> - regmap_read(isc_clk->regmap, ISC_CLKSR, &status);
> -
> - pm_runtime_put_sync(isc_clk->dev);
> -
> - return status & ISC_CLK(isc_clk->id) ? 1 : 0;
> -}
> -
> -static unsigned long
> -isc_clk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> -
> - return DIV_ROUND_CLOSEST(parent_rate, isc_clk->div + 1);
> -}
> -
> -static int isc_clk_determine_rate(struct clk_hw *hw,
> - struct clk_rate_request *req)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> - long best_rate = -EINVAL;
> - int best_diff = -1;
> - unsigned int i, div;
> -
> - for (i = 0; i < clk_hw_get_num_parents(hw); i++) {
> - struct clk_hw *parent;
> - unsigned long parent_rate;
> -
> - parent = clk_hw_get_parent_by_index(hw, i);
> - if (!parent)
> - continue;
> -
> - parent_rate = clk_hw_get_rate(parent);
> - if (!parent_rate)
> - continue;
> -
> - for (div = 1; div < ISC_CLK_MAX_DIV + 2; div++) {
> - unsigned long rate;
> - int diff;
> -
> - rate = DIV_ROUND_CLOSEST(parent_rate, div);
> - diff = abs(req->rate - rate);
> -
> - if (best_diff < 0 || best_diff > diff) {
> - best_rate = rate;
> - best_diff = diff;
> - req->best_parent_rate = parent_rate;
> - req->best_parent_hw = parent;
> - }
> -
> - if (!best_diff || rate < req->rate)
> - break;
> - }
> -
> - if (!best_diff)
> - break;
> - }
> -
> - dev_dbg(isc_clk->dev,
> - "ISC CLK: %s, best_rate = %ld, parent clk: %s @ %ld\n",
> - __func__, best_rate,
> - __clk_get_name((req->best_parent_hw)->clk),
> - req->best_parent_rate);
> -
> - if (best_rate < 0)
> - return best_rate;
> -
> - req->rate = best_rate;
> -
> - return 0;
> -}
> -
> -static int isc_clk_set_parent(struct clk_hw *hw, u8 index)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> -
> - if (index >= clk_hw_get_num_parents(hw))
> - return -EINVAL;
> -
> - isc_clk->parent_id = index;
> -
> - return 0;
> -}
> -
> -static u8 isc_clk_get_parent(struct clk_hw *hw)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> -
> - return isc_clk->parent_id;
> -}
> -
> -static int isc_clk_set_rate(struct clk_hw *hw,
> - unsigned long rate,
> - unsigned long parent_rate)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> - u32 div;
> -
> - if (!rate)
> - return -EINVAL;
> -
> - div = DIV_ROUND_CLOSEST(parent_rate, rate);
> - if (div > (ISC_CLK_MAX_DIV + 1) || !div)
> - return -EINVAL;
> -
> - isc_clk->div = div - 1;
> -
> - return 0;
> -}
> -
> -static const struct clk_ops isc_clk_ops = {
> - .prepare = isc_clk_prepare,
> - .unprepare = isc_clk_unprepare,
> - .enable = isc_clk_enable,
> - .disable = isc_clk_disable,
> - .is_enabled = isc_clk_is_enabled,
> - .recalc_rate = isc_clk_recalc_rate,
> - .determine_rate = isc_clk_determine_rate,
> - .set_parent = isc_clk_set_parent,
> - .get_parent = isc_clk_get_parent,
> - .set_rate = isc_clk_set_rate,
> -};
> -
> -static int isc_clk_register(struct isc_device *isc, unsigned int id)
> -{
> - struct regmap *regmap = isc->regmap;
> - struct device_node *np = isc->dev->of_node;
> - struct isc_clk *isc_clk;
> - struct clk_init_data init;
> - const char *clk_name = np->name;
> - const char *parent_names[3];
> - int num_parents;
> -
> - if (id == ISC_ISPCK && !isc->ispck_required)
> - return 0;
> -
> - num_parents = of_clk_get_parent_count(np);
> - if (num_parents < 1 || num_parents > 3)
> - return -EINVAL;
> -
> - if (num_parents > 2 && id == ISC_ISPCK)
> - num_parents = 2;
> -
> - of_clk_parent_fill(np, parent_names, num_parents);
> -
> - if (id == ISC_MCK)
> - of_property_read_string(np, "clock-output-names", &clk_name);
> - else
> - clk_name = "isc-ispck";
> -
> - init.parent_names = parent_names;
> - init.num_parents = num_parents;
> - init.name = clk_name;
> - init.ops = &isc_clk_ops;
> - init.flags = CLK_SET_RATE_GATE | CLK_SET_PARENT_GATE;
> -
> - isc_clk = &isc->isc_clks[id];
> - isc_clk->hw.init = &init;
> - isc_clk->regmap = regmap;
> - isc_clk->id = id;
> - isc_clk->dev = isc->dev;
> - spin_lock_init(&isc_clk->lock);
> -
> - isc_clk->clk = clk_register(isc->dev, &isc_clk->hw);
> - if (IS_ERR(isc_clk->clk)) {
> - dev_err(isc->dev, "%s: clock register fail\n", clk_name);
> - return PTR_ERR(isc_clk->clk);
> - } else if (id == ISC_MCK)
> - of_clk_add_provider(np, of_clk_src_simple_get, isc_clk->clk);
> -
> - return 0;
> -}
> -
> -int isc_clk_init(struct isc_device *isc)
> -{
> - unsigned int i;
> - int ret;
> -
> - for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++)
> - isc->isc_clks[i].clk = ERR_PTR(-EINVAL);
> -
> - for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
> - ret = isc_clk_register(isc, i);
> - if (ret)
> - return ret;
> - }
> -
> - return 0;
> -}
> -EXPORT_SYMBOL_GPL(isc_clk_init);
> -
> -void isc_clk_cleanup(struct isc_device *isc)
> -{
> - unsigned int i;
> -
> - of_clk_del_provider(isc->dev->of_node);
> -
> - for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
> - struct isc_clk *isc_clk = &isc->isc_clks[i];
> -
> - if (!IS_ERR(isc_clk->clk))
> - clk_unregister(isc_clk->clk);
> - }
> -}
> -EXPORT_SYMBOL_GPL(isc_clk_cleanup);
>
> static int isc_queue_setup(struct vb2_queue *vq,
> unsigned int *nbuffers, unsigned int *nplanes,
> diff --git a/drivers/media/platform/atmel/atmel-isc-clk.c b/drivers/media/platform/atmel/atmel-isc-clk.c
> new file mode 100644
> index 000000000000..2059fe376b00
> --- /dev/null
> +++ b/drivers/media/platform/atmel/atmel-isc-clk.c
> @@ -0,0 +1,311 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Microchip Image Sensor Controller (ISC) common clock driver setup
> + *
> + * Copyright (C) 2016 Microchip Technology, Inc.
> + *
> + * Author: Songjun Wu
> + * Author: Eugen Hristev <eugen.hristev@microchip.com>
> + *
> + */
> +#include <linux/clk.h>
> +#include <linux/clkdev.h>
> +#include <linux/clk-provider.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/regmap.h>
> +
> +#include "atmel-isc-regs.h"
> +#include "atmel-isc.h"
> +
> +static int isc_wait_clk_stable(struct clk_hw *hw)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> + struct regmap *regmap = isc_clk->regmap;
> + unsigned long timeout = jiffies + usecs_to_jiffies(1000);
> + unsigned int status;
> +
> + while (time_before(jiffies, timeout)) {
> + regmap_read(regmap, ISC_CLKSR, &status);
> + if (!(status & ISC_CLKSR_SIP))
> + return 0;
> +
> + usleep_range(10, 250);
> + }
> +
> + return -ETIMEDOUT;
> +}
> +
> +static int isc_clk_prepare(struct clk_hw *hw)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> + int ret;
> +
> + ret = pm_runtime_resume_and_get(isc_clk->dev);
> + if (ret < 0)
> + return ret;
> +
> + return isc_wait_clk_stable(hw);
> +}
> +
> +static void isc_clk_unprepare(struct clk_hw *hw)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> +
> + isc_wait_clk_stable(hw);
> +
> + pm_runtime_put_sync(isc_clk->dev);
> +}
> +
> +static int isc_clk_enable(struct clk_hw *hw)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> + u32 id = isc_clk->id;
> + struct regmap *regmap = isc_clk->regmap;
> + unsigned long flags;
> + unsigned int status;
> +
> + dev_dbg(isc_clk->dev, "ISC CLK: %s, id = %d, div = %d, parent id = %d\n",
> + __func__, id, isc_clk->div, isc_clk->parent_id);
> +
> + spin_lock_irqsave(&isc_clk->lock, flags);
> + regmap_update_bits(regmap, ISC_CLKCFG,
> + ISC_CLKCFG_DIV_MASK(id) | ISC_CLKCFG_SEL_MASK(id),
> + (isc_clk->div << ISC_CLKCFG_DIV_SHIFT(id)) |
> + (isc_clk->parent_id << ISC_CLKCFG_SEL_SHIFT(id)));
> +
> + regmap_write(regmap, ISC_CLKEN, ISC_CLK(id));
> + spin_unlock_irqrestore(&isc_clk->lock, flags);
> +
> + regmap_read(regmap, ISC_CLKSR, &status);
> + if (status & ISC_CLK(id))
> + return 0;
> + else
> + return -EINVAL;
> +}
> +
> +static void isc_clk_disable(struct clk_hw *hw)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> + u32 id = isc_clk->id;
> + unsigned long flags;
> +
> + spin_lock_irqsave(&isc_clk->lock, flags);
> + regmap_write(isc_clk->regmap, ISC_CLKDIS, ISC_CLK(id));
> + spin_unlock_irqrestore(&isc_clk->lock, flags);
> +}
> +
> +static int isc_clk_is_enabled(struct clk_hw *hw)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> + u32 status;
> + int ret;
> +
> + ret = pm_runtime_resume_and_get(isc_clk->dev);
> + if (ret < 0)
> + return 0;
> +
> + regmap_read(isc_clk->regmap, ISC_CLKSR, &status);
> +
> + pm_runtime_put_sync(isc_clk->dev);
> +
> + return status & ISC_CLK(isc_clk->id) ? 1 : 0;
> +}
> +
> +static unsigned long
> +isc_clk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> +
> + return DIV_ROUND_CLOSEST(parent_rate, isc_clk->div + 1);
> +}
> +
> +static int isc_clk_determine_rate(struct clk_hw *hw,
> + struct clk_rate_request *req)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> + long best_rate = -EINVAL;
> + int best_diff = -1;
> + unsigned int i, div;
> +
> + for (i = 0; i < clk_hw_get_num_parents(hw); i++) {
> + struct clk_hw *parent;
> + unsigned long parent_rate;
> +
> + parent = clk_hw_get_parent_by_index(hw, i);
> + if (!parent)
> + continue;
> +
> + parent_rate = clk_hw_get_rate(parent);
> + if (!parent_rate)
> + continue;
> +
> + for (div = 1; div < ISC_CLK_MAX_DIV + 2; div++) {
> + unsigned long rate;
> + int diff;
> +
> + rate = DIV_ROUND_CLOSEST(parent_rate, div);
> + diff = abs(req->rate - rate);
> +
> + if (best_diff < 0 || best_diff > diff) {
> + best_rate = rate;
> + best_diff = diff;
> + req->best_parent_rate = parent_rate;
> + req->best_parent_hw = parent;
> + }
> +
> + if (!best_diff || rate < req->rate)
> + break;
> + }
> +
> + if (!best_diff)
> + break;
> + }
> +
> + dev_dbg(isc_clk->dev,
> + "ISC CLK: %s, best_rate = %ld, parent clk: %s @ %ld\n",
> + __func__, best_rate,
> + __clk_get_name((req->best_parent_hw)->clk),
> + req->best_parent_rate);
> +
> + if (best_rate < 0)
> + return best_rate;
> +
> + req->rate = best_rate;
> +
> + return 0;
> +}
> +
> +static int isc_clk_set_parent(struct clk_hw *hw, u8 index)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> +
> + if (index >= clk_hw_get_num_parents(hw))
> + return -EINVAL;
> +
> + isc_clk->parent_id = index;
> +
> + return 0;
> +}
> +
> +static u8 isc_clk_get_parent(struct clk_hw *hw)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> +
> + return isc_clk->parent_id;
> +}
> +
> +static int isc_clk_set_rate(struct clk_hw *hw,
> + unsigned long rate,
> + unsigned long parent_rate)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> + u32 div;
> +
> + if (!rate)
> + return -EINVAL;
> +
> + div = DIV_ROUND_CLOSEST(parent_rate, rate);
> + if (div > (ISC_CLK_MAX_DIV + 1) || !div)
> + return -EINVAL;
> +
> + isc_clk->div = div - 1;
> +
> + return 0;
> +}
> +
> +static const struct clk_ops isc_clk_ops = {
> + .prepare = isc_clk_prepare,
> + .unprepare = isc_clk_unprepare,
> + .enable = isc_clk_enable,
> + .disable = isc_clk_disable,
> + .is_enabled = isc_clk_is_enabled,
> + .recalc_rate = isc_clk_recalc_rate,
> + .determine_rate = isc_clk_determine_rate,
> + .set_parent = isc_clk_set_parent,
> + .get_parent = isc_clk_get_parent,
> + .set_rate = isc_clk_set_rate,
> +};
> +
> +static int isc_clk_register(struct isc_device *isc, unsigned int id)
> +{
> + struct regmap *regmap = isc->regmap;
> + struct device_node *np = isc->dev->of_node;
> + struct isc_clk *isc_clk;
> + struct clk_init_data init;
> + const char *clk_name = np->name;
> + const char *parent_names[3];
> + int num_parents;
> +
> + if (id == ISC_ISPCK && !isc->ispck_required)
> + return 0;
> +
> + num_parents = of_clk_get_parent_count(np);
> + if (num_parents < 1 || num_parents > 3)
> + return -EINVAL;
> +
> + if (num_parents > 2 && id == ISC_ISPCK)
> + num_parents = 2;
> +
> + of_clk_parent_fill(np, parent_names, num_parents);
> +
> + if (id == ISC_MCK)
> + of_property_read_string(np, "clock-output-names", &clk_name);
> + else
> + clk_name = "isc-ispck";
> +
> + init.parent_names = parent_names;
> + init.num_parents = num_parents;
> + init.name = clk_name;
> + init.ops = &isc_clk_ops;
> + init.flags = CLK_SET_RATE_GATE | CLK_SET_PARENT_GATE;
> +
> + isc_clk = &isc->isc_clks[id];
> + isc_clk->hw.init = &init;
> + isc_clk->regmap = regmap;
> + isc_clk->id = id;
> + isc_clk->dev = isc->dev;
> + spin_lock_init(&isc_clk->lock);
> +
> + isc_clk->clk = clk_register(isc->dev, &isc_clk->hw);
> + if (IS_ERR(isc_clk->clk)) {
> + dev_err(isc->dev, "%s: clock register fail\n", clk_name);
> + return PTR_ERR(isc_clk->clk);
> + } else if (id == ISC_MCK) {
> + of_clk_add_provider(np, of_clk_src_simple_get, isc_clk->clk);
> + }
> +
> + return 0;
> +}
> +
> +int isc_clk_init(struct isc_device *isc)
> +{
> + unsigned int i;
> + int ret;
> +
> + for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++)
> + isc->isc_clks[i].clk = ERR_PTR(-EINVAL);
> +
> + for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
> + ret = isc_clk_register(isc, i);
> + if (ret)
> + return ret;
> + }
> +
> + return 0;
> +}
> +EXPORT_SYMBOL_GPL(isc_clk_init);
> +
> +void isc_clk_cleanup(struct isc_device *isc)
> +{
> + unsigned int i;
> +
> + of_clk_del_provider(isc->dev->of_node);
> +
> + for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
> + struct isc_clk *isc_clk = &isc->isc_clks[i];
> +
> + if (!IS_ERR(isc_clk->clk))
> + clk_unregister(isc_clk->clk);
> + }
> +}
> +EXPORT_SYMBOL_GPL(isc_clk_cleanup);
> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
> index 2bfcb135ef13..32448ccfc636 100644
> --- a/drivers/media/platform/atmel/atmel-isc.h
> +++ b/drivers/media/platform/atmel/atmel-isc.h
> @@ -10,6 +10,13 @@
> */
> #ifndef _ATMEL_ISC_H_
>
> +#include <linux/clk-provider.h>
> +#include <linux/platform_device.h>
> +
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-device.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> #define ISC_CLK_MAX_DIV 255
>
> enum isc_clk_id {
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 04/23] media: atmel: atmel-isc: split the clock code into separate source file
@ 2022-01-11 15:43 ` Jacopo Mondi
0 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-11 15:43 UTC (permalink / raw)
To: Eugen Hristev
Cc: devicetree, linux-kernel, robh+dt, sakari.ailus,
laurent.pinchart, linux-arm-kernel, linux-media
Hi Eugen
On Mon, Dec 13, 2021 at 03:49:21PM +0200, Eugen Hristev wrote:
> The atmel-isc-base is getting crowded. Split the clock functions into
> atmel-isc-clk.c.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Thanks
j
> ---
> Changes in v3:
> - squash with maintainers, add patterns in maintainers
>
> MAINTAINERS | 7 +-
> drivers/media/platform/atmel/Makefile | 3 +-
> drivers/media/platform/atmel/atmel-isc-base.c | 295 -----------------
> drivers/media/platform/atmel/atmel-isc-clk.c | 311 ++++++++++++++++++
> drivers/media/platform/atmel/atmel-isc.h | 7 +
> 5 files changed, 322 insertions(+), 301 deletions(-)
> create mode 100644 drivers/media/platform/atmel/atmel-isc-clk.c
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index afb74460e5c1..6aa274129751 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -12500,11 +12500,8 @@ L: linux-media@vger.kernel.org
> S: Supported
> F: Documentation/devicetree/bindings/media/atmel,isc.yaml
> F: Documentation/devicetree/bindings/media/microchip,xisc.yaml
> -F: drivers/media/platform/atmel/atmel-isc-base.c
> -F: drivers/media/platform/atmel/atmel-isc-regs.h
> -F: drivers/media/platform/atmel/atmel-isc.h
> -F: drivers/media/platform/atmel/atmel-sama5d2-isc.c
> -F: drivers/media/platform/atmel/atmel-sama7g5-isc.c
> +F: drivers/media/platform/atmel/atmel-isc*
> +F: drivers/media/platform/atmel/atmel-sama*-isc*
> F: include/linux/atmel-isc-media.h
>
> MICROCHIP ISI DRIVER
> diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
> index 39f0a7eba702..794e8f739287 100644
> --- a/drivers/media/platform/atmel/Makefile
> +++ b/drivers/media/platform/atmel/Makefile
> @@ -1,9 +1,10 @@
> # SPDX-License-Identifier: GPL-2.0-only
> atmel-isc-objs = atmel-sama5d2-isc.o
> atmel-xisc-objs = atmel-sama7g5-isc.o
> +atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o
>
> obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
> -obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-base.o
> +obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-common.o
> obj-$(CONFIG_VIDEO_ATMEL_ISC) += atmel-isc.o
> obj-$(CONFIG_VIDEO_ATMEL_XISC) += atmel-xisc.o
> obj-$(CONFIG_VIDEO_MICROCHIP_CSI2DC) += microchip-csi2dc.o
> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> index 660cd0ab6749..58f468d32613 100644
> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> @@ -8,10 +8,6 @@
> * Author: Eugen Hristev <eugen.hristev@microchip.com>
> *
> */
> -
> -#include <linux/clk.h>
> -#include <linux/clkdev.h>
> -#include <linux/clk-provider.h>
> #include <linux/delay.h>
> #include <linux/interrupt.h>
> #include <linux/math64.h>
> @@ -100,297 +96,6 @@ static inline void isc_reset_awb_ctrls(struct isc_device *isc)
> }
> }
>
> -static int isc_wait_clk_stable(struct clk_hw *hw)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> - struct regmap *regmap = isc_clk->regmap;
> - unsigned long timeout = jiffies + usecs_to_jiffies(1000);
> - unsigned int status;
> -
> - while (time_before(jiffies, timeout)) {
> - regmap_read(regmap, ISC_CLKSR, &status);
> - if (!(status & ISC_CLKSR_SIP))
> - return 0;
> -
> - usleep_range(10, 250);
> - }
> -
> - return -ETIMEDOUT;
> -}
> -
> -static int isc_clk_prepare(struct clk_hw *hw)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> - int ret;
> -
> - ret = pm_runtime_resume_and_get(isc_clk->dev);
> - if (ret < 0)
> - return ret;
> -
> - return isc_wait_clk_stable(hw);
> -}
> -
> -static void isc_clk_unprepare(struct clk_hw *hw)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> -
> - isc_wait_clk_stable(hw);
> -
> - pm_runtime_put_sync(isc_clk->dev);
> -}
> -
> -static int isc_clk_enable(struct clk_hw *hw)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> - u32 id = isc_clk->id;
> - struct regmap *regmap = isc_clk->regmap;
> - unsigned long flags;
> - unsigned int status;
> -
> - dev_dbg(isc_clk->dev, "ISC CLK: %s, id = %d, div = %d, parent id = %d\n",
> - __func__, id, isc_clk->div, isc_clk->parent_id);
> -
> - spin_lock_irqsave(&isc_clk->lock, flags);
> - regmap_update_bits(regmap, ISC_CLKCFG,
> - ISC_CLKCFG_DIV_MASK(id) | ISC_CLKCFG_SEL_MASK(id),
> - (isc_clk->div << ISC_CLKCFG_DIV_SHIFT(id)) |
> - (isc_clk->parent_id << ISC_CLKCFG_SEL_SHIFT(id)));
> -
> - regmap_write(regmap, ISC_CLKEN, ISC_CLK(id));
> - spin_unlock_irqrestore(&isc_clk->lock, flags);
> -
> - regmap_read(regmap, ISC_CLKSR, &status);
> - if (status & ISC_CLK(id))
> - return 0;
> - else
> - return -EINVAL;
> -}
> -
> -static void isc_clk_disable(struct clk_hw *hw)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> - u32 id = isc_clk->id;
> - unsigned long flags;
> -
> - spin_lock_irqsave(&isc_clk->lock, flags);
> - regmap_write(isc_clk->regmap, ISC_CLKDIS, ISC_CLK(id));
> - spin_unlock_irqrestore(&isc_clk->lock, flags);
> -}
> -
> -static int isc_clk_is_enabled(struct clk_hw *hw)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> - u32 status;
> - int ret;
> -
> - ret = pm_runtime_resume_and_get(isc_clk->dev);
> - if (ret < 0)
> - return 0;
> -
> - regmap_read(isc_clk->regmap, ISC_CLKSR, &status);
> -
> - pm_runtime_put_sync(isc_clk->dev);
> -
> - return status & ISC_CLK(isc_clk->id) ? 1 : 0;
> -}
> -
> -static unsigned long
> -isc_clk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> -
> - return DIV_ROUND_CLOSEST(parent_rate, isc_clk->div + 1);
> -}
> -
> -static int isc_clk_determine_rate(struct clk_hw *hw,
> - struct clk_rate_request *req)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> - long best_rate = -EINVAL;
> - int best_diff = -1;
> - unsigned int i, div;
> -
> - for (i = 0; i < clk_hw_get_num_parents(hw); i++) {
> - struct clk_hw *parent;
> - unsigned long parent_rate;
> -
> - parent = clk_hw_get_parent_by_index(hw, i);
> - if (!parent)
> - continue;
> -
> - parent_rate = clk_hw_get_rate(parent);
> - if (!parent_rate)
> - continue;
> -
> - for (div = 1; div < ISC_CLK_MAX_DIV + 2; div++) {
> - unsigned long rate;
> - int diff;
> -
> - rate = DIV_ROUND_CLOSEST(parent_rate, div);
> - diff = abs(req->rate - rate);
> -
> - if (best_diff < 0 || best_diff > diff) {
> - best_rate = rate;
> - best_diff = diff;
> - req->best_parent_rate = parent_rate;
> - req->best_parent_hw = parent;
> - }
> -
> - if (!best_diff || rate < req->rate)
> - break;
> - }
> -
> - if (!best_diff)
> - break;
> - }
> -
> - dev_dbg(isc_clk->dev,
> - "ISC CLK: %s, best_rate = %ld, parent clk: %s @ %ld\n",
> - __func__, best_rate,
> - __clk_get_name((req->best_parent_hw)->clk),
> - req->best_parent_rate);
> -
> - if (best_rate < 0)
> - return best_rate;
> -
> - req->rate = best_rate;
> -
> - return 0;
> -}
> -
> -static int isc_clk_set_parent(struct clk_hw *hw, u8 index)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> -
> - if (index >= clk_hw_get_num_parents(hw))
> - return -EINVAL;
> -
> - isc_clk->parent_id = index;
> -
> - return 0;
> -}
> -
> -static u8 isc_clk_get_parent(struct clk_hw *hw)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> -
> - return isc_clk->parent_id;
> -}
> -
> -static int isc_clk_set_rate(struct clk_hw *hw,
> - unsigned long rate,
> - unsigned long parent_rate)
> -{
> - struct isc_clk *isc_clk = to_isc_clk(hw);
> - u32 div;
> -
> - if (!rate)
> - return -EINVAL;
> -
> - div = DIV_ROUND_CLOSEST(parent_rate, rate);
> - if (div > (ISC_CLK_MAX_DIV + 1) || !div)
> - return -EINVAL;
> -
> - isc_clk->div = div - 1;
> -
> - return 0;
> -}
> -
> -static const struct clk_ops isc_clk_ops = {
> - .prepare = isc_clk_prepare,
> - .unprepare = isc_clk_unprepare,
> - .enable = isc_clk_enable,
> - .disable = isc_clk_disable,
> - .is_enabled = isc_clk_is_enabled,
> - .recalc_rate = isc_clk_recalc_rate,
> - .determine_rate = isc_clk_determine_rate,
> - .set_parent = isc_clk_set_parent,
> - .get_parent = isc_clk_get_parent,
> - .set_rate = isc_clk_set_rate,
> -};
> -
> -static int isc_clk_register(struct isc_device *isc, unsigned int id)
> -{
> - struct regmap *regmap = isc->regmap;
> - struct device_node *np = isc->dev->of_node;
> - struct isc_clk *isc_clk;
> - struct clk_init_data init;
> - const char *clk_name = np->name;
> - const char *parent_names[3];
> - int num_parents;
> -
> - if (id == ISC_ISPCK && !isc->ispck_required)
> - return 0;
> -
> - num_parents = of_clk_get_parent_count(np);
> - if (num_parents < 1 || num_parents > 3)
> - return -EINVAL;
> -
> - if (num_parents > 2 && id == ISC_ISPCK)
> - num_parents = 2;
> -
> - of_clk_parent_fill(np, parent_names, num_parents);
> -
> - if (id == ISC_MCK)
> - of_property_read_string(np, "clock-output-names", &clk_name);
> - else
> - clk_name = "isc-ispck";
> -
> - init.parent_names = parent_names;
> - init.num_parents = num_parents;
> - init.name = clk_name;
> - init.ops = &isc_clk_ops;
> - init.flags = CLK_SET_RATE_GATE | CLK_SET_PARENT_GATE;
> -
> - isc_clk = &isc->isc_clks[id];
> - isc_clk->hw.init = &init;
> - isc_clk->regmap = regmap;
> - isc_clk->id = id;
> - isc_clk->dev = isc->dev;
> - spin_lock_init(&isc_clk->lock);
> -
> - isc_clk->clk = clk_register(isc->dev, &isc_clk->hw);
> - if (IS_ERR(isc_clk->clk)) {
> - dev_err(isc->dev, "%s: clock register fail\n", clk_name);
> - return PTR_ERR(isc_clk->clk);
> - } else if (id == ISC_MCK)
> - of_clk_add_provider(np, of_clk_src_simple_get, isc_clk->clk);
> -
> - return 0;
> -}
> -
> -int isc_clk_init(struct isc_device *isc)
> -{
> - unsigned int i;
> - int ret;
> -
> - for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++)
> - isc->isc_clks[i].clk = ERR_PTR(-EINVAL);
> -
> - for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
> - ret = isc_clk_register(isc, i);
> - if (ret)
> - return ret;
> - }
> -
> - return 0;
> -}
> -EXPORT_SYMBOL_GPL(isc_clk_init);
> -
> -void isc_clk_cleanup(struct isc_device *isc)
> -{
> - unsigned int i;
> -
> - of_clk_del_provider(isc->dev->of_node);
> -
> - for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
> - struct isc_clk *isc_clk = &isc->isc_clks[i];
> -
> - if (!IS_ERR(isc_clk->clk))
> - clk_unregister(isc_clk->clk);
> - }
> -}
> -EXPORT_SYMBOL_GPL(isc_clk_cleanup);
>
> static int isc_queue_setup(struct vb2_queue *vq,
> unsigned int *nbuffers, unsigned int *nplanes,
> diff --git a/drivers/media/platform/atmel/atmel-isc-clk.c b/drivers/media/platform/atmel/atmel-isc-clk.c
> new file mode 100644
> index 000000000000..2059fe376b00
> --- /dev/null
> +++ b/drivers/media/platform/atmel/atmel-isc-clk.c
> @@ -0,0 +1,311 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Microchip Image Sensor Controller (ISC) common clock driver setup
> + *
> + * Copyright (C) 2016 Microchip Technology, Inc.
> + *
> + * Author: Songjun Wu
> + * Author: Eugen Hristev <eugen.hristev@microchip.com>
> + *
> + */
> +#include <linux/clk.h>
> +#include <linux/clkdev.h>
> +#include <linux/clk-provider.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/regmap.h>
> +
> +#include "atmel-isc-regs.h"
> +#include "atmel-isc.h"
> +
> +static int isc_wait_clk_stable(struct clk_hw *hw)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> + struct regmap *regmap = isc_clk->regmap;
> + unsigned long timeout = jiffies + usecs_to_jiffies(1000);
> + unsigned int status;
> +
> + while (time_before(jiffies, timeout)) {
> + regmap_read(regmap, ISC_CLKSR, &status);
> + if (!(status & ISC_CLKSR_SIP))
> + return 0;
> +
> + usleep_range(10, 250);
> + }
> +
> + return -ETIMEDOUT;
> +}
> +
> +static int isc_clk_prepare(struct clk_hw *hw)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> + int ret;
> +
> + ret = pm_runtime_resume_and_get(isc_clk->dev);
> + if (ret < 0)
> + return ret;
> +
> + return isc_wait_clk_stable(hw);
> +}
> +
> +static void isc_clk_unprepare(struct clk_hw *hw)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> +
> + isc_wait_clk_stable(hw);
> +
> + pm_runtime_put_sync(isc_clk->dev);
> +}
> +
> +static int isc_clk_enable(struct clk_hw *hw)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> + u32 id = isc_clk->id;
> + struct regmap *regmap = isc_clk->regmap;
> + unsigned long flags;
> + unsigned int status;
> +
> + dev_dbg(isc_clk->dev, "ISC CLK: %s, id = %d, div = %d, parent id = %d\n",
> + __func__, id, isc_clk->div, isc_clk->parent_id);
> +
> + spin_lock_irqsave(&isc_clk->lock, flags);
> + regmap_update_bits(regmap, ISC_CLKCFG,
> + ISC_CLKCFG_DIV_MASK(id) | ISC_CLKCFG_SEL_MASK(id),
> + (isc_clk->div << ISC_CLKCFG_DIV_SHIFT(id)) |
> + (isc_clk->parent_id << ISC_CLKCFG_SEL_SHIFT(id)));
> +
> + regmap_write(regmap, ISC_CLKEN, ISC_CLK(id));
> + spin_unlock_irqrestore(&isc_clk->lock, flags);
> +
> + regmap_read(regmap, ISC_CLKSR, &status);
> + if (status & ISC_CLK(id))
> + return 0;
> + else
> + return -EINVAL;
> +}
> +
> +static void isc_clk_disable(struct clk_hw *hw)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> + u32 id = isc_clk->id;
> + unsigned long flags;
> +
> + spin_lock_irqsave(&isc_clk->lock, flags);
> + regmap_write(isc_clk->regmap, ISC_CLKDIS, ISC_CLK(id));
> + spin_unlock_irqrestore(&isc_clk->lock, flags);
> +}
> +
> +static int isc_clk_is_enabled(struct clk_hw *hw)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> + u32 status;
> + int ret;
> +
> + ret = pm_runtime_resume_and_get(isc_clk->dev);
> + if (ret < 0)
> + return 0;
> +
> + regmap_read(isc_clk->regmap, ISC_CLKSR, &status);
> +
> + pm_runtime_put_sync(isc_clk->dev);
> +
> + return status & ISC_CLK(isc_clk->id) ? 1 : 0;
> +}
> +
> +static unsigned long
> +isc_clk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> +
> + return DIV_ROUND_CLOSEST(parent_rate, isc_clk->div + 1);
> +}
> +
> +static int isc_clk_determine_rate(struct clk_hw *hw,
> + struct clk_rate_request *req)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> + long best_rate = -EINVAL;
> + int best_diff = -1;
> + unsigned int i, div;
> +
> + for (i = 0; i < clk_hw_get_num_parents(hw); i++) {
> + struct clk_hw *parent;
> + unsigned long parent_rate;
> +
> + parent = clk_hw_get_parent_by_index(hw, i);
> + if (!parent)
> + continue;
> +
> + parent_rate = clk_hw_get_rate(parent);
> + if (!parent_rate)
> + continue;
> +
> + for (div = 1; div < ISC_CLK_MAX_DIV + 2; div++) {
> + unsigned long rate;
> + int diff;
> +
> + rate = DIV_ROUND_CLOSEST(parent_rate, div);
> + diff = abs(req->rate - rate);
> +
> + if (best_diff < 0 || best_diff > diff) {
> + best_rate = rate;
> + best_diff = diff;
> + req->best_parent_rate = parent_rate;
> + req->best_parent_hw = parent;
> + }
> +
> + if (!best_diff || rate < req->rate)
> + break;
> + }
> +
> + if (!best_diff)
> + break;
> + }
> +
> + dev_dbg(isc_clk->dev,
> + "ISC CLK: %s, best_rate = %ld, parent clk: %s @ %ld\n",
> + __func__, best_rate,
> + __clk_get_name((req->best_parent_hw)->clk),
> + req->best_parent_rate);
> +
> + if (best_rate < 0)
> + return best_rate;
> +
> + req->rate = best_rate;
> +
> + return 0;
> +}
> +
> +static int isc_clk_set_parent(struct clk_hw *hw, u8 index)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> +
> + if (index >= clk_hw_get_num_parents(hw))
> + return -EINVAL;
> +
> + isc_clk->parent_id = index;
> +
> + return 0;
> +}
> +
> +static u8 isc_clk_get_parent(struct clk_hw *hw)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> +
> + return isc_clk->parent_id;
> +}
> +
> +static int isc_clk_set_rate(struct clk_hw *hw,
> + unsigned long rate,
> + unsigned long parent_rate)
> +{
> + struct isc_clk *isc_clk = to_isc_clk(hw);
> + u32 div;
> +
> + if (!rate)
> + return -EINVAL;
> +
> + div = DIV_ROUND_CLOSEST(parent_rate, rate);
> + if (div > (ISC_CLK_MAX_DIV + 1) || !div)
> + return -EINVAL;
> +
> + isc_clk->div = div - 1;
> +
> + return 0;
> +}
> +
> +static const struct clk_ops isc_clk_ops = {
> + .prepare = isc_clk_prepare,
> + .unprepare = isc_clk_unprepare,
> + .enable = isc_clk_enable,
> + .disable = isc_clk_disable,
> + .is_enabled = isc_clk_is_enabled,
> + .recalc_rate = isc_clk_recalc_rate,
> + .determine_rate = isc_clk_determine_rate,
> + .set_parent = isc_clk_set_parent,
> + .get_parent = isc_clk_get_parent,
> + .set_rate = isc_clk_set_rate,
> +};
> +
> +static int isc_clk_register(struct isc_device *isc, unsigned int id)
> +{
> + struct regmap *regmap = isc->regmap;
> + struct device_node *np = isc->dev->of_node;
> + struct isc_clk *isc_clk;
> + struct clk_init_data init;
> + const char *clk_name = np->name;
> + const char *parent_names[3];
> + int num_parents;
> +
> + if (id == ISC_ISPCK && !isc->ispck_required)
> + return 0;
> +
> + num_parents = of_clk_get_parent_count(np);
> + if (num_parents < 1 || num_parents > 3)
> + return -EINVAL;
> +
> + if (num_parents > 2 && id == ISC_ISPCK)
> + num_parents = 2;
> +
> + of_clk_parent_fill(np, parent_names, num_parents);
> +
> + if (id == ISC_MCK)
> + of_property_read_string(np, "clock-output-names", &clk_name);
> + else
> + clk_name = "isc-ispck";
> +
> + init.parent_names = parent_names;
> + init.num_parents = num_parents;
> + init.name = clk_name;
> + init.ops = &isc_clk_ops;
> + init.flags = CLK_SET_RATE_GATE | CLK_SET_PARENT_GATE;
> +
> + isc_clk = &isc->isc_clks[id];
> + isc_clk->hw.init = &init;
> + isc_clk->regmap = regmap;
> + isc_clk->id = id;
> + isc_clk->dev = isc->dev;
> + spin_lock_init(&isc_clk->lock);
> +
> + isc_clk->clk = clk_register(isc->dev, &isc_clk->hw);
> + if (IS_ERR(isc_clk->clk)) {
> + dev_err(isc->dev, "%s: clock register fail\n", clk_name);
> + return PTR_ERR(isc_clk->clk);
> + } else if (id == ISC_MCK) {
> + of_clk_add_provider(np, of_clk_src_simple_get, isc_clk->clk);
> + }
> +
> + return 0;
> +}
> +
> +int isc_clk_init(struct isc_device *isc)
> +{
> + unsigned int i;
> + int ret;
> +
> + for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++)
> + isc->isc_clks[i].clk = ERR_PTR(-EINVAL);
> +
> + for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
> + ret = isc_clk_register(isc, i);
> + if (ret)
> + return ret;
> + }
> +
> + return 0;
> +}
> +EXPORT_SYMBOL_GPL(isc_clk_init);
> +
> +void isc_clk_cleanup(struct isc_device *isc)
> +{
> + unsigned int i;
> +
> + of_clk_del_provider(isc->dev->of_node);
> +
> + for (i = 0; i < ARRAY_SIZE(isc->isc_clks); i++) {
> + struct isc_clk *isc_clk = &isc->isc_clks[i];
> +
> + if (!IS_ERR(isc_clk->clk))
> + clk_unregister(isc_clk->clk);
> + }
> +}
> +EXPORT_SYMBOL_GPL(isc_clk_cleanup);
> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
> index 2bfcb135ef13..32448ccfc636 100644
> --- a/drivers/media/platform/atmel/atmel-isc.h
> +++ b/drivers/media/platform/atmel/atmel-isc.h
> @@ -10,6 +10,13 @@
> */
> #ifndef _ATMEL_ISC_H_
>
> +#include <linux/clk-provider.h>
> +#include <linux/platform_device.h>
> +
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-device.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> #define ISC_CLK_MAX_DIV 255
>
> enum isc_clk_id {
> --
> 2.25.1
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 07/23] media: atmel: atmel-isc-base: use streaming status when queueing buffers
2021-12-13 13:49 ` Eugen Hristev
@ 2022-01-11 15:53 ` Hans Verkuil
-1 siblings, 0 replies; 94+ messages in thread
From: Hans Verkuil @ 2022-01-11 15:53 UTC (permalink / raw)
To: Eugen Hristev, linux-media, robh+dt, jacopo, laurent.pinchart,
sakari.ailus
Cc: linux-kernel, linux-arm-kernel, nicolas.ferre
On 13/12/2021 14:49, Eugen Hristev wrote:
> During experiments with libcamera, it looks like vb2_is_streaming returns
> true before our start streaming is called.
> Order of operations is streamon -> queue -> start_streaming
> ISC would have started the DMA immediately when a buffer is being added
> to the vbqueue if the queue is streaming.
> It is more safe to start the DMA after the start streaming of the driver is
> called.
> Thus, even if vb2queue is streaming, add the buffer to the dma queue of the
> driver instead of actually starting the DMA process, if the start streaming
> has not been called yet.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> ---
> drivers/media/platform/atmel/atmel-isc-base.c | 17 ++++++++++-------
> 1 file changed, 10 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> index 26a6090f056c..e6c9071c04f0 100644
> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> @@ -441,12 +441,14 @@ static void isc_buffer_queue(struct vb2_buffer *vb)
> unsigned long flags;
>
> spin_lock_irqsave(&isc->dma_queue_lock, flags);
> - if (!isc->cur_frm && list_empty(&isc->dma_queue) &&
> - vb2_is_streaming(vb->vb2_queue)) {
> +
> + if (!isc->cur_frm && list_empty(&isc->dma_queue) && !isc->stop) {
> isc->cur_frm = buf;
> isc_start_dma(isc);
> - } else
> + } else {
> list_add_tail(&buf->list, &isc->dma_queue);
> + }
> +
> spin_unlock_irqrestore(&isc->dma_queue_lock, flags);
> }
Both the old and new code doesn't make a lot of sense.
buf_queue is only called by vb2 if start_streaming has already been called or is
about to be called.
Typically all that the buf_queue op does is to call list_add_tail(&buf->list, &isc->dma_queue);
inside the spinlock.
>
> @@ -1014,7 +1016,7 @@ static int isc_s_fmt_vid_cap(struct file *file, void *priv,
> {
> struct isc_device *isc = video_drvdata(file);
>
> - if (vb2_is_streaming(&isc->vb2_vidq))
> + if (!isc->stop)
This is weird as well. Normally this calls vb2_is_busy to check if the
queue is busy (that really means that buffers are already allocated, so
changing the format isn't allowed anymore).
> return -EBUSY;
>
> return isc_set_fmt(isc, f);
> @@ -1536,7 +1538,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
>
> isc_update_awb_ctrls(isc);
>
> - if (vb2_is_streaming(&isc->vb2_vidq)) {
> + if (!isc->stop) {
Ditto.
> /*
> * If we are streaming, we can update profile to
> * have the new settings in place.
> @@ -1552,8 +1554,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
> }
>
> /* if we have autowhitebalance on, start histogram procedure */
> - if (ctrls->awb == ISC_WB_AUTO &&
> - vb2_is_streaming(&isc->vb2_vidq) &&
> + if (ctrls->awb == ISC_WB_AUTO && !isc->stop &&
> ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
> isc_set_histogram(isc, true);
>
> @@ -1829,6 +1830,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
> struct vb2_queue *q = &isc->vb2_vidq;
> int ret = 0;
>
> + isc->stop = true;
> +
I'm really not sure that you need the stop bool at all.
> INIT_WORK(&isc->awb_work, isc_awb_work);
>
> ret = v4l2_device_register_subdev_nodes(&isc->v4l2_dev);
Regards,
Hans
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 07/23] media: atmel: atmel-isc-base: use streaming status when queueing buffers
@ 2022-01-11 15:53 ` Hans Verkuil
0 siblings, 0 replies; 94+ messages in thread
From: Hans Verkuil @ 2022-01-11 15:53 UTC (permalink / raw)
To: Eugen Hristev, linux-media, robh+dt, jacopo, laurent.pinchart,
sakari.ailus
Cc: linux-kernel, linux-arm-kernel
On 13/12/2021 14:49, Eugen Hristev wrote:
> During experiments with libcamera, it looks like vb2_is_streaming returns
> true before our start streaming is called.
> Order of operations is streamon -> queue -> start_streaming
> ISC would have started the DMA immediately when a buffer is being added
> to the vbqueue if the queue is streaming.
> It is more safe to start the DMA after the start streaming of the driver is
> called.
> Thus, even if vb2queue is streaming, add the buffer to the dma queue of the
> driver instead of actually starting the DMA process, if the start streaming
> has not been called yet.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> ---
> drivers/media/platform/atmel/atmel-isc-base.c | 17 ++++++++++-------
> 1 file changed, 10 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> index 26a6090f056c..e6c9071c04f0 100644
> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> @@ -441,12 +441,14 @@ static void isc_buffer_queue(struct vb2_buffer *vb)
> unsigned long flags;
>
> spin_lock_irqsave(&isc->dma_queue_lock, flags);
> - if (!isc->cur_frm && list_empty(&isc->dma_queue) &&
> - vb2_is_streaming(vb->vb2_queue)) {
> +
> + if (!isc->cur_frm && list_empty(&isc->dma_queue) && !isc->stop) {
> isc->cur_frm = buf;
> isc_start_dma(isc);
> - } else
> + } else {
> list_add_tail(&buf->list, &isc->dma_queue);
> + }
> +
> spin_unlock_irqrestore(&isc->dma_queue_lock, flags);
> }
Both the old and new code doesn't make a lot of sense.
buf_queue is only called by vb2 if start_streaming has already been called or is
about to be called.
Typically all that the buf_queue op does is to call list_add_tail(&buf->list, &isc->dma_queue);
inside the spinlock.
>
> @@ -1014,7 +1016,7 @@ static int isc_s_fmt_vid_cap(struct file *file, void *priv,
> {
> struct isc_device *isc = video_drvdata(file);
>
> - if (vb2_is_streaming(&isc->vb2_vidq))
> + if (!isc->stop)
This is weird as well. Normally this calls vb2_is_busy to check if the
queue is busy (that really means that buffers are already allocated, so
changing the format isn't allowed anymore).
> return -EBUSY;
>
> return isc_set_fmt(isc, f);
> @@ -1536,7 +1538,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
>
> isc_update_awb_ctrls(isc);
>
> - if (vb2_is_streaming(&isc->vb2_vidq)) {
> + if (!isc->stop) {
Ditto.
> /*
> * If we are streaming, we can update profile to
> * have the new settings in place.
> @@ -1552,8 +1554,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
> }
>
> /* if we have autowhitebalance on, start histogram procedure */
> - if (ctrls->awb == ISC_WB_AUTO &&
> - vb2_is_streaming(&isc->vb2_vidq) &&
> + if (ctrls->awb == ISC_WB_AUTO && !isc->stop &&
> ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
> isc_set_histogram(isc, true);
>
> @@ -1829,6 +1830,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
> struct vb2_queue *q = &isc->vb2_vidq;
> int ret = 0;
>
> + isc->stop = true;
> +
I'm really not sure that you need the stop bool at all.
> INIT_WORK(&isc->awb_work, isc_awb_work);
>
> ret = v4l2_device_register_subdev_nodes(&isc->v4l2_dev);
Regards,
Hans
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 12/23] media: atmel: atmel-isc: implement media controller
2021-12-13 13:49 ` Eugen Hristev
@ 2022-01-11 16:06 ` Jacopo Mondi
-1 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-11 16:06 UTC (permalink / raw)
To: Eugen Hristev
Cc: linux-media, robh+dt, laurent.pinchart, sakari.ailus, devicetree,
linux-kernel, linux-arm-kernel, nicolas.ferre
Hi Eugen,
On Mon, Dec 13, 2021 at 03:49:29PM +0200, Eugen Hristev wrote:
> Implement the support for media-controller.
> This means that the capabilities of the driver have changed and now
> it also advertises the IO_MC .
> The driver will register it's media device, and add the video entity to this
> media device. The subdevices are registered to the same media device.
> The ISC will have a base entity which is auto-detected as atmel_isc_base.
> It will also register a subdevice that allows cropping of the incoming frame
> to the maximum frame size supported by the ISC.
> The ISC will create a link between the subdevice that is asynchronously
> registered and the atmel_isc_scaler entity.
> Then, the atmel_isc_scaler and atmel_isc_base are connected through another
> link.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> ---
> Changes in v2:
> - implement try formats
>
> drivers/media/platform/atmel/Makefile | 2 +-
> drivers/media/platform/atmel/atmel-isc-base.c | 27 +-
> drivers/media/platform/atmel/atmel-isc-mc.c | 287 ++++++++++++++++++
> drivers/media/platform/atmel/atmel-isc.h | 36 +++
> .../media/platform/atmel/atmel-sama5d2-isc.c | 14 +-
> .../media/platform/atmel/atmel-sama7g5-isc.c | 12 +-
> 6 files changed, 370 insertions(+), 8 deletions(-)
> create mode 100644 drivers/media/platform/atmel/atmel-isc-mc.c
>
> diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
> index 794e8f739287..aa7a18b428f6 100644
> --- a/drivers/media/platform/atmel/Makefile
> +++ b/drivers/media/platform/atmel/Makefile
> @@ -1,7 +1,7 @@
> # SPDX-License-Identifier: GPL-2.0-only
> atmel-isc-objs = atmel-sama5d2-isc.o
> atmel-xisc-objs = atmel-sama7g5-isc.o
> -atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o
> +atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o atmel-isc-mc.o
>
> obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
> obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-common.o
> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> index d0542b97a391..b0c3ed21f372 100644
> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> @@ -1726,6 +1726,7 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
> struct isc_device, v4l2_dev);
> struct isc_subdev_entity *subdev_entity =
> container_of(notifier, struct isc_subdev_entity, notifier);
> + int pad;
>
> if (video_is_registered(&isc->video_dev)) {
> v4l2_err(&isc->v4l2_dev, "only supports one sub-device.\n");
> @@ -1734,6 +1735,16 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
>
> subdev_entity->sd = subdev;
>
> + pad = media_entity_get_fwnode_pad(&subdev->entity, asd->match.fwnode,
> + MEDIA_PAD_FL_SOURCE);
> + if (pad < 0) {
> + v4l2_err(&isc->v4l2_dev, "failed to find pad for %s\n",
> + subdev->name);
> + return pad;
> + }
> +
> + isc->remote_pad = pad;
> +
> return 0;
> }
>
> @@ -1748,8 +1759,8 @@ static void isc_async_unbind(struct v4l2_async_notifier *notifier,
> v4l2_ctrl_handler_free(&isc->ctrls.handler);
> }
>
> -static struct isc_format *find_format_by_code(struct isc_device *isc,
> - unsigned int code, int *index)
> +struct isc_format *isc_find_format_by_code(struct isc_device *isc,
> + unsigned int code, int *index)
> {
> struct isc_format *fmt = &isc->formats_list[0];
> unsigned int i;
> @@ -1765,6 +1776,7 @@ static struct isc_format *find_format_by_code(struct isc_device *isc,
>
> return NULL;
> }
> +EXPORT_SYMBOL_GPL(isc_find_format_by_code);
>
> static int isc_formats_init(struct isc_device *isc)
> {
> @@ -1781,7 +1793,7 @@ static int isc_formats_init(struct isc_device *isc)
> NULL, &mbus_code)) {
> mbus_code.index++;
>
> - fmt = find_format_by_code(isc, mbus_code.code, &i);
> + fmt = isc_find_format_by_code(isc, mbus_code.code, &i);
> if (!fmt) {
> v4l2_warn(&isc->v4l2_dev, "Mbus code %x not supported\n",
> mbus_code.code);
> @@ -1909,7 +1921,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
> vdev->queue = q;
> vdev->lock = &isc->lock;
> vdev->ctrl_handler = &isc->ctrls.handler;
> - vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE;
> + vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE |
> + V4L2_CAP_IO_MC;
> video_set_drvdata(vdev, isc);
>
> ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> @@ -1919,8 +1932,14 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
> goto isc_async_complete_err;
> }
>
> + ret = isc_mc_register(isc);
> + if (ret < 0)
> + goto isc_async_complete_unregister_device;
> return 0;
>
> +isc_async_complete_unregister_device:
> + video_unregister_device(vdev);
> +
> isc_async_complete_err:
> mutex_destroy(&isc->lock);
> return ret;
> diff --git a/drivers/media/platform/atmel/atmel-isc-mc.c b/drivers/media/platform/atmel/atmel-isc-mc.c
> new file mode 100644
> index 000000000000..2aea81d77b2f
> --- /dev/null
> +++ b/drivers/media/platform/atmel/atmel-isc-mc.c
> @@ -0,0 +1,287 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Microchip Image Sensor Controller (ISC) Media Controller support
> + *
> + * Copyright (C) 2021 Microchip Technology, Inc.
> + *
> + * Author: Eugen Hristev <eugen.hristev@microchip.com>
> + *
> + */
> +
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-subdev.h>
Should you include <media/media-entity.h> and <media/media-device.h> ?
Also, should you select MEDIA_CONTROLLER and V4L2_SUBDEV_API for all
the Kconfig symbols ?
> +
> +#include "atmel-isc-regs.h"
> +#include "atmel-isc.h"
> +
> +static const struct media_device_ops isc_media_ops = {
> +};
Do you need this even if it's empty ?
> +
> +static int isc_scaler_get_fmt(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_format *format)
> +{
> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
> + struct v4l2_mbus_framefmt *v4l2_try_fmt;
> +
> + if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
> + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state,
> + format->pad);
> + format->format = *v4l2_try_fmt;
> +
> + return 0;
> + }
> +
> + format->format = isc->scaler_format;
> +
> + return 0;
> +}
> +
> +static int isc_scaler_set_fmt(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_format *req_fmt)
> +{
> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
> + struct v4l2_mbus_framefmt *v4l2_try_fmt;
> + struct isc_format *fmt;
> + unsigned int i;
> +
> + if (req_fmt->pad == ISC_SCALER_PAD_SOURCE)
> + v4l_bound_align_image
> + (&req_fmt->format.width, 16, isc->max_width, 0,
> + &req_fmt->format.height, 16, isc->max_height, 0, 0);
> + else
> + v4l_bound_align_image
> + (&req_fmt->format.width, 16, 10000, 0,
> + &req_fmt->format.height, 16, 10000, 0, 0);
> +
> + req_fmt->format.colorspace = V4L2_COLORSPACE_SRGB;
> + req_fmt->format.field = V4L2_FIELD_NONE;
> + req_fmt->format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> + req_fmt->format.quantization = V4L2_QUANTIZATION_DEFAULT;
> + req_fmt->format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
> +
> + fmt = isc_find_format_by_code(isc, req_fmt->format.code, &i);
> +
> + if (!fmt)
> + fmt = &isc->formats_list[0];
> +
> + req_fmt->format.code = fmt->mbus_code;
> +
> + if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state,
> + req_fmt->pad);
> + *v4l2_try_fmt = req_fmt->format;
> + /* Trying on the pad sink makes the source sink change too */
> + if (req_fmt->pad == ISC_SCALER_PAD_SINK) {
> + v4l2_try_fmt =
> + v4l2_subdev_get_try_format(sd, sd_state,
> + ISC_SCALER_PAD_SOURCE);
> + *v4l2_try_fmt = req_fmt->format;
> +
> + v4l_bound_align_image(&v4l2_try_fmt->width,
> + 16, isc->max_width, 0,
> + &v4l2_try_fmt->height,
> + 16, isc->max_height, 0, 0);
> + }
> + /* if we are just trying, we are done */
> + return 0;
> + }
> +
> + isc->scaler_format = req_fmt->format;
> +
> + return 0;
> +}
> +
> +static int isc_scaler_enum_mbus_code(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_mbus_code_enum *code)
> +{
> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
> + int supported_index = 0;
> + int i;
> +
> + for (i = 0; i < isc->formats_list_size; i++) {
> + if (!isc->formats_list[i].sd_support)
> + continue;
> + if (supported_index == code->index) {
> + code->code = isc->formats_list[i].mbus_code;
> + return 0;
> + }
> + supported_index++;
> + }
> +
> + return -EINVAL;
> +}
> +
> +static int isc_scaler_g_sel(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_selection *sel)
> +{
> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
> +
> + if (sel->pad == ISC_SCALER_PAD_SOURCE)
> + return -EINVAL;
> +
> + if (sel->target != V4L2_SEL_TGT_CROP_BOUNDS &&
> + sel->target != V4L2_SEL_TGT_CROP)
> + return -EINVAL;
> +
> + sel->r.height = isc->max_height;
> + sel->r.width = isc->max_width;
> +
> + sel->r.left = 0;
> + sel->r.top = 0;
> +
> + return 0;
> +}
> +
> +static int isc_scaler_init_cfg(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state)
> +{
> + struct v4l2_mbus_framefmt *v4l2_try_fmt =
> + v4l2_subdev_get_try_format(sd, sd_state, 0);
> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
> +
> + *v4l2_try_fmt = isc->scaler_format;
> +
> + return 0;
> +}
> +
> +static const struct v4l2_subdev_pad_ops isc_scaler_pad_ops = {
> + .enum_mbus_code = isc_scaler_enum_mbus_code,
> + .set_fmt = isc_scaler_set_fmt,
> + .get_fmt = isc_scaler_get_fmt,
> + .get_selection = isc_scaler_g_sel,
> + .init_cfg = isc_scaler_init_cfg,
> +};
> +
> +static const struct v4l2_subdev_ops xisc_scaler_subdev_ops = {
> + .pad = &isc_scaler_pad_ops,
> +};
> +
> +static int isc_init_own_sd(struct isc_device *isc)
> +{
> + int ret;
> +
> + v4l2_subdev_init(&isc->scaler_sd, &xisc_scaler_subdev_ops);
> +
> + isc->scaler_sd.owner = THIS_MODULE;
> + isc->scaler_sd.dev = isc->dev;
> + snprintf(isc->scaler_sd.name, sizeof(isc->scaler_sd.name),
> + "atmel_isc_scaler");
> +
> + isc->scaler_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> + isc->scaler_sd.entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
> + isc->scaler_pads[ISC_SCALER_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> + isc->scaler_pads[ISC_SCALER_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
> +
> + isc->scaler_format.height = isc->max_height;
> + isc->scaler_format.width = isc->max_width;
> + isc->scaler_format.code = isc->formats_list[0].mbus_code;
> + isc->scaler_format.colorspace = V4L2_COLORSPACE_SRGB;
> + isc->scaler_format.field = V4L2_FIELD_NONE;
> + isc->scaler_format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> + isc->scaler_format.quantization = V4L2_QUANTIZATION_DEFAULT;
> + isc->scaler_format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
> +
> + ret = media_entity_pads_init(&isc->scaler_sd.entity,
> + ISC_SCALER_PADS_NUM,
> + isc->scaler_pads);
> + if (ret < 0) {
> + dev_err(isc->dev, "scaler sd media entity init failed\n");
> + return ret;
> + }
> + ret = v4l2_device_register_subdev(&isc->v4l2_dev, &isc->scaler_sd);
> + if (ret < 0) {
> + dev_err(isc->dev, "scaler sd failed to register subdev\n");
> + return ret;
> + }
> +
> + return ret;
> +}
> +
> +int isc_mc_init(struct isc_device *isc, u32 ver)
> +{
> + const struct of_device_id *match;
> + int ret;
> +
> + isc->video_dev.entity.function = MEDIA_ENT_F_IO_V4L;
> + isc->video_dev.entity.flags = MEDIA_ENT_FL_DEFAULT;
> + isc->pads[ISC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> +
> + ret = media_entity_pads_init(&isc->video_dev.entity, ISC_PADS_NUM,
> + isc->pads);
> + if (ret < 0) {
> + dev_err(isc->dev, "media entity init failed\n");
> + return ret;
> + }
> +
> + isc->mdev.dev = isc->dev;
> + isc->mdev.ops = &isc_media_ops;
> +
> + match = of_match_node(isc->dev->driver->of_match_table,
> + isc->dev->of_node);
> +
> + strscpy(isc->mdev.driver_name, KBUILD_MODNAME,
> + sizeof(isc->mdev.driver_name));
> + strscpy(isc->mdev.model, match->compatible, sizeof(isc->mdev.model));
> + snprintf(isc->mdev.bus_info, sizeof(isc->mdev.bus_info), "platform:%s",
> + isc->v4l2_dev.name);
> + isc->mdev.hw_revision = ver;
> +
> + media_device_init(&isc->mdev);
> +
> + isc->v4l2_dev.mdev = &isc->mdev;
> +
> + return isc_init_own_sd(isc);
> +}
> +EXPORT_SYMBOL_GPL(isc_mc_init);
> +
> +int isc_mc_register(struct isc_device *isc)
> +{
> + int ret;
> +
> + ret = media_create_pad_link(&isc->current_subdev->sd->entity,
> + isc->remote_pad, &isc->scaler_sd.entity,
> + ISC_SCALER_PAD_SINK,
> + MEDIA_LNK_FL_ENABLED |
> + MEDIA_LNK_FL_IMMUTABLE);
> +
> + if (ret < 0) {
> + v4l2_err(&isc->v4l2_dev,
> + "Failed to create pad link: %s to %s\n",
> + isc->current_subdev->sd->entity.name,
> + isc->scaler_sd.entity.name);
> + return ret;
> + }
> +
> + dev_dbg(isc->dev, "link with %s pad: %d\n",
> + isc->current_subdev->sd->name, isc->remote_pad);
> +
> + ret = media_create_pad_link(&isc->scaler_sd.entity,
> + ISC_SCALER_PAD_SOURCE,
> + &isc->video_dev.entity, ISC_PAD_SINK,
> + MEDIA_LNK_FL_ENABLED |
> + MEDIA_LNK_FL_IMMUTABLE);
> +
> + if (ret < 0) {
> + v4l2_err(&isc->v4l2_dev,
> + "Failed to create pad link: %s to %s\n",
> + isc->scaler_sd.entity.name,
> + isc->video_dev.entity.name);
> + return ret;
> + }
> +
> + dev_dbg(isc->dev, "link with %s pad: %d\n", isc->scaler_sd.name,
> + ISC_SCALER_PAD_SOURCE);
> +
> + return media_device_register(&isc->mdev);
I feel like the isc-base driver shold instead register the
media-controller and this file shuld actually be only about
isc-scaler or something. I might be missing why it is not possible
though. It doesn't seem to me that mc support is conditional, so it
can happen in the base driver directly ?
Links creation can still happen at async_complete() (I would have
suggested _bound() time, but you have no asyn subdev for the scaler,
so I guess the only async subdev here is the sensor).
All in all: can this function be removed and:
1) Base driver registers the media controller device in probe
2) links are created at async_complete time ?
Thanks
j
> +}
> +EXPORT_SYMBOL_GPL(isc_mc_register);
> +
> +void isc_mc_cleanup(struct isc_device *isc)
> +{
> + media_entity_cleanup(&isc->video_dev.entity);
> +}
> +EXPORT_SYMBOL_GPL(isc_mc_cleanup);
> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
> index 07fa6dbf8460..0b6370d7775f 100644
> --- a/drivers/media/platform/atmel/atmel-isc.h
> +++ b/drivers/media/platform/atmel/atmel-isc.h
> @@ -183,6 +183,17 @@ struct isc_reg_offsets {
> u32 his_entry;
> };
>
> +enum isc_mc_pads {
> + ISC_PAD_SINK = 0,
> + ISC_PADS_NUM = 1,
> +};
> +
> +enum isc_scaler_pads {
> + ISC_SCALER_PAD_SINK = 0,
> + ISC_SCALER_PAD_SOURCE = 1,
> + ISC_SCALER_PADS_NUM = 2,
> +};
> +
> /*
> * struct isc_device - ISC device driver data/config struct
> * @regmap: Register map
> @@ -258,6 +269,12 @@ struct isc_reg_offsets {
> * be used as an input to the controller
> * @controller_formats_size: size of controller_formats array
> * @formats_list_size: size of formats_list array
> + * @pads: media controller pads for isc video entity
> + * @mdev: media device that is registered by the isc
> + * @remote_pad: remote pad on the connected subdevice
> + * @scaler_sd: subdevice for the scaler that isc registers
> + * @scaler_pads: media controller pads for the scaler subdevice
> + * @scaler_format: current format for the scaler subdevice
> */
> struct isc_device {
> struct regmap *regmap;
> @@ -346,6 +363,19 @@ struct isc_device {
> struct isc_format *formats_list;
> u32 controller_formats_size;
> u32 formats_list_size;
> +
> + struct {
> + struct media_pad pads[ISC_PADS_NUM];
> + struct media_device mdev;
> +
> + u32 remote_pad;
> + };
> +
> + struct {
> + struct v4l2_subdev scaler_sd;
> + struct media_pad scaler_pads[ISC_SCALER_PADS_NUM];
> + struct v4l2_mbus_framefmt scaler_format;
> + };
> };
>
> extern const struct regmap_config isc_regmap_config;
> @@ -357,4 +387,10 @@ int isc_clk_init(struct isc_device *isc);
> void isc_subdev_cleanup(struct isc_device *isc);
> void isc_clk_cleanup(struct isc_device *isc);
>
> +int isc_mc_init(struct isc_device *isc, u32 ver);
> +int isc_mc_register(struct isc_device *isc);
> +void isc_mc_cleanup(struct isc_device *isc);
> +
> +struct isc_format *isc_find_format_by_code(struct isc_device *isc,
> + unsigned int code, int *index);
> #endif
> diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> index 1b2063cce0f7..a3e4c32a66c4 100644
> --- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> +++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> @@ -529,6 +529,12 @@ static int atmel_isc_probe(struct platform_device *pdev)
> break;
> }
>
> + regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
> +
> + ret = isc_mc_init(isc, ver);
> + if (ret < 0)
> + goto isc_probe_mc_init_err;
> +
> pm_runtime_set_active(dev);
> pm_runtime_enable(dev);
> pm_request_idle(dev);
> @@ -538,7 +544,7 @@ static int atmel_isc_probe(struct platform_device *pdev)
> ret = clk_prepare_enable(isc->ispck);
> if (ret) {
> dev_err(dev, "failed to enable ispck: %d\n", ret);
> - goto cleanup_subdev;
> + goto isc_probe_mc_init_err;
> }
>
> /* ispck should be greater or equal to hclock */
> @@ -548,7 +554,6 @@ static int atmel_isc_probe(struct platform_device *pdev)
> goto unprepare_clk;
> }
>
> - regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
> dev_info(dev, "Microchip ISC version %x\n", ver);
>
> return 0;
> @@ -556,6 +561,9 @@ static int atmel_isc_probe(struct platform_device *pdev)
> unprepare_clk:
> clk_disable_unprepare(isc->ispck);
>
> +isc_probe_mc_init_err:
> + isc_mc_cleanup(isc);
> +
> cleanup_subdev:
> isc_subdev_cleanup(isc);
>
> @@ -576,6 +584,8 @@ static int atmel_isc_remove(struct platform_device *pdev)
>
> pm_runtime_disable(&pdev->dev);
>
> + isc_mc_cleanup(isc);
> +
> isc_subdev_cleanup(isc);
>
> v4l2_device_unregister(&isc->v4l2_dev);
> diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> index 2b1082295c13..d2ac80b1ccc3 100644
> --- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> +++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> @@ -523,15 +523,23 @@ static int microchip_xisc_probe(struct platform_device *pdev)
> break;
> }
>
> + regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
> +
> + ret = isc_mc_init(isc, ver);
> + if (ret < 0)
> + goto isc_probe_mc_init_err;
> +
> pm_runtime_set_active(dev);
> pm_runtime_enable(dev);
> pm_request_idle(dev);
>
> - regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
> dev_info(dev, "Microchip XISC version %x\n", ver);
>
> return 0;
>
> +isc_probe_mc_init_err:
> + isc_mc_cleanup(isc);
> +
> cleanup_subdev:
> isc_subdev_cleanup(isc);
>
> @@ -552,6 +560,8 @@ static int microchip_xisc_remove(struct platform_device *pdev)
>
> pm_runtime_disable(&pdev->dev);
>
> + isc_mc_cleanup(isc);
> +
> isc_subdev_cleanup(isc);
>
> v4l2_device_unregister(&isc->v4l2_dev);
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 12/23] media: atmel: atmel-isc: implement media controller
@ 2022-01-11 16:06 ` Jacopo Mondi
0 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-11 16:06 UTC (permalink / raw)
To: Eugen Hristev
Cc: devicetree, linux-kernel, robh+dt, sakari.ailus,
laurent.pinchart, linux-arm-kernel, linux-media
Hi Eugen,
On Mon, Dec 13, 2021 at 03:49:29PM +0200, Eugen Hristev wrote:
> Implement the support for media-controller.
> This means that the capabilities of the driver have changed and now
> it also advertises the IO_MC .
> The driver will register it's media device, and add the video entity to this
> media device. The subdevices are registered to the same media device.
> The ISC will have a base entity which is auto-detected as atmel_isc_base.
> It will also register a subdevice that allows cropping of the incoming frame
> to the maximum frame size supported by the ISC.
> The ISC will create a link between the subdevice that is asynchronously
> registered and the atmel_isc_scaler entity.
> Then, the atmel_isc_scaler and atmel_isc_base are connected through another
> link.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> ---
> Changes in v2:
> - implement try formats
>
> drivers/media/platform/atmel/Makefile | 2 +-
> drivers/media/platform/atmel/atmel-isc-base.c | 27 +-
> drivers/media/platform/atmel/atmel-isc-mc.c | 287 ++++++++++++++++++
> drivers/media/platform/atmel/atmel-isc.h | 36 +++
> .../media/platform/atmel/atmel-sama5d2-isc.c | 14 +-
> .../media/platform/atmel/atmel-sama7g5-isc.c | 12 +-
> 6 files changed, 370 insertions(+), 8 deletions(-)
> create mode 100644 drivers/media/platform/atmel/atmel-isc-mc.c
>
> diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
> index 794e8f739287..aa7a18b428f6 100644
> --- a/drivers/media/platform/atmel/Makefile
> +++ b/drivers/media/platform/atmel/Makefile
> @@ -1,7 +1,7 @@
> # SPDX-License-Identifier: GPL-2.0-only
> atmel-isc-objs = atmel-sama5d2-isc.o
> atmel-xisc-objs = atmel-sama7g5-isc.o
> -atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o
> +atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o atmel-isc-mc.o
>
> obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
> obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-common.o
> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> index d0542b97a391..b0c3ed21f372 100644
> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> @@ -1726,6 +1726,7 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
> struct isc_device, v4l2_dev);
> struct isc_subdev_entity *subdev_entity =
> container_of(notifier, struct isc_subdev_entity, notifier);
> + int pad;
>
> if (video_is_registered(&isc->video_dev)) {
> v4l2_err(&isc->v4l2_dev, "only supports one sub-device.\n");
> @@ -1734,6 +1735,16 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
>
> subdev_entity->sd = subdev;
>
> + pad = media_entity_get_fwnode_pad(&subdev->entity, asd->match.fwnode,
> + MEDIA_PAD_FL_SOURCE);
> + if (pad < 0) {
> + v4l2_err(&isc->v4l2_dev, "failed to find pad for %s\n",
> + subdev->name);
> + return pad;
> + }
> +
> + isc->remote_pad = pad;
> +
> return 0;
> }
>
> @@ -1748,8 +1759,8 @@ static void isc_async_unbind(struct v4l2_async_notifier *notifier,
> v4l2_ctrl_handler_free(&isc->ctrls.handler);
> }
>
> -static struct isc_format *find_format_by_code(struct isc_device *isc,
> - unsigned int code, int *index)
> +struct isc_format *isc_find_format_by_code(struct isc_device *isc,
> + unsigned int code, int *index)
> {
> struct isc_format *fmt = &isc->formats_list[0];
> unsigned int i;
> @@ -1765,6 +1776,7 @@ static struct isc_format *find_format_by_code(struct isc_device *isc,
>
> return NULL;
> }
> +EXPORT_SYMBOL_GPL(isc_find_format_by_code);
>
> static int isc_formats_init(struct isc_device *isc)
> {
> @@ -1781,7 +1793,7 @@ static int isc_formats_init(struct isc_device *isc)
> NULL, &mbus_code)) {
> mbus_code.index++;
>
> - fmt = find_format_by_code(isc, mbus_code.code, &i);
> + fmt = isc_find_format_by_code(isc, mbus_code.code, &i);
> if (!fmt) {
> v4l2_warn(&isc->v4l2_dev, "Mbus code %x not supported\n",
> mbus_code.code);
> @@ -1909,7 +1921,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
> vdev->queue = q;
> vdev->lock = &isc->lock;
> vdev->ctrl_handler = &isc->ctrls.handler;
> - vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE;
> + vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE |
> + V4L2_CAP_IO_MC;
> video_set_drvdata(vdev, isc);
>
> ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> @@ -1919,8 +1932,14 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
> goto isc_async_complete_err;
> }
>
> + ret = isc_mc_register(isc);
> + if (ret < 0)
> + goto isc_async_complete_unregister_device;
> return 0;
>
> +isc_async_complete_unregister_device:
> + video_unregister_device(vdev);
> +
> isc_async_complete_err:
> mutex_destroy(&isc->lock);
> return ret;
> diff --git a/drivers/media/platform/atmel/atmel-isc-mc.c b/drivers/media/platform/atmel/atmel-isc-mc.c
> new file mode 100644
> index 000000000000..2aea81d77b2f
> --- /dev/null
> +++ b/drivers/media/platform/atmel/atmel-isc-mc.c
> @@ -0,0 +1,287 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Microchip Image Sensor Controller (ISC) Media Controller support
> + *
> + * Copyright (C) 2021 Microchip Technology, Inc.
> + *
> + * Author: Eugen Hristev <eugen.hristev@microchip.com>
> + *
> + */
> +
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-subdev.h>
Should you include <media/media-entity.h> and <media/media-device.h> ?
Also, should you select MEDIA_CONTROLLER and V4L2_SUBDEV_API for all
the Kconfig symbols ?
> +
> +#include "atmel-isc-regs.h"
> +#include "atmel-isc.h"
> +
> +static const struct media_device_ops isc_media_ops = {
> +};
Do you need this even if it's empty ?
> +
> +static int isc_scaler_get_fmt(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_format *format)
> +{
> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
> + struct v4l2_mbus_framefmt *v4l2_try_fmt;
> +
> + if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
> + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state,
> + format->pad);
> + format->format = *v4l2_try_fmt;
> +
> + return 0;
> + }
> +
> + format->format = isc->scaler_format;
> +
> + return 0;
> +}
> +
> +static int isc_scaler_set_fmt(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_format *req_fmt)
> +{
> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
> + struct v4l2_mbus_framefmt *v4l2_try_fmt;
> + struct isc_format *fmt;
> + unsigned int i;
> +
> + if (req_fmt->pad == ISC_SCALER_PAD_SOURCE)
> + v4l_bound_align_image
> + (&req_fmt->format.width, 16, isc->max_width, 0,
> + &req_fmt->format.height, 16, isc->max_height, 0, 0);
> + else
> + v4l_bound_align_image
> + (&req_fmt->format.width, 16, 10000, 0,
> + &req_fmt->format.height, 16, 10000, 0, 0);
> +
> + req_fmt->format.colorspace = V4L2_COLORSPACE_SRGB;
> + req_fmt->format.field = V4L2_FIELD_NONE;
> + req_fmt->format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> + req_fmt->format.quantization = V4L2_QUANTIZATION_DEFAULT;
> + req_fmt->format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
> +
> + fmt = isc_find_format_by_code(isc, req_fmt->format.code, &i);
> +
> + if (!fmt)
> + fmt = &isc->formats_list[0];
> +
> + req_fmt->format.code = fmt->mbus_code;
> +
> + if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state,
> + req_fmt->pad);
> + *v4l2_try_fmt = req_fmt->format;
> + /* Trying on the pad sink makes the source sink change too */
> + if (req_fmt->pad == ISC_SCALER_PAD_SINK) {
> + v4l2_try_fmt =
> + v4l2_subdev_get_try_format(sd, sd_state,
> + ISC_SCALER_PAD_SOURCE);
> + *v4l2_try_fmt = req_fmt->format;
> +
> + v4l_bound_align_image(&v4l2_try_fmt->width,
> + 16, isc->max_width, 0,
> + &v4l2_try_fmt->height,
> + 16, isc->max_height, 0, 0);
> + }
> + /* if we are just trying, we are done */
> + return 0;
> + }
> +
> + isc->scaler_format = req_fmt->format;
> +
> + return 0;
> +}
> +
> +static int isc_scaler_enum_mbus_code(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_mbus_code_enum *code)
> +{
> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
> + int supported_index = 0;
> + int i;
> +
> + for (i = 0; i < isc->formats_list_size; i++) {
> + if (!isc->formats_list[i].sd_support)
> + continue;
> + if (supported_index == code->index) {
> + code->code = isc->formats_list[i].mbus_code;
> + return 0;
> + }
> + supported_index++;
> + }
> +
> + return -EINVAL;
> +}
> +
> +static int isc_scaler_g_sel(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_selection *sel)
> +{
> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
> +
> + if (sel->pad == ISC_SCALER_PAD_SOURCE)
> + return -EINVAL;
> +
> + if (sel->target != V4L2_SEL_TGT_CROP_BOUNDS &&
> + sel->target != V4L2_SEL_TGT_CROP)
> + return -EINVAL;
> +
> + sel->r.height = isc->max_height;
> + sel->r.width = isc->max_width;
> +
> + sel->r.left = 0;
> + sel->r.top = 0;
> +
> + return 0;
> +}
> +
> +static int isc_scaler_init_cfg(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state)
> +{
> + struct v4l2_mbus_framefmt *v4l2_try_fmt =
> + v4l2_subdev_get_try_format(sd, sd_state, 0);
> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
> +
> + *v4l2_try_fmt = isc->scaler_format;
> +
> + return 0;
> +}
> +
> +static const struct v4l2_subdev_pad_ops isc_scaler_pad_ops = {
> + .enum_mbus_code = isc_scaler_enum_mbus_code,
> + .set_fmt = isc_scaler_set_fmt,
> + .get_fmt = isc_scaler_get_fmt,
> + .get_selection = isc_scaler_g_sel,
> + .init_cfg = isc_scaler_init_cfg,
> +};
> +
> +static const struct v4l2_subdev_ops xisc_scaler_subdev_ops = {
> + .pad = &isc_scaler_pad_ops,
> +};
> +
> +static int isc_init_own_sd(struct isc_device *isc)
> +{
> + int ret;
> +
> + v4l2_subdev_init(&isc->scaler_sd, &xisc_scaler_subdev_ops);
> +
> + isc->scaler_sd.owner = THIS_MODULE;
> + isc->scaler_sd.dev = isc->dev;
> + snprintf(isc->scaler_sd.name, sizeof(isc->scaler_sd.name),
> + "atmel_isc_scaler");
> +
> + isc->scaler_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> + isc->scaler_sd.entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
> + isc->scaler_pads[ISC_SCALER_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> + isc->scaler_pads[ISC_SCALER_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
> +
> + isc->scaler_format.height = isc->max_height;
> + isc->scaler_format.width = isc->max_width;
> + isc->scaler_format.code = isc->formats_list[0].mbus_code;
> + isc->scaler_format.colorspace = V4L2_COLORSPACE_SRGB;
> + isc->scaler_format.field = V4L2_FIELD_NONE;
> + isc->scaler_format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> + isc->scaler_format.quantization = V4L2_QUANTIZATION_DEFAULT;
> + isc->scaler_format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
> +
> + ret = media_entity_pads_init(&isc->scaler_sd.entity,
> + ISC_SCALER_PADS_NUM,
> + isc->scaler_pads);
> + if (ret < 0) {
> + dev_err(isc->dev, "scaler sd media entity init failed\n");
> + return ret;
> + }
> + ret = v4l2_device_register_subdev(&isc->v4l2_dev, &isc->scaler_sd);
> + if (ret < 0) {
> + dev_err(isc->dev, "scaler sd failed to register subdev\n");
> + return ret;
> + }
> +
> + return ret;
> +}
> +
> +int isc_mc_init(struct isc_device *isc, u32 ver)
> +{
> + const struct of_device_id *match;
> + int ret;
> +
> + isc->video_dev.entity.function = MEDIA_ENT_F_IO_V4L;
> + isc->video_dev.entity.flags = MEDIA_ENT_FL_DEFAULT;
> + isc->pads[ISC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> +
> + ret = media_entity_pads_init(&isc->video_dev.entity, ISC_PADS_NUM,
> + isc->pads);
> + if (ret < 0) {
> + dev_err(isc->dev, "media entity init failed\n");
> + return ret;
> + }
> +
> + isc->mdev.dev = isc->dev;
> + isc->mdev.ops = &isc_media_ops;
> +
> + match = of_match_node(isc->dev->driver->of_match_table,
> + isc->dev->of_node);
> +
> + strscpy(isc->mdev.driver_name, KBUILD_MODNAME,
> + sizeof(isc->mdev.driver_name));
> + strscpy(isc->mdev.model, match->compatible, sizeof(isc->mdev.model));
> + snprintf(isc->mdev.bus_info, sizeof(isc->mdev.bus_info), "platform:%s",
> + isc->v4l2_dev.name);
> + isc->mdev.hw_revision = ver;
> +
> + media_device_init(&isc->mdev);
> +
> + isc->v4l2_dev.mdev = &isc->mdev;
> +
> + return isc_init_own_sd(isc);
> +}
> +EXPORT_SYMBOL_GPL(isc_mc_init);
> +
> +int isc_mc_register(struct isc_device *isc)
> +{
> + int ret;
> +
> + ret = media_create_pad_link(&isc->current_subdev->sd->entity,
> + isc->remote_pad, &isc->scaler_sd.entity,
> + ISC_SCALER_PAD_SINK,
> + MEDIA_LNK_FL_ENABLED |
> + MEDIA_LNK_FL_IMMUTABLE);
> +
> + if (ret < 0) {
> + v4l2_err(&isc->v4l2_dev,
> + "Failed to create pad link: %s to %s\n",
> + isc->current_subdev->sd->entity.name,
> + isc->scaler_sd.entity.name);
> + return ret;
> + }
> +
> + dev_dbg(isc->dev, "link with %s pad: %d\n",
> + isc->current_subdev->sd->name, isc->remote_pad);
> +
> + ret = media_create_pad_link(&isc->scaler_sd.entity,
> + ISC_SCALER_PAD_SOURCE,
> + &isc->video_dev.entity, ISC_PAD_SINK,
> + MEDIA_LNK_FL_ENABLED |
> + MEDIA_LNK_FL_IMMUTABLE);
> +
> + if (ret < 0) {
> + v4l2_err(&isc->v4l2_dev,
> + "Failed to create pad link: %s to %s\n",
> + isc->scaler_sd.entity.name,
> + isc->video_dev.entity.name);
> + return ret;
> + }
> +
> + dev_dbg(isc->dev, "link with %s pad: %d\n", isc->scaler_sd.name,
> + ISC_SCALER_PAD_SOURCE);
> +
> + return media_device_register(&isc->mdev);
I feel like the isc-base driver shold instead register the
media-controller and this file shuld actually be only about
isc-scaler or something. I might be missing why it is not possible
though. It doesn't seem to me that mc support is conditional, so it
can happen in the base driver directly ?
Links creation can still happen at async_complete() (I would have
suggested _bound() time, but you have no asyn subdev for the scaler,
so I guess the only async subdev here is the sensor).
All in all: can this function be removed and:
1) Base driver registers the media controller device in probe
2) links are created at async_complete time ?
Thanks
j
> +}
> +EXPORT_SYMBOL_GPL(isc_mc_register);
> +
> +void isc_mc_cleanup(struct isc_device *isc)
> +{
> + media_entity_cleanup(&isc->video_dev.entity);
> +}
> +EXPORT_SYMBOL_GPL(isc_mc_cleanup);
> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
> index 07fa6dbf8460..0b6370d7775f 100644
> --- a/drivers/media/platform/atmel/atmel-isc.h
> +++ b/drivers/media/platform/atmel/atmel-isc.h
> @@ -183,6 +183,17 @@ struct isc_reg_offsets {
> u32 his_entry;
> };
>
> +enum isc_mc_pads {
> + ISC_PAD_SINK = 0,
> + ISC_PADS_NUM = 1,
> +};
> +
> +enum isc_scaler_pads {
> + ISC_SCALER_PAD_SINK = 0,
> + ISC_SCALER_PAD_SOURCE = 1,
> + ISC_SCALER_PADS_NUM = 2,
> +};
> +
> /*
> * struct isc_device - ISC device driver data/config struct
> * @regmap: Register map
> @@ -258,6 +269,12 @@ struct isc_reg_offsets {
> * be used as an input to the controller
> * @controller_formats_size: size of controller_formats array
> * @formats_list_size: size of formats_list array
> + * @pads: media controller pads for isc video entity
> + * @mdev: media device that is registered by the isc
> + * @remote_pad: remote pad on the connected subdevice
> + * @scaler_sd: subdevice for the scaler that isc registers
> + * @scaler_pads: media controller pads for the scaler subdevice
> + * @scaler_format: current format for the scaler subdevice
> */
> struct isc_device {
> struct regmap *regmap;
> @@ -346,6 +363,19 @@ struct isc_device {
> struct isc_format *formats_list;
> u32 controller_formats_size;
> u32 formats_list_size;
> +
> + struct {
> + struct media_pad pads[ISC_PADS_NUM];
> + struct media_device mdev;
> +
> + u32 remote_pad;
> + };
> +
> + struct {
> + struct v4l2_subdev scaler_sd;
> + struct media_pad scaler_pads[ISC_SCALER_PADS_NUM];
> + struct v4l2_mbus_framefmt scaler_format;
> + };
> };
>
> extern const struct regmap_config isc_regmap_config;
> @@ -357,4 +387,10 @@ int isc_clk_init(struct isc_device *isc);
> void isc_subdev_cleanup(struct isc_device *isc);
> void isc_clk_cleanup(struct isc_device *isc);
>
> +int isc_mc_init(struct isc_device *isc, u32 ver);
> +int isc_mc_register(struct isc_device *isc);
> +void isc_mc_cleanup(struct isc_device *isc);
> +
> +struct isc_format *isc_find_format_by_code(struct isc_device *isc,
> + unsigned int code, int *index);
> #endif
> diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> index 1b2063cce0f7..a3e4c32a66c4 100644
> --- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> +++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> @@ -529,6 +529,12 @@ static int atmel_isc_probe(struct platform_device *pdev)
> break;
> }
>
> + regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
> +
> + ret = isc_mc_init(isc, ver);
> + if (ret < 0)
> + goto isc_probe_mc_init_err;
> +
> pm_runtime_set_active(dev);
> pm_runtime_enable(dev);
> pm_request_idle(dev);
> @@ -538,7 +544,7 @@ static int atmel_isc_probe(struct platform_device *pdev)
> ret = clk_prepare_enable(isc->ispck);
> if (ret) {
> dev_err(dev, "failed to enable ispck: %d\n", ret);
> - goto cleanup_subdev;
> + goto isc_probe_mc_init_err;
> }
>
> /* ispck should be greater or equal to hclock */
> @@ -548,7 +554,6 @@ static int atmel_isc_probe(struct platform_device *pdev)
> goto unprepare_clk;
> }
>
> - regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
> dev_info(dev, "Microchip ISC version %x\n", ver);
>
> return 0;
> @@ -556,6 +561,9 @@ static int atmel_isc_probe(struct platform_device *pdev)
> unprepare_clk:
> clk_disable_unprepare(isc->ispck);
>
> +isc_probe_mc_init_err:
> + isc_mc_cleanup(isc);
> +
> cleanup_subdev:
> isc_subdev_cleanup(isc);
>
> @@ -576,6 +584,8 @@ static int atmel_isc_remove(struct platform_device *pdev)
>
> pm_runtime_disable(&pdev->dev);
>
> + isc_mc_cleanup(isc);
> +
> isc_subdev_cleanup(isc);
>
> v4l2_device_unregister(&isc->v4l2_dev);
> diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> index 2b1082295c13..d2ac80b1ccc3 100644
> --- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> +++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> @@ -523,15 +523,23 @@ static int microchip_xisc_probe(struct platform_device *pdev)
> break;
> }
>
> + regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
> +
> + ret = isc_mc_init(isc, ver);
> + if (ret < 0)
> + goto isc_probe_mc_init_err;
> +
> pm_runtime_set_active(dev);
> pm_runtime_enable(dev);
> pm_request_idle(dev);
>
> - regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
> dev_info(dev, "Microchip XISC version %x\n", ver);
>
> return 0;
>
> +isc_probe_mc_init_err:
> + isc_mc_cleanup(isc);
> +
> cleanup_subdev:
> isc_subdev_cleanup(isc);
>
> @@ -552,6 +560,8 @@ static int microchip_xisc_remove(struct platform_device *pdev)
>
> pm_runtime_disable(&pdev->dev);
>
> + isc_mc_cleanup(isc);
> +
> isc_subdev_cleanup(isc);
>
> v4l2_device_unregister(&isc->v4l2_dev);
> --
> 2.25.1
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 12/23] media: atmel: atmel-isc: implement media controller
2022-01-11 16:06 ` Jacopo Mondi
@ 2022-01-11 16:30 ` Eugen.Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen.Hristev @ 2022-01-11 16:30 UTC (permalink / raw)
To: jacopo
Cc: linux-media, robh+dt, laurent.pinchart, sakari.ailus, devicetree,
linux-kernel, linux-arm-kernel, Nicolas.Ferre
On 1/11/22 6:06 PM, Jacopo Mondi wrote:
> Hi Eugen,
>
> On Mon, Dec 13, 2021 at 03:49:29PM +0200, Eugen Hristev wrote:
>> Implement the support for media-controller.
>> This means that the capabilities of the driver have changed and now
>> it also advertises the IO_MC .
>> The driver will register it's media device, and add the video entity to this
>> media device. The subdevices are registered to the same media device.
>> The ISC will have a base entity which is auto-detected as atmel_isc_base.
>> It will also register a subdevice that allows cropping of the incoming frame
>> to the maximum frame size supported by the ISC.
>> The ISC will create a link between the subdevice that is asynchronously
>> registered and the atmel_isc_scaler entity.
>> Then, the atmel_isc_scaler and atmel_isc_base are connected through another
>> link.
>>
>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>> ---
>> Changes in v2:
>> - implement try formats
>>
>> drivers/media/platform/atmel/Makefile | 2 +-
>> drivers/media/platform/atmel/atmel-isc-base.c | 27 +-
>> drivers/media/platform/atmel/atmel-isc-mc.c | 287 ++++++++++++++++++
>> drivers/media/platform/atmel/atmel-isc.h | 36 +++
>> .../media/platform/atmel/atmel-sama5d2-isc.c | 14 +-
>> .../media/platform/atmel/atmel-sama7g5-isc.c | 12 +-
>> 6 files changed, 370 insertions(+), 8 deletions(-)
>> create mode 100644 drivers/media/platform/atmel/atmel-isc-mc.c
>>
>> diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
>> index 794e8f739287..aa7a18b428f6 100644
>> --- a/drivers/media/platform/atmel/Makefile
>> +++ b/drivers/media/platform/atmel/Makefile
>> @@ -1,7 +1,7 @@
>> # SPDX-License-Identifier: GPL-2.0-only
>> atmel-isc-objs = atmel-sama5d2-isc.o
>> atmel-xisc-objs = atmel-sama7g5-isc.o
>> -atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o
>> +atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o atmel-isc-mc.o
>>
>> obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
>> obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-common.o
>> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
>> index d0542b97a391..b0c3ed21f372 100644
>> --- a/drivers/media/platform/atmel/atmel-isc-base.c
>> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
>> @@ -1726,6 +1726,7 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
>> struct isc_device, v4l2_dev);
>> struct isc_subdev_entity *subdev_entity =
>> container_of(notifier, struct isc_subdev_entity, notifier);
>> + int pad;
>>
>> if (video_is_registered(&isc->video_dev)) {
>> v4l2_err(&isc->v4l2_dev, "only supports one sub-device.\n");
>> @@ -1734,6 +1735,16 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
>>
>> subdev_entity->sd = subdev;
>>
>> + pad = media_entity_get_fwnode_pad(&subdev->entity, asd->match.fwnode,
>> + MEDIA_PAD_FL_SOURCE);
>> + if (pad < 0) {
>> + v4l2_err(&isc->v4l2_dev, "failed to find pad for %s\n",
>> + subdev->name);
>> + return pad;
>> + }
>> +
>> + isc->remote_pad = pad;
>> +
>> return 0;
>> }
>>
>> @@ -1748,8 +1759,8 @@ static void isc_async_unbind(struct v4l2_async_notifier *notifier,
>> v4l2_ctrl_handler_free(&isc->ctrls.handler);
>> }
>>
>> -static struct isc_format *find_format_by_code(struct isc_device *isc,
>> - unsigned int code, int *index)
>> +struct isc_format *isc_find_format_by_code(struct isc_device *isc,
>> + unsigned int code, int *index)
>> {
>> struct isc_format *fmt = &isc->formats_list[0];
>> unsigned int i;
>> @@ -1765,6 +1776,7 @@ static struct isc_format *find_format_by_code(struct isc_device *isc,
>>
>> return NULL;
>> }
>> +EXPORT_SYMBOL_GPL(isc_find_format_by_code);
>>
>> static int isc_formats_init(struct isc_device *isc)
>> {
>> @@ -1781,7 +1793,7 @@ static int isc_formats_init(struct isc_device *isc)
>> NULL, &mbus_code)) {
>> mbus_code.index++;
>>
>> - fmt = find_format_by_code(isc, mbus_code.code, &i);
>> + fmt = isc_find_format_by_code(isc, mbus_code.code, &i);
>> if (!fmt) {
>> v4l2_warn(&isc->v4l2_dev, "Mbus code %x not supported\n",
>> mbus_code.code);
>> @@ -1909,7 +1921,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
>> vdev->queue = q;
>> vdev->lock = &isc->lock;
>> vdev->ctrl_handler = &isc->ctrls.handler;
>> - vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE;
>> + vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE |
>> + V4L2_CAP_IO_MC;
>> video_set_drvdata(vdev, isc);
>>
>> ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
>> @@ -1919,8 +1932,14 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
>> goto isc_async_complete_err;
>> }
>>
>> + ret = isc_mc_register(isc);
>> + if (ret < 0)
>> + goto isc_async_complete_unregister_device;
>> return 0;
>>
>> +isc_async_complete_unregister_device:
>> + video_unregister_device(vdev);
>> +
>> isc_async_complete_err:
>> mutex_destroy(&isc->lock);
>> return ret;
>> diff --git a/drivers/media/platform/atmel/atmel-isc-mc.c b/drivers/media/platform/atmel/atmel-isc-mc.c
>> new file mode 100644
>> index 000000000000..2aea81d77b2f
>> --- /dev/null
>> +++ b/drivers/media/platform/atmel/atmel-isc-mc.c
>> @@ -0,0 +1,287 @@
>> +// SPDX-License-Identifier: GPL-2.0-only
>> +/*
>> + * Microchip Image Sensor Controller (ISC) Media Controller support
>> + *
>> + * Copyright (C) 2021 Microchip Technology, Inc.
>> + *
>> + * Author: Eugen Hristev <eugen.hristev@microchip.com>
>> + *
>> + */
>> +
>> +#include <media/v4l2-device.h>
>> +#include <media/v4l2-subdev.h>
>
> Should you include <media/media-entity.h> and <media/media-device.h> ?
Hi Jacopo,
They are probably being included from another header. I will add them.
>
> Also, should you select MEDIA_CONTROLLER and V4L2_SUBDEV_API for all
> the Kconfig symbols ?
VIDEO_ATMEL_ISC directly selects:
select MEDIA_CONTROLLER
select VIDEO_V4L2_SUBDEV_API
and VIDEO_ATMEL_XISC
depends on VIDEO_V4L2 && COMMON_CLK && VIDEO_V4L2_SUBDEV_API
and VIDEO_V4L2_SUBDEV_API depends on MEDIA_CONTROLLER
>
>> +
>> +#include "atmel-isc-regs.h"
>> +#include "atmel-isc.h"
>> +
>> +static const struct media_device_ops isc_media_ops = {
>> +};
>
> Do you need this even if it's empty ?
>
Not sure. I can remove it and pass NULL, but I haven't checked what the
subsystem is doing if there is no ops. I will try.
>> +
>> +static int isc_scaler_get_fmt(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *sd_state,
>> + struct v4l2_subdev_format *format)
>> +{
>> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
>> + struct v4l2_mbus_framefmt *v4l2_try_fmt;
>> +
>> + if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
>> + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state,
>> + format->pad);
>> + format->format = *v4l2_try_fmt;
>> +
>> + return 0;
>> + }
>> +
>> + format->format = isc->scaler_format;
>> +
>> + return 0;
>> +}
>> +
>> +static int isc_scaler_set_fmt(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *sd_state,
>> + struct v4l2_subdev_format *req_fmt)
>> +{
>> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
>> + struct v4l2_mbus_framefmt *v4l2_try_fmt;
>> + struct isc_format *fmt;
>> + unsigned int i;
>> +
>> + if (req_fmt->pad == ISC_SCALER_PAD_SOURCE)
>> + v4l_bound_align_image
>> + (&req_fmt->format.width, 16, isc->max_width, 0,
>> + &req_fmt->format.height, 16, isc->max_height, 0, 0);
>> + else
>> + v4l_bound_align_image
>> + (&req_fmt->format.width, 16, 10000, 0,
>> + &req_fmt->format.height, 16, 10000, 0, 0);
>> +
>> + req_fmt->format.colorspace = V4L2_COLORSPACE_SRGB;
>> + req_fmt->format.field = V4L2_FIELD_NONE;
>> + req_fmt->format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
>> + req_fmt->format.quantization = V4L2_QUANTIZATION_DEFAULT;
>> + req_fmt->format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
>> +
>> + fmt = isc_find_format_by_code(isc, req_fmt->format.code, &i);
>> +
>> + if (!fmt)
>> + fmt = &isc->formats_list[0];
>> +
>> + req_fmt->format.code = fmt->mbus_code;
>> +
>> + if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
>> + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state,
>> + req_fmt->pad);
>> + *v4l2_try_fmt = req_fmt->format;
>> + /* Trying on the pad sink makes the source sink change too */
>> + if (req_fmt->pad == ISC_SCALER_PAD_SINK) {
>> + v4l2_try_fmt =
>> + v4l2_subdev_get_try_format(sd, sd_state,
>> + ISC_SCALER_PAD_SOURCE);
>> + *v4l2_try_fmt = req_fmt->format;
>> +
>> + v4l_bound_align_image(&v4l2_try_fmt->width,
>> + 16, isc->max_width, 0,
>> + &v4l2_try_fmt->height,
>> + 16, isc->max_height, 0, 0);
>> + }
>> + /* if we are just trying, we are done */
>> + return 0;
>> + }
>> +
>> + isc->scaler_format = req_fmt->format;
>> +
>> + return 0;
>> +}
>> +
>> +static int isc_scaler_enum_mbus_code(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *sd_state,
>> + struct v4l2_subdev_mbus_code_enum *code)
>> +{
>> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
>> + int supported_index = 0;
>> + int i;
>> +
>> + for (i = 0; i < isc->formats_list_size; i++) {
>> + if (!isc->formats_list[i].sd_support)
>> + continue;
>> + if (supported_index == code->index) {
>> + code->code = isc->formats_list[i].mbus_code;
>> + return 0;
>> + }
>> + supported_index++;
>> + }
>> +
>> + return -EINVAL;
>> +}
>> +
>> +static int isc_scaler_g_sel(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *sd_state,
>> + struct v4l2_subdev_selection *sel)
>> +{
>> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
>> +
>> + if (sel->pad == ISC_SCALER_PAD_SOURCE)
>> + return -EINVAL;
>> +
>> + if (sel->target != V4L2_SEL_TGT_CROP_BOUNDS &&
>> + sel->target != V4L2_SEL_TGT_CROP)
>> + return -EINVAL;
>> +
>> + sel->r.height = isc->max_height;
>> + sel->r.width = isc->max_width;
>> +
>> + sel->r.left = 0;
>> + sel->r.top = 0;
>> +
>> + return 0;
>> +}
>> +
>> +static int isc_scaler_init_cfg(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *sd_state)
>> +{
>> + struct v4l2_mbus_framefmt *v4l2_try_fmt =
>> + v4l2_subdev_get_try_format(sd, sd_state, 0);
>> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
>> +
>> + *v4l2_try_fmt = isc->scaler_format;
>> +
>> + return 0;
>> +}
>> +
>> +static const struct v4l2_subdev_pad_ops isc_scaler_pad_ops = {
>> + .enum_mbus_code = isc_scaler_enum_mbus_code,
>> + .set_fmt = isc_scaler_set_fmt,
>> + .get_fmt = isc_scaler_get_fmt,
>> + .get_selection = isc_scaler_g_sel,
>> + .init_cfg = isc_scaler_init_cfg,
>> +};
>> +
>> +static const struct v4l2_subdev_ops xisc_scaler_subdev_ops = {
>> + .pad = &isc_scaler_pad_ops,
>> +};
>> +
>> +static int isc_init_own_sd(struct isc_device *isc)
>> +{
>> + int ret;
>> +
>> + v4l2_subdev_init(&isc->scaler_sd, &xisc_scaler_subdev_ops);
>> +
>> + isc->scaler_sd.owner = THIS_MODULE;
>> + isc->scaler_sd.dev = isc->dev;
>> + snprintf(isc->scaler_sd.name, sizeof(isc->scaler_sd.name),
>> + "atmel_isc_scaler");
>> +
>> + isc->scaler_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
>> + isc->scaler_sd.entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
>> + isc->scaler_pads[ISC_SCALER_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>> + isc->scaler_pads[ISC_SCALER_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
>> +
>> + isc->scaler_format.height = isc->max_height;
>> + isc->scaler_format.width = isc->max_width;
>> + isc->scaler_format.code = isc->formats_list[0].mbus_code;
>> + isc->scaler_format.colorspace = V4L2_COLORSPACE_SRGB;
>> + isc->scaler_format.field = V4L2_FIELD_NONE;
>> + isc->scaler_format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
>> + isc->scaler_format.quantization = V4L2_QUANTIZATION_DEFAULT;
>> + isc->scaler_format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
>> +
>> + ret = media_entity_pads_init(&isc->scaler_sd.entity,
>> + ISC_SCALER_PADS_NUM,
>> + isc->scaler_pads);
>> + if (ret < 0) {
>> + dev_err(isc->dev, "scaler sd media entity init failed\n");
>> + return ret;
>> + }
>> + ret = v4l2_device_register_subdev(&isc->v4l2_dev, &isc->scaler_sd);
>> + if (ret < 0) {
>> + dev_err(isc->dev, "scaler sd failed to register subdev\n");
>> + return ret;
>> + }
>> +
>> + return ret;
>> +}
>> +
>> +int isc_mc_init(struct isc_device *isc, u32 ver)
>> +{
>> + const struct of_device_id *match;
>> + int ret;
>> +
>> + isc->video_dev.entity.function = MEDIA_ENT_F_IO_V4L;
>> + isc->video_dev.entity.flags = MEDIA_ENT_FL_DEFAULT;
>> + isc->pads[ISC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>> +
>> + ret = media_entity_pads_init(&isc->video_dev.entity, ISC_PADS_NUM,
>> + isc->pads);
>> + if (ret < 0) {
>> + dev_err(isc->dev, "media entity init failed\n");
>> + return ret;
>> + }
>> +
>> + isc->mdev.dev = isc->dev;
>> + isc->mdev.ops = &isc_media_ops;
>> +
>> + match = of_match_node(isc->dev->driver->of_match_table,
>> + isc->dev->of_node);
>> +
>> + strscpy(isc->mdev.driver_name, KBUILD_MODNAME,
>> + sizeof(isc->mdev.driver_name));
>> + strscpy(isc->mdev.model, match->compatible, sizeof(isc->mdev.model));
>> + snprintf(isc->mdev.bus_info, sizeof(isc->mdev.bus_info), "platform:%s",
>> + isc->v4l2_dev.name);
>> + isc->mdev.hw_revision = ver;
>> +
>> + media_device_init(&isc->mdev);
>> +
>> + isc->v4l2_dev.mdev = &isc->mdev;
>> +
>> + return isc_init_own_sd(isc);
>> +}
>> +EXPORT_SYMBOL_GPL(isc_mc_init);
>> +
>> +int isc_mc_register(struct isc_device *isc)
>> +{
>> + int ret;
>> +
>> + ret = media_create_pad_link(&isc->current_subdev->sd->entity,
>> + isc->remote_pad, &isc->scaler_sd.entity,
>> + ISC_SCALER_PAD_SINK,
>> + MEDIA_LNK_FL_ENABLED |
>> + MEDIA_LNK_FL_IMMUTABLE);
>> +
>> + if (ret < 0) {
>> + v4l2_err(&isc->v4l2_dev,
>> + "Failed to create pad link: %s to %s\n",
>> + isc->current_subdev->sd->entity.name,
>> + isc->scaler_sd.entity.name);
>> + return ret;
>> + }
>> +
>> + dev_dbg(isc->dev, "link with %s pad: %d\n",
>> + isc->current_subdev->sd->name, isc->remote_pad);
>> +
>> + ret = media_create_pad_link(&isc->scaler_sd.entity,
>> + ISC_SCALER_PAD_SOURCE,
>> + &isc->video_dev.entity, ISC_PAD_SINK,
>> + MEDIA_LNK_FL_ENABLED |
>> + MEDIA_LNK_FL_IMMUTABLE);
>> +
>> + if (ret < 0) {
>> + v4l2_err(&isc->v4l2_dev,
>> + "Failed to create pad link: %s to %s\n",
>> + isc->scaler_sd.entity.name,
>> + isc->video_dev.entity.name);
>> + return ret;
>> + }
>> +
>> + dev_dbg(isc->dev, "link with %s pad: %d\n", isc->scaler_sd.name,
>> + ISC_SCALER_PAD_SOURCE);
>> +
>> + return media_device_register(&isc->mdev);
>
> I feel like the isc-base driver shold instead register the
> media-controller and this file shuld actually be only about
> isc-scaler or something. I might be missing why it is not possible
> though. It doesn't seem to me that mc support is conditional, so it
> can happen in the base driver directly ?
This actually is the base driver. Just that the atmel-isc-base.c is
pretty crowded. So I moved the MC part in this file. But it's the same
base driver.
>
> Links creation can still happen at async_complete() (I would have
> suggested _bound() time, but you have no asyn subdev for the scaler,
> so I guess the only async subdev here is the sensor).
Yes, the sensor or any kind of subdevice (XISC normally is connected to
a bridge, ISC to a sensor directly )
>
> All in all: can this function be removed and:
> 1) Base driver registers the media controller device in probe
Should we register the media device even if it's not operational ? and
no video device has been registered ?
I thought it would be safer to have it registered with all the
links/entities/etc once the video node has been completed.
But if you say this is the proper way, I can do that.
I wanted to keep things separated and not overcrowd the base file /probe
function, but if it's just a media_device_register call, I can do it there.
> 2) links are created at async_complete time ?
That works too. I think I could move them there.
Thanks for reviewing,
Eugen
>
> Thanks
> j
>
>
>> +}
>> +EXPORT_SYMBOL_GPL(isc_mc_register);
>> +
>> +void isc_mc_cleanup(struct isc_device *isc)
>> +{
>> + media_entity_cleanup(&isc->video_dev.entity);
>> +}
>> +EXPORT_SYMBOL_GPL(isc_mc_cleanup);
>> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
>> index 07fa6dbf8460..0b6370d7775f 100644
>> --- a/drivers/media/platform/atmel/atmel-isc.h
>> +++ b/drivers/media/platform/atmel/atmel-isc.h
>> @@ -183,6 +183,17 @@ struct isc_reg_offsets {
>> u32 his_entry;
>> };
>>
>> +enum isc_mc_pads {
>> + ISC_PAD_SINK = 0,
>> + ISC_PADS_NUM = 1,
>> +};
>> +
>> +enum isc_scaler_pads {
>> + ISC_SCALER_PAD_SINK = 0,
>> + ISC_SCALER_PAD_SOURCE = 1,
>> + ISC_SCALER_PADS_NUM = 2,
>> +};
>> +
>> /*
>> * struct isc_device - ISC device driver data/config struct
>> * @regmap: Register map
>> @@ -258,6 +269,12 @@ struct isc_reg_offsets {
>> * be used as an input to the controller
>> * @controller_formats_size: size of controller_formats array
>> * @formats_list_size: size of formats_list array
>> + * @pads: media controller pads for isc video entity
>> + * @mdev: media device that is registered by the isc
>> + * @remote_pad: remote pad on the connected subdevice
>> + * @scaler_sd: subdevice for the scaler that isc registers
>> + * @scaler_pads: media controller pads for the scaler subdevice
>> + * @scaler_format: current format for the scaler subdevice
>> */
>> struct isc_device {
>> struct regmap *regmap;
>> @@ -346,6 +363,19 @@ struct isc_device {
>> struct isc_format *formats_list;
>> u32 controller_formats_size;
>> u32 formats_list_size;
>> +
>> + struct {
>> + struct media_pad pads[ISC_PADS_NUM];
>> + struct media_device mdev;
>> +
>> + u32 remote_pad;
>> + };
>> +
>> + struct {
>> + struct v4l2_subdev scaler_sd;
>> + struct media_pad scaler_pads[ISC_SCALER_PADS_NUM];
>> + struct v4l2_mbus_framefmt scaler_format;
>> + };
>> };
>>
>> extern const struct regmap_config isc_regmap_config;
>> @@ -357,4 +387,10 @@ int isc_clk_init(struct isc_device *isc);
>> void isc_subdev_cleanup(struct isc_device *isc);
>> void isc_clk_cleanup(struct isc_device *isc);
>>
>> +int isc_mc_init(struct isc_device *isc, u32 ver);
>> +int isc_mc_register(struct isc_device *isc);
>> +void isc_mc_cleanup(struct isc_device *isc);
>> +
>> +struct isc_format *isc_find_format_by_code(struct isc_device *isc,
>> + unsigned int code, int *index);
>> #endif
>> diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
>> index 1b2063cce0f7..a3e4c32a66c4 100644
>> --- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
>> +++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
>> @@ -529,6 +529,12 @@ static int atmel_isc_probe(struct platform_device *pdev)
>> break;
>> }
>>
>> + regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
>> +
>> + ret = isc_mc_init(isc, ver);
>> + if (ret < 0)
>> + goto isc_probe_mc_init_err;
>> +
>> pm_runtime_set_active(dev);
>> pm_runtime_enable(dev);
>> pm_request_idle(dev);
>> @@ -538,7 +544,7 @@ static int atmel_isc_probe(struct platform_device *pdev)
>> ret = clk_prepare_enable(isc->ispck);
>> if (ret) {
>> dev_err(dev, "failed to enable ispck: %d\n", ret);
>> - goto cleanup_subdev;
>> + goto isc_probe_mc_init_err;
>> }
>>
>> /* ispck should be greater or equal to hclock */
>> @@ -548,7 +554,6 @@ static int atmel_isc_probe(struct platform_device *pdev)
>> goto unprepare_clk;
>> }
>>
>> - regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
>> dev_info(dev, "Microchip ISC version %x\n", ver);
>>
>> return 0;
>> @@ -556,6 +561,9 @@ static int atmel_isc_probe(struct platform_device *pdev)
>> unprepare_clk:
>> clk_disable_unprepare(isc->ispck);
>>
>> +isc_probe_mc_init_err:
>> + isc_mc_cleanup(isc);
>> +
>> cleanup_subdev:
>> isc_subdev_cleanup(isc);
>>
>> @@ -576,6 +584,8 @@ static int atmel_isc_remove(struct platform_device *pdev)
>>
>> pm_runtime_disable(&pdev->dev);
>>
>> + isc_mc_cleanup(isc);
>> +
>> isc_subdev_cleanup(isc);
>>
>> v4l2_device_unregister(&isc->v4l2_dev);
>> diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
>> index 2b1082295c13..d2ac80b1ccc3 100644
>> --- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
>> +++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
>> @@ -523,15 +523,23 @@ static int microchip_xisc_probe(struct platform_device *pdev)
>> break;
>> }
>>
>> + regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
>> +
>> + ret = isc_mc_init(isc, ver);
>> + if (ret < 0)
>> + goto isc_probe_mc_init_err;
>> +
>> pm_runtime_set_active(dev);
>> pm_runtime_enable(dev);
>> pm_request_idle(dev);
>>
>> - regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
>> dev_info(dev, "Microchip XISC version %x\n", ver);
>>
>> return 0;
>>
>> +isc_probe_mc_init_err:
>> + isc_mc_cleanup(isc);
>> +
>> cleanup_subdev:
>> isc_subdev_cleanup(isc);
>>
>> @@ -552,6 +560,8 @@ static int microchip_xisc_remove(struct platform_device *pdev)
>>
>> pm_runtime_disable(&pdev->dev);
>>
>> + isc_mc_cleanup(isc);
>> +
>> isc_subdev_cleanup(isc);
>>
>> v4l2_device_unregister(&isc->v4l2_dev);
>> --
>> 2.25.1
>>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 12/23] media: atmel: atmel-isc: implement media controller
@ 2022-01-11 16:30 ` Eugen.Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen.Hristev @ 2022-01-11 16:30 UTC (permalink / raw)
To: jacopo
Cc: devicetree, linux-kernel, robh+dt, sakari.ailus,
laurent.pinchart, linux-arm-kernel, linux-media
On 1/11/22 6:06 PM, Jacopo Mondi wrote:
> Hi Eugen,
>
> On Mon, Dec 13, 2021 at 03:49:29PM +0200, Eugen Hristev wrote:
>> Implement the support for media-controller.
>> This means that the capabilities of the driver have changed and now
>> it also advertises the IO_MC .
>> The driver will register it's media device, and add the video entity to this
>> media device. The subdevices are registered to the same media device.
>> The ISC will have a base entity which is auto-detected as atmel_isc_base.
>> It will also register a subdevice that allows cropping of the incoming frame
>> to the maximum frame size supported by the ISC.
>> The ISC will create a link between the subdevice that is asynchronously
>> registered and the atmel_isc_scaler entity.
>> Then, the atmel_isc_scaler and atmel_isc_base are connected through another
>> link.
>>
>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>> ---
>> Changes in v2:
>> - implement try formats
>>
>> drivers/media/platform/atmel/Makefile | 2 +-
>> drivers/media/platform/atmel/atmel-isc-base.c | 27 +-
>> drivers/media/platform/atmel/atmel-isc-mc.c | 287 ++++++++++++++++++
>> drivers/media/platform/atmel/atmel-isc.h | 36 +++
>> .../media/platform/atmel/atmel-sama5d2-isc.c | 14 +-
>> .../media/platform/atmel/atmel-sama7g5-isc.c | 12 +-
>> 6 files changed, 370 insertions(+), 8 deletions(-)
>> create mode 100644 drivers/media/platform/atmel/atmel-isc-mc.c
>>
>> diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
>> index 794e8f739287..aa7a18b428f6 100644
>> --- a/drivers/media/platform/atmel/Makefile
>> +++ b/drivers/media/platform/atmel/Makefile
>> @@ -1,7 +1,7 @@
>> # SPDX-License-Identifier: GPL-2.0-only
>> atmel-isc-objs = atmel-sama5d2-isc.o
>> atmel-xisc-objs = atmel-sama7g5-isc.o
>> -atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o
>> +atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o atmel-isc-mc.o
>>
>> obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
>> obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-common.o
>> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
>> index d0542b97a391..b0c3ed21f372 100644
>> --- a/drivers/media/platform/atmel/atmel-isc-base.c
>> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
>> @@ -1726,6 +1726,7 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
>> struct isc_device, v4l2_dev);
>> struct isc_subdev_entity *subdev_entity =
>> container_of(notifier, struct isc_subdev_entity, notifier);
>> + int pad;
>>
>> if (video_is_registered(&isc->video_dev)) {
>> v4l2_err(&isc->v4l2_dev, "only supports one sub-device.\n");
>> @@ -1734,6 +1735,16 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
>>
>> subdev_entity->sd = subdev;
>>
>> + pad = media_entity_get_fwnode_pad(&subdev->entity, asd->match.fwnode,
>> + MEDIA_PAD_FL_SOURCE);
>> + if (pad < 0) {
>> + v4l2_err(&isc->v4l2_dev, "failed to find pad for %s\n",
>> + subdev->name);
>> + return pad;
>> + }
>> +
>> + isc->remote_pad = pad;
>> +
>> return 0;
>> }
>>
>> @@ -1748,8 +1759,8 @@ static void isc_async_unbind(struct v4l2_async_notifier *notifier,
>> v4l2_ctrl_handler_free(&isc->ctrls.handler);
>> }
>>
>> -static struct isc_format *find_format_by_code(struct isc_device *isc,
>> - unsigned int code, int *index)
>> +struct isc_format *isc_find_format_by_code(struct isc_device *isc,
>> + unsigned int code, int *index)
>> {
>> struct isc_format *fmt = &isc->formats_list[0];
>> unsigned int i;
>> @@ -1765,6 +1776,7 @@ static struct isc_format *find_format_by_code(struct isc_device *isc,
>>
>> return NULL;
>> }
>> +EXPORT_SYMBOL_GPL(isc_find_format_by_code);
>>
>> static int isc_formats_init(struct isc_device *isc)
>> {
>> @@ -1781,7 +1793,7 @@ static int isc_formats_init(struct isc_device *isc)
>> NULL, &mbus_code)) {
>> mbus_code.index++;
>>
>> - fmt = find_format_by_code(isc, mbus_code.code, &i);
>> + fmt = isc_find_format_by_code(isc, mbus_code.code, &i);
>> if (!fmt) {
>> v4l2_warn(&isc->v4l2_dev, "Mbus code %x not supported\n",
>> mbus_code.code);
>> @@ -1909,7 +1921,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
>> vdev->queue = q;
>> vdev->lock = &isc->lock;
>> vdev->ctrl_handler = &isc->ctrls.handler;
>> - vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE;
>> + vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE |
>> + V4L2_CAP_IO_MC;
>> video_set_drvdata(vdev, isc);
>>
>> ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
>> @@ -1919,8 +1932,14 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
>> goto isc_async_complete_err;
>> }
>>
>> + ret = isc_mc_register(isc);
>> + if (ret < 0)
>> + goto isc_async_complete_unregister_device;
>> return 0;
>>
>> +isc_async_complete_unregister_device:
>> + video_unregister_device(vdev);
>> +
>> isc_async_complete_err:
>> mutex_destroy(&isc->lock);
>> return ret;
>> diff --git a/drivers/media/platform/atmel/atmel-isc-mc.c b/drivers/media/platform/atmel/atmel-isc-mc.c
>> new file mode 100644
>> index 000000000000..2aea81d77b2f
>> --- /dev/null
>> +++ b/drivers/media/platform/atmel/atmel-isc-mc.c
>> @@ -0,0 +1,287 @@
>> +// SPDX-License-Identifier: GPL-2.0-only
>> +/*
>> + * Microchip Image Sensor Controller (ISC) Media Controller support
>> + *
>> + * Copyright (C) 2021 Microchip Technology, Inc.
>> + *
>> + * Author: Eugen Hristev <eugen.hristev@microchip.com>
>> + *
>> + */
>> +
>> +#include <media/v4l2-device.h>
>> +#include <media/v4l2-subdev.h>
>
> Should you include <media/media-entity.h> and <media/media-device.h> ?
Hi Jacopo,
They are probably being included from another header. I will add them.
>
> Also, should you select MEDIA_CONTROLLER and V4L2_SUBDEV_API for all
> the Kconfig symbols ?
VIDEO_ATMEL_ISC directly selects:
select MEDIA_CONTROLLER
select VIDEO_V4L2_SUBDEV_API
and VIDEO_ATMEL_XISC
depends on VIDEO_V4L2 && COMMON_CLK && VIDEO_V4L2_SUBDEV_API
and VIDEO_V4L2_SUBDEV_API depends on MEDIA_CONTROLLER
>
>> +
>> +#include "atmel-isc-regs.h"
>> +#include "atmel-isc.h"
>> +
>> +static const struct media_device_ops isc_media_ops = {
>> +};
>
> Do you need this even if it's empty ?
>
Not sure. I can remove it and pass NULL, but I haven't checked what the
subsystem is doing if there is no ops. I will try.
>> +
>> +static int isc_scaler_get_fmt(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *sd_state,
>> + struct v4l2_subdev_format *format)
>> +{
>> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
>> + struct v4l2_mbus_framefmt *v4l2_try_fmt;
>> +
>> + if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
>> + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state,
>> + format->pad);
>> + format->format = *v4l2_try_fmt;
>> +
>> + return 0;
>> + }
>> +
>> + format->format = isc->scaler_format;
>> +
>> + return 0;
>> +}
>> +
>> +static int isc_scaler_set_fmt(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *sd_state,
>> + struct v4l2_subdev_format *req_fmt)
>> +{
>> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
>> + struct v4l2_mbus_framefmt *v4l2_try_fmt;
>> + struct isc_format *fmt;
>> + unsigned int i;
>> +
>> + if (req_fmt->pad == ISC_SCALER_PAD_SOURCE)
>> + v4l_bound_align_image
>> + (&req_fmt->format.width, 16, isc->max_width, 0,
>> + &req_fmt->format.height, 16, isc->max_height, 0, 0);
>> + else
>> + v4l_bound_align_image
>> + (&req_fmt->format.width, 16, 10000, 0,
>> + &req_fmt->format.height, 16, 10000, 0, 0);
>> +
>> + req_fmt->format.colorspace = V4L2_COLORSPACE_SRGB;
>> + req_fmt->format.field = V4L2_FIELD_NONE;
>> + req_fmt->format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
>> + req_fmt->format.quantization = V4L2_QUANTIZATION_DEFAULT;
>> + req_fmt->format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
>> +
>> + fmt = isc_find_format_by_code(isc, req_fmt->format.code, &i);
>> +
>> + if (!fmt)
>> + fmt = &isc->formats_list[0];
>> +
>> + req_fmt->format.code = fmt->mbus_code;
>> +
>> + if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
>> + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state,
>> + req_fmt->pad);
>> + *v4l2_try_fmt = req_fmt->format;
>> + /* Trying on the pad sink makes the source sink change too */
>> + if (req_fmt->pad == ISC_SCALER_PAD_SINK) {
>> + v4l2_try_fmt =
>> + v4l2_subdev_get_try_format(sd, sd_state,
>> + ISC_SCALER_PAD_SOURCE);
>> + *v4l2_try_fmt = req_fmt->format;
>> +
>> + v4l_bound_align_image(&v4l2_try_fmt->width,
>> + 16, isc->max_width, 0,
>> + &v4l2_try_fmt->height,
>> + 16, isc->max_height, 0, 0);
>> + }
>> + /* if we are just trying, we are done */
>> + return 0;
>> + }
>> +
>> + isc->scaler_format = req_fmt->format;
>> +
>> + return 0;
>> +}
>> +
>> +static int isc_scaler_enum_mbus_code(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *sd_state,
>> + struct v4l2_subdev_mbus_code_enum *code)
>> +{
>> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
>> + int supported_index = 0;
>> + int i;
>> +
>> + for (i = 0; i < isc->formats_list_size; i++) {
>> + if (!isc->formats_list[i].sd_support)
>> + continue;
>> + if (supported_index == code->index) {
>> + code->code = isc->formats_list[i].mbus_code;
>> + return 0;
>> + }
>> + supported_index++;
>> + }
>> +
>> + return -EINVAL;
>> +}
>> +
>> +static int isc_scaler_g_sel(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *sd_state,
>> + struct v4l2_subdev_selection *sel)
>> +{
>> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
>> +
>> + if (sel->pad == ISC_SCALER_PAD_SOURCE)
>> + return -EINVAL;
>> +
>> + if (sel->target != V4L2_SEL_TGT_CROP_BOUNDS &&
>> + sel->target != V4L2_SEL_TGT_CROP)
>> + return -EINVAL;
>> +
>> + sel->r.height = isc->max_height;
>> + sel->r.width = isc->max_width;
>> +
>> + sel->r.left = 0;
>> + sel->r.top = 0;
>> +
>> + return 0;
>> +}
>> +
>> +static int isc_scaler_init_cfg(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *sd_state)
>> +{
>> + struct v4l2_mbus_framefmt *v4l2_try_fmt =
>> + v4l2_subdev_get_try_format(sd, sd_state, 0);
>> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
>> +
>> + *v4l2_try_fmt = isc->scaler_format;
>> +
>> + return 0;
>> +}
>> +
>> +static const struct v4l2_subdev_pad_ops isc_scaler_pad_ops = {
>> + .enum_mbus_code = isc_scaler_enum_mbus_code,
>> + .set_fmt = isc_scaler_set_fmt,
>> + .get_fmt = isc_scaler_get_fmt,
>> + .get_selection = isc_scaler_g_sel,
>> + .init_cfg = isc_scaler_init_cfg,
>> +};
>> +
>> +static const struct v4l2_subdev_ops xisc_scaler_subdev_ops = {
>> + .pad = &isc_scaler_pad_ops,
>> +};
>> +
>> +static int isc_init_own_sd(struct isc_device *isc)
>> +{
>> + int ret;
>> +
>> + v4l2_subdev_init(&isc->scaler_sd, &xisc_scaler_subdev_ops);
>> +
>> + isc->scaler_sd.owner = THIS_MODULE;
>> + isc->scaler_sd.dev = isc->dev;
>> + snprintf(isc->scaler_sd.name, sizeof(isc->scaler_sd.name),
>> + "atmel_isc_scaler");
>> +
>> + isc->scaler_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
>> + isc->scaler_sd.entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
>> + isc->scaler_pads[ISC_SCALER_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>> + isc->scaler_pads[ISC_SCALER_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
>> +
>> + isc->scaler_format.height = isc->max_height;
>> + isc->scaler_format.width = isc->max_width;
>> + isc->scaler_format.code = isc->formats_list[0].mbus_code;
>> + isc->scaler_format.colorspace = V4L2_COLORSPACE_SRGB;
>> + isc->scaler_format.field = V4L2_FIELD_NONE;
>> + isc->scaler_format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
>> + isc->scaler_format.quantization = V4L2_QUANTIZATION_DEFAULT;
>> + isc->scaler_format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
>> +
>> + ret = media_entity_pads_init(&isc->scaler_sd.entity,
>> + ISC_SCALER_PADS_NUM,
>> + isc->scaler_pads);
>> + if (ret < 0) {
>> + dev_err(isc->dev, "scaler sd media entity init failed\n");
>> + return ret;
>> + }
>> + ret = v4l2_device_register_subdev(&isc->v4l2_dev, &isc->scaler_sd);
>> + if (ret < 0) {
>> + dev_err(isc->dev, "scaler sd failed to register subdev\n");
>> + return ret;
>> + }
>> +
>> + return ret;
>> +}
>> +
>> +int isc_mc_init(struct isc_device *isc, u32 ver)
>> +{
>> + const struct of_device_id *match;
>> + int ret;
>> +
>> + isc->video_dev.entity.function = MEDIA_ENT_F_IO_V4L;
>> + isc->video_dev.entity.flags = MEDIA_ENT_FL_DEFAULT;
>> + isc->pads[ISC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>> +
>> + ret = media_entity_pads_init(&isc->video_dev.entity, ISC_PADS_NUM,
>> + isc->pads);
>> + if (ret < 0) {
>> + dev_err(isc->dev, "media entity init failed\n");
>> + return ret;
>> + }
>> +
>> + isc->mdev.dev = isc->dev;
>> + isc->mdev.ops = &isc_media_ops;
>> +
>> + match = of_match_node(isc->dev->driver->of_match_table,
>> + isc->dev->of_node);
>> +
>> + strscpy(isc->mdev.driver_name, KBUILD_MODNAME,
>> + sizeof(isc->mdev.driver_name));
>> + strscpy(isc->mdev.model, match->compatible, sizeof(isc->mdev.model));
>> + snprintf(isc->mdev.bus_info, sizeof(isc->mdev.bus_info), "platform:%s",
>> + isc->v4l2_dev.name);
>> + isc->mdev.hw_revision = ver;
>> +
>> + media_device_init(&isc->mdev);
>> +
>> + isc->v4l2_dev.mdev = &isc->mdev;
>> +
>> + return isc_init_own_sd(isc);
>> +}
>> +EXPORT_SYMBOL_GPL(isc_mc_init);
>> +
>> +int isc_mc_register(struct isc_device *isc)
>> +{
>> + int ret;
>> +
>> + ret = media_create_pad_link(&isc->current_subdev->sd->entity,
>> + isc->remote_pad, &isc->scaler_sd.entity,
>> + ISC_SCALER_PAD_SINK,
>> + MEDIA_LNK_FL_ENABLED |
>> + MEDIA_LNK_FL_IMMUTABLE);
>> +
>> + if (ret < 0) {
>> + v4l2_err(&isc->v4l2_dev,
>> + "Failed to create pad link: %s to %s\n",
>> + isc->current_subdev->sd->entity.name,
>> + isc->scaler_sd.entity.name);
>> + return ret;
>> + }
>> +
>> + dev_dbg(isc->dev, "link with %s pad: %d\n",
>> + isc->current_subdev->sd->name, isc->remote_pad);
>> +
>> + ret = media_create_pad_link(&isc->scaler_sd.entity,
>> + ISC_SCALER_PAD_SOURCE,
>> + &isc->video_dev.entity, ISC_PAD_SINK,
>> + MEDIA_LNK_FL_ENABLED |
>> + MEDIA_LNK_FL_IMMUTABLE);
>> +
>> + if (ret < 0) {
>> + v4l2_err(&isc->v4l2_dev,
>> + "Failed to create pad link: %s to %s\n",
>> + isc->scaler_sd.entity.name,
>> + isc->video_dev.entity.name);
>> + return ret;
>> + }
>> +
>> + dev_dbg(isc->dev, "link with %s pad: %d\n", isc->scaler_sd.name,
>> + ISC_SCALER_PAD_SOURCE);
>> +
>> + return media_device_register(&isc->mdev);
>
> I feel like the isc-base driver shold instead register the
> media-controller and this file shuld actually be only about
> isc-scaler or something. I might be missing why it is not possible
> though. It doesn't seem to me that mc support is conditional, so it
> can happen in the base driver directly ?
This actually is the base driver. Just that the atmel-isc-base.c is
pretty crowded. So I moved the MC part in this file. But it's the same
base driver.
>
> Links creation can still happen at async_complete() (I would have
> suggested _bound() time, but you have no asyn subdev for the scaler,
> so I guess the only async subdev here is the sensor).
Yes, the sensor or any kind of subdevice (XISC normally is connected to
a bridge, ISC to a sensor directly )
>
> All in all: can this function be removed and:
> 1) Base driver registers the media controller device in probe
Should we register the media device even if it's not operational ? and
no video device has been registered ?
I thought it would be safer to have it registered with all the
links/entities/etc once the video node has been completed.
But if you say this is the proper way, I can do that.
I wanted to keep things separated and not overcrowd the base file /probe
function, but if it's just a media_device_register call, I can do it there.
> 2) links are created at async_complete time ?
That works too. I think I could move them there.
Thanks for reviewing,
Eugen
>
> Thanks
> j
>
>
>> +}
>> +EXPORT_SYMBOL_GPL(isc_mc_register);
>> +
>> +void isc_mc_cleanup(struct isc_device *isc)
>> +{
>> + media_entity_cleanup(&isc->video_dev.entity);
>> +}
>> +EXPORT_SYMBOL_GPL(isc_mc_cleanup);
>> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
>> index 07fa6dbf8460..0b6370d7775f 100644
>> --- a/drivers/media/platform/atmel/atmel-isc.h
>> +++ b/drivers/media/platform/atmel/atmel-isc.h
>> @@ -183,6 +183,17 @@ struct isc_reg_offsets {
>> u32 his_entry;
>> };
>>
>> +enum isc_mc_pads {
>> + ISC_PAD_SINK = 0,
>> + ISC_PADS_NUM = 1,
>> +};
>> +
>> +enum isc_scaler_pads {
>> + ISC_SCALER_PAD_SINK = 0,
>> + ISC_SCALER_PAD_SOURCE = 1,
>> + ISC_SCALER_PADS_NUM = 2,
>> +};
>> +
>> /*
>> * struct isc_device - ISC device driver data/config struct
>> * @regmap: Register map
>> @@ -258,6 +269,12 @@ struct isc_reg_offsets {
>> * be used as an input to the controller
>> * @controller_formats_size: size of controller_formats array
>> * @formats_list_size: size of formats_list array
>> + * @pads: media controller pads for isc video entity
>> + * @mdev: media device that is registered by the isc
>> + * @remote_pad: remote pad on the connected subdevice
>> + * @scaler_sd: subdevice for the scaler that isc registers
>> + * @scaler_pads: media controller pads for the scaler subdevice
>> + * @scaler_format: current format for the scaler subdevice
>> */
>> struct isc_device {
>> struct regmap *regmap;
>> @@ -346,6 +363,19 @@ struct isc_device {
>> struct isc_format *formats_list;
>> u32 controller_formats_size;
>> u32 formats_list_size;
>> +
>> + struct {
>> + struct media_pad pads[ISC_PADS_NUM];
>> + struct media_device mdev;
>> +
>> + u32 remote_pad;
>> + };
>> +
>> + struct {
>> + struct v4l2_subdev scaler_sd;
>> + struct media_pad scaler_pads[ISC_SCALER_PADS_NUM];
>> + struct v4l2_mbus_framefmt scaler_format;
>> + };
>> };
>>
>> extern const struct regmap_config isc_regmap_config;
>> @@ -357,4 +387,10 @@ int isc_clk_init(struct isc_device *isc);
>> void isc_subdev_cleanup(struct isc_device *isc);
>> void isc_clk_cleanup(struct isc_device *isc);
>>
>> +int isc_mc_init(struct isc_device *isc, u32 ver);
>> +int isc_mc_register(struct isc_device *isc);
>> +void isc_mc_cleanup(struct isc_device *isc);
>> +
>> +struct isc_format *isc_find_format_by_code(struct isc_device *isc,
>> + unsigned int code, int *index);
>> #endif
>> diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
>> index 1b2063cce0f7..a3e4c32a66c4 100644
>> --- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
>> +++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
>> @@ -529,6 +529,12 @@ static int atmel_isc_probe(struct platform_device *pdev)
>> break;
>> }
>>
>> + regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
>> +
>> + ret = isc_mc_init(isc, ver);
>> + if (ret < 0)
>> + goto isc_probe_mc_init_err;
>> +
>> pm_runtime_set_active(dev);
>> pm_runtime_enable(dev);
>> pm_request_idle(dev);
>> @@ -538,7 +544,7 @@ static int atmel_isc_probe(struct platform_device *pdev)
>> ret = clk_prepare_enable(isc->ispck);
>> if (ret) {
>> dev_err(dev, "failed to enable ispck: %d\n", ret);
>> - goto cleanup_subdev;
>> + goto isc_probe_mc_init_err;
>> }
>>
>> /* ispck should be greater or equal to hclock */
>> @@ -548,7 +554,6 @@ static int atmel_isc_probe(struct platform_device *pdev)
>> goto unprepare_clk;
>> }
>>
>> - regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
>> dev_info(dev, "Microchip ISC version %x\n", ver);
>>
>> return 0;
>> @@ -556,6 +561,9 @@ static int atmel_isc_probe(struct platform_device *pdev)
>> unprepare_clk:
>> clk_disable_unprepare(isc->ispck);
>>
>> +isc_probe_mc_init_err:
>> + isc_mc_cleanup(isc);
>> +
>> cleanup_subdev:
>> isc_subdev_cleanup(isc);
>>
>> @@ -576,6 +584,8 @@ static int atmel_isc_remove(struct platform_device *pdev)
>>
>> pm_runtime_disable(&pdev->dev);
>>
>> + isc_mc_cleanup(isc);
>> +
>> isc_subdev_cleanup(isc);
>>
>> v4l2_device_unregister(&isc->v4l2_dev);
>> diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
>> index 2b1082295c13..d2ac80b1ccc3 100644
>> --- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
>> +++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
>> @@ -523,15 +523,23 @@ static int microchip_xisc_probe(struct platform_device *pdev)
>> break;
>> }
>>
>> + regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
>> +
>> + ret = isc_mc_init(isc, ver);
>> + if (ret < 0)
>> + goto isc_probe_mc_init_err;
>> +
>> pm_runtime_set_active(dev);
>> pm_runtime_enable(dev);
>> pm_request_idle(dev);
>>
>> - regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
>> dev_info(dev, "Microchip XISC version %x\n", ver);
>>
>> return 0;
>>
>> +isc_probe_mc_init_err:
>> + isc_mc_cleanup(isc);
>> +
>> cleanup_subdev:
>> isc_subdev_cleanup(isc);
>>
>> @@ -552,6 +560,8 @@ static int microchip_xisc_remove(struct platform_device *pdev)
>>
>> pm_runtime_disable(&pdev->dev);
>>
>> + isc_mc_cleanup(isc);
>> +
>> isc_subdev_cleanup(isc);
>>
>> v4l2_device_unregister(&isc->v4l2_dev);
>> --
>> 2.25.1
>>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 07/23] media: atmel: atmel-isc-base: use streaming status when queueing buffers
2022-01-11 15:53 ` Hans Verkuil
@ 2022-01-11 16:38 ` Eugen.Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen.Hristev @ 2022-01-11 16:38 UTC (permalink / raw)
To: hverkuil, jacopo, laurent.pinchart
Cc: linux-media, robh+dt, sakari.ailus, linux-kernel,
linux-arm-kernel, Nicolas.Ferre
On 1/11/22 5:53 PM, Hans Verkuil wrote:
> On 13/12/2021 14:49, Eugen Hristev wrote:
>> During experiments with libcamera, it looks like vb2_is_streaming returns
>> true before our start streaming is called.
>> Order of operations is streamon -> queue -> start_streaming
>> ISC would have started the DMA immediately when a buffer is being added
>> to the vbqueue if the queue is streaming.
>> It is more safe to start the DMA after the start streaming of the driver is
>> called.
>> Thus, even if vb2queue is streaming, add the buffer to the dma queue of the
>> driver instead of actually starting the DMA process, if the start streaming
>> has not been called yet.
>>
>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>> ---
>> drivers/media/platform/atmel/atmel-isc-base.c | 17 ++++++++++-------
>> 1 file changed, 10 insertions(+), 7 deletions(-)
>>
>> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
>> index 26a6090f056c..e6c9071c04f0 100644
>> --- a/drivers/media/platform/atmel/atmel-isc-base.c
>> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
>> @@ -441,12 +441,14 @@ static void isc_buffer_queue(struct vb2_buffer *vb)
>> unsigned long flags;
>>
>> spin_lock_irqsave(&isc->dma_queue_lock, flags);
>> - if (!isc->cur_frm && list_empty(&isc->dma_queue) &&
>> - vb2_is_streaming(vb->vb2_queue)) {
>> +
>> + if (!isc->cur_frm && list_empty(&isc->dma_queue) && !isc->stop) {
>> isc->cur_frm = buf;
>> isc_start_dma(isc);
>> - } else
>> + } else {
>> list_add_tail(&buf->list, &isc->dma_queue);
>> + }
>> +
>> spin_unlock_irqrestore(&isc->dma_queue_lock, flags);
>> }
>
> Both the old and new code doesn't make a lot of sense.
Hello Hans,
I debated this a bit with Laurent when I initially wrote the patch.
The problem with the current code, or the current way things are
working, is the fact that vb2_is_streaming returns true from the moment
of streamon.
The DMA engine of the ISC is being started at the moment of
start_streaming, and if buffers are being sent earlier/later, they are
just added to the queue and DMA engine is restarted if it finished
previously.
However if frames are added beforehand, like, first streamon, then add
some frames, and then later call the start_streaming, ISC will crash.
(as I said, DMA engine is prepared at start_streaming).
To avoid this, I changed the vb2_is_streaming calls to keeping an
internal state of the streaming.
Do you think I should try another different approach to solve this ?
Perhaps do not prepare the ISC dma engine at start_streaming, but at
another place ?
Laurent do you remember the talk we had about this problem a while back ?
>
> buf_queue is only called by vb2 if start_streaming has already been called or is
> about to be called.
That's a huge difference for ISC driver as it is today. start_streaming
being called afterwards, means the ISC DMA engine is not started (not
started streaming), hence a crash.
>
> Typically all that the buf_queue op does is to call list_add_tail(&buf->list, &isc->dma_queue);
> inside the spinlock.
>
>>
>> @@ -1014,7 +1016,7 @@ static int isc_s_fmt_vid_cap(struct file *file, void *priv,
>> {
>> struct isc_device *isc = video_drvdata(file);
>>
>> - if (vb2_is_streaming(&isc->vb2_vidq))
>> + if (!isc->stop)
>
> This is weird as well. Normally this calls vb2_is_busy to check if the
> queue is busy (that really means that buffers are already allocated, so
> changing the format isn't allowed anymore).
You think the query to the streaming status makes no sense here hence it
should be removed completely ?
I can do that with a prequel patch if it's the case.
Thanks for reviewing,
Eugen
>
>> return -EBUSY;
>>
>> return isc_set_fmt(isc, f);
>> @@ -1536,7 +1538,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
>>
>> isc_update_awb_ctrls(isc);
>>
>> - if (vb2_is_streaming(&isc->vb2_vidq)) {
>> + if (!isc->stop) {
>
> Ditto.
>
>> /*
>> * If we are streaming, we can update profile to
>> * have the new settings in place.
>> @@ -1552,8 +1554,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
>> }
>>
>> /* if we have autowhitebalance on, start histogram procedure */
>> - if (ctrls->awb == ISC_WB_AUTO &&
>> - vb2_is_streaming(&isc->vb2_vidq) &&
>> + if (ctrls->awb == ISC_WB_AUTO && !isc->stop &&
>> ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
>> isc_set_histogram(isc, true);
>>
>> @@ -1829,6 +1830,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
>> struct vb2_queue *q = &isc->vb2_vidq;
>> int ret = 0;
>>
>> + isc->stop = true;
>> +
>
> I'm really not sure that you need the stop bool at all.
>
>> INIT_WORK(&isc->awb_work, isc_awb_work);
>>
>> ret = v4l2_device_register_subdev_nodes(&isc->v4l2_dev);
>
> Regards,
>
> Hans
>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 07/23] media: atmel: atmel-isc-base: use streaming status when queueing buffers
@ 2022-01-11 16:38 ` Eugen.Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen.Hristev @ 2022-01-11 16:38 UTC (permalink / raw)
To: hverkuil, jacopo, laurent.pinchart
Cc: linux-kernel, robh+dt, sakari.ailus, linux-arm-kernel, linux-media
On 1/11/22 5:53 PM, Hans Verkuil wrote:
> On 13/12/2021 14:49, Eugen Hristev wrote:
>> During experiments with libcamera, it looks like vb2_is_streaming returns
>> true before our start streaming is called.
>> Order of operations is streamon -> queue -> start_streaming
>> ISC would have started the DMA immediately when a buffer is being added
>> to the vbqueue if the queue is streaming.
>> It is more safe to start the DMA after the start streaming of the driver is
>> called.
>> Thus, even if vb2queue is streaming, add the buffer to the dma queue of the
>> driver instead of actually starting the DMA process, if the start streaming
>> has not been called yet.
>>
>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>> ---
>> drivers/media/platform/atmel/atmel-isc-base.c | 17 ++++++++++-------
>> 1 file changed, 10 insertions(+), 7 deletions(-)
>>
>> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
>> index 26a6090f056c..e6c9071c04f0 100644
>> --- a/drivers/media/platform/atmel/atmel-isc-base.c
>> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
>> @@ -441,12 +441,14 @@ static void isc_buffer_queue(struct vb2_buffer *vb)
>> unsigned long flags;
>>
>> spin_lock_irqsave(&isc->dma_queue_lock, flags);
>> - if (!isc->cur_frm && list_empty(&isc->dma_queue) &&
>> - vb2_is_streaming(vb->vb2_queue)) {
>> +
>> + if (!isc->cur_frm && list_empty(&isc->dma_queue) && !isc->stop) {
>> isc->cur_frm = buf;
>> isc_start_dma(isc);
>> - } else
>> + } else {
>> list_add_tail(&buf->list, &isc->dma_queue);
>> + }
>> +
>> spin_unlock_irqrestore(&isc->dma_queue_lock, flags);
>> }
>
> Both the old and new code doesn't make a lot of sense.
Hello Hans,
I debated this a bit with Laurent when I initially wrote the patch.
The problem with the current code, or the current way things are
working, is the fact that vb2_is_streaming returns true from the moment
of streamon.
The DMA engine of the ISC is being started at the moment of
start_streaming, and if buffers are being sent earlier/later, they are
just added to the queue and DMA engine is restarted if it finished
previously.
However if frames are added beforehand, like, first streamon, then add
some frames, and then later call the start_streaming, ISC will crash.
(as I said, DMA engine is prepared at start_streaming).
To avoid this, I changed the vb2_is_streaming calls to keeping an
internal state of the streaming.
Do you think I should try another different approach to solve this ?
Perhaps do not prepare the ISC dma engine at start_streaming, but at
another place ?
Laurent do you remember the talk we had about this problem a while back ?
>
> buf_queue is only called by vb2 if start_streaming has already been called or is
> about to be called.
That's a huge difference for ISC driver as it is today. start_streaming
being called afterwards, means the ISC DMA engine is not started (not
started streaming), hence a crash.
>
> Typically all that the buf_queue op does is to call list_add_tail(&buf->list, &isc->dma_queue);
> inside the spinlock.
>
>>
>> @@ -1014,7 +1016,7 @@ static int isc_s_fmt_vid_cap(struct file *file, void *priv,
>> {
>> struct isc_device *isc = video_drvdata(file);
>>
>> - if (vb2_is_streaming(&isc->vb2_vidq))
>> + if (!isc->stop)
>
> This is weird as well. Normally this calls vb2_is_busy to check if the
> queue is busy (that really means that buffers are already allocated, so
> changing the format isn't allowed anymore).
You think the query to the streaming status makes no sense here hence it
should be removed completely ?
I can do that with a prequel patch if it's the case.
Thanks for reviewing,
Eugen
>
>> return -EBUSY;
>>
>> return isc_set_fmt(isc, f);
>> @@ -1536,7 +1538,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
>>
>> isc_update_awb_ctrls(isc);
>>
>> - if (vb2_is_streaming(&isc->vb2_vidq)) {
>> + if (!isc->stop) {
>
> Ditto.
>
>> /*
>> * If we are streaming, we can update profile to
>> * have the new settings in place.
>> @@ -1552,8 +1554,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
>> }
>>
>> /* if we have autowhitebalance on, start histogram procedure */
>> - if (ctrls->awb == ISC_WB_AUTO &&
>> - vb2_is_streaming(&isc->vb2_vidq) &&
>> + if (ctrls->awb == ISC_WB_AUTO && !isc->stop &&
>> ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
>> isc_set_histogram(isc, true);
>>
>> @@ -1829,6 +1830,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
>> struct vb2_queue *q = &isc->vb2_vidq;
>> int ret = 0;
>>
>> + isc->stop = true;
>> +
>
> I'm really not sure that you need the stop bool at all.
>
>> INIT_WORK(&isc->awb_work, isc_awb_work);
>>
>> ret = v4l2_device_register_subdev_nodes(&isc->v4l2_dev);
>
> Regards,
>
> Hans
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 12/23] media: atmel: atmel-isc: implement media controller
2022-01-11 16:30 ` Eugen.Hristev
@ 2022-01-11 16:51 ` Jacopo Mondi
-1 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-11 16:51 UTC (permalink / raw)
To: Eugen.Hristev
Cc: linux-media, robh+dt, laurent.pinchart, sakari.ailus, devicetree,
linux-kernel, linux-arm-kernel, Nicolas.Ferre
On Tue, Jan 11, 2022 at 04:30:15PM +0000, Eugen.Hristev@microchip.com wrote:
> On 1/11/22 6:06 PM, Jacopo Mondi wrote:
> > Hi Eugen,
> >
> > On Mon, Dec 13, 2021 at 03:49:29PM +0200, Eugen Hristev wrote:
> >> Implement the support for media-controller.
> >> This means that the capabilities of the driver have changed and now
> >> it also advertises the IO_MC .
> >> The driver will register it's media device, and add the video entity to this
> >> media device. The subdevices are registered to the same media device.
> >> The ISC will have a base entity which is auto-detected as atmel_isc_base.
> >> It will also register a subdevice that allows cropping of the incoming frame
> >> to the maximum frame size supported by the ISC.
> >> The ISC will create a link between the subdevice that is asynchronously
> >> registered and the atmel_isc_scaler entity.
> >> Then, the atmel_isc_scaler and atmel_isc_base are connected through another
> >> link.
> >>
> >> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> >> ---
> >> Changes in v2:
> >> - implement try formats
> >>
> >> drivers/media/platform/atmel/Makefile | 2 +-
> >> drivers/media/platform/atmel/atmel-isc-base.c | 27 +-
> >> drivers/media/platform/atmel/atmel-isc-mc.c | 287 ++++++++++++++++++
> >> drivers/media/platform/atmel/atmel-isc.h | 36 +++
> >> .../media/platform/atmel/atmel-sama5d2-isc.c | 14 +-
> >> .../media/platform/atmel/atmel-sama7g5-isc.c | 12 +-
> >> 6 files changed, 370 insertions(+), 8 deletions(-)
> >> create mode 100644 drivers/media/platform/atmel/atmel-isc-mc.c
> >>
> >> diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
> >> index 794e8f739287..aa7a18b428f6 100644
> >> --- a/drivers/media/platform/atmel/Makefile
> >> +++ b/drivers/media/platform/atmel/Makefile
> >> @@ -1,7 +1,7 @@
> >> # SPDX-License-Identifier: GPL-2.0-only
> >> atmel-isc-objs = atmel-sama5d2-isc.o
> >> atmel-xisc-objs = atmel-sama7g5-isc.o
> >> -atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o
> >> +atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o atmel-isc-mc.o
> >>
> >> obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
> >> obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-common.o
> >> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> >> index d0542b97a391..b0c3ed21f372 100644
> >> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> >> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> >> @@ -1726,6 +1726,7 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
> >> struct isc_device, v4l2_dev);
> >> struct isc_subdev_entity *subdev_entity =
> >> container_of(notifier, struct isc_subdev_entity, notifier);
> >> + int pad;
> >>
> >> if (video_is_registered(&isc->video_dev)) {
> >> v4l2_err(&isc->v4l2_dev, "only supports one sub-device.\n");
> >> @@ -1734,6 +1735,16 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
> >>
> >> subdev_entity->sd = subdev;
> >>
> >> + pad = media_entity_get_fwnode_pad(&subdev->entity, asd->match.fwnode,
> >> + MEDIA_PAD_FL_SOURCE);
> >> + if (pad < 0) {
> >> + v4l2_err(&isc->v4l2_dev, "failed to find pad for %s\n",
> >> + subdev->name);
> >> + return pad;
> >> + }
> >> +
> >> + isc->remote_pad = pad;
> >> +
> >> return 0;
> >> }
> >>
> >> @@ -1748,8 +1759,8 @@ static void isc_async_unbind(struct v4l2_async_notifier *notifier,
> >> v4l2_ctrl_handler_free(&isc->ctrls.handler);
> >> }
> >>
> >> -static struct isc_format *find_format_by_code(struct isc_device *isc,
> >> - unsigned int code, int *index)
> >> +struct isc_format *isc_find_format_by_code(struct isc_device *isc,
> >> + unsigned int code, int *index)
> >> {
> >> struct isc_format *fmt = &isc->formats_list[0];
> >> unsigned int i;
> >> @@ -1765,6 +1776,7 @@ static struct isc_format *find_format_by_code(struct isc_device *isc,
> >>
> >> return NULL;
> >> }
> >> +EXPORT_SYMBOL_GPL(isc_find_format_by_code);
> >>
> >> static int isc_formats_init(struct isc_device *isc)
> >> {
> >> @@ -1781,7 +1793,7 @@ static int isc_formats_init(struct isc_device *isc)
> >> NULL, &mbus_code)) {
> >> mbus_code.index++;
> >>
> >> - fmt = find_format_by_code(isc, mbus_code.code, &i);
> >> + fmt = isc_find_format_by_code(isc, mbus_code.code, &i);
> >> if (!fmt) {
> >> v4l2_warn(&isc->v4l2_dev, "Mbus code %x not supported\n",
> >> mbus_code.code);
> >> @@ -1909,7 +1921,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
> >> vdev->queue = q;
> >> vdev->lock = &isc->lock;
> >> vdev->ctrl_handler = &isc->ctrls.handler;
> >> - vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE;
> >> + vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE |
> >> + V4L2_CAP_IO_MC;
> >> video_set_drvdata(vdev, isc);
> >>
> >> ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> >> @@ -1919,8 +1932,14 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
> >> goto isc_async_complete_err;
> >> }
> >>
> >> + ret = isc_mc_register(isc);
> >> + if (ret < 0)
> >> + goto isc_async_complete_unregister_device;
> >> return 0;
> >>
> >> +isc_async_complete_unregister_device:
> >> + video_unregister_device(vdev);
> >> +
> >> isc_async_complete_err:
> >> mutex_destroy(&isc->lock);
> >> return ret;
> >> diff --git a/drivers/media/platform/atmel/atmel-isc-mc.c b/drivers/media/platform/atmel/atmel-isc-mc.c
> >> new file mode 100644
> >> index 000000000000..2aea81d77b2f
> >> --- /dev/null
> >> +++ b/drivers/media/platform/atmel/atmel-isc-mc.c
> >> @@ -0,0 +1,287 @@
> >> +// SPDX-License-Identifier: GPL-2.0-only
> >> +/*
> >> + * Microchip Image Sensor Controller (ISC) Media Controller support
> >> + *
> >> + * Copyright (C) 2021 Microchip Technology, Inc.
> >> + *
> >> + * Author: Eugen Hristev <eugen.hristev@microchip.com>
> >> + *
> >> + */
> >> +
> >> +#include <media/v4l2-device.h>
> >> +#include <media/v4l2-subdev.h>
> >
> > Should you include <media/media-entity.h> and <media/media-device.h> ?
>
> Hi Jacopo,
>
> They are probably being included from another header. I will add them.
>
Thank you :)
> >
> > Also, should you select MEDIA_CONTROLLER and V4L2_SUBDEV_API for all
> > the Kconfig symbols ?
>
>
> VIDEO_ATMEL_ISC directly selects:
> select MEDIA_CONTROLLER
> select VIDEO_V4L2_SUBDEV_API
>
> and VIDEO_ATMEL_XISC
> depends on VIDEO_V4L2 && COMMON_CLK && VIDEO_V4L2_SUBDEV_API
And selects VIDEO_ATMEL_ISC_BASE so it's fine!
VIDEO_ATMEL_ISI does not. I'm fine if this is intended, I might have
got lost in the several pieces this driver is composed of :)
>
> and VIDEO_V4L2_SUBDEV_API depends on MEDIA_CONTROLLER
>
> >
> >> +
> >> +#include "atmel-isc-regs.h"
> >> +#include "atmel-isc.h"
> >> +
> >> +static const struct media_device_ops isc_media_ops = {
> >> +};
> >
> > Do you need this even if it's empty ?
> >
>
> Not sure. I can remove it and pass NULL, but I haven't checked what the
> subsystem is doing if there is no ops. I will try.
>
Seems like it is safe as the core checks for if (mdev->ops) before
accessing it (except for mc-request for which operations are
mandatory)
> >> +
> >> +static int isc_scaler_get_fmt(struct v4l2_subdev *sd,
> >> + struct v4l2_subdev_state *sd_state,
> >> + struct v4l2_subdev_format *format)
> >> +{
> >> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
> >> + struct v4l2_mbus_framefmt *v4l2_try_fmt;
> >> +
> >> + if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
> >> + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state,
> >> + format->pad);
> >> + format->format = *v4l2_try_fmt;
> >> +
> >> + return 0;
> >> + }
> >> +
> >> + format->format = isc->scaler_format;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int isc_scaler_set_fmt(struct v4l2_subdev *sd,
> >> + struct v4l2_subdev_state *sd_state,
> >> + struct v4l2_subdev_format *req_fmt)
> >> +{
> >> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
> >> + struct v4l2_mbus_framefmt *v4l2_try_fmt;
> >> + struct isc_format *fmt;
> >> + unsigned int i;
> >> +
> >> + if (req_fmt->pad == ISC_SCALER_PAD_SOURCE)
> >> + v4l_bound_align_image
> >> + (&req_fmt->format.width, 16, isc->max_width, 0,
> >> + &req_fmt->format.height, 16, isc->max_height, 0, 0);
> >> + else
> >> + v4l_bound_align_image
> >> + (&req_fmt->format.width, 16, 10000, 0,
> >> + &req_fmt->format.height, 16, 10000, 0, 0);
> >> +
> >> + req_fmt->format.colorspace = V4L2_COLORSPACE_SRGB;
> >> + req_fmt->format.field = V4L2_FIELD_NONE;
> >> + req_fmt->format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> >> + req_fmt->format.quantization = V4L2_QUANTIZATION_DEFAULT;
> >> + req_fmt->format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
> >> +
> >> + fmt = isc_find_format_by_code(isc, req_fmt->format.code, &i);
> >> +
> >> + if (!fmt)
> >> + fmt = &isc->formats_list[0];
> >> +
> >> + req_fmt->format.code = fmt->mbus_code;
> >> +
> >> + if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> >> + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state,
> >> + req_fmt->pad);
> >> + *v4l2_try_fmt = req_fmt->format;
> >> + /* Trying on the pad sink makes the source sink change too */
> >> + if (req_fmt->pad == ISC_SCALER_PAD_SINK) {
> >> + v4l2_try_fmt =
> >> + v4l2_subdev_get_try_format(sd, sd_state,
> >> + ISC_SCALER_PAD_SOURCE);
> >> + *v4l2_try_fmt = req_fmt->format;
> >> +
> >> + v4l_bound_align_image(&v4l2_try_fmt->width,
> >> + 16, isc->max_width, 0,
> >> + &v4l2_try_fmt->height,
> >> + 16, isc->max_height, 0, 0);
> >> + }
> >> + /* if we are just trying, we are done */
> >> + return 0;
> >> + }
> >> +
> >> + isc->scaler_format = req_fmt->format;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int isc_scaler_enum_mbus_code(struct v4l2_subdev *sd,
> >> + struct v4l2_subdev_state *sd_state,
> >> + struct v4l2_subdev_mbus_code_enum *code)
> >> +{
> >> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
> >> + int supported_index = 0;
> >> + int i;
> >> +
> >> + for (i = 0; i < isc->formats_list_size; i++) {
> >> + if (!isc->formats_list[i].sd_support)
> >> + continue;
> >> + if (supported_index == code->index) {
> >> + code->code = isc->formats_list[i].mbus_code;
> >> + return 0;
> >> + }
> >> + supported_index++;
> >> + }
> >> +
> >> + return -EINVAL;
> >> +}
> >> +
> >> +static int isc_scaler_g_sel(struct v4l2_subdev *sd,
> >> + struct v4l2_subdev_state *sd_state,
> >> + struct v4l2_subdev_selection *sel)
> >> +{
> >> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
> >> +
> >> + if (sel->pad == ISC_SCALER_PAD_SOURCE)
> >> + return -EINVAL;
> >> +
> >> + if (sel->target != V4L2_SEL_TGT_CROP_BOUNDS &&
> >> + sel->target != V4L2_SEL_TGT_CROP)
> >> + return -EINVAL;
> >> +
> >> + sel->r.height = isc->max_height;
> >> + sel->r.width = isc->max_width;
> >> +
> >> + sel->r.left = 0;
> >> + sel->r.top = 0;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int isc_scaler_init_cfg(struct v4l2_subdev *sd,
> >> + struct v4l2_subdev_state *sd_state)
> >> +{
> >> + struct v4l2_mbus_framefmt *v4l2_try_fmt =
> >> + v4l2_subdev_get_try_format(sd, sd_state, 0);
> >> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
> >> +
> >> + *v4l2_try_fmt = isc->scaler_format;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static const struct v4l2_subdev_pad_ops isc_scaler_pad_ops = {
> >> + .enum_mbus_code = isc_scaler_enum_mbus_code,
> >> + .set_fmt = isc_scaler_set_fmt,
> >> + .get_fmt = isc_scaler_get_fmt,
> >> + .get_selection = isc_scaler_g_sel,
> >> + .init_cfg = isc_scaler_init_cfg,
> >> +};
> >> +
> >> +static const struct v4l2_subdev_ops xisc_scaler_subdev_ops = {
> >> + .pad = &isc_scaler_pad_ops,
> >> +};
> >> +
> >> +static int isc_init_own_sd(struct isc_device *isc)
> >> +{
> >> + int ret;
> >> +
> >> + v4l2_subdev_init(&isc->scaler_sd, &xisc_scaler_subdev_ops);
> >> +
> >> + isc->scaler_sd.owner = THIS_MODULE;
> >> + isc->scaler_sd.dev = isc->dev;
> >> + snprintf(isc->scaler_sd.name, sizeof(isc->scaler_sd.name),
> >> + "atmel_isc_scaler");
> >> +
> >> + isc->scaler_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> >> + isc->scaler_sd.entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
> >> + isc->scaler_pads[ISC_SCALER_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> >> + isc->scaler_pads[ISC_SCALER_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
> >> +
> >> + isc->scaler_format.height = isc->max_height;
> >> + isc->scaler_format.width = isc->max_width;
> >> + isc->scaler_format.code = isc->formats_list[0].mbus_code;
> >> + isc->scaler_format.colorspace = V4L2_COLORSPACE_SRGB;
> >> + isc->scaler_format.field = V4L2_FIELD_NONE;
> >> + isc->scaler_format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> >> + isc->scaler_format.quantization = V4L2_QUANTIZATION_DEFAULT;
> >> + isc->scaler_format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
> >> +
> >> + ret = media_entity_pads_init(&isc->scaler_sd.entity,
> >> + ISC_SCALER_PADS_NUM,
> >> + isc->scaler_pads);
> >> + if (ret < 0) {
> >> + dev_err(isc->dev, "scaler sd media entity init failed\n");
> >> + return ret;
> >> + }
> >> + ret = v4l2_device_register_subdev(&isc->v4l2_dev, &isc->scaler_sd);
> >> + if (ret < 0) {
> >> + dev_err(isc->dev, "scaler sd failed to register subdev\n");
> >> + return ret;
> >> + }
> >> +
> >> + return ret;
> >> +}
> >> +
> >> +int isc_mc_init(struct isc_device *isc, u32 ver)
> >> +{
> >> + const struct of_device_id *match;
> >> + int ret;
> >> +
> >> + isc->video_dev.entity.function = MEDIA_ENT_F_IO_V4L;
> >> + isc->video_dev.entity.flags = MEDIA_ENT_FL_DEFAULT;
> >> + isc->pads[ISC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> >> +
> >> + ret = media_entity_pads_init(&isc->video_dev.entity, ISC_PADS_NUM,
> >> + isc->pads);
> >> + if (ret < 0) {
> >> + dev_err(isc->dev, "media entity init failed\n");
> >> + return ret;
> >> + }
> >> +
> >> + isc->mdev.dev = isc->dev;
> >> + isc->mdev.ops = &isc_media_ops;
> >> +
> >> + match = of_match_node(isc->dev->driver->of_match_table,
> >> + isc->dev->of_node);
> >> +
> >> + strscpy(isc->mdev.driver_name, KBUILD_MODNAME,
> >> + sizeof(isc->mdev.driver_name));
> >> + strscpy(isc->mdev.model, match->compatible, sizeof(isc->mdev.model));
> >> + snprintf(isc->mdev.bus_info, sizeof(isc->mdev.bus_info), "platform:%s",
> >> + isc->v4l2_dev.name);
> >> + isc->mdev.hw_revision = ver;
> >> +
> >> + media_device_init(&isc->mdev);
> >> +
> >> + isc->v4l2_dev.mdev = &isc->mdev;
> >> +
> >> + return isc_init_own_sd(isc);
> >> +}
> >> +EXPORT_SYMBOL_GPL(isc_mc_init);
> >> +
> >> +int isc_mc_register(struct isc_device *isc)
> >> +{
> >> + int ret;
> >> +
> >> + ret = media_create_pad_link(&isc->current_subdev->sd->entity,
> >> + isc->remote_pad, &isc->scaler_sd.entity,
> >> + ISC_SCALER_PAD_SINK,
> >> + MEDIA_LNK_FL_ENABLED |
> >> + MEDIA_LNK_FL_IMMUTABLE);
> >> +
> >> + if (ret < 0) {
> >> + v4l2_err(&isc->v4l2_dev,
> >> + "Failed to create pad link: %s to %s\n",
> >> + isc->current_subdev->sd->entity.name,
> >> + isc->scaler_sd.entity.name);
> >> + return ret;
> >> + }
> >> +
> >> + dev_dbg(isc->dev, "link with %s pad: %d\n",
> >> + isc->current_subdev->sd->name, isc->remote_pad);
> >> +
> >> + ret = media_create_pad_link(&isc->scaler_sd.entity,
> >> + ISC_SCALER_PAD_SOURCE,
> >> + &isc->video_dev.entity, ISC_PAD_SINK,
> >> + MEDIA_LNK_FL_ENABLED |
> >> + MEDIA_LNK_FL_IMMUTABLE);
> >> +
> >> + if (ret < 0) {
> >> + v4l2_err(&isc->v4l2_dev,
> >> + "Failed to create pad link: %s to %s\n",
> >> + isc->scaler_sd.entity.name,
> >> + isc->video_dev.entity.name);
> >> + return ret;
> >> + }
> >> +
> >> + dev_dbg(isc->dev, "link with %s pad: %d\n", isc->scaler_sd.name,
> >> + ISC_SCALER_PAD_SOURCE);
> >> +
> >> + return media_device_register(&isc->mdev);
> >
> > I feel like the isc-base driver shold instead register the
> > media-controller and this file shuld actually be only about
> > isc-scaler or something. I might be missing why it is not possible
> > though. It doesn't seem to me that mc support is conditional, so it
> > can happen in the base driver directly ?
>
> This actually is the base driver. Just that the atmel-isc-base.c is
> pretty crowded. So I moved the MC part in this file. But it's the same
> base driver.
>
I understand. isc-base is pretty big, you're right.
However breaking mc apart in a file that mixes registering an entity
for the scaler and initializing mc for the whole driver feels weird,
at least to me, as I don't immediately get why the mc part is handled
by the scaler file...
> >
> > Links creation can still happen at async_complete() (I would have
> > suggested _bound() time, but you have no asyn subdev for the scaler,
> > so I guess the only async subdev here is the sensor).
>
> Yes, the sensor or any kind of subdevice (XISC normally is connected to
> a bridge, ISC to a sensor directly )
>
> >
> > All in all: can this function be removed and:
> > 1) Base driver registers the media controller device in probe
>
> Should we register the media device even if it's not operational ? and
> no video device has been registered ?
> I thought it would be safer to have it registered with all the
> links/entities/etc once the video node has been completed.
>
> But if you say this is the proper way, I can do that.
> I wanted to keep things separated and not overcrowd the base file /probe
> function, but if it's just a media_device_register call, I can do it there.
You are right. Calling media_device_register() at async_complete time
is the right thing to do.
Thanks
j
>
> > 2) links are created at async_complete time ?
>
> That works too. I think I could move them there.
>
> Thanks for reviewing,
>
> Eugen
>
> >
> > Thanks
> > j
> >
> >
> >> +}
> >> +EXPORT_SYMBOL_GPL(isc_mc_register);
> >> +
> >> +void isc_mc_cleanup(struct isc_device *isc)
> >> +{
> >> + media_entity_cleanup(&isc->video_dev.entity);
> >> +}
> >> +EXPORT_SYMBOL_GPL(isc_mc_cleanup);
> >> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
> >> index 07fa6dbf8460..0b6370d7775f 100644
> >> --- a/drivers/media/platform/atmel/atmel-isc.h
> >> +++ b/drivers/media/platform/atmel/atmel-isc.h
> >> @@ -183,6 +183,17 @@ struct isc_reg_offsets {
> >> u32 his_entry;
> >> };
> >>
> >> +enum isc_mc_pads {
> >> + ISC_PAD_SINK = 0,
> >> + ISC_PADS_NUM = 1,
> >> +};
> >> +
> >> +enum isc_scaler_pads {
> >> + ISC_SCALER_PAD_SINK = 0,
> >> + ISC_SCALER_PAD_SOURCE = 1,
> >> + ISC_SCALER_PADS_NUM = 2,
> >> +};
> >> +
> >> /*
> >> * struct isc_device - ISC device driver data/config struct
> >> * @regmap: Register map
> >> @@ -258,6 +269,12 @@ struct isc_reg_offsets {
> >> * be used as an input to the controller
> >> * @controller_formats_size: size of controller_formats array
> >> * @formats_list_size: size of formats_list array
> >> + * @pads: media controller pads for isc video entity
> >> + * @mdev: media device that is registered by the isc
> >> + * @remote_pad: remote pad on the connected subdevice
> >> + * @scaler_sd: subdevice for the scaler that isc registers
> >> + * @scaler_pads: media controller pads for the scaler subdevice
> >> + * @scaler_format: current format for the scaler subdevice
> >> */
> >> struct isc_device {
> >> struct regmap *regmap;
> >> @@ -346,6 +363,19 @@ struct isc_device {
> >> struct isc_format *formats_list;
> >> u32 controller_formats_size;
> >> u32 formats_list_size;
> >> +
> >> + struct {
> >> + struct media_pad pads[ISC_PADS_NUM];
> >> + struct media_device mdev;
> >> +
> >> + u32 remote_pad;
> >> + };
> >> +
> >> + struct {
> >> + struct v4l2_subdev scaler_sd;
> >> + struct media_pad scaler_pads[ISC_SCALER_PADS_NUM];
> >> + struct v4l2_mbus_framefmt scaler_format;
> >> + };
> >> };
> >>
> >> extern const struct regmap_config isc_regmap_config;
> >> @@ -357,4 +387,10 @@ int isc_clk_init(struct isc_device *isc);
> >> void isc_subdev_cleanup(struct isc_device *isc);
> >> void isc_clk_cleanup(struct isc_device *isc);
> >>
> >> +int isc_mc_init(struct isc_device *isc, u32 ver);
> >> +int isc_mc_register(struct isc_device *isc);
> >> +void isc_mc_cleanup(struct isc_device *isc);
> >> +
> >> +struct isc_format *isc_find_format_by_code(struct isc_device *isc,
> >> + unsigned int code, int *index);
> >> #endif
> >> diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> >> index 1b2063cce0f7..a3e4c32a66c4 100644
> >> --- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> >> +++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> >> @@ -529,6 +529,12 @@ static int atmel_isc_probe(struct platform_device *pdev)
> >> break;
> >> }
> >>
> >> + regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
> >> +
> >> + ret = isc_mc_init(isc, ver);
> >> + if (ret < 0)
> >> + goto isc_probe_mc_init_err;
> >> +
> >> pm_runtime_set_active(dev);
> >> pm_runtime_enable(dev);
> >> pm_request_idle(dev);
> >> @@ -538,7 +544,7 @@ static int atmel_isc_probe(struct platform_device *pdev)
> >> ret = clk_prepare_enable(isc->ispck);
> >> if (ret) {
> >> dev_err(dev, "failed to enable ispck: %d\n", ret);
> >> - goto cleanup_subdev;
> >> + goto isc_probe_mc_init_err;
> >> }
> >>
> >> /* ispck should be greater or equal to hclock */
> >> @@ -548,7 +554,6 @@ static int atmel_isc_probe(struct platform_device *pdev)
> >> goto unprepare_clk;
> >> }
> >>
> >> - regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
> >> dev_info(dev, "Microchip ISC version %x\n", ver);
> >>
> >> return 0;
> >> @@ -556,6 +561,9 @@ static int atmel_isc_probe(struct platform_device *pdev)
> >> unprepare_clk:
> >> clk_disable_unprepare(isc->ispck);
> >>
> >> +isc_probe_mc_init_err:
> >> + isc_mc_cleanup(isc);
> >> +
> >> cleanup_subdev:
> >> isc_subdev_cleanup(isc);
> >>
> >> @@ -576,6 +584,8 @@ static int atmel_isc_remove(struct platform_device *pdev)
> >>
> >> pm_runtime_disable(&pdev->dev);
> >>
> >> + isc_mc_cleanup(isc);
> >> +
> >> isc_subdev_cleanup(isc);
> >>
> >> v4l2_device_unregister(&isc->v4l2_dev);
> >> diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> >> index 2b1082295c13..d2ac80b1ccc3 100644
> >> --- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> >> +++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> >> @@ -523,15 +523,23 @@ static int microchip_xisc_probe(struct platform_device *pdev)
> >> break;
> >> }
> >>
> >> + regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
> >> +
> >> + ret = isc_mc_init(isc, ver);
> >> + if (ret < 0)
> >> + goto isc_probe_mc_init_err;
> >> +
> >> pm_runtime_set_active(dev);
> >> pm_runtime_enable(dev);
> >> pm_request_idle(dev);
> >>
> >> - regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
> >> dev_info(dev, "Microchip XISC version %x\n", ver);
> >>
> >> return 0;
> >>
> >> +isc_probe_mc_init_err:
> >> + isc_mc_cleanup(isc);
> >> +
> >> cleanup_subdev:
> >> isc_subdev_cleanup(isc);
> >>
> >> @@ -552,6 +560,8 @@ static int microchip_xisc_remove(struct platform_device *pdev)
> >>
> >> pm_runtime_disable(&pdev->dev);
> >>
> >> + isc_mc_cleanup(isc);
> >> +
> >> isc_subdev_cleanup(isc);
> >>
> >> v4l2_device_unregister(&isc->v4l2_dev);
> >> --
> >> 2.25.1
> >>
>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 12/23] media: atmel: atmel-isc: implement media controller
@ 2022-01-11 16:51 ` Jacopo Mondi
0 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-11 16:51 UTC (permalink / raw)
To: Eugen.Hristev
Cc: devicetree, linux-kernel, robh+dt, sakari.ailus,
laurent.pinchart, linux-arm-kernel, linux-media
On Tue, Jan 11, 2022 at 04:30:15PM +0000, Eugen.Hristev@microchip.com wrote:
> On 1/11/22 6:06 PM, Jacopo Mondi wrote:
> > Hi Eugen,
> >
> > On Mon, Dec 13, 2021 at 03:49:29PM +0200, Eugen Hristev wrote:
> >> Implement the support for media-controller.
> >> This means that the capabilities of the driver have changed and now
> >> it also advertises the IO_MC .
> >> The driver will register it's media device, and add the video entity to this
> >> media device. The subdevices are registered to the same media device.
> >> The ISC will have a base entity which is auto-detected as atmel_isc_base.
> >> It will also register a subdevice that allows cropping of the incoming frame
> >> to the maximum frame size supported by the ISC.
> >> The ISC will create a link between the subdevice that is asynchronously
> >> registered and the atmel_isc_scaler entity.
> >> Then, the atmel_isc_scaler and atmel_isc_base are connected through another
> >> link.
> >>
> >> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> >> ---
> >> Changes in v2:
> >> - implement try formats
> >>
> >> drivers/media/platform/atmel/Makefile | 2 +-
> >> drivers/media/platform/atmel/atmel-isc-base.c | 27 +-
> >> drivers/media/platform/atmel/atmel-isc-mc.c | 287 ++++++++++++++++++
> >> drivers/media/platform/atmel/atmel-isc.h | 36 +++
> >> .../media/platform/atmel/atmel-sama5d2-isc.c | 14 +-
> >> .../media/platform/atmel/atmel-sama7g5-isc.c | 12 +-
> >> 6 files changed, 370 insertions(+), 8 deletions(-)
> >> create mode 100644 drivers/media/platform/atmel/atmel-isc-mc.c
> >>
> >> diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
> >> index 794e8f739287..aa7a18b428f6 100644
> >> --- a/drivers/media/platform/atmel/Makefile
> >> +++ b/drivers/media/platform/atmel/Makefile
> >> @@ -1,7 +1,7 @@
> >> # SPDX-License-Identifier: GPL-2.0-only
> >> atmel-isc-objs = atmel-sama5d2-isc.o
> >> atmel-xisc-objs = atmel-sama7g5-isc.o
> >> -atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o
> >> +atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o atmel-isc-mc.o
> >>
> >> obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
> >> obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-common.o
> >> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> >> index d0542b97a391..b0c3ed21f372 100644
> >> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> >> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> >> @@ -1726,6 +1726,7 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
> >> struct isc_device, v4l2_dev);
> >> struct isc_subdev_entity *subdev_entity =
> >> container_of(notifier, struct isc_subdev_entity, notifier);
> >> + int pad;
> >>
> >> if (video_is_registered(&isc->video_dev)) {
> >> v4l2_err(&isc->v4l2_dev, "only supports one sub-device.\n");
> >> @@ -1734,6 +1735,16 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
> >>
> >> subdev_entity->sd = subdev;
> >>
> >> + pad = media_entity_get_fwnode_pad(&subdev->entity, asd->match.fwnode,
> >> + MEDIA_PAD_FL_SOURCE);
> >> + if (pad < 0) {
> >> + v4l2_err(&isc->v4l2_dev, "failed to find pad for %s\n",
> >> + subdev->name);
> >> + return pad;
> >> + }
> >> +
> >> + isc->remote_pad = pad;
> >> +
> >> return 0;
> >> }
> >>
> >> @@ -1748,8 +1759,8 @@ static void isc_async_unbind(struct v4l2_async_notifier *notifier,
> >> v4l2_ctrl_handler_free(&isc->ctrls.handler);
> >> }
> >>
> >> -static struct isc_format *find_format_by_code(struct isc_device *isc,
> >> - unsigned int code, int *index)
> >> +struct isc_format *isc_find_format_by_code(struct isc_device *isc,
> >> + unsigned int code, int *index)
> >> {
> >> struct isc_format *fmt = &isc->formats_list[0];
> >> unsigned int i;
> >> @@ -1765,6 +1776,7 @@ static struct isc_format *find_format_by_code(struct isc_device *isc,
> >>
> >> return NULL;
> >> }
> >> +EXPORT_SYMBOL_GPL(isc_find_format_by_code);
> >>
> >> static int isc_formats_init(struct isc_device *isc)
> >> {
> >> @@ -1781,7 +1793,7 @@ static int isc_formats_init(struct isc_device *isc)
> >> NULL, &mbus_code)) {
> >> mbus_code.index++;
> >>
> >> - fmt = find_format_by_code(isc, mbus_code.code, &i);
> >> + fmt = isc_find_format_by_code(isc, mbus_code.code, &i);
> >> if (!fmt) {
> >> v4l2_warn(&isc->v4l2_dev, "Mbus code %x not supported\n",
> >> mbus_code.code);
> >> @@ -1909,7 +1921,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
> >> vdev->queue = q;
> >> vdev->lock = &isc->lock;
> >> vdev->ctrl_handler = &isc->ctrls.handler;
> >> - vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE;
> >> + vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE |
> >> + V4L2_CAP_IO_MC;
> >> video_set_drvdata(vdev, isc);
> >>
> >> ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> >> @@ -1919,8 +1932,14 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
> >> goto isc_async_complete_err;
> >> }
> >>
> >> + ret = isc_mc_register(isc);
> >> + if (ret < 0)
> >> + goto isc_async_complete_unregister_device;
> >> return 0;
> >>
> >> +isc_async_complete_unregister_device:
> >> + video_unregister_device(vdev);
> >> +
> >> isc_async_complete_err:
> >> mutex_destroy(&isc->lock);
> >> return ret;
> >> diff --git a/drivers/media/platform/atmel/atmel-isc-mc.c b/drivers/media/platform/atmel/atmel-isc-mc.c
> >> new file mode 100644
> >> index 000000000000..2aea81d77b2f
> >> --- /dev/null
> >> +++ b/drivers/media/platform/atmel/atmel-isc-mc.c
> >> @@ -0,0 +1,287 @@
> >> +// SPDX-License-Identifier: GPL-2.0-only
> >> +/*
> >> + * Microchip Image Sensor Controller (ISC) Media Controller support
> >> + *
> >> + * Copyright (C) 2021 Microchip Technology, Inc.
> >> + *
> >> + * Author: Eugen Hristev <eugen.hristev@microchip.com>
> >> + *
> >> + */
> >> +
> >> +#include <media/v4l2-device.h>
> >> +#include <media/v4l2-subdev.h>
> >
> > Should you include <media/media-entity.h> and <media/media-device.h> ?
>
> Hi Jacopo,
>
> They are probably being included from another header. I will add them.
>
Thank you :)
> >
> > Also, should you select MEDIA_CONTROLLER and V4L2_SUBDEV_API for all
> > the Kconfig symbols ?
>
>
> VIDEO_ATMEL_ISC directly selects:
> select MEDIA_CONTROLLER
> select VIDEO_V4L2_SUBDEV_API
>
> and VIDEO_ATMEL_XISC
> depends on VIDEO_V4L2 && COMMON_CLK && VIDEO_V4L2_SUBDEV_API
And selects VIDEO_ATMEL_ISC_BASE so it's fine!
VIDEO_ATMEL_ISI does not. I'm fine if this is intended, I might have
got lost in the several pieces this driver is composed of :)
>
> and VIDEO_V4L2_SUBDEV_API depends on MEDIA_CONTROLLER
>
> >
> >> +
> >> +#include "atmel-isc-regs.h"
> >> +#include "atmel-isc.h"
> >> +
> >> +static const struct media_device_ops isc_media_ops = {
> >> +};
> >
> > Do you need this even if it's empty ?
> >
>
> Not sure. I can remove it and pass NULL, but I haven't checked what the
> subsystem is doing if there is no ops. I will try.
>
Seems like it is safe as the core checks for if (mdev->ops) before
accessing it (except for mc-request for which operations are
mandatory)
> >> +
> >> +static int isc_scaler_get_fmt(struct v4l2_subdev *sd,
> >> + struct v4l2_subdev_state *sd_state,
> >> + struct v4l2_subdev_format *format)
> >> +{
> >> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
> >> + struct v4l2_mbus_framefmt *v4l2_try_fmt;
> >> +
> >> + if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
> >> + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state,
> >> + format->pad);
> >> + format->format = *v4l2_try_fmt;
> >> +
> >> + return 0;
> >> + }
> >> +
> >> + format->format = isc->scaler_format;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int isc_scaler_set_fmt(struct v4l2_subdev *sd,
> >> + struct v4l2_subdev_state *sd_state,
> >> + struct v4l2_subdev_format *req_fmt)
> >> +{
> >> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
> >> + struct v4l2_mbus_framefmt *v4l2_try_fmt;
> >> + struct isc_format *fmt;
> >> + unsigned int i;
> >> +
> >> + if (req_fmt->pad == ISC_SCALER_PAD_SOURCE)
> >> + v4l_bound_align_image
> >> + (&req_fmt->format.width, 16, isc->max_width, 0,
> >> + &req_fmt->format.height, 16, isc->max_height, 0, 0);
> >> + else
> >> + v4l_bound_align_image
> >> + (&req_fmt->format.width, 16, 10000, 0,
> >> + &req_fmt->format.height, 16, 10000, 0, 0);
> >> +
> >> + req_fmt->format.colorspace = V4L2_COLORSPACE_SRGB;
> >> + req_fmt->format.field = V4L2_FIELD_NONE;
> >> + req_fmt->format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> >> + req_fmt->format.quantization = V4L2_QUANTIZATION_DEFAULT;
> >> + req_fmt->format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
> >> +
> >> + fmt = isc_find_format_by_code(isc, req_fmt->format.code, &i);
> >> +
> >> + if (!fmt)
> >> + fmt = &isc->formats_list[0];
> >> +
> >> + req_fmt->format.code = fmt->mbus_code;
> >> +
> >> + if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> >> + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state,
> >> + req_fmt->pad);
> >> + *v4l2_try_fmt = req_fmt->format;
> >> + /* Trying on the pad sink makes the source sink change too */
> >> + if (req_fmt->pad == ISC_SCALER_PAD_SINK) {
> >> + v4l2_try_fmt =
> >> + v4l2_subdev_get_try_format(sd, sd_state,
> >> + ISC_SCALER_PAD_SOURCE);
> >> + *v4l2_try_fmt = req_fmt->format;
> >> +
> >> + v4l_bound_align_image(&v4l2_try_fmt->width,
> >> + 16, isc->max_width, 0,
> >> + &v4l2_try_fmt->height,
> >> + 16, isc->max_height, 0, 0);
> >> + }
> >> + /* if we are just trying, we are done */
> >> + return 0;
> >> + }
> >> +
> >> + isc->scaler_format = req_fmt->format;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int isc_scaler_enum_mbus_code(struct v4l2_subdev *sd,
> >> + struct v4l2_subdev_state *sd_state,
> >> + struct v4l2_subdev_mbus_code_enum *code)
> >> +{
> >> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
> >> + int supported_index = 0;
> >> + int i;
> >> +
> >> + for (i = 0; i < isc->formats_list_size; i++) {
> >> + if (!isc->formats_list[i].sd_support)
> >> + continue;
> >> + if (supported_index == code->index) {
> >> + code->code = isc->formats_list[i].mbus_code;
> >> + return 0;
> >> + }
> >> + supported_index++;
> >> + }
> >> +
> >> + return -EINVAL;
> >> +}
> >> +
> >> +static int isc_scaler_g_sel(struct v4l2_subdev *sd,
> >> + struct v4l2_subdev_state *sd_state,
> >> + struct v4l2_subdev_selection *sel)
> >> +{
> >> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
> >> +
> >> + if (sel->pad == ISC_SCALER_PAD_SOURCE)
> >> + return -EINVAL;
> >> +
> >> + if (sel->target != V4L2_SEL_TGT_CROP_BOUNDS &&
> >> + sel->target != V4L2_SEL_TGT_CROP)
> >> + return -EINVAL;
> >> +
> >> + sel->r.height = isc->max_height;
> >> + sel->r.width = isc->max_width;
> >> +
> >> + sel->r.left = 0;
> >> + sel->r.top = 0;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int isc_scaler_init_cfg(struct v4l2_subdev *sd,
> >> + struct v4l2_subdev_state *sd_state)
> >> +{
> >> + struct v4l2_mbus_framefmt *v4l2_try_fmt =
> >> + v4l2_subdev_get_try_format(sd, sd_state, 0);
> >> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
> >> +
> >> + *v4l2_try_fmt = isc->scaler_format;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static const struct v4l2_subdev_pad_ops isc_scaler_pad_ops = {
> >> + .enum_mbus_code = isc_scaler_enum_mbus_code,
> >> + .set_fmt = isc_scaler_set_fmt,
> >> + .get_fmt = isc_scaler_get_fmt,
> >> + .get_selection = isc_scaler_g_sel,
> >> + .init_cfg = isc_scaler_init_cfg,
> >> +};
> >> +
> >> +static const struct v4l2_subdev_ops xisc_scaler_subdev_ops = {
> >> + .pad = &isc_scaler_pad_ops,
> >> +};
> >> +
> >> +static int isc_init_own_sd(struct isc_device *isc)
> >> +{
> >> + int ret;
> >> +
> >> + v4l2_subdev_init(&isc->scaler_sd, &xisc_scaler_subdev_ops);
> >> +
> >> + isc->scaler_sd.owner = THIS_MODULE;
> >> + isc->scaler_sd.dev = isc->dev;
> >> + snprintf(isc->scaler_sd.name, sizeof(isc->scaler_sd.name),
> >> + "atmel_isc_scaler");
> >> +
> >> + isc->scaler_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> >> + isc->scaler_sd.entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
> >> + isc->scaler_pads[ISC_SCALER_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> >> + isc->scaler_pads[ISC_SCALER_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
> >> +
> >> + isc->scaler_format.height = isc->max_height;
> >> + isc->scaler_format.width = isc->max_width;
> >> + isc->scaler_format.code = isc->formats_list[0].mbus_code;
> >> + isc->scaler_format.colorspace = V4L2_COLORSPACE_SRGB;
> >> + isc->scaler_format.field = V4L2_FIELD_NONE;
> >> + isc->scaler_format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> >> + isc->scaler_format.quantization = V4L2_QUANTIZATION_DEFAULT;
> >> + isc->scaler_format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
> >> +
> >> + ret = media_entity_pads_init(&isc->scaler_sd.entity,
> >> + ISC_SCALER_PADS_NUM,
> >> + isc->scaler_pads);
> >> + if (ret < 0) {
> >> + dev_err(isc->dev, "scaler sd media entity init failed\n");
> >> + return ret;
> >> + }
> >> + ret = v4l2_device_register_subdev(&isc->v4l2_dev, &isc->scaler_sd);
> >> + if (ret < 0) {
> >> + dev_err(isc->dev, "scaler sd failed to register subdev\n");
> >> + return ret;
> >> + }
> >> +
> >> + return ret;
> >> +}
> >> +
> >> +int isc_mc_init(struct isc_device *isc, u32 ver)
> >> +{
> >> + const struct of_device_id *match;
> >> + int ret;
> >> +
> >> + isc->video_dev.entity.function = MEDIA_ENT_F_IO_V4L;
> >> + isc->video_dev.entity.flags = MEDIA_ENT_FL_DEFAULT;
> >> + isc->pads[ISC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> >> +
> >> + ret = media_entity_pads_init(&isc->video_dev.entity, ISC_PADS_NUM,
> >> + isc->pads);
> >> + if (ret < 0) {
> >> + dev_err(isc->dev, "media entity init failed\n");
> >> + return ret;
> >> + }
> >> +
> >> + isc->mdev.dev = isc->dev;
> >> + isc->mdev.ops = &isc_media_ops;
> >> +
> >> + match = of_match_node(isc->dev->driver->of_match_table,
> >> + isc->dev->of_node);
> >> +
> >> + strscpy(isc->mdev.driver_name, KBUILD_MODNAME,
> >> + sizeof(isc->mdev.driver_name));
> >> + strscpy(isc->mdev.model, match->compatible, sizeof(isc->mdev.model));
> >> + snprintf(isc->mdev.bus_info, sizeof(isc->mdev.bus_info), "platform:%s",
> >> + isc->v4l2_dev.name);
> >> + isc->mdev.hw_revision = ver;
> >> +
> >> + media_device_init(&isc->mdev);
> >> +
> >> + isc->v4l2_dev.mdev = &isc->mdev;
> >> +
> >> + return isc_init_own_sd(isc);
> >> +}
> >> +EXPORT_SYMBOL_GPL(isc_mc_init);
> >> +
> >> +int isc_mc_register(struct isc_device *isc)
> >> +{
> >> + int ret;
> >> +
> >> + ret = media_create_pad_link(&isc->current_subdev->sd->entity,
> >> + isc->remote_pad, &isc->scaler_sd.entity,
> >> + ISC_SCALER_PAD_SINK,
> >> + MEDIA_LNK_FL_ENABLED |
> >> + MEDIA_LNK_FL_IMMUTABLE);
> >> +
> >> + if (ret < 0) {
> >> + v4l2_err(&isc->v4l2_dev,
> >> + "Failed to create pad link: %s to %s\n",
> >> + isc->current_subdev->sd->entity.name,
> >> + isc->scaler_sd.entity.name);
> >> + return ret;
> >> + }
> >> +
> >> + dev_dbg(isc->dev, "link with %s pad: %d\n",
> >> + isc->current_subdev->sd->name, isc->remote_pad);
> >> +
> >> + ret = media_create_pad_link(&isc->scaler_sd.entity,
> >> + ISC_SCALER_PAD_SOURCE,
> >> + &isc->video_dev.entity, ISC_PAD_SINK,
> >> + MEDIA_LNK_FL_ENABLED |
> >> + MEDIA_LNK_FL_IMMUTABLE);
> >> +
> >> + if (ret < 0) {
> >> + v4l2_err(&isc->v4l2_dev,
> >> + "Failed to create pad link: %s to %s\n",
> >> + isc->scaler_sd.entity.name,
> >> + isc->video_dev.entity.name);
> >> + return ret;
> >> + }
> >> +
> >> + dev_dbg(isc->dev, "link with %s pad: %d\n", isc->scaler_sd.name,
> >> + ISC_SCALER_PAD_SOURCE);
> >> +
> >> + return media_device_register(&isc->mdev);
> >
> > I feel like the isc-base driver shold instead register the
> > media-controller and this file shuld actually be only about
> > isc-scaler or something. I might be missing why it is not possible
> > though. It doesn't seem to me that mc support is conditional, so it
> > can happen in the base driver directly ?
>
> This actually is the base driver. Just that the atmel-isc-base.c is
> pretty crowded. So I moved the MC part in this file. But it's the same
> base driver.
>
I understand. isc-base is pretty big, you're right.
However breaking mc apart in a file that mixes registering an entity
for the scaler and initializing mc for the whole driver feels weird,
at least to me, as I don't immediately get why the mc part is handled
by the scaler file...
> >
> > Links creation can still happen at async_complete() (I would have
> > suggested _bound() time, but you have no asyn subdev for the scaler,
> > so I guess the only async subdev here is the sensor).
>
> Yes, the sensor or any kind of subdevice (XISC normally is connected to
> a bridge, ISC to a sensor directly )
>
> >
> > All in all: can this function be removed and:
> > 1) Base driver registers the media controller device in probe
>
> Should we register the media device even if it's not operational ? and
> no video device has been registered ?
> I thought it would be safer to have it registered with all the
> links/entities/etc once the video node has been completed.
>
> But if you say this is the proper way, I can do that.
> I wanted to keep things separated and not overcrowd the base file /probe
> function, but if it's just a media_device_register call, I can do it there.
You are right. Calling media_device_register() at async_complete time
is the right thing to do.
Thanks
j
>
> > 2) links are created at async_complete time ?
>
> That works too. I think I could move them there.
>
> Thanks for reviewing,
>
> Eugen
>
> >
> > Thanks
> > j
> >
> >
> >> +}
> >> +EXPORT_SYMBOL_GPL(isc_mc_register);
> >> +
> >> +void isc_mc_cleanup(struct isc_device *isc)
> >> +{
> >> + media_entity_cleanup(&isc->video_dev.entity);
> >> +}
> >> +EXPORT_SYMBOL_GPL(isc_mc_cleanup);
> >> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
> >> index 07fa6dbf8460..0b6370d7775f 100644
> >> --- a/drivers/media/platform/atmel/atmel-isc.h
> >> +++ b/drivers/media/platform/atmel/atmel-isc.h
> >> @@ -183,6 +183,17 @@ struct isc_reg_offsets {
> >> u32 his_entry;
> >> };
> >>
> >> +enum isc_mc_pads {
> >> + ISC_PAD_SINK = 0,
> >> + ISC_PADS_NUM = 1,
> >> +};
> >> +
> >> +enum isc_scaler_pads {
> >> + ISC_SCALER_PAD_SINK = 0,
> >> + ISC_SCALER_PAD_SOURCE = 1,
> >> + ISC_SCALER_PADS_NUM = 2,
> >> +};
> >> +
> >> /*
> >> * struct isc_device - ISC device driver data/config struct
> >> * @regmap: Register map
> >> @@ -258,6 +269,12 @@ struct isc_reg_offsets {
> >> * be used as an input to the controller
> >> * @controller_formats_size: size of controller_formats array
> >> * @formats_list_size: size of formats_list array
> >> + * @pads: media controller pads for isc video entity
> >> + * @mdev: media device that is registered by the isc
> >> + * @remote_pad: remote pad on the connected subdevice
> >> + * @scaler_sd: subdevice for the scaler that isc registers
> >> + * @scaler_pads: media controller pads for the scaler subdevice
> >> + * @scaler_format: current format for the scaler subdevice
> >> */
> >> struct isc_device {
> >> struct regmap *regmap;
> >> @@ -346,6 +363,19 @@ struct isc_device {
> >> struct isc_format *formats_list;
> >> u32 controller_formats_size;
> >> u32 formats_list_size;
> >> +
> >> + struct {
> >> + struct media_pad pads[ISC_PADS_NUM];
> >> + struct media_device mdev;
> >> +
> >> + u32 remote_pad;
> >> + };
> >> +
> >> + struct {
> >> + struct v4l2_subdev scaler_sd;
> >> + struct media_pad scaler_pads[ISC_SCALER_PADS_NUM];
> >> + struct v4l2_mbus_framefmt scaler_format;
> >> + };
> >> };
> >>
> >> extern const struct regmap_config isc_regmap_config;
> >> @@ -357,4 +387,10 @@ int isc_clk_init(struct isc_device *isc);
> >> void isc_subdev_cleanup(struct isc_device *isc);
> >> void isc_clk_cleanup(struct isc_device *isc);
> >>
> >> +int isc_mc_init(struct isc_device *isc, u32 ver);
> >> +int isc_mc_register(struct isc_device *isc);
> >> +void isc_mc_cleanup(struct isc_device *isc);
> >> +
> >> +struct isc_format *isc_find_format_by_code(struct isc_device *isc,
> >> + unsigned int code, int *index);
> >> #endif
> >> diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> >> index 1b2063cce0f7..a3e4c32a66c4 100644
> >> --- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> >> +++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> >> @@ -529,6 +529,12 @@ static int atmel_isc_probe(struct platform_device *pdev)
> >> break;
> >> }
> >>
> >> + regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
> >> +
> >> + ret = isc_mc_init(isc, ver);
> >> + if (ret < 0)
> >> + goto isc_probe_mc_init_err;
> >> +
> >> pm_runtime_set_active(dev);
> >> pm_runtime_enable(dev);
> >> pm_request_idle(dev);
> >> @@ -538,7 +544,7 @@ static int atmel_isc_probe(struct platform_device *pdev)
> >> ret = clk_prepare_enable(isc->ispck);
> >> if (ret) {
> >> dev_err(dev, "failed to enable ispck: %d\n", ret);
> >> - goto cleanup_subdev;
> >> + goto isc_probe_mc_init_err;
> >> }
> >>
> >> /* ispck should be greater or equal to hclock */
> >> @@ -548,7 +554,6 @@ static int atmel_isc_probe(struct platform_device *pdev)
> >> goto unprepare_clk;
> >> }
> >>
> >> - regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
> >> dev_info(dev, "Microchip ISC version %x\n", ver);
> >>
> >> return 0;
> >> @@ -556,6 +561,9 @@ static int atmel_isc_probe(struct platform_device *pdev)
> >> unprepare_clk:
> >> clk_disable_unprepare(isc->ispck);
> >>
> >> +isc_probe_mc_init_err:
> >> + isc_mc_cleanup(isc);
> >> +
> >> cleanup_subdev:
> >> isc_subdev_cleanup(isc);
> >>
> >> @@ -576,6 +584,8 @@ static int atmel_isc_remove(struct platform_device *pdev)
> >>
> >> pm_runtime_disable(&pdev->dev);
> >>
> >> + isc_mc_cleanup(isc);
> >> +
> >> isc_subdev_cleanup(isc);
> >>
> >> v4l2_device_unregister(&isc->v4l2_dev);
> >> diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> >> index 2b1082295c13..d2ac80b1ccc3 100644
> >> --- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> >> +++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> >> @@ -523,15 +523,23 @@ static int microchip_xisc_probe(struct platform_device *pdev)
> >> break;
> >> }
> >>
> >> + regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
> >> +
> >> + ret = isc_mc_init(isc, ver);
> >> + if (ret < 0)
> >> + goto isc_probe_mc_init_err;
> >> +
> >> pm_runtime_set_active(dev);
> >> pm_runtime_enable(dev);
> >> pm_request_idle(dev);
> >>
> >> - regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
> >> dev_info(dev, "Microchip XISC version %x\n", ver);
> >>
> >> return 0;
> >>
> >> +isc_probe_mc_init_err:
> >> + isc_mc_cleanup(isc);
> >> +
> >> cleanup_subdev:
> >> isc_subdev_cleanup(isc);
> >>
> >> @@ -552,6 +560,8 @@ static int microchip_xisc_remove(struct platform_device *pdev)
> >>
> >> pm_runtime_disable(&pdev->dev);
> >>
> >> + isc_mc_cleanup(isc);
> >> +
> >> isc_subdev_cleanup(isc);
> >>
> >> v4l2_device_unregister(&isc->v4l2_dev);
> >> --
> >> 2.25.1
> >>
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 12/23] media: atmel: atmel-isc: implement media controller
2022-01-11 16:51 ` Jacopo Mondi
@ 2022-01-11 17:00 ` Eugen.Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen.Hristev @ 2022-01-11 17:00 UTC (permalink / raw)
To: jacopo
Cc: linux-media, robh+dt, laurent.pinchart, sakari.ailus, devicetree,
linux-kernel, linux-arm-kernel, Nicolas.Ferre
On 1/11/22 6:51 PM, Jacopo Mondi wrote:
> On Tue, Jan 11, 2022 at 04:30:15PM +0000, Eugen.Hristev@microchip.com wrote:
>> On 1/11/22 6:06 PM, Jacopo Mondi wrote:
>>> Hi Eugen,
>>>
>>> On Mon, Dec 13, 2021 at 03:49:29PM +0200, Eugen Hristev wrote:
>>>> Implement the support for media-controller.
>>>> This means that the capabilities of the driver have changed and now
>>>> it also advertises the IO_MC .
>>>> The driver will register it's media device, and add the video entity to this
>>>> media device. The subdevices are registered to the same media device.
>>>> The ISC will have a base entity which is auto-detected as atmel_isc_base.
>>>> It will also register a subdevice that allows cropping of the incoming frame
>>>> to the maximum frame size supported by the ISC.
>>>> The ISC will create a link between the subdevice that is asynchronously
>>>> registered and the atmel_isc_scaler entity.
>>>> Then, the atmel_isc_scaler and atmel_isc_base are connected through another
>>>> link.
>>>>
>>>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>>>> ---
>>>> Changes in v2:
>>>> - implement try formats
>>>>
>>>> drivers/media/platform/atmel/Makefile | 2 +-
>>>> drivers/media/platform/atmel/atmel-isc-base.c | 27 +-
>>>> drivers/media/platform/atmel/atmel-isc-mc.c | 287 ++++++++++++++++++
>>>> drivers/media/platform/atmel/atmel-isc.h | 36 +++
>>>> .../media/platform/atmel/atmel-sama5d2-isc.c | 14 +-
>>>> .../media/platform/atmel/atmel-sama7g5-isc.c | 12 +-
>>>> 6 files changed, 370 insertions(+), 8 deletions(-)
>>>> create mode 100644 drivers/media/platform/atmel/atmel-isc-mc.c
>>>>
>>>> diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
>>>> index 794e8f739287..aa7a18b428f6 100644
>>>> --- a/drivers/media/platform/atmel/Makefile
>>>> +++ b/drivers/media/platform/atmel/Makefile
>>>> @@ -1,7 +1,7 @@
>>>> # SPDX-License-Identifier: GPL-2.0-only
>>>> atmel-isc-objs = atmel-sama5d2-isc.o
>>>> atmel-xisc-objs = atmel-sama7g5-isc.o
>>>> -atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o
>>>> +atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o atmel-isc-mc.o
>>>>
>>>> obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
>>>> obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-common.o
>>>> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
>>>> index d0542b97a391..b0c3ed21f372 100644
>>>> --- a/drivers/media/platform/atmel/atmel-isc-base.c
>>>> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
>>>> @@ -1726,6 +1726,7 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
>>>> struct isc_device, v4l2_dev);
>>>> struct isc_subdev_entity *subdev_entity =
>>>> container_of(notifier, struct isc_subdev_entity, notifier);
>>>> + int pad;
>>>>
>>>> if (video_is_registered(&isc->video_dev)) {
>>>> v4l2_err(&isc->v4l2_dev, "only supports one sub-device.\n");
>>>> @@ -1734,6 +1735,16 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
>>>>
>>>> subdev_entity->sd = subdev;
>>>>
>>>> + pad = media_entity_get_fwnode_pad(&subdev->entity, asd->match.fwnode,
>>>> + MEDIA_PAD_FL_SOURCE);
>>>> + if (pad < 0) {
>>>> + v4l2_err(&isc->v4l2_dev, "failed to find pad for %s\n",
>>>> + subdev->name);
>>>> + return pad;
>>>> + }
>>>> +
>>>> + isc->remote_pad = pad;
>>>> +
>>>> return 0;
>>>> }
>>>>
>>>> @@ -1748,8 +1759,8 @@ static void isc_async_unbind(struct v4l2_async_notifier *notifier,
>>>> v4l2_ctrl_handler_free(&isc->ctrls.handler);
>>>> }
>>>>
>>>> -static struct isc_format *find_format_by_code(struct isc_device *isc,
>>>> - unsigned int code, int *index)
>>>> +struct isc_format *isc_find_format_by_code(struct isc_device *isc,
>>>> + unsigned int code, int *index)
>>>> {
>>>> struct isc_format *fmt = &isc->formats_list[0];
>>>> unsigned int i;
>>>> @@ -1765,6 +1776,7 @@ static struct isc_format *find_format_by_code(struct isc_device *isc,
>>>>
>>>> return NULL;
>>>> }
>>>> +EXPORT_SYMBOL_GPL(isc_find_format_by_code);
>>>>
>>>> static int isc_formats_init(struct isc_device *isc)
>>>> {
>>>> @@ -1781,7 +1793,7 @@ static int isc_formats_init(struct isc_device *isc)
>>>> NULL, &mbus_code)) {
>>>> mbus_code.index++;
>>>>
>>>> - fmt = find_format_by_code(isc, mbus_code.code, &i);
>>>> + fmt = isc_find_format_by_code(isc, mbus_code.code, &i);
>>>> if (!fmt) {
>>>> v4l2_warn(&isc->v4l2_dev, "Mbus code %x not supported\n",
>>>> mbus_code.code);
>>>> @@ -1909,7 +1921,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
>>>> vdev->queue = q;
>>>> vdev->lock = &isc->lock;
>>>> vdev->ctrl_handler = &isc->ctrls.handler;
>>>> - vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE;
>>>> + vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE |
>>>> + V4L2_CAP_IO_MC;
>>>> video_set_drvdata(vdev, isc);
>>>>
>>>> ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
>>>> @@ -1919,8 +1932,14 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
>>>> goto isc_async_complete_err;
>>>> }
>>>>
>>>> + ret = isc_mc_register(isc);
>>>> + if (ret < 0)
>>>> + goto isc_async_complete_unregister_device;
>>>> return 0;
>>>>
>>>> +isc_async_complete_unregister_device:
>>>> + video_unregister_device(vdev);
>>>> +
>>>> isc_async_complete_err:
>>>> mutex_destroy(&isc->lock);
>>>> return ret;
>>>> diff --git a/drivers/media/platform/atmel/atmel-isc-mc.c b/drivers/media/platform/atmel/atmel-isc-mc.c
>>>> new file mode 100644
>>>> index 000000000000..2aea81d77b2f
>>>> --- /dev/null
>>>> +++ b/drivers/media/platform/atmel/atmel-isc-mc.c
>>>> @@ -0,0 +1,287 @@
>>>> +// SPDX-License-Identifier: GPL-2.0-only
>>>> +/*
>>>> + * Microchip Image Sensor Controller (ISC) Media Controller support
>>>> + *
>>>> + * Copyright (C) 2021 Microchip Technology, Inc.
>>>> + *
>>>> + * Author: Eugen Hristev <eugen.hristev@microchip.com>
>>>> + *
>>>> + */
>>>> +
>>>> +#include <media/v4l2-device.h>
>>>> +#include <media/v4l2-subdev.h>
>>>
>>> Should you include <media/media-entity.h> and <media/media-device.h> ?
>>
>> Hi Jacopo,
>>
>> They are probably being included from another header. I will add them.
>>
>
> Thank you :)
>
>>>
>>> Also, should you select MEDIA_CONTROLLER and V4L2_SUBDEV_API for all
>>> the Kconfig symbols ?
>>
>>
>> VIDEO_ATMEL_ISC directly selects:
>> select MEDIA_CONTROLLER
>> select VIDEO_V4L2_SUBDEV_API
>>
>> and VIDEO_ATMEL_XISC
>> depends on VIDEO_V4L2 && COMMON_CLK && VIDEO_V4L2_SUBDEV_API
>
> And selects VIDEO_ATMEL_ISC_BASE so it's fine!
>
> VIDEO_ATMEL_ISI does not. I'm fine if this is intended, I might have
> got lost in the several pieces this driver is composed of :)
The ISI is a totally different driver with no code shared with the ISC.
It's a different hardware, on some older platforms.
It does not support media controller and will not in the near future.
Initially there was ISI, then ISC appeared and a new driver was created.
Then a newer version of the ISC appeared named XISC, but a lot was
common with the old ISC, so I created a new driver XISC with shared
common base with the old ISC.
So now we have ISI, and ISC/XISC who share ATMEL_ISC_BASE
>
>>
>> and VIDEO_V4L2_SUBDEV_API depends on MEDIA_CONTROLLER
>>
>>>
>>>> +
>>>> +#include "atmel-isc-regs.h"
>>>> +#include "atmel-isc.h"
>>>> +
>>>> +static const struct media_device_ops isc_media_ops = {
>>>> +};
>>>
>>> Do you need this even if it's empty ?
>>>
>>
>> Not sure. I can remove it and pass NULL, but I haven't checked what the
>> subsystem is doing if there is no ops. I will try.
>>
>
> Seems like it is safe as the core checks for if (mdev->ops) before
> accessing it (except for mc-request for which operations are
> mandatory)
>
>>>> +
>>>> +static int isc_scaler_get_fmt(struct v4l2_subdev *sd,
>>>> + struct v4l2_subdev_state *sd_state,
>>>> + struct v4l2_subdev_format *format)
>>>> +{
>>>> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
>>>> + struct v4l2_mbus_framefmt *v4l2_try_fmt;
>>>> +
>>>> + if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
>>>> + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state,
>>>> + format->pad);
>>>> + format->format = *v4l2_try_fmt;
>>>> +
>>>> + return 0;
>>>> + }
>>>> +
>>>> + format->format = isc->scaler_format;
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> +static int isc_scaler_set_fmt(struct v4l2_subdev *sd,
>>>> + struct v4l2_subdev_state *sd_state,
>>>> + struct v4l2_subdev_format *req_fmt)
>>>> +{
>>>> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
>>>> + struct v4l2_mbus_framefmt *v4l2_try_fmt;
>>>> + struct isc_format *fmt;
>>>> + unsigned int i;
>>>> +
>>>> + if (req_fmt->pad == ISC_SCALER_PAD_SOURCE)
>>>> + v4l_bound_align_image
>>>> + (&req_fmt->format.width, 16, isc->max_width, 0,
>>>> + &req_fmt->format.height, 16, isc->max_height, 0, 0);
>>>> + else
>>>> + v4l_bound_align_image
>>>> + (&req_fmt->format.width, 16, 10000, 0,
>>>> + &req_fmt->format.height, 16, 10000, 0, 0);
>>>> +
>>>> + req_fmt->format.colorspace = V4L2_COLORSPACE_SRGB;
>>>> + req_fmt->format.field = V4L2_FIELD_NONE;
>>>> + req_fmt->format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
>>>> + req_fmt->format.quantization = V4L2_QUANTIZATION_DEFAULT;
>>>> + req_fmt->format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
>>>> +
>>>> + fmt = isc_find_format_by_code(isc, req_fmt->format.code, &i);
>>>> +
>>>> + if (!fmt)
>>>> + fmt = &isc->formats_list[0];
>>>> +
>>>> + req_fmt->format.code = fmt->mbus_code;
>>>> +
>>>> + if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
>>>> + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state,
>>>> + req_fmt->pad);
>>>> + *v4l2_try_fmt = req_fmt->format;
>>>> + /* Trying on the pad sink makes the source sink change too */
>>>> + if (req_fmt->pad == ISC_SCALER_PAD_SINK) {
>>>> + v4l2_try_fmt =
>>>> + v4l2_subdev_get_try_format(sd, sd_state,
>>>> + ISC_SCALER_PAD_SOURCE);
>>>> + *v4l2_try_fmt = req_fmt->format;
>>>> +
>>>> + v4l_bound_align_image(&v4l2_try_fmt->width,
>>>> + 16, isc->max_width, 0,
>>>> + &v4l2_try_fmt->height,
>>>> + 16, isc->max_height, 0, 0);
>>>> + }
>>>> + /* if we are just trying, we are done */
>>>> + return 0;
>>>> + }
>>>> +
>>>> + isc->scaler_format = req_fmt->format;
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> +static int isc_scaler_enum_mbus_code(struct v4l2_subdev *sd,
>>>> + struct v4l2_subdev_state *sd_state,
>>>> + struct v4l2_subdev_mbus_code_enum *code)
>>>> +{
>>>> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
>>>> + int supported_index = 0;
>>>> + int i;
>>>> +
>>>> + for (i = 0; i < isc->formats_list_size; i++) {
>>>> + if (!isc->formats_list[i].sd_support)
>>>> + continue;
>>>> + if (supported_index == code->index) {
>>>> + code->code = isc->formats_list[i].mbus_code;
>>>> + return 0;
>>>> + }
>>>> + supported_index++;
>>>> + }
>>>> +
>>>> + return -EINVAL;
>>>> +}
>>>> +
>>>> +static int isc_scaler_g_sel(struct v4l2_subdev *sd,
>>>> + struct v4l2_subdev_state *sd_state,
>>>> + struct v4l2_subdev_selection *sel)
>>>> +{
>>>> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
>>>> +
>>>> + if (sel->pad == ISC_SCALER_PAD_SOURCE)
>>>> + return -EINVAL;
>>>> +
>>>> + if (sel->target != V4L2_SEL_TGT_CROP_BOUNDS &&
>>>> + sel->target != V4L2_SEL_TGT_CROP)
>>>> + return -EINVAL;
>>>> +
>>>> + sel->r.height = isc->max_height;
>>>> + sel->r.width = isc->max_width;
>>>> +
>>>> + sel->r.left = 0;
>>>> + sel->r.top = 0;
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> +static int isc_scaler_init_cfg(struct v4l2_subdev *sd,
>>>> + struct v4l2_subdev_state *sd_state)
>>>> +{
>>>> + struct v4l2_mbus_framefmt *v4l2_try_fmt =
>>>> + v4l2_subdev_get_try_format(sd, sd_state, 0);
>>>> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
>>>> +
>>>> + *v4l2_try_fmt = isc->scaler_format;
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> +static const struct v4l2_subdev_pad_ops isc_scaler_pad_ops = {
>>>> + .enum_mbus_code = isc_scaler_enum_mbus_code,
>>>> + .set_fmt = isc_scaler_set_fmt,
>>>> + .get_fmt = isc_scaler_get_fmt,
>>>> + .get_selection = isc_scaler_g_sel,
>>>> + .init_cfg = isc_scaler_init_cfg,
>>>> +};
>>>> +
>>>> +static const struct v4l2_subdev_ops xisc_scaler_subdev_ops = {
>>>> + .pad = &isc_scaler_pad_ops,
>>>> +};
>>>> +
>>>> +static int isc_init_own_sd(struct isc_device *isc)
>>>> +{
>>>> + int ret;
>>>> +
>>>> + v4l2_subdev_init(&isc->scaler_sd, &xisc_scaler_subdev_ops);
>>>> +
>>>> + isc->scaler_sd.owner = THIS_MODULE;
>>>> + isc->scaler_sd.dev = isc->dev;
>>>> + snprintf(isc->scaler_sd.name, sizeof(isc->scaler_sd.name),
>>>> + "atmel_isc_scaler");
>>>> +
>>>> + isc->scaler_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
>>>> + isc->scaler_sd.entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
>>>> + isc->scaler_pads[ISC_SCALER_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>>>> + isc->scaler_pads[ISC_SCALER_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
>>>> +
>>>> + isc->scaler_format.height = isc->max_height;
>>>> + isc->scaler_format.width = isc->max_width;
>>>> + isc->scaler_format.code = isc->formats_list[0].mbus_code;
>>>> + isc->scaler_format.colorspace = V4L2_COLORSPACE_SRGB;
>>>> + isc->scaler_format.field = V4L2_FIELD_NONE;
>>>> + isc->scaler_format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
>>>> + isc->scaler_format.quantization = V4L2_QUANTIZATION_DEFAULT;
>>>> + isc->scaler_format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
>>>> +
>>>> + ret = media_entity_pads_init(&isc->scaler_sd.entity,
>>>> + ISC_SCALER_PADS_NUM,
>>>> + isc->scaler_pads);
>>>> + if (ret < 0) {
>>>> + dev_err(isc->dev, "scaler sd media entity init failed\n");
>>>> + return ret;
>>>> + }
>>>> + ret = v4l2_device_register_subdev(&isc->v4l2_dev, &isc->scaler_sd);
>>>> + if (ret < 0) {
>>>> + dev_err(isc->dev, "scaler sd failed to register subdev\n");
>>>> + return ret;
>>>> + }
>>>> +
>>>> + return ret;
>>>> +}
>>>> +
>>>> +int isc_mc_init(struct isc_device *isc, u32 ver)
>>>> +{
>>>> + const struct of_device_id *match;
>>>> + int ret;
>>>> +
>>>> + isc->video_dev.entity.function = MEDIA_ENT_F_IO_V4L;
>>>> + isc->video_dev.entity.flags = MEDIA_ENT_FL_DEFAULT;
>>>> + isc->pads[ISC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>>>> +
>>>> + ret = media_entity_pads_init(&isc->video_dev.entity, ISC_PADS_NUM,
>>>> + isc->pads);
>>>> + if (ret < 0) {
>>>> + dev_err(isc->dev, "media entity init failed\n");
>>>> + return ret;
>>>> + }
>>>> +
>>>> + isc->mdev.dev = isc->dev;
>>>> + isc->mdev.ops = &isc_media_ops;
>>>> +
>>>> + match = of_match_node(isc->dev->driver->of_match_table,
>>>> + isc->dev->of_node);
>>>> +
>>>> + strscpy(isc->mdev.driver_name, KBUILD_MODNAME,
>>>> + sizeof(isc->mdev.driver_name));
>>>> + strscpy(isc->mdev.model, match->compatible, sizeof(isc->mdev.model));
>>>> + snprintf(isc->mdev.bus_info, sizeof(isc->mdev.bus_info), "platform:%s",
>>>> + isc->v4l2_dev.name);
>>>> + isc->mdev.hw_revision = ver;
>>>> +
>>>> + media_device_init(&isc->mdev);
>>>> +
>>>> + isc->v4l2_dev.mdev = &isc->mdev;
>>>> +
>>>> + return isc_init_own_sd(isc);
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(isc_mc_init);
>>>> +
>>>> +int isc_mc_register(struct isc_device *isc)
>>>> +{
>>>> + int ret;
>>>> +
>>>> + ret = media_create_pad_link(&isc->current_subdev->sd->entity,
>>>> + isc->remote_pad, &isc->scaler_sd.entity,
>>>> + ISC_SCALER_PAD_SINK,
>>>> + MEDIA_LNK_FL_ENABLED |
>>>> + MEDIA_LNK_FL_IMMUTABLE);
>>>> +
>>>> + if (ret < 0) {
>>>> + v4l2_err(&isc->v4l2_dev,
>>>> + "Failed to create pad link: %s to %s\n",
>>>> + isc->current_subdev->sd->entity.name,
>>>> + isc->scaler_sd.entity.name);
>>>> + return ret;
>>>> + }
>>>> +
>>>> + dev_dbg(isc->dev, "link with %s pad: %d\n",
>>>> + isc->current_subdev->sd->name, isc->remote_pad);
>>>> +
>>>> + ret = media_create_pad_link(&isc->scaler_sd.entity,
>>>> + ISC_SCALER_PAD_SOURCE,
>>>> + &isc->video_dev.entity, ISC_PAD_SINK,
>>>> + MEDIA_LNK_FL_ENABLED |
>>>> + MEDIA_LNK_FL_IMMUTABLE);
>>>> +
>>>> + if (ret < 0) {
>>>> + v4l2_err(&isc->v4l2_dev,
>>>> + "Failed to create pad link: %s to %s\n",
>>>> + isc->scaler_sd.entity.name,
>>>> + isc->video_dev.entity.name);
>>>> + return ret;
>>>> + }
>>>> +
>>>> + dev_dbg(isc->dev, "link with %s pad: %d\n", isc->scaler_sd.name,
>>>> + ISC_SCALER_PAD_SOURCE);
>>>> +
>>>> + return media_device_register(&isc->mdev);
>>>
>>> I feel like the isc-base driver shold instead register the
>>> media-controller and this file shuld actually be only about
>>> isc-scaler or something. I might be missing why it is not possible
>>> though. It doesn't seem to me that mc support is conditional, so it
>>> can happen in the base driver directly ?
>>
>> This actually is the base driver. Just that the atmel-isc-base.c is
>> pretty crowded. So I moved the MC part in this file. But it's the same
>> base driver.
>>
>
> I understand. isc-base is pretty big, you're right.
> However breaking mc apart in a file that mixes registering an entity
> for the scaler and initializing mc for the whole driver feels weird,
> at least to me, as I don't immediately get why the mc part is handled
> by the scaler file...
>
>>>
>>> Links creation can still happen at async_complete() (I would have
>>> suggested _bound() time, but you have no asyn subdev for the scaler,
>>> so I guess the only async subdev here is the sensor).
>>
>> Yes, the sensor or any kind of subdevice (XISC normally is connected to
>> a bridge, ISC to a sensor directly )
>>
>>>
>>> All in all: can this function be removed and:
>>> 1) Base driver registers the media controller device in probe
>>
>> Should we register the media device even if it's not operational ? and
>> no video device has been registered ?
>> I thought it would be safer to have it registered with all the
>> links/entities/etc once the video node has been completed.
>>
>> But if you say this is the proper way, I can do that.
>> I wanted to keep things separated and not overcrowd the base file /probe
>> function, but if it's just a media_device_register call, I can do it there.
>
> You are right. Calling media_device_register() at async_complete time
> is the right thing to do.
>
> Thanks
> j
>
>
>>
>>> 2) links are created at async_complete time ?
>>
>> That works too. I think I could move them there.
>>
>> Thanks for reviewing,
>>
>> Eugen
>>
>>>
>>> Thanks
>>> j
>>>
>>>
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(isc_mc_register);
>>>> +
>>>> +void isc_mc_cleanup(struct isc_device *isc)
>>>> +{
>>>> + media_entity_cleanup(&isc->video_dev.entity);
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(isc_mc_cleanup);
>>>> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
>>>> index 07fa6dbf8460..0b6370d7775f 100644
>>>> --- a/drivers/media/platform/atmel/atmel-isc.h
>>>> +++ b/drivers/media/platform/atmel/atmel-isc.h
>>>> @@ -183,6 +183,17 @@ struct isc_reg_offsets {
>>>> u32 his_entry;
>>>> };
>>>>
>>>> +enum isc_mc_pads {
>>>> + ISC_PAD_SINK = 0,
>>>> + ISC_PADS_NUM = 1,
>>>> +};
>>>> +
>>>> +enum isc_scaler_pads {
>>>> + ISC_SCALER_PAD_SINK = 0,
>>>> + ISC_SCALER_PAD_SOURCE = 1,
>>>> + ISC_SCALER_PADS_NUM = 2,
>>>> +};
>>>> +
>>>> /*
>>>> * struct isc_device - ISC device driver data/config struct
>>>> * @regmap: Register map
>>>> @@ -258,6 +269,12 @@ struct isc_reg_offsets {
>>>> * be used as an input to the controller
>>>> * @controller_formats_size: size of controller_formats array
>>>> * @formats_list_size: size of formats_list array
>>>> + * @pads: media controller pads for isc video entity
>>>> + * @mdev: media device that is registered by the isc
>>>> + * @remote_pad: remote pad on the connected subdevice
>>>> + * @scaler_sd: subdevice for the scaler that isc registers
>>>> + * @scaler_pads: media controller pads for the scaler subdevice
>>>> + * @scaler_format: current format for the scaler subdevice
>>>> */
>>>> struct isc_device {
>>>> struct regmap *regmap;
>>>> @@ -346,6 +363,19 @@ struct isc_device {
>>>> struct isc_format *formats_list;
>>>> u32 controller_formats_size;
>>>> u32 formats_list_size;
>>>> +
>>>> + struct {
>>>> + struct media_pad pads[ISC_PADS_NUM];
>>>> + struct media_device mdev;
>>>> +
>>>> + u32 remote_pad;
>>>> + };
>>>> +
>>>> + struct {
>>>> + struct v4l2_subdev scaler_sd;
>>>> + struct media_pad scaler_pads[ISC_SCALER_PADS_NUM];
>>>> + struct v4l2_mbus_framefmt scaler_format;
>>>> + };
>>>> };
>>>>
>>>> extern const struct regmap_config isc_regmap_config;
>>>> @@ -357,4 +387,10 @@ int isc_clk_init(struct isc_device *isc);
>>>> void isc_subdev_cleanup(struct isc_device *isc);
>>>> void isc_clk_cleanup(struct isc_device *isc);
>>>>
>>>> +int isc_mc_init(struct isc_device *isc, u32 ver);
>>>> +int isc_mc_register(struct isc_device *isc);
>>>> +void isc_mc_cleanup(struct isc_device *isc);
>>>> +
>>>> +struct isc_format *isc_find_format_by_code(struct isc_device *isc,
>>>> + unsigned int code, int *index);
>>>> #endif
>>>> diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
>>>> index 1b2063cce0f7..a3e4c32a66c4 100644
>>>> --- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
>>>> +++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
>>>> @@ -529,6 +529,12 @@ static int atmel_isc_probe(struct platform_device *pdev)
>>>> break;
>>>> }
>>>>
>>>> + regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
>>>> +
>>>> + ret = isc_mc_init(isc, ver);
>>>> + if (ret < 0)
>>>> + goto isc_probe_mc_init_err;
>>>> +
>>>> pm_runtime_set_active(dev);
>>>> pm_runtime_enable(dev);
>>>> pm_request_idle(dev);
>>>> @@ -538,7 +544,7 @@ static int atmel_isc_probe(struct platform_device *pdev)
>>>> ret = clk_prepare_enable(isc->ispck);
>>>> if (ret) {
>>>> dev_err(dev, "failed to enable ispck: %d\n", ret);
>>>> - goto cleanup_subdev;
>>>> + goto isc_probe_mc_init_err;
>>>> }
>>>>
>>>> /* ispck should be greater or equal to hclock */
>>>> @@ -548,7 +554,6 @@ static int atmel_isc_probe(struct platform_device *pdev)
>>>> goto unprepare_clk;
>>>> }
>>>>
>>>> - regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
>>>> dev_info(dev, "Microchip ISC version %x\n", ver);
>>>>
>>>> return 0;
>>>> @@ -556,6 +561,9 @@ static int atmel_isc_probe(struct platform_device *pdev)
>>>> unprepare_clk:
>>>> clk_disable_unprepare(isc->ispck);
>>>>
>>>> +isc_probe_mc_init_err:
>>>> + isc_mc_cleanup(isc);
>>>> +
>>>> cleanup_subdev:
>>>> isc_subdev_cleanup(isc);
>>>>
>>>> @@ -576,6 +584,8 @@ static int atmel_isc_remove(struct platform_device *pdev)
>>>>
>>>> pm_runtime_disable(&pdev->dev);
>>>>
>>>> + isc_mc_cleanup(isc);
>>>> +
>>>> isc_subdev_cleanup(isc);
>>>>
>>>> v4l2_device_unregister(&isc->v4l2_dev);
>>>> diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
>>>> index 2b1082295c13..d2ac80b1ccc3 100644
>>>> --- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
>>>> +++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
>>>> @@ -523,15 +523,23 @@ static int microchip_xisc_probe(struct platform_device *pdev)
>>>> break;
>>>> }
>>>>
>>>> + regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
>>>> +
>>>> + ret = isc_mc_init(isc, ver);
>>>> + if (ret < 0)
>>>> + goto isc_probe_mc_init_err;
>>>> +
>>>> pm_runtime_set_active(dev);
>>>> pm_runtime_enable(dev);
>>>> pm_request_idle(dev);
>>>>
>>>> - regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
>>>> dev_info(dev, "Microchip XISC version %x\n", ver);
>>>>
>>>> return 0;
>>>>
>>>> +isc_probe_mc_init_err:
>>>> + isc_mc_cleanup(isc);
>>>> +
>>>> cleanup_subdev:
>>>> isc_subdev_cleanup(isc);
>>>>
>>>> @@ -552,6 +560,8 @@ static int microchip_xisc_remove(struct platform_device *pdev)
>>>>
>>>> pm_runtime_disable(&pdev->dev);
>>>>
>>>> + isc_mc_cleanup(isc);
>>>> +
>>>> isc_subdev_cleanup(isc);
>>>>
>>>> v4l2_device_unregister(&isc->v4l2_dev);
>>>> --
>>>> 2.25.1
>>>>
>>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 12/23] media: atmel: atmel-isc: implement media controller
@ 2022-01-11 17:00 ` Eugen.Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen.Hristev @ 2022-01-11 17:00 UTC (permalink / raw)
To: jacopo
Cc: devicetree, linux-kernel, robh+dt, sakari.ailus,
laurent.pinchart, linux-arm-kernel, linux-media
On 1/11/22 6:51 PM, Jacopo Mondi wrote:
> On Tue, Jan 11, 2022 at 04:30:15PM +0000, Eugen.Hristev@microchip.com wrote:
>> On 1/11/22 6:06 PM, Jacopo Mondi wrote:
>>> Hi Eugen,
>>>
>>> On Mon, Dec 13, 2021 at 03:49:29PM +0200, Eugen Hristev wrote:
>>>> Implement the support for media-controller.
>>>> This means that the capabilities of the driver have changed and now
>>>> it also advertises the IO_MC .
>>>> The driver will register it's media device, and add the video entity to this
>>>> media device. The subdevices are registered to the same media device.
>>>> The ISC will have a base entity which is auto-detected as atmel_isc_base.
>>>> It will also register a subdevice that allows cropping of the incoming frame
>>>> to the maximum frame size supported by the ISC.
>>>> The ISC will create a link between the subdevice that is asynchronously
>>>> registered and the atmel_isc_scaler entity.
>>>> Then, the atmel_isc_scaler and atmel_isc_base are connected through another
>>>> link.
>>>>
>>>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>>>> ---
>>>> Changes in v2:
>>>> - implement try formats
>>>>
>>>> drivers/media/platform/atmel/Makefile | 2 +-
>>>> drivers/media/platform/atmel/atmel-isc-base.c | 27 +-
>>>> drivers/media/platform/atmel/atmel-isc-mc.c | 287 ++++++++++++++++++
>>>> drivers/media/platform/atmel/atmel-isc.h | 36 +++
>>>> .../media/platform/atmel/atmel-sama5d2-isc.c | 14 +-
>>>> .../media/platform/atmel/atmel-sama7g5-isc.c | 12 +-
>>>> 6 files changed, 370 insertions(+), 8 deletions(-)
>>>> create mode 100644 drivers/media/platform/atmel/atmel-isc-mc.c
>>>>
>>>> diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile
>>>> index 794e8f739287..aa7a18b428f6 100644
>>>> --- a/drivers/media/platform/atmel/Makefile
>>>> +++ b/drivers/media/platform/atmel/Makefile
>>>> @@ -1,7 +1,7 @@
>>>> # SPDX-License-Identifier: GPL-2.0-only
>>>> atmel-isc-objs = atmel-sama5d2-isc.o
>>>> atmel-xisc-objs = atmel-sama7g5-isc.o
>>>> -atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o
>>>> +atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o atmel-isc-mc.o
>>>>
>>>> obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o
>>>> obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-common.o
>>>> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
>>>> index d0542b97a391..b0c3ed21f372 100644
>>>> --- a/drivers/media/platform/atmel/atmel-isc-base.c
>>>> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
>>>> @@ -1726,6 +1726,7 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
>>>> struct isc_device, v4l2_dev);
>>>> struct isc_subdev_entity *subdev_entity =
>>>> container_of(notifier, struct isc_subdev_entity, notifier);
>>>> + int pad;
>>>>
>>>> if (video_is_registered(&isc->video_dev)) {
>>>> v4l2_err(&isc->v4l2_dev, "only supports one sub-device.\n");
>>>> @@ -1734,6 +1735,16 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier,
>>>>
>>>> subdev_entity->sd = subdev;
>>>>
>>>> + pad = media_entity_get_fwnode_pad(&subdev->entity, asd->match.fwnode,
>>>> + MEDIA_PAD_FL_SOURCE);
>>>> + if (pad < 0) {
>>>> + v4l2_err(&isc->v4l2_dev, "failed to find pad for %s\n",
>>>> + subdev->name);
>>>> + return pad;
>>>> + }
>>>> +
>>>> + isc->remote_pad = pad;
>>>> +
>>>> return 0;
>>>> }
>>>>
>>>> @@ -1748,8 +1759,8 @@ static void isc_async_unbind(struct v4l2_async_notifier *notifier,
>>>> v4l2_ctrl_handler_free(&isc->ctrls.handler);
>>>> }
>>>>
>>>> -static struct isc_format *find_format_by_code(struct isc_device *isc,
>>>> - unsigned int code, int *index)
>>>> +struct isc_format *isc_find_format_by_code(struct isc_device *isc,
>>>> + unsigned int code, int *index)
>>>> {
>>>> struct isc_format *fmt = &isc->formats_list[0];
>>>> unsigned int i;
>>>> @@ -1765,6 +1776,7 @@ static struct isc_format *find_format_by_code(struct isc_device *isc,
>>>>
>>>> return NULL;
>>>> }
>>>> +EXPORT_SYMBOL_GPL(isc_find_format_by_code);
>>>>
>>>> static int isc_formats_init(struct isc_device *isc)
>>>> {
>>>> @@ -1781,7 +1793,7 @@ static int isc_formats_init(struct isc_device *isc)
>>>> NULL, &mbus_code)) {
>>>> mbus_code.index++;
>>>>
>>>> - fmt = find_format_by_code(isc, mbus_code.code, &i);
>>>> + fmt = isc_find_format_by_code(isc, mbus_code.code, &i);
>>>> if (!fmt) {
>>>> v4l2_warn(&isc->v4l2_dev, "Mbus code %x not supported\n",
>>>> mbus_code.code);
>>>> @@ -1909,7 +1921,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
>>>> vdev->queue = q;
>>>> vdev->lock = &isc->lock;
>>>> vdev->ctrl_handler = &isc->ctrls.handler;
>>>> - vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE;
>>>> + vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE |
>>>> + V4L2_CAP_IO_MC;
>>>> video_set_drvdata(vdev, isc);
>>>>
>>>> ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
>>>> @@ -1919,8 +1932,14 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
>>>> goto isc_async_complete_err;
>>>> }
>>>>
>>>> + ret = isc_mc_register(isc);
>>>> + if (ret < 0)
>>>> + goto isc_async_complete_unregister_device;
>>>> return 0;
>>>>
>>>> +isc_async_complete_unregister_device:
>>>> + video_unregister_device(vdev);
>>>> +
>>>> isc_async_complete_err:
>>>> mutex_destroy(&isc->lock);
>>>> return ret;
>>>> diff --git a/drivers/media/platform/atmel/atmel-isc-mc.c b/drivers/media/platform/atmel/atmel-isc-mc.c
>>>> new file mode 100644
>>>> index 000000000000..2aea81d77b2f
>>>> --- /dev/null
>>>> +++ b/drivers/media/platform/atmel/atmel-isc-mc.c
>>>> @@ -0,0 +1,287 @@
>>>> +// SPDX-License-Identifier: GPL-2.0-only
>>>> +/*
>>>> + * Microchip Image Sensor Controller (ISC) Media Controller support
>>>> + *
>>>> + * Copyright (C) 2021 Microchip Technology, Inc.
>>>> + *
>>>> + * Author: Eugen Hristev <eugen.hristev@microchip.com>
>>>> + *
>>>> + */
>>>> +
>>>> +#include <media/v4l2-device.h>
>>>> +#include <media/v4l2-subdev.h>
>>>
>>> Should you include <media/media-entity.h> and <media/media-device.h> ?
>>
>> Hi Jacopo,
>>
>> They are probably being included from another header. I will add them.
>>
>
> Thank you :)
>
>>>
>>> Also, should you select MEDIA_CONTROLLER and V4L2_SUBDEV_API for all
>>> the Kconfig symbols ?
>>
>>
>> VIDEO_ATMEL_ISC directly selects:
>> select MEDIA_CONTROLLER
>> select VIDEO_V4L2_SUBDEV_API
>>
>> and VIDEO_ATMEL_XISC
>> depends on VIDEO_V4L2 && COMMON_CLK && VIDEO_V4L2_SUBDEV_API
>
> And selects VIDEO_ATMEL_ISC_BASE so it's fine!
>
> VIDEO_ATMEL_ISI does not. I'm fine if this is intended, I might have
> got lost in the several pieces this driver is composed of :)
The ISI is a totally different driver with no code shared with the ISC.
It's a different hardware, on some older platforms.
It does not support media controller and will not in the near future.
Initially there was ISI, then ISC appeared and a new driver was created.
Then a newer version of the ISC appeared named XISC, but a lot was
common with the old ISC, so I created a new driver XISC with shared
common base with the old ISC.
So now we have ISI, and ISC/XISC who share ATMEL_ISC_BASE
>
>>
>> and VIDEO_V4L2_SUBDEV_API depends on MEDIA_CONTROLLER
>>
>>>
>>>> +
>>>> +#include "atmel-isc-regs.h"
>>>> +#include "atmel-isc.h"
>>>> +
>>>> +static const struct media_device_ops isc_media_ops = {
>>>> +};
>>>
>>> Do you need this even if it's empty ?
>>>
>>
>> Not sure. I can remove it and pass NULL, but I haven't checked what the
>> subsystem is doing if there is no ops. I will try.
>>
>
> Seems like it is safe as the core checks for if (mdev->ops) before
> accessing it (except for mc-request for which operations are
> mandatory)
>
>>>> +
>>>> +static int isc_scaler_get_fmt(struct v4l2_subdev *sd,
>>>> + struct v4l2_subdev_state *sd_state,
>>>> + struct v4l2_subdev_format *format)
>>>> +{
>>>> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
>>>> + struct v4l2_mbus_framefmt *v4l2_try_fmt;
>>>> +
>>>> + if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
>>>> + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state,
>>>> + format->pad);
>>>> + format->format = *v4l2_try_fmt;
>>>> +
>>>> + return 0;
>>>> + }
>>>> +
>>>> + format->format = isc->scaler_format;
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> +static int isc_scaler_set_fmt(struct v4l2_subdev *sd,
>>>> + struct v4l2_subdev_state *sd_state,
>>>> + struct v4l2_subdev_format *req_fmt)
>>>> +{
>>>> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
>>>> + struct v4l2_mbus_framefmt *v4l2_try_fmt;
>>>> + struct isc_format *fmt;
>>>> + unsigned int i;
>>>> +
>>>> + if (req_fmt->pad == ISC_SCALER_PAD_SOURCE)
>>>> + v4l_bound_align_image
>>>> + (&req_fmt->format.width, 16, isc->max_width, 0,
>>>> + &req_fmt->format.height, 16, isc->max_height, 0, 0);
>>>> + else
>>>> + v4l_bound_align_image
>>>> + (&req_fmt->format.width, 16, 10000, 0,
>>>> + &req_fmt->format.height, 16, 10000, 0, 0);
>>>> +
>>>> + req_fmt->format.colorspace = V4L2_COLORSPACE_SRGB;
>>>> + req_fmt->format.field = V4L2_FIELD_NONE;
>>>> + req_fmt->format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
>>>> + req_fmt->format.quantization = V4L2_QUANTIZATION_DEFAULT;
>>>> + req_fmt->format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
>>>> +
>>>> + fmt = isc_find_format_by_code(isc, req_fmt->format.code, &i);
>>>> +
>>>> + if (!fmt)
>>>> + fmt = &isc->formats_list[0];
>>>> +
>>>> + req_fmt->format.code = fmt->mbus_code;
>>>> +
>>>> + if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
>>>> + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state,
>>>> + req_fmt->pad);
>>>> + *v4l2_try_fmt = req_fmt->format;
>>>> + /* Trying on the pad sink makes the source sink change too */
>>>> + if (req_fmt->pad == ISC_SCALER_PAD_SINK) {
>>>> + v4l2_try_fmt =
>>>> + v4l2_subdev_get_try_format(sd, sd_state,
>>>> + ISC_SCALER_PAD_SOURCE);
>>>> + *v4l2_try_fmt = req_fmt->format;
>>>> +
>>>> + v4l_bound_align_image(&v4l2_try_fmt->width,
>>>> + 16, isc->max_width, 0,
>>>> + &v4l2_try_fmt->height,
>>>> + 16, isc->max_height, 0, 0);
>>>> + }
>>>> + /* if we are just trying, we are done */
>>>> + return 0;
>>>> + }
>>>> +
>>>> + isc->scaler_format = req_fmt->format;
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> +static int isc_scaler_enum_mbus_code(struct v4l2_subdev *sd,
>>>> + struct v4l2_subdev_state *sd_state,
>>>> + struct v4l2_subdev_mbus_code_enum *code)
>>>> +{
>>>> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
>>>> + int supported_index = 0;
>>>> + int i;
>>>> +
>>>> + for (i = 0; i < isc->formats_list_size; i++) {
>>>> + if (!isc->formats_list[i].sd_support)
>>>> + continue;
>>>> + if (supported_index == code->index) {
>>>> + code->code = isc->formats_list[i].mbus_code;
>>>> + return 0;
>>>> + }
>>>> + supported_index++;
>>>> + }
>>>> +
>>>> + return -EINVAL;
>>>> +}
>>>> +
>>>> +static int isc_scaler_g_sel(struct v4l2_subdev *sd,
>>>> + struct v4l2_subdev_state *sd_state,
>>>> + struct v4l2_subdev_selection *sel)
>>>> +{
>>>> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
>>>> +
>>>> + if (sel->pad == ISC_SCALER_PAD_SOURCE)
>>>> + return -EINVAL;
>>>> +
>>>> + if (sel->target != V4L2_SEL_TGT_CROP_BOUNDS &&
>>>> + sel->target != V4L2_SEL_TGT_CROP)
>>>> + return -EINVAL;
>>>> +
>>>> + sel->r.height = isc->max_height;
>>>> + sel->r.width = isc->max_width;
>>>> +
>>>> + sel->r.left = 0;
>>>> + sel->r.top = 0;
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> +static int isc_scaler_init_cfg(struct v4l2_subdev *sd,
>>>> + struct v4l2_subdev_state *sd_state)
>>>> +{
>>>> + struct v4l2_mbus_framefmt *v4l2_try_fmt =
>>>> + v4l2_subdev_get_try_format(sd, sd_state, 0);
>>>> + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd);
>>>> +
>>>> + *v4l2_try_fmt = isc->scaler_format;
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> +static const struct v4l2_subdev_pad_ops isc_scaler_pad_ops = {
>>>> + .enum_mbus_code = isc_scaler_enum_mbus_code,
>>>> + .set_fmt = isc_scaler_set_fmt,
>>>> + .get_fmt = isc_scaler_get_fmt,
>>>> + .get_selection = isc_scaler_g_sel,
>>>> + .init_cfg = isc_scaler_init_cfg,
>>>> +};
>>>> +
>>>> +static const struct v4l2_subdev_ops xisc_scaler_subdev_ops = {
>>>> + .pad = &isc_scaler_pad_ops,
>>>> +};
>>>> +
>>>> +static int isc_init_own_sd(struct isc_device *isc)
>>>> +{
>>>> + int ret;
>>>> +
>>>> + v4l2_subdev_init(&isc->scaler_sd, &xisc_scaler_subdev_ops);
>>>> +
>>>> + isc->scaler_sd.owner = THIS_MODULE;
>>>> + isc->scaler_sd.dev = isc->dev;
>>>> + snprintf(isc->scaler_sd.name, sizeof(isc->scaler_sd.name),
>>>> + "atmel_isc_scaler");
>>>> +
>>>> + isc->scaler_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
>>>> + isc->scaler_sd.entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
>>>> + isc->scaler_pads[ISC_SCALER_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>>>> + isc->scaler_pads[ISC_SCALER_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
>>>> +
>>>> + isc->scaler_format.height = isc->max_height;
>>>> + isc->scaler_format.width = isc->max_width;
>>>> + isc->scaler_format.code = isc->formats_list[0].mbus_code;
>>>> + isc->scaler_format.colorspace = V4L2_COLORSPACE_SRGB;
>>>> + isc->scaler_format.field = V4L2_FIELD_NONE;
>>>> + isc->scaler_format.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
>>>> + isc->scaler_format.quantization = V4L2_QUANTIZATION_DEFAULT;
>>>> + isc->scaler_format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
>>>> +
>>>> + ret = media_entity_pads_init(&isc->scaler_sd.entity,
>>>> + ISC_SCALER_PADS_NUM,
>>>> + isc->scaler_pads);
>>>> + if (ret < 0) {
>>>> + dev_err(isc->dev, "scaler sd media entity init failed\n");
>>>> + return ret;
>>>> + }
>>>> + ret = v4l2_device_register_subdev(&isc->v4l2_dev, &isc->scaler_sd);
>>>> + if (ret < 0) {
>>>> + dev_err(isc->dev, "scaler sd failed to register subdev\n");
>>>> + return ret;
>>>> + }
>>>> +
>>>> + return ret;
>>>> +}
>>>> +
>>>> +int isc_mc_init(struct isc_device *isc, u32 ver)
>>>> +{
>>>> + const struct of_device_id *match;
>>>> + int ret;
>>>> +
>>>> + isc->video_dev.entity.function = MEDIA_ENT_F_IO_V4L;
>>>> + isc->video_dev.entity.flags = MEDIA_ENT_FL_DEFAULT;
>>>> + isc->pads[ISC_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>>>> +
>>>> + ret = media_entity_pads_init(&isc->video_dev.entity, ISC_PADS_NUM,
>>>> + isc->pads);
>>>> + if (ret < 0) {
>>>> + dev_err(isc->dev, "media entity init failed\n");
>>>> + return ret;
>>>> + }
>>>> +
>>>> + isc->mdev.dev = isc->dev;
>>>> + isc->mdev.ops = &isc_media_ops;
>>>> +
>>>> + match = of_match_node(isc->dev->driver->of_match_table,
>>>> + isc->dev->of_node);
>>>> +
>>>> + strscpy(isc->mdev.driver_name, KBUILD_MODNAME,
>>>> + sizeof(isc->mdev.driver_name));
>>>> + strscpy(isc->mdev.model, match->compatible, sizeof(isc->mdev.model));
>>>> + snprintf(isc->mdev.bus_info, sizeof(isc->mdev.bus_info), "platform:%s",
>>>> + isc->v4l2_dev.name);
>>>> + isc->mdev.hw_revision = ver;
>>>> +
>>>> + media_device_init(&isc->mdev);
>>>> +
>>>> + isc->v4l2_dev.mdev = &isc->mdev;
>>>> +
>>>> + return isc_init_own_sd(isc);
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(isc_mc_init);
>>>> +
>>>> +int isc_mc_register(struct isc_device *isc)
>>>> +{
>>>> + int ret;
>>>> +
>>>> + ret = media_create_pad_link(&isc->current_subdev->sd->entity,
>>>> + isc->remote_pad, &isc->scaler_sd.entity,
>>>> + ISC_SCALER_PAD_SINK,
>>>> + MEDIA_LNK_FL_ENABLED |
>>>> + MEDIA_LNK_FL_IMMUTABLE);
>>>> +
>>>> + if (ret < 0) {
>>>> + v4l2_err(&isc->v4l2_dev,
>>>> + "Failed to create pad link: %s to %s\n",
>>>> + isc->current_subdev->sd->entity.name,
>>>> + isc->scaler_sd.entity.name);
>>>> + return ret;
>>>> + }
>>>> +
>>>> + dev_dbg(isc->dev, "link with %s pad: %d\n",
>>>> + isc->current_subdev->sd->name, isc->remote_pad);
>>>> +
>>>> + ret = media_create_pad_link(&isc->scaler_sd.entity,
>>>> + ISC_SCALER_PAD_SOURCE,
>>>> + &isc->video_dev.entity, ISC_PAD_SINK,
>>>> + MEDIA_LNK_FL_ENABLED |
>>>> + MEDIA_LNK_FL_IMMUTABLE);
>>>> +
>>>> + if (ret < 0) {
>>>> + v4l2_err(&isc->v4l2_dev,
>>>> + "Failed to create pad link: %s to %s\n",
>>>> + isc->scaler_sd.entity.name,
>>>> + isc->video_dev.entity.name);
>>>> + return ret;
>>>> + }
>>>> +
>>>> + dev_dbg(isc->dev, "link with %s pad: %d\n", isc->scaler_sd.name,
>>>> + ISC_SCALER_PAD_SOURCE);
>>>> +
>>>> + return media_device_register(&isc->mdev);
>>>
>>> I feel like the isc-base driver shold instead register the
>>> media-controller and this file shuld actually be only about
>>> isc-scaler or something. I might be missing why it is not possible
>>> though. It doesn't seem to me that mc support is conditional, so it
>>> can happen in the base driver directly ?
>>
>> This actually is the base driver. Just that the atmel-isc-base.c is
>> pretty crowded. So I moved the MC part in this file. But it's the same
>> base driver.
>>
>
> I understand. isc-base is pretty big, you're right.
> However breaking mc apart in a file that mixes registering an entity
> for the scaler and initializing mc for the whole driver feels weird,
> at least to me, as I don't immediately get why the mc part is handled
> by the scaler file...
>
>>>
>>> Links creation can still happen at async_complete() (I would have
>>> suggested _bound() time, but you have no asyn subdev for the scaler,
>>> so I guess the only async subdev here is the sensor).
>>
>> Yes, the sensor or any kind of subdevice (XISC normally is connected to
>> a bridge, ISC to a sensor directly )
>>
>>>
>>> All in all: can this function be removed and:
>>> 1) Base driver registers the media controller device in probe
>>
>> Should we register the media device even if it's not operational ? and
>> no video device has been registered ?
>> I thought it would be safer to have it registered with all the
>> links/entities/etc once the video node has been completed.
>>
>> But if you say this is the proper way, I can do that.
>> I wanted to keep things separated and not overcrowd the base file /probe
>> function, but if it's just a media_device_register call, I can do it there.
>
> You are right. Calling media_device_register() at async_complete time
> is the right thing to do.
>
> Thanks
> j
>
>
>>
>>> 2) links are created at async_complete time ?
>>
>> That works too. I think I could move them there.
>>
>> Thanks for reviewing,
>>
>> Eugen
>>
>>>
>>> Thanks
>>> j
>>>
>>>
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(isc_mc_register);
>>>> +
>>>> +void isc_mc_cleanup(struct isc_device *isc)
>>>> +{
>>>> + media_entity_cleanup(&isc->video_dev.entity);
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(isc_mc_cleanup);
>>>> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
>>>> index 07fa6dbf8460..0b6370d7775f 100644
>>>> --- a/drivers/media/platform/atmel/atmel-isc.h
>>>> +++ b/drivers/media/platform/atmel/atmel-isc.h
>>>> @@ -183,6 +183,17 @@ struct isc_reg_offsets {
>>>> u32 his_entry;
>>>> };
>>>>
>>>> +enum isc_mc_pads {
>>>> + ISC_PAD_SINK = 0,
>>>> + ISC_PADS_NUM = 1,
>>>> +};
>>>> +
>>>> +enum isc_scaler_pads {
>>>> + ISC_SCALER_PAD_SINK = 0,
>>>> + ISC_SCALER_PAD_SOURCE = 1,
>>>> + ISC_SCALER_PADS_NUM = 2,
>>>> +};
>>>> +
>>>> /*
>>>> * struct isc_device - ISC device driver data/config struct
>>>> * @regmap: Register map
>>>> @@ -258,6 +269,12 @@ struct isc_reg_offsets {
>>>> * be used as an input to the controller
>>>> * @controller_formats_size: size of controller_formats array
>>>> * @formats_list_size: size of formats_list array
>>>> + * @pads: media controller pads for isc video entity
>>>> + * @mdev: media device that is registered by the isc
>>>> + * @remote_pad: remote pad on the connected subdevice
>>>> + * @scaler_sd: subdevice for the scaler that isc registers
>>>> + * @scaler_pads: media controller pads for the scaler subdevice
>>>> + * @scaler_format: current format for the scaler subdevice
>>>> */
>>>> struct isc_device {
>>>> struct regmap *regmap;
>>>> @@ -346,6 +363,19 @@ struct isc_device {
>>>> struct isc_format *formats_list;
>>>> u32 controller_formats_size;
>>>> u32 formats_list_size;
>>>> +
>>>> + struct {
>>>> + struct media_pad pads[ISC_PADS_NUM];
>>>> + struct media_device mdev;
>>>> +
>>>> + u32 remote_pad;
>>>> + };
>>>> +
>>>> + struct {
>>>> + struct v4l2_subdev scaler_sd;
>>>> + struct media_pad scaler_pads[ISC_SCALER_PADS_NUM];
>>>> + struct v4l2_mbus_framefmt scaler_format;
>>>> + };
>>>> };
>>>>
>>>> extern const struct regmap_config isc_regmap_config;
>>>> @@ -357,4 +387,10 @@ int isc_clk_init(struct isc_device *isc);
>>>> void isc_subdev_cleanup(struct isc_device *isc);
>>>> void isc_clk_cleanup(struct isc_device *isc);
>>>>
>>>> +int isc_mc_init(struct isc_device *isc, u32 ver);
>>>> +int isc_mc_register(struct isc_device *isc);
>>>> +void isc_mc_cleanup(struct isc_device *isc);
>>>> +
>>>> +struct isc_format *isc_find_format_by_code(struct isc_device *isc,
>>>> + unsigned int code, int *index);
>>>> #endif
>>>> diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
>>>> index 1b2063cce0f7..a3e4c32a66c4 100644
>>>> --- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
>>>> +++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
>>>> @@ -529,6 +529,12 @@ static int atmel_isc_probe(struct platform_device *pdev)
>>>> break;
>>>> }
>>>>
>>>> + regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
>>>> +
>>>> + ret = isc_mc_init(isc, ver);
>>>> + if (ret < 0)
>>>> + goto isc_probe_mc_init_err;
>>>> +
>>>> pm_runtime_set_active(dev);
>>>> pm_runtime_enable(dev);
>>>> pm_request_idle(dev);
>>>> @@ -538,7 +544,7 @@ static int atmel_isc_probe(struct platform_device *pdev)
>>>> ret = clk_prepare_enable(isc->ispck);
>>>> if (ret) {
>>>> dev_err(dev, "failed to enable ispck: %d\n", ret);
>>>> - goto cleanup_subdev;
>>>> + goto isc_probe_mc_init_err;
>>>> }
>>>>
>>>> /* ispck should be greater or equal to hclock */
>>>> @@ -548,7 +554,6 @@ static int atmel_isc_probe(struct platform_device *pdev)
>>>> goto unprepare_clk;
>>>> }
>>>>
>>>> - regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
>>>> dev_info(dev, "Microchip ISC version %x\n", ver);
>>>>
>>>> return 0;
>>>> @@ -556,6 +561,9 @@ static int atmel_isc_probe(struct platform_device *pdev)
>>>> unprepare_clk:
>>>> clk_disable_unprepare(isc->ispck);
>>>>
>>>> +isc_probe_mc_init_err:
>>>> + isc_mc_cleanup(isc);
>>>> +
>>>> cleanup_subdev:
>>>> isc_subdev_cleanup(isc);
>>>>
>>>> @@ -576,6 +584,8 @@ static int atmel_isc_remove(struct platform_device *pdev)
>>>>
>>>> pm_runtime_disable(&pdev->dev);
>>>>
>>>> + isc_mc_cleanup(isc);
>>>> +
>>>> isc_subdev_cleanup(isc);
>>>>
>>>> v4l2_device_unregister(&isc->v4l2_dev);
>>>> diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
>>>> index 2b1082295c13..d2ac80b1ccc3 100644
>>>> --- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
>>>> +++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
>>>> @@ -523,15 +523,23 @@ static int microchip_xisc_probe(struct platform_device *pdev)
>>>> break;
>>>> }
>>>>
>>>> + regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
>>>> +
>>>> + ret = isc_mc_init(isc, ver);
>>>> + if (ret < 0)
>>>> + goto isc_probe_mc_init_err;
>>>> +
>>>> pm_runtime_set_active(dev);
>>>> pm_runtime_enable(dev);
>>>> pm_request_idle(dev);
>>>>
>>>> - regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver);
>>>> dev_info(dev, "Microchip XISC version %x\n", ver);
>>>>
>>>> return 0;
>>>>
>>>> +isc_probe_mc_init_err:
>>>> + isc_mc_cleanup(isc);
>>>> +
>>>> cleanup_subdev:
>>>> isc_subdev_cleanup(isc);
>>>>
>>>> @@ -552,6 +560,8 @@ static int microchip_xisc_remove(struct platform_device *pdev)
>>>>
>>>> pm_runtime_disable(&pdev->dev);
>>>>
>>>> + isc_mc_cleanup(isc);
>>>> +
>>>> isc_subdev_cleanup(isc);
>>>>
>>>> v4l2_device_unregister(&isc->v4l2_dev);
>>>> --
>>>> 2.25.1
>>>>
>>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 17/23] media: atmel: atmel-isc-base: use mutex to lock awb workqueue from streaming
2021-12-13 13:49 ` Eugen Hristev
@ 2022-01-12 8:58 ` Jacopo Mondi
-1 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-12 8:58 UTC (permalink / raw)
To: Eugen Hristev
Cc: linux-media, robh+dt, laurent.pinchart, sakari.ailus, devicetree,
linux-kernel, linux-arm-kernel, nicolas.ferre
Hi Eugen
On Mon, Dec 13, 2021 at 03:49:34PM +0200, Eugen Hristev wrote:
> The AWB workqueue runs in a kernel thread and needs to be synchronized
> w.r.t. the streaming status.
> It is possible that streaming is stopped while the AWB workq is running.
> In this case it is likely that the check for isc->stop is done at one point
> in time, but the AWB computations are done later, including a call to
> isc_update_profile, which requires streaming to be started.
> Thus , isc_update_profile will fail if during this operation sequence the
> streaming was stopped.
> To solve this issue, a mutex is added, that will serialize the awb work and
> streaming stopping, with the mention that either streaming is stopped
> completely including termination of the last frame is done, and after that
> the AWB work can check stream status and stop; either first AWB work is
> completed and after that the streaming can stop correctly.
> The awb spin lock cannot be used since this spinlock is taken in the same
> context and using it in the stop streaming will result in a recursion BUG.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> ---
> drivers/media/platform/atmel/atmel-isc-base.c | 31 ++++++++++++++++---
> drivers/media/platform/atmel/atmel-isc.h | 1 +
> 2 files changed, 28 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> index b0c3ed21f372..53cac1aac0fd 100644
> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> @@ -401,6 +401,7 @@ static void isc_stop_streaming(struct vb2_queue *vq)
> struct isc_buffer *buf;
> int ret;
>
> + mutex_lock(&isc->awb_mutex);
> v4l2_ctrl_activate(isc->do_wb_ctrl, false);
>
> isc->stop = true;
> @@ -410,6 +411,8 @@ static void isc_stop_streaming(struct vb2_queue *vq)
> v4l2_err(&isc->v4l2_dev,
> "Timeout waiting for end of the capture\n");
>
> + mutex_unlock(&isc->awb_mutex);
> +
> /* Disable DMA interrupt */
> regmap_write(isc->regmap, ISC_INTDIS, ISC_INT_DDONE);
>
> @@ -1416,10 +1419,6 @@ static void isc_awb_work(struct work_struct *w)
> u32 min, max;
> int ret;
>
> - /* streaming is not active anymore */
> - if (isc->stop)
> - return;
> -
> if (ctrls->hist_stat != HIST_ENABLED)
> return;
>
> @@ -1470,7 +1469,24 @@ static void isc_awb_work(struct work_struct *w)
> }
> regmap_write(regmap, ISC_HIS_CFG + isc->offsets.his,
> hist_id | baysel | ISC_HIS_CFG_RAR);
isc_stop_streaming() calls runtime_put and here you access the hw.
Feels like it's safer to hold the mutex for the whole duration of the
AWB routine ?
> +
> + /*
> + * We have to make sure the streaming has not stopped meanwhile.
> + * ISC requires a frame to clock the internal profile update.
> + * To avoid issues, lock the sequence with a mutex
> + */
> + mutex_lock(&isc->awb_mutex);
> +
> + /* streaming is not active anymore */
> + if (isc->stop) {
> + mutex_unlock(&isc->awb_mutex);
> + return;
> + };
> +
> isc_update_profile(isc);
> +
> + mutex_unlock(&isc->awb_mutex);
> +
> /* if awb has been disabled, we don't need to start another histogram */
> if (ctrls->awb)
> regmap_write(regmap, ISC_CTRLEN, ISC_CTRL_HISREQ);
> @@ -1549,6 +1565,8 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
>
> isc_update_awb_ctrls(isc);
>
> + mutex_lock(&isc->awb_mutex);
> +
> if (!isc->stop) {
> /*
> * If we are streaming, we can update profile to
> @@ -1563,6 +1581,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
> */
> v4l2_ctrl_activate(isc->do_wb_ctrl, false);
> }
> + mutex_unlock(&isc->awb_mutex);
>
> /* if we have autowhitebalance on, start histogram procedure */
> if (ctrls->awb == ISC_WB_AUTO && !isc->stop &&
> @@ -1754,6 +1773,7 @@ static void isc_async_unbind(struct v4l2_async_notifier *notifier,
> {
> struct isc_device *isc = container_of(notifier->v4l2_dev,
> struct isc_device, v4l2_dev);
> + mutex_destroy(&isc->awb_mutex);
> cancel_work_sync(&isc->awb_work);
> video_unregister_device(&isc->video_dev);
> v4l2_ctrl_handler_free(&isc->ctrls.handler);
> @@ -1866,6 +1886,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
> isc->current_subdev = container_of(notifier,
> struct isc_subdev_entity, notifier);
> mutex_init(&isc->lock);
> + mutex_init(&isc->awb_mutex);
> +
> init_completion(&isc->comp);
>
> /* Initialize videobuf2 queue */
> @@ -1941,6 +1963,7 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
> video_unregister_device(vdev);
>
> isc_async_complete_err:
> + mutex_destroy(&isc->awb_mutex);
> mutex_destroy(&isc->lock);
> return ret;
> }
> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
> index 0b6370d7775f..c2cb805faff3 100644
> --- a/drivers/media/platform/atmel/atmel-isc.h
> +++ b/drivers/media/platform/atmel/atmel-isc.h
> @@ -307,6 +307,7 @@ struct isc_device {
> struct work_struct awb_work;
>
> struct mutex lock; /* serialize access to file operations */
> + struct mutex awb_mutex; /* serialize access to streaming status from awb work queue */
> spinlock_t awb_lock; /* serialize access to DMA buffers from awb work queue */
>
> struct regmap_field *pipeline[ISC_PIPE_LINE_NODE_NUM];
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 17/23] media: atmel: atmel-isc-base: use mutex to lock awb workqueue from streaming
@ 2022-01-12 8:58 ` Jacopo Mondi
0 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-12 8:58 UTC (permalink / raw)
To: Eugen Hristev
Cc: devicetree, linux-kernel, robh+dt, sakari.ailus,
laurent.pinchart, linux-arm-kernel, linux-media
Hi Eugen
On Mon, Dec 13, 2021 at 03:49:34PM +0200, Eugen Hristev wrote:
> The AWB workqueue runs in a kernel thread and needs to be synchronized
> w.r.t. the streaming status.
> It is possible that streaming is stopped while the AWB workq is running.
> In this case it is likely that the check for isc->stop is done at one point
> in time, but the AWB computations are done later, including a call to
> isc_update_profile, which requires streaming to be started.
> Thus , isc_update_profile will fail if during this operation sequence the
> streaming was stopped.
> To solve this issue, a mutex is added, that will serialize the awb work and
> streaming stopping, with the mention that either streaming is stopped
> completely including termination of the last frame is done, and after that
> the AWB work can check stream status and stop; either first AWB work is
> completed and after that the streaming can stop correctly.
> The awb spin lock cannot be used since this spinlock is taken in the same
> context and using it in the stop streaming will result in a recursion BUG.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> ---
> drivers/media/platform/atmel/atmel-isc-base.c | 31 ++++++++++++++++---
> drivers/media/platform/atmel/atmel-isc.h | 1 +
> 2 files changed, 28 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> index b0c3ed21f372..53cac1aac0fd 100644
> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> @@ -401,6 +401,7 @@ static void isc_stop_streaming(struct vb2_queue *vq)
> struct isc_buffer *buf;
> int ret;
>
> + mutex_lock(&isc->awb_mutex);
> v4l2_ctrl_activate(isc->do_wb_ctrl, false);
>
> isc->stop = true;
> @@ -410,6 +411,8 @@ static void isc_stop_streaming(struct vb2_queue *vq)
> v4l2_err(&isc->v4l2_dev,
> "Timeout waiting for end of the capture\n");
>
> + mutex_unlock(&isc->awb_mutex);
> +
> /* Disable DMA interrupt */
> regmap_write(isc->regmap, ISC_INTDIS, ISC_INT_DDONE);
>
> @@ -1416,10 +1419,6 @@ static void isc_awb_work(struct work_struct *w)
> u32 min, max;
> int ret;
>
> - /* streaming is not active anymore */
> - if (isc->stop)
> - return;
> -
> if (ctrls->hist_stat != HIST_ENABLED)
> return;
>
> @@ -1470,7 +1469,24 @@ static void isc_awb_work(struct work_struct *w)
> }
> regmap_write(regmap, ISC_HIS_CFG + isc->offsets.his,
> hist_id | baysel | ISC_HIS_CFG_RAR);
isc_stop_streaming() calls runtime_put and here you access the hw.
Feels like it's safer to hold the mutex for the whole duration of the
AWB routine ?
> +
> + /*
> + * We have to make sure the streaming has not stopped meanwhile.
> + * ISC requires a frame to clock the internal profile update.
> + * To avoid issues, lock the sequence with a mutex
> + */
> + mutex_lock(&isc->awb_mutex);
> +
> + /* streaming is not active anymore */
> + if (isc->stop) {
> + mutex_unlock(&isc->awb_mutex);
> + return;
> + };
> +
> isc_update_profile(isc);
> +
> + mutex_unlock(&isc->awb_mutex);
> +
> /* if awb has been disabled, we don't need to start another histogram */
> if (ctrls->awb)
> regmap_write(regmap, ISC_CTRLEN, ISC_CTRL_HISREQ);
> @@ -1549,6 +1565,8 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
>
> isc_update_awb_ctrls(isc);
>
> + mutex_lock(&isc->awb_mutex);
> +
> if (!isc->stop) {
> /*
> * If we are streaming, we can update profile to
> @@ -1563,6 +1581,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
> */
> v4l2_ctrl_activate(isc->do_wb_ctrl, false);
> }
> + mutex_unlock(&isc->awb_mutex);
>
> /* if we have autowhitebalance on, start histogram procedure */
> if (ctrls->awb == ISC_WB_AUTO && !isc->stop &&
> @@ -1754,6 +1773,7 @@ static void isc_async_unbind(struct v4l2_async_notifier *notifier,
> {
> struct isc_device *isc = container_of(notifier->v4l2_dev,
> struct isc_device, v4l2_dev);
> + mutex_destroy(&isc->awb_mutex);
> cancel_work_sync(&isc->awb_work);
> video_unregister_device(&isc->video_dev);
> v4l2_ctrl_handler_free(&isc->ctrls.handler);
> @@ -1866,6 +1886,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
> isc->current_subdev = container_of(notifier,
> struct isc_subdev_entity, notifier);
> mutex_init(&isc->lock);
> + mutex_init(&isc->awb_mutex);
> +
> init_completion(&isc->comp);
>
> /* Initialize videobuf2 queue */
> @@ -1941,6 +1963,7 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
> video_unregister_device(vdev);
>
> isc_async_complete_err:
> + mutex_destroy(&isc->awb_mutex);
> mutex_destroy(&isc->lock);
> return ret;
> }
> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
> index 0b6370d7775f..c2cb805faff3 100644
> --- a/drivers/media/platform/atmel/atmel-isc.h
> +++ b/drivers/media/platform/atmel/atmel-isc.h
> @@ -307,6 +307,7 @@ struct isc_device {
> struct work_struct awb_work;
>
> struct mutex lock; /* serialize access to file operations */
> + struct mutex awb_mutex; /* serialize access to streaming status from awb work queue */
> spinlock_t awb_lock; /* serialize access to DMA buffers from awb work queue */
>
> struct regmap_field *pipeline[ISC_PIPE_LINE_NODE_NUM];
> --
> 2.25.1
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 18/23] media: atmel: atmel-isc-base: add wb debug messages
2021-12-13 13:49 ` Eugen Hristev
@ 2022-01-12 9:00 ` Jacopo Mondi
-1 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-12 9:00 UTC (permalink / raw)
To: Eugen Hristev
Cc: linux-media, robh+dt, laurent.pinchart, sakari.ailus, devicetree,
linux-kernel, linux-arm-kernel, nicolas.ferre
Hello Eugen,
On Mon, Dec 13, 2021 at 03:49:35PM +0200, Eugen Hristev wrote:
> Add debug messages that make it easier to debug white balance algorithm.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
The dev_dbg() infrastructure is preferred, as far as I know v4l2_dbg()
is legacy and should be removed.
However the driver uses v4l2_dbg() already, so
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Thanks
j
> ---
> drivers/media/platform/atmel/atmel-isc-base.c | 18 ++++++++++++++++++
> 1 file changed, 18 insertions(+)
>
> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> index 53cac1aac0fd..f1f1019f9d82 100644
> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> @@ -1329,10 +1329,15 @@ static void isc_hist_count(struct isc_device *isc, u32 *min, u32 *max)
>
> if (!*min)
> *min = 1;
> +
> + v4l2_dbg(1, debug, &isc->v4l2_dev,
> + "isc wb: hist_id %u, hist_count %u",
> + ctrls->hist_id, *hist_count);
> }
>
> static void isc_wb_update(struct isc_ctrls *ctrls)
> {
> + struct isc_device *isc = container_of(ctrls, struct isc_device, ctrls);
> u32 *hist_count = &ctrls->hist_count[0];
> u32 c, offset[4];
> u64 avg = 0;
> @@ -1349,6 +1354,9 @@ static void isc_wb_update(struct isc_ctrls *ctrls)
> (u64)hist_count[ISC_HIS_CFG_MODE_GB];
> avg >>= 1;
>
> + v4l2_dbg(1, debug, &isc->v4l2_dev,
> + "isc wb: green components average %llu\n", avg);
> +
> /* Green histogram is null, nothing to do */
> if (!avg)
> return;
> @@ -1401,9 +1409,15 @@ static void isc_wb_update(struct isc_ctrls *ctrls)
> else
> gw_gain[c] = 1 << 9;
>
> + v4l2_dbg(1, debug, &isc->v4l2_dev,
> + "isc wb: component %d, s_gain %u, gw_gain %u\n",
> + c, s_gain[c], gw_gain[c]);
> /* multiply both gains and adjust for decimals */
> ctrls->gain[c] = s_gain[c] * gw_gain[c];
> ctrls->gain[c] >>= 9;
> + v4l2_dbg(1, debug, &isc->v4l2_dev,
> + "isc wb: component %d, final gain %u\n",
> + c, ctrls->gain[c]);
> }
> }
>
> @@ -1423,6 +1437,10 @@ static void isc_awb_work(struct work_struct *w)
> return;
>
> isc_hist_count(isc, &min, &max);
> +
> + v4l2_dbg(1, debug, &isc->v4l2_dev,
> + "isc wb mode %d: hist min %u , max %u\n", hist_id, min, max);
> +
> ctrls->hist_minmax[hist_id][HIST_MIN_INDEX] = min;
> ctrls->hist_minmax[hist_id][HIST_MAX_INDEX] = max;
>
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 18/23] media: atmel: atmel-isc-base: add wb debug messages
@ 2022-01-12 9:00 ` Jacopo Mondi
0 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-12 9:00 UTC (permalink / raw)
To: Eugen Hristev
Cc: devicetree, linux-kernel, robh+dt, sakari.ailus,
laurent.pinchart, linux-arm-kernel, linux-media
Hello Eugen,
On Mon, Dec 13, 2021 at 03:49:35PM +0200, Eugen Hristev wrote:
> Add debug messages that make it easier to debug white balance algorithm.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
The dev_dbg() infrastructure is preferred, as far as I know v4l2_dbg()
is legacy and should be removed.
However the driver uses v4l2_dbg() already, so
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Thanks
j
> ---
> drivers/media/platform/atmel/atmel-isc-base.c | 18 ++++++++++++++++++
> 1 file changed, 18 insertions(+)
>
> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> index 53cac1aac0fd..f1f1019f9d82 100644
> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> @@ -1329,10 +1329,15 @@ static void isc_hist_count(struct isc_device *isc, u32 *min, u32 *max)
>
> if (!*min)
> *min = 1;
> +
> + v4l2_dbg(1, debug, &isc->v4l2_dev,
> + "isc wb: hist_id %u, hist_count %u",
> + ctrls->hist_id, *hist_count);
> }
>
> static void isc_wb_update(struct isc_ctrls *ctrls)
> {
> + struct isc_device *isc = container_of(ctrls, struct isc_device, ctrls);
> u32 *hist_count = &ctrls->hist_count[0];
> u32 c, offset[4];
> u64 avg = 0;
> @@ -1349,6 +1354,9 @@ static void isc_wb_update(struct isc_ctrls *ctrls)
> (u64)hist_count[ISC_HIS_CFG_MODE_GB];
> avg >>= 1;
>
> + v4l2_dbg(1, debug, &isc->v4l2_dev,
> + "isc wb: green components average %llu\n", avg);
> +
> /* Green histogram is null, nothing to do */
> if (!avg)
> return;
> @@ -1401,9 +1409,15 @@ static void isc_wb_update(struct isc_ctrls *ctrls)
> else
> gw_gain[c] = 1 << 9;
>
> + v4l2_dbg(1, debug, &isc->v4l2_dev,
> + "isc wb: component %d, s_gain %u, gw_gain %u\n",
> + c, s_gain[c], gw_gain[c]);
> /* multiply both gains and adjust for decimals */
> ctrls->gain[c] = s_gain[c] * gw_gain[c];
> ctrls->gain[c] >>= 9;
> + v4l2_dbg(1, debug, &isc->v4l2_dev,
> + "isc wb: component %d, final gain %u\n",
> + c, ctrls->gain[c]);
> }
> }
>
> @@ -1423,6 +1437,10 @@ static void isc_awb_work(struct work_struct *w)
> return;
>
> isc_hist_count(isc, &min, &max);
> +
> + v4l2_dbg(1, debug, &isc->v4l2_dev,
> + "isc wb mode %d: hist min %u , max %u\n", hist_id, min, max);
> +
> ctrls->hist_minmax[hist_id][HIST_MIN_INDEX] = min;
> ctrls->hist_minmax[hist_id][HIST_MAX_INDEX] = max;
>
> --
> 2.25.1
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 19/23] media: atmel: atmel-isc-base: clamp wb gain coefficients
2021-12-13 13:49 ` Eugen Hristev
@ 2022-01-12 9:04 ` Jacopo Mondi
-1 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-12 9:04 UTC (permalink / raw)
To: Eugen Hristev
Cc: linux-media, robh+dt, laurent.pinchart, sakari.ailus, devicetree,
linux-kernel, linux-arm-kernel, nicolas.ferre
Hi Eugen
On Mon, Dec 13, 2021 at 03:49:36PM +0200, Eugen Hristev wrote:
> White balance computed gains can overflow above the 13 bits hardware
> coefficient that can be used, in some specific scenarios like a subexposure
> from the sensor when the image is mostly black.
> In this case the computed gain has to be clamped to the maximum value
> allowed by the hardware.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> ---
> drivers/media/platform/atmel/atmel-isc-base.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> index f1f1019f9d82..31c8e3029eee 100644
> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> @@ -1415,6 +1415,10 @@ static void isc_wb_update(struct isc_ctrls *ctrls)
> /* multiply both gains and adjust for decimals */
> ctrls->gain[c] = s_gain[c] * gw_gain[c];
> ctrls->gain[c] >>= 9;
> +
> + /* make sure we are not out of range */
> + ctrls->gain[c] = clamp_val(ctrls->gain[c], 0, GENMASK(12, 0));
#include <linux/minmax.h>
What is the type of gain[c} ? If it's unsigned clamping with [0 is not
required. I would in that case gain[c] = min(gain[c], ...)
It's fine anyway
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Thanks
j
> +
> v4l2_dbg(1, debug, &isc->v4l2_dev,
> "isc wb: component %d, final gain %u\n",
> c, ctrls->gain[c]);
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 19/23] media: atmel: atmel-isc-base: clamp wb gain coefficients
@ 2022-01-12 9:04 ` Jacopo Mondi
0 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-12 9:04 UTC (permalink / raw)
To: Eugen Hristev
Cc: devicetree, linux-kernel, robh+dt, sakari.ailus,
laurent.pinchart, linux-arm-kernel, linux-media
Hi Eugen
On Mon, Dec 13, 2021 at 03:49:36PM +0200, Eugen Hristev wrote:
> White balance computed gains can overflow above the 13 bits hardware
> coefficient that can be used, in some specific scenarios like a subexposure
> from the sensor when the image is mostly black.
> In this case the computed gain has to be clamped to the maximum value
> allowed by the hardware.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> ---
> drivers/media/platform/atmel/atmel-isc-base.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> index f1f1019f9d82..31c8e3029eee 100644
> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> @@ -1415,6 +1415,10 @@ static void isc_wb_update(struct isc_ctrls *ctrls)
> /* multiply both gains and adjust for decimals */
> ctrls->gain[c] = s_gain[c] * gw_gain[c];
> ctrls->gain[c] >>= 9;
> +
> + /* make sure we are not out of range */
> + ctrls->gain[c] = clamp_val(ctrls->gain[c], 0, GENMASK(12, 0));
#include <linux/minmax.h>
What is the type of gain[c} ? If it's unsigned clamping with [0 is not
required. I would in that case gain[c] = min(gain[c], ...)
It's fine anyway
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Thanks
j
> +
> v4l2_dbg(1, debug, &isc->v4l2_dev,
> "isc wb: component %d, final gain %u\n",
> c, ctrls->gain[c]);
> --
> 2.25.1
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 20/23] media: atmel: atmel-sama7g5-isc: fix UYVY input format mbus_code typo
2021-12-13 13:49 ` Eugen Hristev
@ 2022-01-12 9:04 ` Jacopo Mondi
-1 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-12 9:04 UTC (permalink / raw)
To: Eugen Hristev
Cc: linux-media, robh+dt, laurent.pinchart, sakari.ailus, devicetree,
linux-kernel, linux-arm-kernel, nicolas.ferre
Hi Eugen
On Mon, Dec 13, 2021 at 03:49:37PM +0200, Eugen Hristev wrote:
> Fix the mbus code for UYVY input format type to the right color rotation.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Thanks
j
> ---
> drivers/media/platform/atmel/atmel-sama7g5-isc.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> index d2ac80b1ccc3..38721bd902e2 100644
> --- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> +++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> @@ -188,7 +188,7 @@ static struct isc_format sama7g5_formats_list[] = {
> },
> {
> .fourcc = V4L2_PIX_FMT_UYVY,
> - .mbus_code = MEDIA_BUS_FMT_YUYV8_2X8,
> + .mbus_code = MEDIA_BUS_FMT_UYVY8_2X8,
> .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_EIGHT,
> },
> {
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 20/23] media: atmel: atmel-sama7g5-isc: fix UYVY input format mbus_code typo
@ 2022-01-12 9:04 ` Jacopo Mondi
0 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-12 9:04 UTC (permalink / raw)
To: Eugen Hristev
Cc: devicetree, linux-kernel, robh+dt, sakari.ailus,
laurent.pinchart, linux-arm-kernel, linux-media
Hi Eugen
On Mon, Dec 13, 2021 at 03:49:37PM +0200, Eugen Hristev wrote:
> Fix the mbus code for UYVY input format type to the right color rotation.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Thanks
j
> ---
> drivers/media/platform/atmel/atmel-sama7g5-isc.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> index d2ac80b1ccc3..38721bd902e2 100644
> --- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> +++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> @@ -188,7 +188,7 @@ static struct isc_format sama7g5_formats_list[] = {
> },
> {
> .fourcc = V4L2_PIX_FMT_UYVY,
> - .mbus_code = MEDIA_BUS_FMT_YUYV8_2X8,
> + .mbus_code = MEDIA_BUS_FMT_UYVY8_2X8,
> .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_EIGHT,
> },
> {
> --
> 2.25.1
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 21/23] media: atmel: atmel-isc: add raw Bayer 8bit 10bit output formats
2021-12-13 13:49 ` Eugen Hristev
@ 2022-01-12 9:05 ` Jacopo Mondi
-1 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-12 9:05 UTC (permalink / raw)
To: Eugen Hristev
Cc: linux-media, robh+dt, laurent.pinchart, sakari.ailus, devicetree,
linux-kernel, linux-arm-kernel, nicolas.ferre
Hi Eugen
On Mon, Dec 13, 2021 at 03:49:38PM +0200, Eugen Hristev wrote:
> The ISC can dump the 8 bit and 10 bit raw bayer formats directly to
> the memory.
> Thus, add them to the supported output format list.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Thanks
j
> ---
> .../media/platform/atmel/atmel-sama5d2-isc.c | 24 +++++++++++++++++++
> .../media/platform/atmel/atmel-sama7g5-isc.c | 24 +++++++++++++++++++
> 2 files changed, 48 insertions(+)
>
> diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> index a8d4ba60d3ac..025c3e8a7e95 100644
> --- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> +++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> @@ -88,6 +88,30 @@ static const struct isc_format sama5d2_controller_formats[] = {
> {
> .fourcc = V4L2_PIX_FMT_Y10,
> },
> + {
> + .fourcc = V4L2_PIX_FMT_SBGGR8,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SGBRG8,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SGRBG8,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SRGGB8,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SBGGR10,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SGBRG10,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SGRBG10,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SRGGB10,
> + },
> };
>
> /* This is a list of formats that the ISC can receive as *input* */
> diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> index 38721bd902e2..9dc75eed0098 100644
> --- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> +++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> @@ -100,6 +100,30 @@ static const struct isc_format sama7g5_controller_formats[] = {
> {
> .fourcc = V4L2_PIX_FMT_Y16,
> },
> + {
> + .fourcc = V4L2_PIX_FMT_SBGGR8,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SGBRG8,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SGRBG8,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SRGGB8,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SBGGR10,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SGBRG10,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SGRBG10,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SRGGB10,
> + },
> };
>
> /* This is a list of formats that the ISC can receive as *input* */
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 21/23] media: atmel: atmel-isc: add raw Bayer 8bit 10bit output formats
@ 2022-01-12 9:05 ` Jacopo Mondi
0 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-12 9:05 UTC (permalink / raw)
To: Eugen Hristev
Cc: devicetree, linux-kernel, robh+dt, sakari.ailus,
laurent.pinchart, linux-arm-kernel, linux-media
Hi Eugen
On Mon, Dec 13, 2021 at 03:49:38PM +0200, Eugen Hristev wrote:
> The ISC can dump the 8 bit and 10 bit raw bayer formats directly to
> the memory.
> Thus, add them to the supported output format list.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Thanks
j
> ---
> .../media/platform/atmel/atmel-sama5d2-isc.c | 24 +++++++++++++++++++
> .../media/platform/atmel/atmel-sama7g5-isc.c | 24 +++++++++++++++++++
> 2 files changed, 48 insertions(+)
>
> diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> index a8d4ba60d3ac..025c3e8a7e95 100644
> --- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> +++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> @@ -88,6 +88,30 @@ static const struct isc_format sama5d2_controller_formats[] = {
> {
> .fourcc = V4L2_PIX_FMT_Y10,
> },
> + {
> + .fourcc = V4L2_PIX_FMT_SBGGR8,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SGBRG8,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SGRBG8,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SRGGB8,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SBGGR10,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SGBRG10,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SGRBG10,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SRGGB10,
> + },
> };
>
> /* This is a list of formats that the ISC can receive as *input* */
> diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> index 38721bd902e2..9dc75eed0098 100644
> --- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> +++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> @@ -100,6 +100,30 @@ static const struct isc_format sama7g5_controller_formats[] = {
> {
> .fourcc = V4L2_PIX_FMT_Y16,
> },
> + {
> + .fourcc = V4L2_PIX_FMT_SBGGR8,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SGBRG8,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SGRBG8,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SRGGB8,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SBGGR10,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SGBRG10,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SGRBG10,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_SRGGB10,
> + },
> };
>
> /* This is a list of formats that the ISC can receive as *input* */
> --
> 2.25.1
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 22/23] media: atmel: atmel-isc: compact the controller formats list
2021-12-13 13:49 ` Eugen Hristev
@ 2022-01-12 9:06 ` Jacopo Mondi
-1 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-12 9:06 UTC (permalink / raw)
To: Eugen Hristev
Cc: linux-media, robh+dt, laurent.pinchart, sakari.ailus, devicetree,
linux-kernel, linux-arm-kernel, nicolas.ferre
Hi Eugen
On Mon, Dec 13, 2021 at 03:49:39PM +0200, Eugen Hristev wrote:
> Compact the list array to be more readable.
> No other changes, only cosmetic.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Thanks
j
> ---
> .../media/platform/atmel/atmel-sama5d2-isc.c | 51 ++++++----------
> .../media/platform/atmel/atmel-sama7g5-isc.c | 60 +++++++------------
> 2 files changed, 37 insertions(+), 74 deletions(-)
>
> diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> index 025c3e8a7e95..d96ee3373889 100644
> --- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> +++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> @@ -60,56 +60,39 @@
> static const struct isc_format sama5d2_controller_formats[] = {
> {
> .fourcc = V4L2_PIX_FMT_ARGB444,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_ARGB555,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_RGB565,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_ABGR32,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_XBGR32,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_YUV420,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_YUYV,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_YUV422P,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_GREY,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_Y10,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SBGGR8,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SGBRG8,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SGRBG8,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SRGGB8,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SBGGR10,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SGBRG10,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SGRBG10,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SRGGB10,
> },
> };
> diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> index 9dc75eed0098..e07ae188c15f 100644
> --- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> +++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> @@ -63,65 +63,45 @@
> static const struct isc_format sama7g5_controller_formats[] = {
> {
> .fourcc = V4L2_PIX_FMT_ARGB444,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_ARGB555,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_RGB565,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_ABGR32,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_XBGR32,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_YUV420,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_UYVY,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_VYUY,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_YUYV,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_YUV422P,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_GREY,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_Y10,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_Y16,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SBGGR8,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SGBRG8,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SGRBG8,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SRGGB8,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SBGGR10,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SGBRG10,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SGRBG10,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SRGGB10,
> },
> };
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 22/23] media: atmel: atmel-isc: compact the controller formats list
@ 2022-01-12 9:06 ` Jacopo Mondi
0 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-12 9:06 UTC (permalink / raw)
To: Eugen Hristev
Cc: devicetree, linux-kernel, robh+dt, sakari.ailus,
laurent.pinchart, linux-arm-kernel, linux-media
Hi Eugen
On Mon, Dec 13, 2021 at 03:49:39PM +0200, Eugen Hristev wrote:
> Compact the list array to be more readable.
> No other changes, only cosmetic.
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Thanks
j
> ---
> .../media/platform/atmel/atmel-sama5d2-isc.c | 51 ++++++----------
> .../media/platform/atmel/atmel-sama7g5-isc.c | 60 +++++++------------
> 2 files changed, 37 insertions(+), 74 deletions(-)
>
> diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> index 025c3e8a7e95..d96ee3373889 100644
> --- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> +++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c
> @@ -60,56 +60,39 @@
> static const struct isc_format sama5d2_controller_formats[] = {
> {
> .fourcc = V4L2_PIX_FMT_ARGB444,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_ARGB555,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_RGB565,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_ABGR32,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_XBGR32,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_YUV420,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_YUYV,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_YUV422P,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_GREY,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_Y10,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SBGGR8,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SGBRG8,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SGRBG8,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SRGGB8,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SBGGR10,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SGBRG10,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SGRBG10,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SRGGB10,
> },
> };
> diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> index 9dc75eed0098..e07ae188c15f 100644
> --- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> +++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c
> @@ -63,65 +63,45 @@
> static const struct isc_format sama7g5_controller_formats[] = {
> {
> .fourcc = V4L2_PIX_FMT_ARGB444,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_ARGB555,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_RGB565,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_ABGR32,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_XBGR32,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_YUV420,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_UYVY,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_VYUY,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_YUYV,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_YUV422P,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_GREY,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_Y10,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_Y16,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SBGGR8,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SGBRG8,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SGRBG8,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SRGGB8,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SBGGR10,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SGBRG10,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SGRBG10,
> - },
> - {
> + }, {
> .fourcc = V4L2_PIX_FMT_SRGGB10,
> },
> };
> --
> 2.25.1
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 23/23] media: atmel: atmel-isc: change format propagation to subdev into only verification
2021-12-13 13:49 ` Eugen Hristev
@ 2022-01-12 9:21 ` Jacopo Mondi
-1 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-12 9:21 UTC (permalink / raw)
To: Eugen Hristev
Cc: linux-media, robh+dt, laurent.pinchart, sakari.ailus, devicetree,
linux-kernel, linux-arm-kernel, nicolas.ferre
Hi Eugen
On Mon, Dec 13, 2021 at 03:49:40PM +0200, Eugen Hristev wrote:
> As a top MC video driver, the atmel-isc should not propagate the format to the
> subdevice.
> It should rather check at streamon() time if the subdev is properly configured
> with a compatible format.
> Removed the whole format finding logic, and reworked the format verification
> at streamon time, such that the ISC will return an error if the subdevice
> is not properly configured.
> With this being done, the module parameter 'sensor_prefered' makes no sense
> anymore. The ISC should not decide which format the sensor is using. The
> ISC should only cope with the situation and inform userspace if the streaming
> is possible in the current configuration.
Sounds great!
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> ---
> Changes in v3:
> - clamp to maximum resolution once the frame size from the subdev is found
>
> drivers/media/platform/atmel/atmel-isc-base.c | 271 ++++++++----------
> drivers/media/platform/atmel/atmel-isc.h | 1 +
> 2 files changed, 126 insertions(+), 146 deletions(-)
>
> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> index 31c8e3029eee..00c8c9588a78 100644
> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> @@ -36,11 +36,6 @@ static unsigned int debug;
> module_param(debug, int, 0644);
> MODULE_PARM_DESC(debug, "debug level (0-2)");
>
> -static unsigned int sensor_preferred = 1;
> -module_param(sensor_preferred, uint, 0644);
> -MODULE_PARM_DESC(sensor_preferred,
> - "Sensor is preferred to output the specified format (1-on 0-off), default 1");
> -
> #define ISC_IS_FORMAT_RAW(mbus_code) \
> (((mbus_code) & 0xf000) == 0x3000)
>
> @@ -532,7 +527,7 @@ static int isc_enum_fmt_vid_cap(struct file *file, void *priv,
> * convert it to any of the formats that we usually can with a
> * RAW sensor. Thus, do not advertise them.
> */
> - if (!isc->config.sd_format ||
> + if (isc->config.sd_format &&
Is this change intentional ?
> !ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
> return -EINVAL;
>
> @@ -621,20 +616,30 @@ static int isc_try_validate_formats(struct isc_device *isc)
> break;
> default:
> /* any other different formats are not supported */
> + v4l2_err(&isc->v4l2_dev, "Requested unsupported format.\n");
> ret = -EINVAL;
> }
> v4l2_dbg(1, debug, &isc->v4l2_dev,
> "Format validation, requested rgb=%u, yuv=%u, grey=%u, bayer=%u\n",
> rgb, yuv, grey, bayer);
Would it make sense to move this before the switch so that the
error messages, if any, appear later ?
>
> - /* we cannot output RAW if we do not receive RAW */
> - if ((bayer) && !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code))
> + if ((bayer) &&
> + !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code)) {
> + v4l2_err(&isc->v4l2_dev, "Cannot output RAW if we do not receive RAW.\n");
> return -EINVAL;
> + }
>
> - /* we cannot output GREY if we do not receive RAW/GREY */
> if (grey && !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code) &&
> - !ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code))
> + !ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) {
> + v4l2_err(&isc->v4l2_dev, "Cannot output GREY if we do not receive RAW/GREY.\n");
> return -EINVAL;
> + }
> +
> + if ((rgb || bayer || yuv) &&
> + ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) {
> + v4l2_err(&isc->v4l2_dev, "Cannot convert GREY to another format.\n");
> + return -EINVAL;
> + }
>
> return ret;
> }
> @@ -862,7 +867,7 @@ static void isc_try_fse(struct isc_device *isc,
> * If we do not know yet which format the subdev is using, we cannot
> * do anything.
> */
> - if (!isc->try_config.sd_format)
> + if (!isc->config.sd_format)
> return;
>
> fse.code = isc->try_config.sd_format->mbus_code;
> @@ -883,180 +888,141 @@ static void isc_try_fse(struct isc_device *isc,
> }
> }
>
> -static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f,
> - u32 *code)
> +static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f)
> {
> - int i;
> - struct isc_format *sd_fmt = NULL, *direct_fmt = NULL;
> struct v4l2_pix_format *pixfmt = &f->fmt.pix;
> - struct v4l2_subdev_pad_config pad_cfg = {};
> - struct v4l2_subdev_state pad_state = {
> - .pads = &pad_cfg
> - };
> - struct v4l2_subdev_format format = {
> - .which = V4L2_SUBDEV_FORMAT_TRY,
> - };
> - u32 mbus_code;
> - int ret;
> - bool rlp_dma_direct_dump = false;
> + unsigned int i;
>
> if (f->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
> return -EINVAL;
>
> - /* Step 1: find a RAW format that is supported */
> - for (i = 0; i < isc->num_user_formats; i++) {
> - if (ISC_IS_FORMAT_RAW(isc->user_formats[i]->mbus_code)) {
> - sd_fmt = isc->user_formats[i];
> + isc->try_config.fourcc = isc->user_formats[0]->fourcc;
> +
> + /* find if the format requested is supported */
> + for (i = 0; i < isc->controller_formats_size; i++)
> + if (isc->controller_formats[i].fourcc == pixfmt->pixelformat) {
> + isc->try_config.fourcc = pixfmt->pixelformat;
> break;
> }
> - }
> - /* Step 2: We can continue with this RAW format, or we can look
> - * for better: maybe sensor supports directly what we need.
> - */
> - direct_fmt = find_format_by_fourcc(isc, pixfmt->pixelformat);
> -
> - /* Step 3: We have both. We decide given the module parameter which
> - * one to use.
> - */
> - if (direct_fmt && sd_fmt && sensor_preferred)
> - sd_fmt = direct_fmt;
> -
> - /* Step 4: we do not have RAW but we have a direct format. Use it. */
> - if (direct_fmt && !sd_fmt)
> - sd_fmt = direct_fmt;
> -
> - /* Step 5: if we are using a direct format, we need to package
> - * everything as 8 bit data and just dump it
> - */
> - if (sd_fmt == direct_fmt)
> - rlp_dma_direct_dump = true;
> -
> - /* Step 6: We have no format. This can happen if the userspace
> - * requests some weird/invalid format.
> - * In this case, default to whatever we have
> - */
> - if (!sd_fmt && !direct_fmt) {
> - sd_fmt = isc->user_formats[isc->num_user_formats - 1];
> - v4l2_dbg(1, debug, &isc->v4l2_dev,
> - "Sensor not supporting %.4s, using %.4s\n",
> - (char *)&pixfmt->pixelformat, (char *)&sd_fmt->fourcc);
> - }
> -
> - if (!sd_fmt) {
> - ret = -EINVAL;
> - goto isc_try_fmt_err;
> - }
> -
> - /* Step 7: Print out what we decided for debugging */
> - v4l2_dbg(1, debug, &isc->v4l2_dev,
> - "Preferring to have sensor using format %.4s\n",
> - (char *)&sd_fmt->fourcc);
> -
> - /* Step 8: at this moment we decided which format the subdev will use */
> - isc->try_config.sd_format = sd_fmt;
> -
> - /* Limit to Atmel ISC hardware capabilities */
> - if (pixfmt->width > isc->max_width)
> - pixfmt->width = isc->max_width;
> - if (pixfmt->height > isc->max_height)
> - pixfmt->height = isc->max_height;
> -
> - /*
> - * The mbus format is the one the subdev outputs.
> - * The pixels will be transferred in this format Sensor -> ISC
> - */
> - mbus_code = sd_fmt->mbus_code;
> -
> - /*
> - * Validate formats. If the required format is not OK, default to raw.
> - */
> -
> - isc->try_config.fourcc = pixfmt->pixelformat;
> -
> - if (isc_try_validate_formats(isc)) {
> - pixfmt->pixelformat = isc->try_config.fourcc = sd_fmt->fourcc;
> - /* Re-try to validate the new format */
> - ret = isc_try_validate_formats(isc);
> - if (ret)
> - goto isc_try_fmt_err;
> - }
> -
> - ret = isc_try_configure_rlp_dma(isc, rlp_dma_direct_dump);
> - if (ret)
> - goto isc_try_fmt_err;
> -
> - ret = isc_try_configure_pipeline(isc);
> - if (ret)
> - goto isc_try_fmt_err;
>
> - /* Obtain frame sizes if possible to have crop requirements ready */
> - isc_try_fse(isc, &pad_state);
> -
> - v4l2_fill_mbus_format(&format.format, pixfmt, mbus_code);
> - ret = v4l2_subdev_call(isc->current_subdev->sd, pad, set_fmt,
> - &pad_state, &format);
> - if (ret < 0)
> - goto isc_try_fmt_subdev_err;
> + /* If we did not find the requested format, we will fallback here */
> + pixfmt->pixelformat = isc->try_config.fourcc;
> + pixfmt->colorspace = V4L2_COLORSPACE_SRGB;
> + pixfmt->field = V4L2_FIELD_NONE;
>
> - v4l2_fill_pix_format(pixfmt, &format.format);
> + isc_try_configure_rlp_dma(isc, false);
>
> /* Limit to Atmel ISC hardware capabilities */
> - if (pixfmt->width > isc->max_width)
> - pixfmt->width = isc->max_width;
> - if (pixfmt->height > isc->max_height)
> - pixfmt->height = isc->max_height;
> + v4l_bound_align_image(&pixfmt->width, 16, isc->max_width, 0,
> + &pixfmt->height, 16, isc->max_height, 0, 0);
>
> pixfmt->field = V4L2_FIELD_NONE;
> pixfmt->bytesperline = (pixfmt->width * isc->try_config.bpp_v4l2) >> 3;
> pixfmt->sizeimage = ((pixfmt->width * isc->try_config.bpp) >> 3) *
> pixfmt->height;
>
> - if (code)
> - *code = mbus_code;
> + isc->try_fmt = *f;
>
> return 0;
> +}
>
> -isc_try_fmt_err:
> - v4l2_err(&isc->v4l2_dev, "Could not find any possible format for a working pipeline\n");
> -isc_try_fmt_subdev_err:
> - memset(&isc->try_config, 0, sizeof(isc->try_config));
> +static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
> +{
> + isc_try_fmt(isc, f);
>
> - return ret;
> + /* make the try configuration active */
> + isc->config = isc->try_config;
> + isc->fmt = isc->try_fmt;
> +
> + v4l2_dbg(1, debug, &isc->v4l2_dev, "ISC set_fmt to %.4s @%dx%d\n",
> + (char *)&f->fmt.pix.pixelformat,
> + f->fmt.pix.width, f->fmt.pix.height);
> +
> + return 0;
> }
>
> -static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
> +static int isc_validate(struct isc_device *isc)
> {
> + int ret;
> + int i;
> + struct isc_format *sd_fmt = NULL;
> + struct v4l2_pix_format *pixfmt = &isc->fmt.fmt.pix;
> struct v4l2_subdev_format format = {
> .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + .pad = isc->remote_pad,
> + };
> + struct v4l2_subdev_pad_config pad_cfg = {};
> + struct v4l2_subdev_state pad_state = {
> + .pads = &pad_cfg,
> };
> - u32 mbus_code = 0;
> - int ret;
>
> - ret = isc_try_fmt(isc, f, &mbus_code);
> + /* Get current format from subdev */
> + ret = v4l2_subdev_call(isc->current_subdev->sd, pad, get_fmt, NULL,
> + &format);
Ah! Haven't you just said we don't care anymore about the subdev
format ? :)
Kidding, I might got a bit lost in the logic, but if I look at your
above isc_try_validate_formats() it seems like an ideal candidate for
.link_validate() media_entity operation.
Just to make sure we're on the same page, here's how it should ideally
look like:
- set format does care about subdev format. It only checks that the
format required from the user is one of the ISC supported one. Ie.
no v4l2_subdev_call()
- as s_stream time your top driver calls media_pipeline_start()
- media_pipeline_start() walks all the entities in the pipeline and
validates the format of connected pads. To validate formats the
__media_pipeline_start() functions calls link_validate() on each
entity. You should in your driver set
static const struct media_entity_operations your_media_entity_ops = {
.link_validate = v4l2_subdev_link_validate,
};
if you want to use the default link validation procedure, or set the
callback to your custom validation function, which can behave more
or less like isc_try_validate_formats()
Does it match your understanding too ?
Thanks
j
> if (ret)
> return ret;
>
> - v4l2_fill_mbus_format(&format.format, &f->fmt.pix, mbus_code);
> - ret = v4l2_subdev_call(isc->current_subdev->sd, pad,
> - set_fmt, NULL, &format);
> - if (ret < 0)
> - return ret;
> + /* Identify the subdev's format configuration */
> + for (i = 0; i < isc->num_user_formats; i++)
> + if (isc->user_formats[i]->mbus_code == format.format.code) {
> + sd_fmt = isc->user_formats[i];
> + break;
> + }
> +
> + /* Check if the format is not supported */
> + if (!sd_fmt) {
> + v4l2_err(&isc->v4l2_dev,
> + "Current subdevice is streaming a media bus code that is not supported 0x%x\n",
> + format.format.code);
> + return -EPIPE;
> + }
> +
> + /* At this moment we know which format the subdev will use */
> + isc->try_config.sd_format = sd_fmt;
> +
> + /* If the sensor is not RAW, we can only do a direct dump */
> + if (!ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code))
> + isc_try_configure_rlp_dma(isc, true);
>
> /* Limit to Atmel ISC hardware capabilities */
> - if (f->fmt.pix.width > isc->max_width)
> - f->fmt.pix.width = isc->max_width;
> - if (f->fmt.pix.height > isc->max_height)
> - f->fmt.pix.height = isc->max_height;
> + v4l_bound_align_image(&format.format.width, 16, isc->max_width, 0,
> + &format.format.height, 16, isc->max_height, 0, 0);
>
> - isc->fmt = *f;
> + /* Check if the frame size is the same. Otherwise we may overflow */
> + if (pixfmt->height != format.format.height ||
> + pixfmt->width != format.format.width) {
> + v4l2_err(&isc->v4l2_dev,
> + "ISC not configured with the proper frame size: %dx%d\n",
> + format.format.width, format.format.height);
> + return -EPIPE;
> + }
>
> + v4l2_dbg(1, debug, &isc->v4l2_dev,
> + "Identified subdev using format %.4s with %dx%d %d bpp\n",
> + (char *)&sd_fmt->fourcc, pixfmt->width, pixfmt->height,
> + isc->try_config.bpp);
> +
> + /* Reset and restart AWB if the subdevice changed the format */
> if (isc->try_config.sd_format && isc->config.sd_format &&
> isc->try_config.sd_format != isc->config.sd_format) {
> isc->ctrls.hist_stat = HIST_INIT;
> isc_reset_awb_ctrls(isc);
> isc_update_v4l2_ctrls(isc);
> }
> - /* make the try configuration active */
> +
> + /* Validate formats */
> + ret = isc_try_validate_formats(isc);
> + if (ret)
> + return ret;
> +
> + /* Obtain frame sizes if possible to have crop requirements ready */
> + isc_try_fse(isc, &pad_state);
> +
> + /* Configure ISC pipeline for the config */
> + ret = isc_try_configure_pipeline(isc);
> + if (ret)
> + return ret;
> +
> isc->config = isc->try_config;
>
> v4l2_dbg(1, debug, &isc->v4l2_dev, "New ISC configuration in place\n");
> @@ -1064,6 +1030,19 @@ static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
> return 0;
> }
>
> +static int isc_streamon(struct file *file, void *priv, enum v4l2_buf_type bt)
> +{
> + struct isc_device *isc = video_drvdata(file);
> + int ret;
> +
> + ret = isc_validate(isc);
> +
> + if (ret)
> + return ret;
> +
> + return vb2_ioctl_streamon(file, priv, bt);
> +}
> +
> static int isc_s_fmt_vid_cap(struct file *file, void *priv,
> struct v4l2_format *f)
> {
> @@ -1080,7 +1059,7 @@ static int isc_try_fmt_vid_cap(struct file *file, void *priv,
> {
> struct isc_device *isc = video_drvdata(file);
>
> - return isc_try_fmt(isc, f, NULL);
> + return isc_try_fmt(isc, f);
> }
>
> static int isc_enum_input(struct file *file, void *priv,
> @@ -1176,7 +1155,7 @@ static const struct v4l2_ioctl_ops isc_ioctl_ops = {
> .vidioc_dqbuf = vb2_ioctl_dqbuf,
> .vidioc_create_bufs = vb2_ioctl_create_bufs,
> .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> - .vidioc_streamon = vb2_ioctl_streamon,
> + .vidioc_streamon = isc_streamon,
> .vidioc_streamoff = vb2_ioctl_streamoff,
>
> .vidioc_g_parm = isc_g_parm,
> @@ -1879,7 +1858,7 @@ static int isc_set_default_fmt(struct isc_device *isc)
> };
> int ret;
>
> - ret = isc_try_fmt(isc, &f, NULL);
> + ret = isc_try_fmt(isc, &f);
> if (ret)
> return ret;
>
> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
> index c2cb805faff3..7081698adddd 100644
> --- a/drivers/media/platform/atmel/atmel-isc.h
> +++ b/drivers/media/platform/atmel/atmel-isc.h
> @@ -297,6 +297,7 @@ struct isc_device {
> struct completion comp;
>
> struct v4l2_format fmt;
> + struct v4l2_format try_fmt;
> struct isc_format **user_formats;
> unsigned int num_user_formats;
>
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 23/23] media: atmel: atmel-isc: change format propagation to subdev into only verification
@ 2022-01-12 9:21 ` Jacopo Mondi
0 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-12 9:21 UTC (permalink / raw)
To: Eugen Hristev
Cc: devicetree, linux-kernel, robh+dt, sakari.ailus,
laurent.pinchart, linux-arm-kernel, linux-media
Hi Eugen
On Mon, Dec 13, 2021 at 03:49:40PM +0200, Eugen Hristev wrote:
> As a top MC video driver, the atmel-isc should not propagate the format to the
> subdevice.
> It should rather check at streamon() time if the subdev is properly configured
> with a compatible format.
> Removed the whole format finding logic, and reworked the format verification
> at streamon time, such that the ISC will return an error if the subdevice
> is not properly configured.
> With this being done, the module parameter 'sensor_prefered' makes no sense
> anymore. The ISC should not decide which format the sensor is using. The
> ISC should only cope with the situation and inform userspace if the streaming
> is possible in the current configuration.
Sounds great!
>
> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> ---
> Changes in v3:
> - clamp to maximum resolution once the frame size from the subdev is found
>
> drivers/media/platform/atmel/atmel-isc-base.c | 271 ++++++++----------
> drivers/media/platform/atmel/atmel-isc.h | 1 +
> 2 files changed, 126 insertions(+), 146 deletions(-)
>
> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> index 31c8e3029eee..00c8c9588a78 100644
> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> @@ -36,11 +36,6 @@ static unsigned int debug;
> module_param(debug, int, 0644);
> MODULE_PARM_DESC(debug, "debug level (0-2)");
>
> -static unsigned int sensor_preferred = 1;
> -module_param(sensor_preferred, uint, 0644);
> -MODULE_PARM_DESC(sensor_preferred,
> - "Sensor is preferred to output the specified format (1-on 0-off), default 1");
> -
> #define ISC_IS_FORMAT_RAW(mbus_code) \
> (((mbus_code) & 0xf000) == 0x3000)
>
> @@ -532,7 +527,7 @@ static int isc_enum_fmt_vid_cap(struct file *file, void *priv,
> * convert it to any of the formats that we usually can with a
> * RAW sensor. Thus, do not advertise them.
> */
> - if (!isc->config.sd_format ||
> + if (isc->config.sd_format &&
Is this change intentional ?
> !ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
> return -EINVAL;
>
> @@ -621,20 +616,30 @@ static int isc_try_validate_formats(struct isc_device *isc)
> break;
> default:
> /* any other different formats are not supported */
> + v4l2_err(&isc->v4l2_dev, "Requested unsupported format.\n");
> ret = -EINVAL;
> }
> v4l2_dbg(1, debug, &isc->v4l2_dev,
> "Format validation, requested rgb=%u, yuv=%u, grey=%u, bayer=%u\n",
> rgb, yuv, grey, bayer);
Would it make sense to move this before the switch so that the
error messages, if any, appear later ?
>
> - /* we cannot output RAW if we do not receive RAW */
> - if ((bayer) && !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code))
> + if ((bayer) &&
> + !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code)) {
> + v4l2_err(&isc->v4l2_dev, "Cannot output RAW if we do not receive RAW.\n");
> return -EINVAL;
> + }
>
> - /* we cannot output GREY if we do not receive RAW/GREY */
> if (grey && !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code) &&
> - !ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code))
> + !ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) {
> + v4l2_err(&isc->v4l2_dev, "Cannot output GREY if we do not receive RAW/GREY.\n");
> return -EINVAL;
> + }
> +
> + if ((rgb || bayer || yuv) &&
> + ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) {
> + v4l2_err(&isc->v4l2_dev, "Cannot convert GREY to another format.\n");
> + return -EINVAL;
> + }
>
> return ret;
> }
> @@ -862,7 +867,7 @@ static void isc_try_fse(struct isc_device *isc,
> * If we do not know yet which format the subdev is using, we cannot
> * do anything.
> */
> - if (!isc->try_config.sd_format)
> + if (!isc->config.sd_format)
> return;
>
> fse.code = isc->try_config.sd_format->mbus_code;
> @@ -883,180 +888,141 @@ static void isc_try_fse(struct isc_device *isc,
> }
> }
>
> -static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f,
> - u32 *code)
> +static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f)
> {
> - int i;
> - struct isc_format *sd_fmt = NULL, *direct_fmt = NULL;
> struct v4l2_pix_format *pixfmt = &f->fmt.pix;
> - struct v4l2_subdev_pad_config pad_cfg = {};
> - struct v4l2_subdev_state pad_state = {
> - .pads = &pad_cfg
> - };
> - struct v4l2_subdev_format format = {
> - .which = V4L2_SUBDEV_FORMAT_TRY,
> - };
> - u32 mbus_code;
> - int ret;
> - bool rlp_dma_direct_dump = false;
> + unsigned int i;
>
> if (f->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
> return -EINVAL;
>
> - /* Step 1: find a RAW format that is supported */
> - for (i = 0; i < isc->num_user_formats; i++) {
> - if (ISC_IS_FORMAT_RAW(isc->user_formats[i]->mbus_code)) {
> - sd_fmt = isc->user_formats[i];
> + isc->try_config.fourcc = isc->user_formats[0]->fourcc;
> +
> + /* find if the format requested is supported */
> + for (i = 0; i < isc->controller_formats_size; i++)
> + if (isc->controller_formats[i].fourcc == pixfmt->pixelformat) {
> + isc->try_config.fourcc = pixfmt->pixelformat;
> break;
> }
> - }
> - /* Step 2: We can continue with this RAW format, or we can look
> - * for better: maybe sensor supports directly what we need.
> - */
> - direct_fmt = find_format_by_fourcc(isc, pixfmt->pixelformat);
> -
> - /* Step 3: We have both. We decide given the module parameter which
> - * one to use.
> - */
> - if (direct_fmt && sd_fmt && sensor_preferred)
> - sd_fmt = direct_fmt;
> -
> - /* Step 4: we do not have RAW but we have a direct format. Use it. */
> - if (direct_fmt && !sd_fmt)
> - sd_fmt = direct_fmt;
> -
> - /* Step 5: if we are using a direct format, we need to package
> - * everything as 8 bit data and just dump it
> - */
> - if (sd_fmt == direct_fmt)
> - rlp_dma_direct_dump = true;
> -
> - /* Step 6: We have no format. This can happen if the userspace
> - * requests some weird/invalid format.
> - * In this case, default to whatever we have
> - */
> - if (!sd_fmt && !direct_fmt) {
> - sd_fmt = isc->user_formats[isc->num_user_formats - 1];
> - v4l2_dbg(1, debug, &isc->v4l2_dev,
> - "Sensor not supporting %.4s, using %.4s\n",
> - (char *)&pixfmt->pixelformat, (char *)&sd_fmt->fourcc);
> - }
> -
> - if (!sd_fmt) {
> - ret = -EINVAL;
> - goto isc_try_fmt_err;
> - }
> -
> - /* Step 7: Print out what we decided for debugging */
> - v4l2_dbg(1, debug, &isc->v4l2_dev,
> - "Preferring to have sensor using format %.4s\n",
> - (char *)&sd_fmt->fourcc);
> -
> - /* Step 8: at this moment we decided which format the subdev will use */
> - isc->try_config.sd_format = sd_fmt;
> -
> - /* Limit to Atmel ISC hardware capabilities */
> - if (pixfmt->width > isc->max_width)
> - pixfmt->width = isc->max_width;
> - if (pixfmt->height > isc->max_height)
> - pixfmt->height = isc->max_height;
> -
> - /*
> - * The mbus format is the one the subdev outputs.
> - * The pixels will be transferred in this format Sensor -> ISC
> - */
> - mbus_code = sd_fmt->mbus_code;
> -
> - /*
> - * Validate formats. If the required format is not OK, default to raw.
> - */
> -
> - isc->try_config.fourcc = pixfmt->pixelformat;
> -
> - if (isc_try_validate_formats(isc)) {
> - pixfmt->pixelformat = isc->try_config.fourcc = sd_fmt->fourcc;
> - /* Re-try to validate the new format */
> - ret = isc_try_validate_formats(isc);
> - if (ret)
> - goto isc_try_fmt_err;
> - }
> -
> - ret = isc_try_configure_rlp_dma(isc, rlp_dma_direct_dump);
> - if (ret)
> - goto isc_try_fmt_err;
> -
> - ret = isc_try_configure_pipeline(isc);
> - if (ret)
> - goto isc_try_fmt_err;
>
> - /* Obtain frame sizes if possible to have crop requirements ready */
> - isc_try_fse(isc, &pad_state);
> -
> - v4l2_fill_mbus_format(&format.format, pixfmt, mbus_code);
> - ret = v4l2_subdev_call(isc->current_subdev->sd, pad, set_fmt,
> - &pad_state, &format);
> - if (ret < 0)
> - goto isc_try_fmt_subdev_err;
> + /* If we did not find the requested format, we will fallback here */
> + pixfmt->pixelformat = isc->try_config.fourcc;
> + pixfmt->colorspace = V4L2_COLORSPACE_SRGB;
> + pixfmt->field = V4L2_FIELD_NONE;
>
> - v4l2_fill_pix_format(pixfmt, &format.format);
> + isc_try_configure_rlp_dma(isc, false);
>
> /* Limit to Atmel ISC hardware capabilities */
> - if (pixfmt->width > isc->max_width)
> - pixfmt->width = isc->max_width;
> - if (pixfmt->height > isc->max_height)
> - pixfmt->height = isc->max_height;
> + v4l_bound_align_image(&pixfmt->width, 16, isc->max_width, 0,
> + &pixfmt->height, 16, isc->max_height, 0, 0);
>
> pixfmt->field = V4L2_FIELD_NONE;
> pixfmt->bytesperline = (pixfmt->width * isc->try_config.bpp_v4l2) >> 3;
> pixfmt->sizeimage = ((pixfmt->width * isc->try_config.bpp) >> 3) *
> pixfmt->height;
>
> - if (code)
> - *code = mbus_code;
> + isc->try_fmt = *f;
>
> return 0;
> +}
>
> -isc_try_fmt_err:
> - v4l2_err(&isc->v4l2_dev, "Could not find any possible format for a working pipeline\n");
> -isc_try_fmt_subdev_err:
> - memset(&isc->try_config, 0, sizeof(isc->try_config));
> +static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
> +{
> + isc_try_fmt(isc, f);
>
> - return ret;
> + /* make the try configuration active */
> + isc->config = isc->try_config;
> + isc->fmt = isc->try_fmt;
> +
> + v4l2_dbg(1, debug, &isc->v4l2_dev, "ISC set_fmt to %.4s @%dx%d\n",
> + (char *)&f->fmt.pix.pixelformat,
> + f->fmt.pix.width, f->fmt.pix.height);
> +
> + return 0;
> }
>
> -static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
> +static int isc_validate(struct isc_device *isc)
> {
> + int ret;
> + int i;
> + struct isc_format *sd_fmt = NULL;
> + struct v4l2_pix_format *pixfmt = &isc->fmt.fmt.pix;
> struct v4l2_subdev_format format = {
> .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + .pad = isc->remote_pad,
> + };
> + struct v4l2_subdev_pad_config pad_cfg = {};
> + struct v4l2_subdev_state pad_state = {
> + .pads = &pad_cfg,
> };
> - u32 mbus_code = 0;
> - int ret;
>
> - ret = isc_try_fmt(isc, f, &mbus_code);
> + /* Get current format from subdev */
> + ret = v4l2_subdev_call(isc->current_subdev->sd, pad, get_fmt, NULL,
> + &format);
Ah! Haven't you just said we don't care anymore about the subdev
format ? :)
Kidding, I might got a bit lost in the logic, but if I look at your
above isc_try_validate_formats() it seems like an ideal candidate for
.link_validate() media_entity operation.
Just to make sure we're on the same page, here's how it should ideally
look like:
- set format does care about subdev format. It only checks that the
format required from the user is one of the ISC supported one. Ie.
no v4l2_subdev_call()
- as s_stream time your top driver calls media_pipeline_start()
- media_pipeline_start() walks all the entities in the pipeline and
validates the format of connected pads. To validate formats the
__media_pipeline_start() functions calls link_validate() on each
entity. You should in your driver set
static const struct media_entity_operations your_media_entity_ops = {
.link_validate = v4l2_subdev_link_validate,
};
if you want to use the default link validation procedure, or set the
callback to your custom validation function, which can behave more
or less like isc_try_validate_formats()
Does it match your understanding too ?
Thanks
j
> if (ret)
> return ret;
>
> - v4l2_fill_mbus_format(&format.format, &f->fmt.pix, mbus_code);
> - ret = v4l2_subdev_call(isc->current_subdev->sd, pad,
> - set_fmt, NULL, &format);
> - if (ret < 0)
> - return ret;
> + /* Identify the subdev's format configuration */
> + for (i = 0; i < isc->num_user_formats; i++)
> + if (isc->user_formats[i]->mbus_code == format.format.code) {
> + sd_fmt = isc->user_formats[i];
> + break;
> + }
> +
> + /* Check if the format is not supported */
> + if (!sd_fmt) {
> + v4l2_err(&isc->v4l2_dev,
> + "Current subdevice is streaming a media bus code that is not supported 0x%x\n",
> + format.format.code);
> + return -EPIPE;
> + }
> +
> + /* At this moment we know which format the subdev will use */
> + isc->try_config.sd_format = sd_fmt;
> +
> + /* If the sensor is not RAW, we can only do a direct dump */
> + if (!ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code))
> + isc_try_configure_rlp_dma(isc, true);
>
> /* Limit to Atmel ISC hardware capabilities */
> - if (f->fmt.pix.width > isc->max_width)
> - f->fmt.pix.width = isc->max_width;
> - if (f->fmt.pix.height > isc->max_height)
> - f->fmt.pix.height = isc->max_height;
> + v4l_bound_align_image(&format.format.width, 16, isc->max_width, 0,
> + &format.format.height, 16, isc->max_height, 0, 0);
>
> - isc->fmt = *f;
> + /* Check if the frame size is the same. Otherwise we may overflow */
> + if (pixfmt->height != format.format.height ||
> + pixfmt->width != format.format.width) {
> + v4l2_err(&isc->v4l2_dev,
> + "ISC not configured with the proper frame size: %dx%d\n",
> + format.format.width, format.format.height);
> + return -EPIPE;
> + }
>
> + v4l2_dbg(1, debug, &isc->v4l2_dev,
> + "Identified subdev using format %.4s with %dx%d %d bpp\n",
> + (char *)&sd_fmt->fourcc, pixfmt->width, pixfmt->height,
> + isc->try_config.bpp);
> +
> + /* Reset and restart AWB if the subdevice changed the format */
> if (isc->try_config.sd_format && isc->config.sd_format &&
> isc->try_config.sd_format != isc->config.sd_format) {
> isc->ctrls.hist_stat = HIST_INIT;
> isc_reset_awb_ctrls(isc);
> isc_update_v4l2_ctrls(isc);
> }
> - /* make the try configuration active */
> +
> + /* Validate formats */
> + ret = isc_try_validate_formats(isc);
> + if (ret)
> + return ret;
> +
> + /* Obtain frame sizes if possible to have crop requirements ready */
> + isc_try_fse(isc, &pad_state);
> +
> + /* Configure ISC pipeline for the config */
> + ret = isc_try_configure_pipeline(isc);
> + if (ret)
> + return ret;
> +
> isc->config = isc->try_config;
>
> v4l2_dbg(1, debug, &isc->v4l2_dev, "New ISC configuration in place\n");
> @@ -1064,6 +1030,19 @@ static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
> return 0;
> }
>
> +static int isc_streamon(struct file *file, void *priv, enum v4l2_buf_type bt)
> +{
> + struct isc_device *isc = video_drvdata(file);
> + int ret;
> +
> + ret = isc_validate(isc);
> +
> + if (ret)
> + return ret;
> +
> + return vb2_ioctl_streamon(file, priv, bt);
> +}
> +
> static int isc_s_fmt_vid_cap(struct file *file, void *priv,
> struct v4l2_format *f)
> {
> @@ -1080,7 +1059,7 @@ static int isc_try_fmt_vid_cap(struct file *file, void *priv,
> {
> struct isc_device *isc = video_drvdata(file);
>
> - return isc_try_fmt(isc, f, NULL);
> + return isc_try_fmt(isc, f);
> }
>
> static int isc_enum_input(struct file *file, void *priv,
> @@ -1176,7 +1155,7 @@ static const struct v4l2_ioctl_ops isc_ioctl_ops = {
> .vidioc_dqbuf = vb2_ioctl_dqbuf,
> .vidioc_create_bufs = vb2_ioctl_create_bufs,
> .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> - .vidioc_streamon = vb2_ioctl_streamon,
> + .vidioc_streamon = isc_streamon,
> .vidioc_streamoff = vb2_ioctl_streamoff,
>
> .vidioc_g_parm = isc_g_parm,
> @@ -1879,7 +1858,7 @@ static int isc_set_default_fmt(struct isc_device *isc)
> };
> int ret;
>
> - ret = isc_try_fmt(isc, &f, NULL);
> + ret = isc_try_fmt(isc, &f);
> if (ret)
> return ret;
>
> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
> index c2cb805faff3..7081698adddd 100644
> --- a/drivers/media/platform/atmel/atmel-isc.h
> +++ b/drivers/media/platform/atmel/atmel-isc.h
> @@ -297,6 +297,7 @@ struct isc_device {
> struct completion comp;
>
> struct v4l2_format fmt;
> + struct v4l2_format try_fmt;
> struct isc_format **user_formats;
> unsigned int num_user_formats;
>
> --
> 2.25.1
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 07/23] media: atmel: atmel-isc-base: use streaming status when queueing buffers
2022-01-11 16:38 ` Eugen.Hristev
@ 2022-01-12 9:52 ` Hans Verkuil
-1 siblings, 0 replies; 94+ messages in thread
From: Hans Verkuil @ 2022-01-12 9:52 UTC (permalink / raw)
To: Eugen.Hristev, jacopo, laurent.pinchart
Cc: linux-media, robh+dt, sakari.ailus, linux-kernel,
linux-arm-kernel, Nicolas.Ferre
On 11/01/2022 17:38, Eugen.Hristev@microchip.com wrote:
> On 1/11/22 5:53 PM, Hans Verkuil wrote:
>> On 13/12/2021 14:49, Eugen Hristev wrote:
>>> During experiments with libcamera, it looks like vb2_is_streaming returns
>>> true before our start streaming is called.
>>> Order of operations is streamon -> queue -> start_streaming
>>> ISC would have started the DMA immediately when a buffer is being added
>>> to the vbqueue if the queue is streaming.
>>> It is more safe to start the DMA after the start streaming of the driver is
>>> called.
>>> Thus, even if vb2queue is streaming, add the buffer to the dma queue of the
>>> driver instead of actually starting the DMA process, if the start streaming
>>> has not been called yet.
>>>
>>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>>> ---
>>> drivers/media/platform/atmel/atmel-isc-base.c | 17 ++++++++++-------
>>> 1 file changed, 10 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
>>> index 26a6090f056c..e6c9071c04f0 100644
>>> --- a/drivers/media/platform/atmel/atmel-isc-base.c
>>> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
>>> @@ -441,12 +441,14 @@ static void isc_buffer_queue(struct vb2_buffer *vb)
>>> unsigned long flags;
>>>
>>> spin_lock_irqsave(&isc->dma_queue_lock, flags);
>>> - if (!isc->cur_frm && list_empty(&isc->dma_queue) &&
>>> - vb2_is_streaming(vb->vb2_queue)) {
>>> +
>>> + if (!isc->cur_frm && list_empty(&isc->dma_queue) && !isc->stop) {
>>> isc->cur_frm = buf;
>>> isc_start_dma(isc);
>>> - } else
>>> + } else {
>>> list_add_tail(&buf->list, &isc->dma_queue);
>>> + }
>>> +
>>> spin_unlock_irqrestore(&isc->dma_queue_lock, flags);
>>> }
>>
>> Both the old and new code doesn't make a lot of sense.
>
> Hello Hans,
>
> I debated this a bit with Laurent when I initially wrote the patch.
>
> The problem with the current code, or the current way things are
> working, is the fact that vb2_is_streaming returns true from the moment
> of streamon.
> The DMA engine of the ISC is being started at the moment of
> start_streaming, and if buffers are being sent earlier/later, they are
> just added to the queue and DMA engine is restarted if it finished
> previously.
> However if frames are added beforehand, like, first streamon, then add
beforehand -> afterwards?
> some frames, and then later call the start_streaming, ISC will crash.
> (as I said, DMA engine is prepared at start_streaming).
> To avoid this, I changed the vb2_is_streaming calls to keeping an
> internal state of the streaming.
> Do you think I should try another different approach to solve this ?
> Perhaps do not prepare the ISC dma engine at start_streaming, but at
> another place ?
>
> Laurent do you remember the talk we had about this problem a while back ?
>
>>
>> buf_queue is only called by vb2 if start_streaming has already been called or is
>> about to be called.
>
> That's a huge difference for ISC driver as it is today. start_streaming
> being called afterwards, means the ISC DMA engine is not started (not
> started streaming), hence a crash.
The start_streaming op is called when streamon is called and if there are
sufficient buffers (>= q->min_buffers_needed). If there are insufficient
buffers at streamon time, then start_streaming is only called when the number
of queued buffers reaches the minimum.
vb2 has three different state checks that you can use:
vb2_is_busy() is true if buffers are allocated. Typically this means that you
can no longer do format changes or anything that might change the buffer
sizes or layout.
vb2_is_streaming() is true if streamon was called. If this is true, then vb2_is_busy()
is also true.
And finally vb2_start_streaming_called() is true if the start_streaming op
was called (i.e. DMA is in progress). If this is true, then vb2_is_streaming is
also true.
Basically vb2_is_streaming() reports the streaming state that userspace sees,
while vb2_start_streaming_called() reports the internal DMA streaming state.
The confusion is probably that you were using vb2_is_streaming instead of
vb2_start_streaming_called.
>
>>
>> Typically all that the buf_queue op does is to call list_add_tail(&buf->list, &isc->dma_queue);
>> inside the spinlock.
>>
>>>
>>> @@ -1014,7 +1016,7 @@ static int isc_s_fmt_vid_cap(struct file *file, void *priv,
>>> {
>>> struct isc_device *isc = video_drvdata(file);
>>>
>>> - if (vb2_is_streaming(&isc->vb2_vidq))
>>> + if (!isc->stop)
>>
>> This is weird as well. Normally this calls vb2_is_busy to check if the
>> queue is busy (that really means that buffers are already allocated, so
>> changing the format isn't allowed anymore).
>
> You think the query to the streaming status makes no sense here hence it
> should be removed completely ?
Right. And instead check vb2_is_busy().
I actually think that this is something that would fail with v4l2-compliance.
Do you still run v4l2-compliance?
> I can do that with a prequel patch if it's the case.
Probably wise.
Regards,
Hans
>
> Thanks for reviewing,
>
> Eugen
>
>>
>>> return -EBUSY;
>>>
>>> return isc_set_fmt(isc, f);
>>> @@ -1536,7 +1538,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
>>>
>>> isc_update_awb_ctrls(isc);
>>>
>>> - if (vb2_is_streaming(&isc->vb2_vidq)) {
>>> + if (!isc->stop) {
>>
>> Ditto.
>>
>>> /*
>>> * If we are streaming, we can update profile to
>>> * have the new settings in place.
>>> @@ -1552,8 +1554,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
>>> }
>>>
>>> /* if we have autowhitebalance on, start histogram procedure */
>>> - if (ctrls->awb == ISC_WB_AUTO &&
>>> - vb2_is_streaming(&isc->vb2_vidq) &&
>>> + if (ctrls->awb == ISC_WB_AUTO && !isc->stop &&
>>> ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
>>> isc_set_histogram(isc, true);
>>>
>>> @@ -1829,6 +1830,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
>>> struct vb2_queue *q = &isc->vb2_vidq;
>>> int ret = 0;
>>>
>>> + isc->stop = true;
>>> +
>>
>> I'm really not sure that you need the stop bool at all.
>>
>>> INIT_WORK(&isc->awb_work, isc_awb_work);
>>>
>>> ret = v4l2_device_register_subdev_nodes(&isc->v4l2_dev);
>>
>> Regards,
>>
>> Hans
>>
>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 07/23] media: atmel: atmel-isc-base: use streaming status when queueing buffers
@ 2022-01-12 9:52 ` Hans Verkuil
0 siblings, 0 replies; 94+ messages in thread
From: Hans Verkuil @ 2022-01-12 9:52 UTC (permalink / raw)
To: Eugen.Hristev, jacopo, laurent.pinchart
Cc: linux-kernel, robh+dt, sakari.ailus, linux-arm-kernel, linux-media
On 11/01/2022 17:38, Eugen.Hristev@microchip.com wrote:
> On 1/11/22 5:53 PM, Hans Verkuil wrote:
>> On 13/12/2021 14:49, Eugen Hristev wrote:
>>> During experiments with libcamera, it looks like vb2_is_streaming returns
>>> true before our start streaming is called.
>>> Order of operations is streamon -> queue -> start_streaming
>>> ISC would have started the DMA immediately when a buffer is being added
>>> to the vbqueue if the queue is streaming.
>>> It is more safe to start the DMA after the start streaming of the driver is
>>> called.
>>> Thus, even if vb2queue is streaming, add the buffer to the dma queue of the
>>> driver instead of actually starting the DMA process, if the start streaming
>>> has not been called yet.
>>>
>>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>>> ---
>>> drivers/media/platform/atmel/atmel-isc-base.c | 17 ++++++++++-------
>>> 1 file changed, 10 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
>>> index 26a6090f056c..e6c9071c04f0 100644
>>> --- a/drivers/media/platform/atmel/atmel-isc-base.c
>>> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
>>> @@ -441,12 +441,14 @@ static void isc_buffer_queue(struct vb2_buffer *vb)
>>> unsigned long flags;
>>>
>>> spin_lock_irqsave(&isc->dma_queue_lock, flags);
>>> - if (!isc->cur_frm && list_empty(&isc->dma_queue) &&
>>> - vb2_is_streaming(vb->vb2_queue)) {
>>> +
>>> + if (!isc->cur_frm && list_empty(&isc->dma_queue) && !isc->stop) {
>>> isc->cur_frm = buf;
>>> isc_start_dma(isc);
>>> - } else
>>> + } else {
>>> list_add_tail(&buf->list, &isc->dma_queue);
>>> + }
>>> +
>>> spin_unlock_irqrestore(&isc->dma_queue_lock, flags);
>>> }
>>
>> Both the old and new code doesn't make a lot of sense.
>
> Hello Hans,
>
> I debated this a bit with Laurent when I initially wrote the patch.
>
> The problem with the current code, or the current way things are
> working, is the fact that vb2_is_streaming returns true from the moment
> of streamon.
> The DMA engine of the ISC is being started at the moment of
> start_streaming, and if buffers are being sent earlier/later, they are
> just added to the queue and DMA engine is restarted if it finished
> previously.
> However if frames are added beforehand, like, first streamon, then add
beforehand -> afterwards?
> some frames, and then later call the start_streaming, ISC will crash.
> (as I said, DMA engine is prepared at start_streaming).
> To avoid this, I changed the vb2_is_streaming calls to keeping an
> internal state of the streaming.
> Do you think I should try another different approach to solve this ?
> Perhaps do not prepare the ISC dma engine at start_streaming, but at
> another place ?
>
> Laurent do you remember the talk we had about this problem a while back ?
>
>>
>> buf_queue is only called by vb2 if start_streaming has already been called or is
>> about to be called.
>
> That's a huge difference for ISC driver as it is today. start_streaming
> being called afterwards, means the ISC DMA engine is not started (not
> started streaming), hence a crash.
The start_streaming op is called when streamon is called and if there are
sufficient buffers (>= q->min_buffers_needed). If there are insufficient
buffers at streamon time, then start_streaming is only called when the number
of queued buffers reaches the minimum.
vb2 has three different state checks that you can use:
vb2_is_busy() is true if buffers are allocated. Typically this means that you
can no longer do format changes or anything that might change the buffer
sizes or layout.
vb2_is_streaming() is true if streamon was called. If this is true, then vb2_is_busy()
is also true.
And finally vb2_start_streaming_called() is true if the start_streaming op
was called (i.e. DMA is in progress). If this is true, then vb2_is_streaming is
also true.
Basically vb2_is_streaming() reports the streaming state that userspace sees,
while vb2_start_streaming_called() reports the internal DMA streaming state.
The confusion is probably that you were using vb2_is_streaming instead of
vb2_start_streaming_called.
>
>>
>> Typically all that the buf_queue op does is to call list_add_tail(&buf->list, &isc->dma_queue);
>> inside the spinlock.
>>
>>>
>>> @@ -1014,7 +1016,7 @@ static int isc_s_fmt_vid_cap(struct file *file, void *priv,
>>> {
>>> struct isc_device *isc = video_drvdata(file);
>>>
>>> - if (vb2_is_streaming(&isc->vb2_vidq))
>>> + if (!isc->stop)
>>
>> This is weird as well. Normally this calls vb2_is_busy to check if the
>> queue is busy (that really means that buffers are already allocated, so
>> changing the format isn't allowed anymore).
>
> You think the query to the streaming status makes no sense here hence it
> should be removed completely ?
Right. And instead check vb2_is_busy().
I actually think that this is something that would fail with v4l2-compliance.
Do you still run v4l2-compliance?
> I can do that with a prequel patch if it's the case.
Probably wise.
Regards,
Hans
>
> Thanks for reviewing,
>
> Eugen
>
>>
>>> return -EBUSY;
>>>
>>> return isc_set_fmt(isc, f);
>>> @@ -1536,7 +1538,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
>>>
>>> isc_update_awb_ctrls(isc);
>>>
>>> - if (vb2_is_streaming(&isc->vb2_vidq)) {
>>> + if (!isc->stop) {
>>
>> Ditto.
>>
>>> /*
>>> * If we are streaming, we can update profile to
>>> * have the new settings in place.
>>> @@ -1552,8 +1554,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
>>> }
>>>
>>> /* if we have autowhitebalance on, start histogram procedure */
>>> - if (ctrls->awb == ISC_WB_AUTO &&
>>> - vb2_is_streaming(&isc->vb2_vidq) &&
>>> + if (ctrls->awb == ISC_WB_AUTO && !isc->stop &&
>>> ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
>>> isc_set_histogram(isc, true);
>>>
>>> @@ -1829,6 +1830,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
>>> struct vb2_queue *q = &isc->vb2_vidq;
>>> int ret = 0;
>>>
>>> + isc->stop = true;
>>> +
>>
>> I'm really not sure that you need the stop bool at all.
>>
>>> INIT_WORK(&isc->awb_work, isc_awb_work);
>>>
>>> ret = v4l2_device_register_subdev_nodes(&isc->v4l2_dev);
>>
>> Regards,
>>
>> Hans
>>
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 00/23] media: atmel: atmel-isc: implement media controller
2021-12-13 13:49 ` Eugen Hristev
@ 2022-01-12 12:07 ` Hans Verkuil
-1 siblings, 0 replies; 94+ messages in thread
From: Hans Verkuil @ 2022-01-12 12:07 UTC (permalink / raw)
To: Eugen Hristev, linux-media, robh+dt, jacopo, laurent.pinchart,
sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel, nicolas.ferre
Hi Eugen,
On 13/12/2021 14:49, Eugen Hristev wrote:
> This series is the v3 of the series that attempts to support media controller
> in the atmel ISC and XISC drivers.
> This series also includes the csi2dc driver which was previously sent in a
> separate series:
> https://www.spinics.net/lists/linux-media/msg181042.html
> https://www.spinics.net/lists/linux-media/msg181044.html
> The driver now addresses comments received in latest v5 series from last year.
>
> The series includes some minor changes and fixes that improve the isc common
> code base, like removing the enum frameintervals VIDIOC, fixing bytesperline
> for planar formats, etc.
>
> Many thanks to folks from libcamera who helped a lot with understanding
> how a media controller driver should behave.
>
> Feedback is welcome !
>
> Changes in v3:
> - change in bindings, small fixes in csi2dc driver and conversion to mc
> for the isc-base.
> - removed some MAINTAINERS patches and used patterns in MAINTAINERS
>
> Changes in v2:
> - integrated many changes suggested by Jacopo in the review of the v1 series.
> - add a few new patches
Based on the current review status I think it would make sense to add
the following patches from this series to a PR:
1-6, 8-9, 11, 16, 18-22.
Are there any patches (esp. 11 and later) that require other patches
not in this list? In other words, does this make sense?
Regards,
Hans
>
>
> Eugen Hristev (23):
> MAINTAINERS: add microchip csi2dc
> dt-bindings: media: atmel: csi2dc: add bindings for microchip csi2dc
> media: atmel: introduce microchip csi2dc driver
> media: atmel: atmel-isc: split the clock code into separate source
> file
> media: atmel: atmel-isc: replace video device name with module name
> media: atmel: atmel-sama7g5-isc: fix ispck leftover
> media: atmel: atmel-isc-base: use streaming status when queueing
> buffers
> media: atmel: atmel-isc-base: remove frameintervals VIDIOC
> media: atmel: atmel-isc-base: report frame sizes as full supported
> range
> media: atmel: atmel-isc-base: implement mbus_code support in enumfmt
> media: atmel: atmel-isc-base: fix bytesperline value for planar
> formats
> media: atmel: atmel-isc: implement media controller
> ARM: dts: at91: sama7g5: add nodes for video capture
> ARM: configs: at91: sama7: add xisc and csi2dc
> ARM: multi_v7_defconfig: add atmel video pipeline modules
> media: atmel: atmel-sama5d2-isc: fix wrong mask in YUYV format check
> media: atmel: atmel-isc-base: use mutex to lock awb workqueue from
> streaming
> media: atmel: atmel-isc-base: add wb debug messages
> media: atmel: atmel-isc-base: clamp wb gain coefficients
> media: atmel: atmel-sama7g5-isc: fix UYVY input format mbus_code typo
> media: atmel: atmel-isc: add raw Bayer 8bit 10bit output formats
> media: atmel: atmel-isc: compact the controller formats list
> media: atmel: atmel-isc: change format propagation to subdev into only
> verification
>
> .../bindings/media/microchip,csi2dc.yaml | 197 +++++
> MAINTAINERS | 14 +-
> arch/arm/boot/dts/sama7g5.dtsi | 48 ++
> arch/arm/configs/multi_v7_defconfig | 3 +
> arch/arm/configs/sama7_defconfig | 2 +
> drivers/media/platform/Makefile | 1 +
> drivers/media/platform/atmel/Kconfig | 15 +
> drivers/media/platform/atmel/Makefile | 4 +-
> drivers/media/platform/atmel/atmel-isc-base.c | 790 ++++++-----------
> drivers/media/platform/atmel/atmel-isc-clk.c | 311 +++++++
> drivers/media/platform/atmel/atmel-isc-mc.c | 287 +++++++
> drivers/media/platform/atmel/atmel-isc.h | 49 ++
> .../media/platform/atmel/atmel-sama5d2-isc.c | 59 +-
> .../media/platform/atmel/atmel-sama7g5-isc.c | 72 +-
> .../media/platform/atmel/microchip-csi2dc.c | 806 ++++++++++++++++++
> 15 files changed, 2085 insertions(+), 573 deletions(-)
> create mode 100644 Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
> create mode 100644 drivers/media/platform/atmel/atmel-isc-clk.c
> create mode 100644 drivers/media/platform/atmel/atmel-isc-mc.c
> create mode 100644 drivers/media/platform/atmel/microchip-csi2dc.c
>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 00/23] media: atmel: atmel-isc: implement media controller
@ 2022-01-12 12:07 ` Hans Verkuil
0 siblings, 0 replies; 94+ messages in thread
From: Hans Verkuil @ 2022-01-12 12:07 UTC (permalink / raw)
To: Eugen Hristev, linux-media, robh+dt, jacopo, laurent.pinchart,
sakari.ailus
Cc: devicetree, linux-kernel, linux-arm-kernel
Hi Eugen,
On 13/12/2021 14:49, Eugen Hristev wrote:
> This series is the v3 of the series that attempts to support media controller
> in the atmel ISC and XISC drivers.
> This series also includes the csi2dc driver which was previously sent in a
> separate series:
> https://www.spinics.net/lists/linux-media/msg181042.html
> https://www.spinics.net/lists/linux-media/msg181044.html
> The driver now addresses comments received in latest v5 series from last year.
>
> The series includes some minor changes and fixes that improve the isc common
> code base, like removing the enum frameintervals VIDIOC, fixing bytesperline
> for planar formats, etc.
>
> Many thanks to folks from libcamera who helped a lot with understanding
> how a media controller driver should behave.
>
> Feedback is welcome !
>
> Changes in v3:
> - change in bindings, small fixes in csi2dc driver and conversion to mc
> for the isc-base.
> - removed some MAINTAINERS patches and used patterns in MAINTAINERS
>
> Changes in v2:
> - integrated many changes suggested by Jacopo in the review of the v1 series.
> - add a few new patches
Based on the current review status I think it would make sense to add
the following patches from this series to a PR:
1-6, 8-9, 11, 16, 18-22.
Are there any patches (esp. 11 and later) that require other patches
not in this list? In other words, does this make sense?
Regards,
Hans
>
>
> Eugen Hristev (23):
> MAINTAINERS: add microchip csi2dc
> dt-bindings: media: atmel: csi2dc: add bindings for microchip csi2dc
> media: atmel: introduce microchip csi2dc driver
> media: atmel: atmel-isc: split the clock code into separate source
> file
> media: atmel: atmel-isc: replace video device name with module name
> media: atmel: atmel-sama7g5-isc: fix ispck leftover
> media: atmel: atmel-isc-base: use streaming status when queueing
> buffers
> media: atmel: atmel-isc-base: remove frameintervals VIDIOC
> media: atmel: atmel-isc-base: report frame sizes as full supported
> range
> media: atmel: atmel-isc-base: implement mbus_code support in enumfmt
> media: atmel: atmel-isc-base: fix bytesperline value for planar
> formats
> media: atmel: atmel-isc: implement media controller
> ARM: dts: at91: sama7g5: add nodes for video capture
> ARM: configs: at91: sama7: add xisc and csi2dc
> ARM: multi_v7_defconfig: add atmel video pipeline modules
> media: atmel: atmel-sama5d2-isc: fix wrong mask in YUYV format check
> media: atmel: atmel-isc-base: use mutex to lock awb workqueue from
> streaming
> media: atmel: atmel-isc-base: add wb debug messages
> media: atmel: atmel-isc-base: clamp wb gain coefficients
> media: atmel: atmel-sama7g5-isc: fix UYVY input format mbus_code typo
> media: atmel: atmel-isc: add raw Bayer 8bit 10bit output formats
> media: atmel: atmel-isc: compact the controller formats list
> media: atmel: atmel-isc: change format propagation to subdev into only
> verification
>
> .../bindings/media/microchip,csi2dc.yaml | 197 +++++
> MAINTAINERS | 14 +-
> arch/arm/boot/dts/sama7g5.dtsi | 48 ++
> arch/arm/configs/multi_v7_defconfig | 3 +
> arch/arm/configs/sama7_defconfig | 2 +
> drivers/media/platform/Makefile | 1 +
> drivers/media/platform/atmel/Kconfig | 15 +
> drivers/media/platform/atmel/Makefile | 4 +-
> drivers/media/platform/atmel/atmel-isc-base.c | 790 ++++++-----------
> drivers/media/platform/atmel/atmel-isc-clk.c | 311 +++++++
> drivers/media/platform/atmel/atmel-isc-mc.c | 287 +++++++
> drivers/media/platform/atmel/atmel-isc.h | 49 ++
> .../media/platform/atmel/atmel-sama5d2-isc.c | 59 +-
> .../media/platform/atmel/atmel-sama7g5-isc.c | 72 +-
> .../media/platform/atmel/microchip-csi2dc.c | 806 ++++++++++++++++++
> 15 files changed, 2085 insertions(+), 573 deletions(-)
> create mode 100644 Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
> create mode 100644 drivers/media/platform/atmel/atmel-isc-clk.c
> create mode 100644 drivers/media/platform/atmel/atmel-isc-mc.c
> create mode 100644 drivers/media/platform/atmel/microchip-csi2dc.c
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 00/23] media: atmel: atmel-isc: implement media controller
2022-01-12 12:07 ` Hans Verkuil
@ 2022-01-12 12:46 ` Eugen.Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen.Hristev @ 2022-01-12 12:46 UTC (permalink / raw)
To: hverkuil, linux-media, jacopo
Cc: robh+dt, laurent.pinchart, sakari.ailus, devicetree,
linux-kernel, linux-arm-kernel, Nicolas.Ferre
On 1/12/22 2:07 PM, Hans Verkuil wrote:
> Hi Eugen,
>
> On 13/12/2021 14:49, Eugen Hristev wrote:
>> This series is the v3 of the series that attempts to support media controller
>> in the atmel ISC and XISC drivers.
>> This series also includes the csi2dc driver which was previously sent in a
>> separate series:
>> https://www.spinics.net/lists/linux-media/msg181042.html
>> https://www.spinics.net/lists/linux-media/msg181044.html
>> The driver now addresses comments received in latest v5 series from last year.
>>
>> The series includes some minor changes and fixes that improve the isc common
>> code base, like removing the enum frameintervals VIDIOC, fixing bytesperline
>> for planar formats, etc.
>>
>> Many thanks to folks from libcamera who helped a lot with understanding
>> how a media controller driver should behave.
>>
>> Feedback is welcome !
>>
>> Changes in v3:
>> - change in bindings, small fixes in csi2dc driver and conversion to mc
>> for the isc-base.
>> - removed some MAINTAINERS patches and used patterns in MAINTAINERS
>>
>> Changes in v2:
>> - integrated many changes suggested by Jacopo in the review of the v1 series.
>> - add a few new patches
>
> Based on the current review status I think it would make sense to add
> the following patches from this series to a PR:
>
> 1-6, 8-9, 11, 16, 18-22.
>
> Are there any patches (esp. 11 and later) that require other patches
> not in this list? In other words, does this make sense?
Hello Hans,
Yes, this sounds good . I will rework the patches that are left next week.
Then I will send a v4 of this series with only the patches that are not
yet applied. And I will rebase it on top of the media tree.
Thanks for reviewing !
Eugen
>
> Regards,
>
> Hans
>
>>
>>
>> Eugen Hristev (23):
>> MAINTAINERS: add microchip csi2dc
>> dt-bindings: media: atmel: csi2dc: add bindings for microchip csi2dc
>> media: atmel: introduce microchip csi2dc driver
>> media: atmel: atmel-isc: split the clock code into separate source
>> file
>> media: atmel: atmel-isc: replace video device name with module name
>> media: atmel: atmel-sama7g5-isc: fix ispck leftover
>> media: atmel: atmel-isc-base: use streaming status when queueing
>> buffers
>> media: atmel: atmel-isc-base: remove frameintervals VIDIOC
>> media: atmel: atmel-isc-base: report frame sizes as full supported
>> range
>> media: atmel: atmel-isc-base: implement mbus_code support in enumfmt
>> media: atmel: atmel-isc-base: fix bytesperline value for planar
>> formats
>> media: atmel: atmel-isc: implement media controller
>> ARM: dts: at91: sama7g5: add nodes for video capture
>> ARM: configs: at91: sama7: add xisc and csi2dc
>> ARM: multi_v7_defconfig: add atmel video pipeline modules
>> media: atmel: atmel-sama5d2-isc: fix wrong mask in YUYV format check
>> media: atmel: atmel-isc-base: use mutex to lock awb workqueue from
>> streaming
>> media: atmel: atmel-isc-base: add wb debug messages
>> media: atmel: atmel-isc-base: clamp wb gain coefficients
>> media: atmel: atmel-sama7g5-isc: fix UYVY input format mbus_code typo
>> media: atmel: atmel-isc: add raw Bayer 8bit 10bit output formats
>> media: atmel: atmel-isc: compact the controller formats list
>> media: atmel: atmel-isc: change format propagation to subdev into only
>> verification
>>
>> .../bindings/media/microchip,csi2dc.yaml | 197 +++++
>> MAINTAINERS | 14 +-
>> arch/arm/boot/dts/sama7g5.dtsi | 48 ++
>> arch/arm/configs/multi_v7_defconfig | 3 +
>> arch/arm/configs/sama7_defconfig | 2 +
>> drivers/media/platform/Makefile | 1 +
>> drivers/media/platform/atmel/Kconfig | 15 +
>> drivers/media/platform/atmel/Makefile | 4 +-
>> drivers/media/platform/atmel/atmel-isc-base.c | 790 ++++++-----------
>> drivers/media/platform/atmel/atmel-isc-clk.c | 311 +++++++
>> drivers/media/platform/atmel/atmel-isc-mc.c | 287 +++++++
>> drivers/media/platform/atmel/atmel-isc.h | 49 ++
>> .../media/platform/atmel/atmel-sama5d2-isc.c | 59 +-
>> .../media/platform/atmel/atmel-sama7g5-isc.c | 72 +-
>> .../media/platform/atmel/microchip-csi2dc.c | 806 ++++++++++++++++++
>> 15 files changed, 2085 insertions(+), 573 deletions(-)
>> create mode 100644 Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
>> create mode 100644 drivers/media/platform/atmel/atmel-isc-clk.c
>> create mode 100644 drivers/media/platform/atmel/atmel-isc-mc.c
>> create mode 100644 drivers/media/platform/atmel/microchip-csi2dc.c
>>
>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 00/23] media: atmel: atmel-isc: implement media controller
@ 2022-01-12 12:46 ` Eugen.Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen.Hristev @ 2022-01-12 12:46 UTC (permalink / raw)
To: hverkuil, linux-media, jacopo
Cc: devicetree, linux-kernel, robh+dt, sakari.ailus,
laurent.pinchart, linux-arm-kernel
On 1/12/22 2:07 PM, Hans Verkuil wrote:
> Hi Eugen,
>
> On 13/12/2021 14:49, Eugen Hristev wrote:
>> This series is the v3 of the series that attempts to support media controller
>> in the atmel ISC and XISC drivers.
>> This series also includes the csi2dc driver which was previously sent in a
>> separate series:
>> https://www.spinics.net/lists/linux-media/msg181042.html
>> https://www.spinics.net/lists/linux-media/msg181044.html
>> The driver now addresses comments received in latest v5 series from last year.
>>
>> The series includes some minor changes and fixes that improve the isc common
>> code base, like removing the enum frameintervals VIDIOC, fixing bytesperline
>> for planar formats, etc.
>>
>> Many thanks to folks from libcamera who helped a lot with understanding
>> how a media controller driver should behave.
>>
>> Feedback is welcome !
>>
>> Changes in v3:
>> - change in bindings, small fixes in csi2dc driver and conversion to mc
>> for the isc-base.
>> - removed some MAINTAINERS patches and used patterns in MAINTAINERS
>>
>> Changes in v2:
>> - integrated many changes suggested by Jacopo in the review of the v1 series.
>> - add a few new patches
>
> Based on the current review status I think it would make sense to add
> the following patches from this series to a PR:
>
> 1-6, 8-9, 11, 16, 18-22.
>
> Are there any patches (esp. 11 and later) that require other patches
> not in this list? In other words, does this make sense?
Hello Hans,
Yes, this sounds good . I will rework the patches that are left next week.
Then I will send a v4 of this series with only the patches that are not
yet applied. And I will rebase it on top of the media tree.
Thanks for reviewing !
Eugen
>
> Regards,
>
> Hans
>
>>
>>
>> Eugen Hristev (23):
>> MAINTAINERS: add microchip csi2dc
>> dt-bindings: media: atmel: csi2dc: add bindings for microchip csi2dc
>> media: atmel: introduce microchip csi2dc driver
>> media: atmel: atmel-isc: split the clock code into separate source
>> file
>> media: atmel: atmel-isc: replace video device name with module name
>> media: atmel: atmel-sama7g5-isc: fix ispck leftover
>> media: atmel: atmel-isc-base: use streaming status when queueing
>> buffers
>> media: atmel: atmel-isc-base: remove frameintervals VIDIOC
>> media: atmel: atmel-isc-base: report frame sizes as full supported
>> range
>> media: atmel: atmel-isc-base: implement mbus_code support in enumfmt
>> media: atmel: atmel-isc-base: fix bytesperline value for planar
>> formats
>> media: atmel: atmel-isc: implement media controller
>> ARM: dts: at91: sama7g5: add nodes for video capture
>> ARM: configs: at91: sama7: add xisc and csi2dc
>> ARM: multi_v7_defconfig: add atmel video pipeline modules
>> media: atmel: atmel-sama5d2-isc: fix wrong mask in YUYV format check
>> media: atmel: atmel-isc-base: use mutex to lock awb workqueue from
>> streaming
>> media: atmel: atmel-isc-base: add wb debug messages
>> media: atmel: atmel-isc-base: clamp wb gain coefficients
>> media: atmel: atmel-sama7g5-isc: fix UYVY input format mbus_code typo
>> media: atmel: atmel-isc: add raw Bayer 8bit 10bit output formats
>> media: atmel: atmel-isc: compact the controller formats list
>> media: atmel: atmel-isc: change format propagation to subdev into only
>> verification
>>
>> .../bindings/media/microchip,csi2dc.yaml | 197 +++++
>> MAINTAINERS | 14 +-
>> arch/arm/boot/dts/sama7g5.dtsi | 48 ++
>> arch/arm/configs/multi_v7_defconfig | 3 +
>> arch/arm/configs/sama7_defconfig | 2 +
>> drivers/media/platform/Makefile | 1 +
>> drivers/media/platform/atmel/Kconfig | 15 +
>> drivers/media/platform/atmel/Makefile | 4 +-
>> drivers/media/platform/atmel/atmel-isc-base.c | 790 ++++++-----------
>> drivers/media/platform/atmel/atmel-isc-clk.c | 311 +++++++
>> drivers/media/platform/atmel/atmel-isc-mc.c | 287 +++++++
>> drivers/media/platform/atmel/atmel-isc.h | 49 ++
>> .../media/platform/atmel/atmel-sama5d2-isc.c | 59 +-
>> .../media/platform/atmel/atmel-sama7g5-isc.c | 72 +-
>> .../media/platform/atmel/microchip-csi2dc.c | 806 ++++++++++++++++++
>> 15 files changed, 2085 insertions(+), 573 deletions(-)
>> create mode 100644 Documentation/devicetree/bindings/media/microchip,csi2dc.yaml
>> create mode 100644 drivers/media/platform/atmel/atmel-isc-clk.c
>> create mode 100644 drivers/media/platform/atmel/atmel-isc-mc.c
>> create mode 100644 drivers/media/platform/atmel/microchip-csi2dc.c
>>
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 17/23] media: atmel: atmel-isc-base: use mutex to lock awb workqueue from streaming
2022-01-12 8:58 ` Jacopo Mondi
@ 2022-01-19 15:40 ` Eugen.Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen.Hristev @ 2022-01-19 15:40 UTC (permalink / raw)
To: jacopo
Cc: linux-media, robh+dt, laurent.pinchart, sakari.ailus, devicetree,
linux-kernel, linux-arm-kernel, Nicolas.Ferre
On 1/12/22 10:58 AM, Jacopo Mondi wrote:
> Hi Eugen
>
> On Mon, Dec 13, 2021 at 03:49:34PM +0200, Eugen Hristev wrote:
>> The AWB workqueue runs in a kernel thread and needs to be synchronized
>> w.r.t. the streaming status.
>> It is possible that streaming is stopped while the AWB workq is running.
>> In this case it is likely that the check for isc->stop is done at one point
>> in time, but the AWB computations are done later, including a call to
>> isc_update_profile, which requires streaming to be started.
>> Thus , isc_update_profile will fail if during this operation sequence the
>> streaming was stopped.
>> To solve this issue, a mutex is added, that will serialize the awb work and
>> streaming stopping, with the mention that either streaming is stopped
>> completely including termination of the last frame is done, and after that
>> the AWB work can check stream status and stop; either first AWB work is
>> completed and after that the streaming can stop correctly.
>> The awb spin lock cannot be used since this spinlock is taken in the same
>> context and using it in the stop streaming will result in a recursion BUG.
>>
>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>> ---
>> drivers/media/platform/atmel/atmel-isc-base.c | 31 ++++++++++++++++---
>> drivers/media/platform/atmel/atmel-isc.h | 1 +
>> 2 files changed, 28 insertions(+), 4 deletions(-)
>>
>> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
>> index b0c3ed21f372..53cac1aac0fd 100644
>> --- a/drivers/media/platform/atmel/atmel-isc-base.c
>> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
>> @@ -401,6 +401,7 @@ static void isc_stop_streaming(struct vb2_queue *vq)
>> struct isc_buffer *buf;
>> int ret;
>>
>> + mutex_lock(&isc->awb_mutex);
>> v4l2_ctrl_activate(isc->do_wb_ctrl, false);
>>
>> isc->stop = true;
>> @@ -410,6 +411,8 @@ static void isc_stop_streaming(struct vb2_queue *vq)
>> v4l2_err(&isc->v4l2_dev,
>> "Timeout waiting for end of the capture\n");
>>
>> + mutex_unlock(&isc->awb_mutex);
>> +
>> /* Disable DMA interrupt */
>> regmap_write(isc->regmap, ISC_INTDIS, ISC_INT_DDONE);
>>
>> @@ -1416,10 +1419,6 @@ static void isc_awb_work(struct work_struct *w)
>> u32 min, max;
>> int ret;
>>
>> - /* streaming is not active anymore */
>> - if (isc->stop)
>> - return;
>> -
>> if (ctrls->hist_stat != HIST_ENABLED)
>> return;
>>
>> @@ -1470,7 +1469,24 @@ static void isc_awb_work(struct work_struct *w)
>> }
>> regmap_write(regmap, ISC_HIS_CFG + isc->offsets.his,
>> hist_id | baysel | ISC_HIS_CFG_RAR);
>
> isc_stop_streaming() calls runtime_put and here you access the hw.
Hi Jacopo,
That is correct. However the awb routine will call resume and get here
(before accessing the hardware) :
https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/atmel/atmel-isc-base.c#L1722
So I think that we are good as we are now
>
> Feels like it's safer to hold the mutex for the whole duration of the
> AWB routine ?
>
I prefer to have the critical section as little as possible. As we are
only synchronizing the 'streaming status' , only this should be under
the lock.
If you have a different opinion, let me know.
Eugen
>> +
>> + /*
>> + * We have to make sure the streaming has not stopped meanwhile.
>> + * ISC requires a frame to clock the internal profile update.
>> + * To avoid issues, lock the sequence with a mutex
>> + */
>> + mutex_lock(&isc->awb_mutex);
>> +
>> + /* streaming is not active anymore */
>> + if (isc->stop) {
>> + mutex_unlock(&isc->awb_mutex);
>> + return;
>> + };
>> +
>> isc_update_profile(isc);
>> +
>> + mutex_unlock(&isc->awb_mutex);
>> +
>> /* if awb has been disabled, we don't need to start another histogram */
>> if (ctrls->awb)
>> regmap_write(regmap, ISC_CTRLEN, ISC_CTRL_HISREQ);
>> @@ -1549,6 +1565,8 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
>>
>> isc_update_awb_ctrls(isc);
>>
>> + mutex_lock(&isc->awb_mutex);
>> +
>> if (!isc->stop) {
>> /*
>> * If we are streaming, we can update profile to
>> @@ -1563,6 +1581,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
>> */
>> v4l2_ctrl_activate(isc->do_wb_ctrl, false);
>> }
>> + mutex_unlock(&isc->awb_mutex);
>>
>> /* if we have autowhitebalance on, start histogram procedure */
>> if (ctrls->awb == ISC_WB_AUTO && !isc->stop &&
>> @@ -1754,6 +1773,7 @@ static void isc_async_unbind(struct v4l2_async_notifier *notifier,
>> {
>> struct isc_device *isc = container_of(notifier->v4l2_dev,
>> struct isc_device, v4l2_dev);
>> + mutex_destroy(&isc->awb_mutex);
>> cancel_work_sync(&isc->awb_work);
>> video_unregister_device(&isc->video_dev);
>> v4l2_ctrl_handler_free(&isc->ctrls.handler);
>> @@ -1866,6 +1886,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
>> isc->current_subdev = container_of(notifier,
>> struct isc_subdev_entity, notifier);
>> mutex_init(&isc->lock);
>> + mutex_init(&isc->awb_mutex);
>> +
>> init_completion(&isc->comp);
>>
>> /* Initialize videobuf2 queue */
>> @@ -1941,6 +1963,7 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
>> video_unregister_device(vdev);
>>
>> isc_async_complete_err:
>> + mutex_destroy(&isc->awb_mutex);
>> mutex_destroy(&isc->lock);
>> return ret;
>> }
>> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
>> index 0b6370d7775f..c2cb805faff3 100644
>> --- a/drivers/media/platform/atmel/atmel-isc.h
>> +++ b/drivers/media/platform/atmel/atmel-isc.h
>> @@ -307,6 +307,7 @@ struct isc_device {
>> struct work_struct awb_work;
>>
>> struct mutex lock; /* serialize access to file operations */
>> + struct mutex awb_mutex; /* serialize access to streaming status from awb work queue */
>> spinlock_t awb_lock; /* serialize access to DMA buffers from awb work queue */
>>
>> struct regmap_field *pipeline[ISC_PIPE_LINE_NODE_NUM];
>> --
>> 2.25.1
>>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 17/23] media: atmel: atmel-isc-base: use mutex to lock awb workqueue from streaming
@ 2022-01-19 15:40 ` Eugen.Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen.Hristev @ 2022-01-19 15:40 UTC (permalink / raw)
To: jacopo
Cc: devicetree, linux-kernel, robh+dt, sakari.ailus,
laurent.pinchart, linux-arm-kernel, linux-media
On 1/12/22 10:58 AM, Jacopo Mondi wrote:
> Hi Eugen
>
> On Mon, Dec 13, 2021 at 03:49:34PM +0200, Eugen Hristev wrote:
>> The AWB workqueue runs in a kernel thread and needs to be synchronized
>> w.r.t. the streaming status.
>> It is possible that streaming is stopped while the AWB workq is running.
>> In this case it is likely that the check for isc->stop is done at one point
>> in time, but the AWB computations are done later, including a call to
>> isc_update_profile, which requires streaming to be started.
>> Thus , isc_update_profile will fail if during this operation sequence the
>> streaming was stopped.
>> To solve this issue, a mutex is added, that will serialize the awb work and
>> streaming stopping, with the mention that either streaming is stopped
>> completely including termination of the last frame is done, and after that
>> the AWB work can check stream status and stop; either first AWB work is
>> completed and after that the streaming can stop correctly.
>> The awb spin lock cannot be used since this spinlock is taken in the same
>> context and using it in the stop streaming will result in a recursion BUG.
>>
>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>> ---
>> drivers/media/platform/atmel/atmel-isc-base.c | 31 ++++++++++++++++---
>> drivers/media/platform/atmel/atmel-isc.h | 1 +
>> 2 files changed, 28 insertions(+), 4 deletions(-)
>>
>> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
>> index b0c3ed21f372..53cac1aac0fd 100644
>> --- a/drivers/media/platform/atmel/atmel-isc-base.c
>> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
>> @@ -401,6 +401,7 @@ static void isc_stop_streaming(struct vb2_queue *vq)
>> struct isc_buffer *buf;
>> int ret;
>>
>> + mutex_lock(&isc->awb_mutex);
>> v4l2_ctrl_activate(isc->do_wb_ctrl, false);
>>
>> isc->stop = true;
>> @@ -410,6 +411,8 @@ static void isc_stop_streaming(struct vb2_queue *vq)
>> v4l2_err(&isc->v4l2_dev,
>> "Timeout waiting for end of the capture\n");
>>
>> + mutex_unlock(&isc->awb_mutex);
>> +
>> /* Disable DMA interrupt */
>> regmap_write(isc->regmap, ISC_INTDIS, ISC_INT_DDONE);
>>
>> @@ -1416,10 +1419,6 @@ static void isc_awb_work(struct work_struct *w)
>> u32 min, max;
>> int ret;
>>
>> - /* streaming is not active anymore */
>> - if (isc->stop)
>> - return;
>> -
>> if (ctrls->hist_stat != HIST_ENABLED)
>> return;
>>
>> @@ -1470,7 +1469,24 @@ static void isc_awb_work(struct work_struct *w)
>> }
>> regmap_write(regmap, ISC_HIS_CFG + isc->offsets.his,
>> hist_id | baysel | ISC_HIS_CFG_RAR);
>
> isc_stop_streaming() calls runtime_put and here you access the hw.
Hi Jacopo,
That is correct. However the awb routine will call resume and get here
(before accessing the hardware) :
https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/atmel/atmel-isc-base.c#L1722
So I think that we are good as we are now
>
> Feels like it's safer to hold the mutex for the whole duration of the
> AWB routine ?
>
I prefer to have the critical section as little as possible. As we are
only synchronizing the 'streaming status' , only this should be under
the lock.
If you have a different opinion, let me know.
Eugen
>> +
>> + /*
>> + * We have to make sure the streaming has not stopped meanwhile.
>> + * ISC requires a frame to clock the internal profile update.
>> + * To avoid issues, lock the sequence with a mutex
>> + */
>> + mutex_lock(&isc->awb_mutex);
>> +
>> + /* streaming is not active anymore */
>> + if (isc->stop) {
>> + mutex_unlock(&isc->awb_mutex);
>> + return;
>> + };
>> +
>> isc_update_profile(isc);
>> +
>> + mutex_unlock(&isc->awb_mutex);
>> +
>> /* if awb has been disabled, we don't need to start another histogram */
>> if (ctrls->awb)
>> regmap_write(regmap, ISC_CTRLEN, ISC_CTRL_HISREQ);
>> @@ -1549,6 +1565,8 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
>>
>> isc_update_awb_ctrls(isc);
>>
>> + mutex_lock(&isc->awb_mutex);
>> +
>> if (!isc->stop) {
>> /*
>> * If we are streaming, we can update profile to
>> @@ -1563,6 +1581,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
>> */
>> v4l2_ctrl_activate(isc->do_wb_ctrl, false);
>> }
>> + mutex_unlock(&isc->awb_mutex);
>>
>> /* if we have autowhitebalance on, start histogram procedure */
>> if (ctrls->awb == ISC_WB_AUTO && !isc->stop &&
>> @@ -1754,6 +1773,7 @@ static void isc_async_unbind(struct v4l2_async_notifier *notifier,
>> {
>> struct isc_device *isc = container_of(notifier->v4l2_dev,
>> struct isc_device, v4l2_dev);
>> + mutex_destroy(&isc->awb_mutex);
>> cancel_work_sync(&isc->awb_work);
>> video_unregister_device(&isc->video_dev);
>> v4l2_ctrl_handler_free(&isc->ctrls.handler);
>> @@ -1866,6 +1886,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
>> isc->current_subdev = container_of(notifier,
>> struct isc_subdev_entity, notifier);
>> mutex_init(&isc->lock);
>> + mutex_init(&isc->awb_mutex);
>> +
>> init_completion(&isc->comp);
>>
>> /* Initialize videobuf2 queue */
>> @@ -1941,6 +1963,7 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
>> video_unregister_device(vdev);
>>
>> isc_async_complete_err:
>> + mutex_destroy(&isc->awb_mutex);
>> mutex_destroy(&isc->lock);
>> return ret;
>> }
>> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
>> index 0b6370d7775f..c2cb805faff3 100644
>> --- a/drivers/media/platform/atmel/atmel-isc.h
>> +++ b/drivers/media/platform/atmel/atmel-isc.h
>> @@ -307,6 +307,7 @@ struct isc_device {
>> struct work_struct awb_work;
>>
>> struct mutex lock; /* serialize access to file operations */
>> + struct mutex awb_mutex; /* serialize access to streaming status from awb work queue */
>> spinlock_t awb_lock; /* serialize access to DMA buffers from awb work queue */
>>
>> struct regmap_field *pipeline[ISC_PIPE_LINE_NODE_NUM];
>> --
>> 2.25.1
>>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 23/23] media: atmel: atmel-isc: change format propagation to subdev into only verification
2022-01-12 9:21 ` Jacopo Mondi
@ 2022-01-20 8:43 ` Eugen.Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen.Hristev @ 2022-01-20 8:43 UTC (permalink / raw)
To: jacopo
Cc: linux-media, robh+dt, laurent.pinchart, sakari.ailus, devicetree,
linux-kernel, linux-arm-kernel, Nicolas.Ferre
On 1/12/22 11:21 AM, Jacopo Mondi wrote:
> Hi Eugen
Hi Jacopo,
>
> On Mon, Dec 13, 2021 at 03:49:40PM +0200, Eugen Hristev wrote:
>> As a top MC video driver, the atmel-isc should not propagate the format to the
>> subdevice.
>> It should rather check at streamon() time if the subdev is properly configured
>> with a compatible format.
>> Removed the whole format finding logic, and reworked the format verification
>> at streamon time, such that the ISC will return an error if the subdevice
>> is not properly configured.
>> With this being done, the module parameter 'sensor_prefered' makes no sense
>> anymore. The ISC should not decide which format the sensor is using. The
>> ISC should only cope with the situation and inform userspace if the streaming
>> is possible in the current configuration.
>
> Sounds great!
>
>>
>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>> ---
>> Changes in v3:
>> - clamp to maximum resolution once the frame size from the subdev is found
>>
>> drivers/media/platform/atmel/atmel-isc-base.c | 271 ++++++++----------
>> drivers/media/platform/atmel/atmel-isc.h | 1 +
>> 2 files changed, 126 insertions(+), 146 deletions(-)
>>
>> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
>> index 31c8e3029eee..00c8c9588a78 100644
>> --- a/drivers/media/platform/atmel/atmel-isc-base.c
>> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
>> @@ -36,11 +36,6 @@ static unsigned int debug;
>> module_param(debug, int, 0644);
>> MODULE_PARM_DESC(debug, "debug level (0-2)");
>>
>> -static unsigned int sensor_preferred = 1;
>> -module_param(sensor_preferred, uint, 0644);
>> -MODULE_PARM_DESC(sensor_preferred,
>> - "Sensor is preferred to output the specified format (1-on 0-off), default 1");
>> -
>> #define ISC_IS_FORMAT_RAW(mbus_code) \
>> (((mbus_code) & 0xf000) == 0x3000)
>>
>> @@ -532,7 +527,7 @@ static int isc_enum_fmt_vid_cap(struct file *file, void *priv,
>> * convert it to any of the formats that we usually can with a
>> * RAW sensor. Thus, do not advertise them.
>> */
>> - if (!isc->config.sd_format ||
>> + if (isc->config.sd_format &&
>
> Is this change intentional ?
>
>> !ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
>> return -EINVAL;
>>
>> @@ -621,20 +616,30 @@ static int isc_try_validate_formats(struct isc_device *isc)
>> break;
>> default:
>> /* any other different formats are not supported */
>> + v4l2_err(&isc->v4l2_dev, "Requested unsupported format.\n");
>> ret = -EINVAL;
>> }
>> v4l2_dbg(1, debug, &isc->v4l2_dev,
>> "Format validation, requested rgb=%u, yuv=%u, grey=%u, bayer=%u\n",
>> rgb, yuv, grey, bayer);
>
> Would it make sense to move this before the switch so that the
> error messages, if any, appear later ?
Actually, no, because the variables rgb, yuv, grey, bayer, are set
according to what happens in the switch
>
>>
>> - /* we cannot output RAW if we do not receive RAW */
>> - if ((bayer) && !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code))
>> + if ((bayer) &&
>> + !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code)) {
>> + v4l2_err(&isc->v4l2_dev, "Cannot output RAW if we do not receive RAW.\n");
>> return -EINVAL;
>> + }
>>
>> - /* we cannot output GREY if we do not receive RAW/GREY */
>> if (grey && !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code) &&
>> - !ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code))
>> + !ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) {
>> + v4l2_err(&isc->v4l2_dev, "Cannot output GREY if we do not receive RAW/GREY.\n");
>> return -EINVAL;
>> + }
>> +
>> + if ((rgb || bayer || yuv) &&
>> + ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) {
>> + v4l2_err(&isc->v4l2_dev, "Cannot convert GREY to another format.\n");
>> + return -EINVAL;
>> + }
>>
>> return ret;
>> }
>> @@ -862,7 +867,7 @@ static void isc_try_fse(struct isc_device *isc,
>> * If we do not know yet which format the subdev is using, we cannot
>> * do anything.
>> */
>> - if (!isc->try_config.sd_format)
>> + if (!isc->config.sd_format)
>> return;
>>
>> fse.code = isc->try_config.sd_format->mbus_code;
>> @@ -883,180 +888,141 @@ static void isc_try_fse(struct isc_device *isc,
>> }
>> }
>>
>> -static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f,
>> - u32 *code)
>> +static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f)
>> {
>> - int i;
>> - struct isc_format *sd_fmt = NULL, *direct_fmt = NULL;
>> struct v4l2_pix_format *pixfmt = &f->fmt.pix;
>> - struct v4l2_subdev_pad_config pad_cfg = {};
>> - struct v4l2_subdev_state pad_state = {
>> - .pads = &pad_cfg
>> - };
>> - struct v4l2_subdev_format format = {
>> - .which = V4L2_SUBDEV_FORMAT_TRY,
>> - };
>> - u32 mbus_code;
>> - int ret;
>> - bool rlp_dma_direct_dump = false;
>> + unsigned int i;
>>
>> if (f->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
>> return -EINVAL;
>>
>> - /* Step 1: find a RAW format that is supported */
>> - for (i = 0; i < isc->num_user_formats; i++) {
>> - if (ISC_IS_FORMAT_RAW(isc->user_formats[i]->mbus_code)) {
>> - sd_fmt = isc->user_formats[i];
>> + isc->try_config.fourcc = isc->user_formats[0]->fourcc;
>> +
>> + /* find if the format requested is supported */
>> + for (i = 0; i < isc->controller_formats_size; i++)
>> + if (isc->controller_formats[i].fourcc == pixfmt->pixelformat) {
>> + isc->try_config.fourcc = pixfmt->pixelformat;
>> break;
>> }
>> - }
>> - /* Step 2: We can continue with this RAW format, or we can look
>> - * for better: maybe sensor supports directly what we need.
>> - */
>> - direct_fmt = find_format_by_fourcc(isc, pixfmt->pixelformat);
>> -
>> - /* Step 3: We have both. We decide given the module parameter which
>> - * one to use.
>> - */
>> - if (direct_fmt && sd_fmt && sensor_preferred)
>> - sd_fmt = direct_fmt;
>> -
>> - /* Step 4: we do not have RAW but we have a direct format. Use it. */
>> - if (direct_fmt && !sd_fmt)
>> - sd_fmt = direct_fmt;
>> -
>> - /* Step 5: if we are using a direct format, we need to package
>> - * everything as 8 bit data and just dump it
>> - */
>> - if (sd_fmt == direct_fmt)
>> - rlp_dma_direct_dump = true;
>> -
>> - /* Step 6: We have no format. This can happen if the userspace
>> - * requests some weird/invalid format.
>> - * In this case, default to whatever we have
>> - */
>> - if (!sd_fmt && !direct_fmt) {
>> - sd_fmt = isc->user_formats[isc->num_user_formats - 1];
>> - v4l2_dbg(1, debug, &isc->v4l2_dev,
>> - "Sensor not supporting %.4s, using %.4s\n",
>> - (char *)&pixfmt->pixelformat, (char *)&sd_fmt->fourcc);
>> - }
>> -
>> - if (!sd_fmt) {
>> - ret = -EINVAL;
>> - goto isc_try_fmt_err;
>> - }
>> -
>> - /* Step 7: Print out what we decided for debugging */
>> - v4l2_dbg(1, debug, &isc->v4l2_dev,
>> - "Preferring to have sensor using format %.4s\n",
>> - (char *)&sd_fmt->fourcc);
>> -
>> - /* Step 8: at this moment we decided which format the subdev will use */
>> - isc->try_config.sd_format = sd_fmt;
>> -
>> - /* Limit to Atmel ISC hardware capabilities */
>> - if (pixfmt->width > isc->max_width)
>> - pixfmt->width = isc->max_width;
>> - if (pixfmt->height > isc->max_height)
>> - pixfmt->height = isc->max_height;
>> -
>> - /*
>> - * The mbus format is the one the subdev outputs.
>> - * The pixels will be transferred in this format Sensor -> ISC
>> - */
>> - mbus_code = sd_fmt->mbus_code;
>> -
>> - /*
>> - * Validate formats. If the required format is not OK, default to raw.
>> - */
>> -
>> - isc->try_config.fourcc = pixfmt->pixelformat;
>> -
>> - if (isc_try_validate_formats(isc)) {
>> - pixfmt->pixelformat = isc->try_config.fourcc = sd_fmt->fourcc;
>> - /* Re-try to validate the new format */
>> - ret = isc_try_validate_formats(isc);
>> - if (ret)
>> - goto isc_try_fmt_err;
>> - }
>> -
>> - ret = isc_try_configure_rlp_dma(isc, rlp_dma_direct_dump);
>> - if (ret)
>> - goto isc_try_fmt_err;
>> -
>> - ret = isc_try_configure_pipeline(isc);
>> - if (ret)
>> - goto isc_try_fmt_err;
>>
>> - /* Obtain frame sizes if possible to have crop requirements ready */
>> - isc_try_fse(isc, &pad_state);
>> -
>> - v4l2_fill_mbus_format(&format.format, pixfmt, mbus_code);
>> - ret = v4l2_subdev_call(isc->current_subdev->sd, pad, set_fmt,
>> - &pad_state, &format);
>> - if (ret < 0)
>> - goto isc_try_fmt_subdev_err;
>> + /* If we did not find the requested format, we will fallback here */
>> + pixfmt->pixelformat = isc->try_config.fourcc;
>> + pixfmt->colorspace = V4L2_COLORSPACE_SRGB;
>> + pixfmt->field = V4L2_FIELD_NONE;
>>
>> - v4l2_fill_pix_format(pixfmt, &format.format);
>> + isc_try_configure_rlp_dma(isc, false);
>>
>> /* Limit to Atmel ISC hardware capabilities */
>> - if (pixfmt->width > isc->max_width)
>> - pixfmt->width = isc->max_width;
>> - if (pixfmt->height > isc->max_height)
>> - pixfmt->height = isc->max_height;
>> + v4l_bound_align_image(&pixfmt->width, 16, isc->max_width, 0,
>> + &pixfmt->height, 16, isc->max_height, 0, 0);
>>
>> pixfmt->field = V4L2_FIELD_NONE;
>> pixfmt->bytesperline = (pixfmt->width * isc->try_config.bpp_v4l2) >> 3;
>> pixfmt->sizeimage = ((pixfmt->width * isc->try_config.bpp) >> 3) *
>> pixfmt->height;
>>
>> - if (code)
>> - *code = mbus_code;
>> + isc->try_fmt = *f;
>>
>> return 0;
>> +}
>>
>> -isc_try_fmt_err:
>> - v4l2_err(&isc->v4l2_dev, "Could not find any possible format for a working pipeline\n");
>> -isc_try_fmt_subdev_err:
>> - memset(&isc->try_config, 0, sizeof(isc->try_config));
>> +static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
>> +{
>> + isc_try_fmt(isc, f);
>>
>> - return ret;
>> + /* make the try configuration active */
>> + isc->config = isc->try_config;
>> + isc->fmt = isc->try_fmt;
>> +
>> + v4l2_dbg(1, debug, &isc->v4l2_dev, "ISC set_fmt to %.4s @%dx%d\n",
>> + (char *)&f->fmt.pix.pixelformat,
>> + f->fmt.pix.width, f->fmt.pix.height);
>> +
>> + return 0;
>> }
>>
>> -static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
>> +static int isc_validate(struct isc_device *isc)
>> {
>> + int ret;
>> + int i;
>> + struct isc_format *sd_fmt = NULL;
>> + struct v4l2_pix_format *pixfmt = &isc->fmt.fmt.pix;
>> struct v4l2_subdev_format format = {
>> .which = V4L2_SUBDEV_FORMAT_ACTIVE,
>> + .pad = isc->remote_pad,
>> + };
>> + struct v4l2_subdev_pad_config pad_cfg = {};
>> + struct v4l2_subdev_state pad_state = {
>> + .pads = &pad_cfg,
>> };
>> - u32 mbus_code = 0;
>> - int ret;
>>
>> - ret = isc_try_fmt(isc, f, &mbus_code);
>> + /* Get current format from subdev */
>> + ret = v4l2_subdev_call(isc->current_subdev->sd, pad, get_fmt, NULL,
>> + &format);
>
> Ah! Haven't you just said we don't care anymore about the subdev
> format ? :)
We don't ! but we can't stream anything if the sensor streams a format
that we don't understand do we ?
>
> Kidding, I might got a bit lost in the logic, but if I look at your
> above isc_try_validate_formats() it seems like an ideal candidate for
> .link_validate() media_entity operation.
>
> Just to make sure we're on the same page, here's how it should ideally
> look like:
> - set format does care about subdev format. It only checks that the
> format required from the user is one of the ISC supported one. Ie.
> no v4l2_subdev_call()
>
> - as s_stream time your top driver calls media_pipeline_start()
>
> - media_pipeline_start() walks all the entities in the pipeline and
> validates the format of connected pads. To validate formats the
> __media_pipeline_start() functions calls link_validate() on each
> entity. You should in your driver set
Is this done automatically ? if this would be the case, where all links
are correctly set, then, in theory, it would not be possible that some
entities have a format set that is not supported by the other side
>
> static const struct media_entity_operations your_media_entity_ops = {
> .link_validate = v4l2_subdev_link_validate,
> };
>
> if you want to use the default link validation procedure, or set the
> callback to your custom validation function, which can behave more
> or less like isc_try_validate_formats()
Does this mean that I can ask the subdev for the format ?
Or how can I obtain the format on the other side of the link ?
link validate can do this for me ?
Thanks for reviewing,
Eugen
>
> Does it match your understanding too ?
>
> Thanks
> j
>
>
>
>> if (ret)
>> return ret;
>>
>> - v4l2_fill_mbus_format(&format.format, &f->fmt.pix, mbus_code);
>> - ret = v4l2_subdev_call(isc->current_subdev->sd, pad,
>> - set_fmt, NULL, &format);
>> - if (ret < 0)
>> - return ret;
>> + /* Identify the subdev's format configuration */
>> + for (i = 0; i < isc->num_user_formats; i++)
>> + if (isc->user_formats[i]->mbus_code == format.format.code) {
>> + sd_fmt = isc->user_formats[i];
>> + break;
>> + }
>> +
>> + /* Check if the format is not supported */
>> + if (!sd_fmt) {
>> + v4l2_err(&isc->v4l2_dev,
>> + "Current subdevice is streaming a media bus code that is not supported 0x%x\n",
>> + format.format.code);
>> + return -EPIPE;
>> + }
>> +
>> + /* At this moment we know which format the subdev will use */
>> + isc->try_config.sd_format = sd_fmt;
>> +
>> + /* If the sensor is not RAW, we can only do a direct dump */
>> + if (!ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code))
>> + isc_try_configure_rlp_dma(isc, true);
>>
>> /* Limit to Atmel ISC hardware capabilities */
>> - if (f->fmt.pix.width > isc->max_width)
>> - f->fmt.pix.width = isc->max_width;
>> - if (f->fmt.pix.height > isc->max_height)
>> - f->fmt.pix.height = isc->max_height;
>> + v4l_bound_align_image(&format.format.width, 16, isc->max_width, 0,
>> + &format.format.height, 16, isc->max_height, 0, 0);
>>
>> - isc->fmt = *f;
>> + /* Check if the frame size is the same. Otherwise we may overflow */
>> + if (pixfmt->height != format.format.height ||
>> + pixfmt->width != format.format.width) {
>> + v4l2_err(&isc->v4l2_dev,
>> + "ISC not configured with the proper frame size: %dx%d\n",
>> + format.format.width, format.format.height);
>> + return -EPIPE;
>> + }
>>
>> + v4l2_dbg(1, debug, &isc->v4l2_dev,
>> + "Identified subdev using format %.4s with %dx%d %d bpp\n",
>> + (char *)&sd_fmt->fourcc, pixfmt->width, pixfmt->height,
>> + isc->try_config.bpp);
>> +
>> + /* Reset and restart AWB if the subdevice changed the format */
>> if (isc->try_config.sd_format && isc->config.sd_format &&
>> isc->try_config.sd_format != isc->config.sd_format) {
>> isc->ctrls.hist_stat = HIST_INIT;
>> isc_reset_awb_ctrls(isc);
>> isc_update_v4l2_ctrls(isc);
>> }
>> - /* make the try configuration active */
>> +
>> + /* Validate formats */
>> + ret = isc_try_validate_formats(isc);
>> + if (ret)
>> + return ret;
>> +
>> + /* Obtain frame sizes if possible to have crop requirements ready */
>> + isc_try_fse(isc, &pad_state);
>> +
>> + /* Configure ISC pipeline for the config */
>> + ret = isc_try_configure_pipeline(isc);
>> + if (ret)
>> + return ret;
>> +
>> isc->config = isc->try_config;
>>
>> v4l2_dbg(1, debug, &isc->v4l2_dev, "New ISC configuration in place\n");
>> @@ -1064,6 +1030,19 @@ static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
>> return 0;
>> }
>>
>> +static int isc_streamon(struct file *file, void *priv, enum v4l2_buf_type bt)
>> +{
>> + struct isc_device *isc = video_drvdata(file);
>> + int ret;
>> +
>> + ret = isc_validate(isc);
>> +
>> + if (ret)
>> + return ret;
>> +
>> + return vb2_ioctl_streamon(file, priv, bt);
>> +}
>> +
>> static int isc_s_fmt_vid_cap(struct file *file, void *priv,
>> struct v4l2_format *f)
>> {
>> @@ -1080,7 +1059,7 @@ static int isc_try_fmt_vid_cap(struct file *file, void *priv,
>> {
>> struct isc_device *isc = video_drvdata(file);
>>
>> - return isc_try_fmt(isc, f, NULL);
>> + return isc_try_fmt(isc, f);
>> }
>>
>> static int isc_enum_input(struct file *file, void *priv,
>> @@ -1176,7 +1155,7 @@ static const struct v4l2_ioctl_ops isc_ioctl_ops = {
>> .vidioc_dqbuf = vb2_ioctl_dqbuf,
>> .vidioc_create_bufs = vb2_ioctl_create_bufs,
>> .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>> - .vidioc_streamon = vb2_ioctl_streamon,
>> + .vidioc_streamon = isc_streamon,
>> .vidioc_streamoff = vb2_ioctl_streamoff,
>>
>> .vidioc_g_parm = isc_g_parm,
>> @@ -1879,7 +1858,7 @@ static int isc_set_default_fmt(struct isc_device *isc)
>> };
>> int ret;
>>
>> - ret = isc_try_fmt(isc, &f, NULL);
>> + ret = isc_try_fmt(isc, &f);
>> if (ret)
>> return ret;
>>
>> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
>> index c2cb805faff3..7081698adddd 100644
>> --- a/drivers/media/platform/atmel/atmel-isc.h
>> +++ b/drivers/media/platform/atmel/atmel-isc.h
>> @@ -297,6 +297,7 @@ struct isc_device {
>> struct completion comp;
>>
>> struct v4l2_format fmt;
>> + struct v4l2_format try_fmt;
>> struct isc_format **user_formats;
>> unsigned int num_user_formats;
>>
>> --
>> 2.25.1
>>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 23/23] media: atmel: atmel-isc: change format propagation to subdev into only verification
@ 2022-01-20 8:43 ` Eugen.Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen.Hristev @ 2022-01-20 8:43 UTC (permalink / raw)
To: jacopo
Cc: devicetree, linux-kernel, robh+dt, sakari.ailus,
laurent.pinchart, linux-arm-kernel, linux-media
On 1/12/22 11:21 AM, Jacopo Mondi wrote:
> Hi Eugen
Hi Jacopo,
>
> On Mon, Dec 13, 2021 at 03:49:40PM +0200, Eugen Hristev wrote:
>> As a top MC video driver, the atmel-isc should not propagate the format to the
>> subdevice.
>> It should rather check at streamon() time if the subdev is properly configured
>> with a compatible format.
>> Removed the whole format finding logic, and reworked the format verification
>> at streamon time, such that the ISC will return an error if the subdevice
>> is not properly configured.
>> With this being done, the module parameter 'sensor_prefered' makes no sense
>> anymore. The ISC should not decide which format the sensor is using. The
>> ISC should only cope with the situation and inform userspace if the streaming
>> is possible in the current configuration.
>
> Sounds great!
>
>>
>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>> ---
>> Changes in v3:
>> - clamp to maximum resolution once the frame size from the subdev is found
>>
>> drivers/media/platform/atmel/atmel-isc-base.c | 271 ++++++++----------
>> drivers/media/platform/atmel/atmel-isc.h | 1 +
>> 2 files changed, 126 insertions(+), 146 deletions(-)
>>
>> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
>> index 31c8e3029eee..00c8c9588a78 100644
>> --- a/drivers/media/platform/atmel/atmel-isc-base.c
>> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
>> @@ -36,11 +36,6 @@ static unsigned int debug;
>> module_param(debug, int, 0644);
>> MODULE_PARM_DESC(debug, "debug level (0-2)");
>>
>> -static unsigned int sensor_preferred = 1;
>> -module_param(sensor_preferred, uint, 0644);
>> -MODULE_PARM_DESC(sensor_preferred,
>> - "Sensor is preferred to output the specified format (1-on 0-off), default 1");
>> -
>> #define ISC_IS_FORMAT_RAW(mbus_code) \
>> (((mbus_code) & 0xf000) == 0x3000)
>>
>> @@ -532,7 +527,7 @@ static int isc_enum_fmt_vid_cap(struct file *file, void *priv,
>> * convert it to any of the formats that we usually can with a
>> * RAW sensor. Thus, do not advertise them.
>> */
>> - if (!isc->config.sd_format ||
>> + if (isc->config.sd_format &&
>
> Is this change intentional ?
>
>> !ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
>> return -EINVAL;
>>
>> @@ -621,20 +616,30 @@ static int isc_try_validate_formats(struct isc_device *isc)
>> break;
>> default:
>> /* any other different formats are not supported */
>> + v4l2_err(&isc->v4l2_dev, "Requested unsupported format.\n");
>> ret = -EINVAL;
>> }
>> v4l2_dbg(1, debug, &isc->v4l2_dev,
>> "Format validation, requested rgb=%u, yuv=%u, grey=%u, bayer=%u\n",
>> rgb, yuv, grey, bayer);
>
> Would it make sense to move this before the switch so that the
> error messages, if any, appear later ?
Actually, no, because the variables rgb, yuv, grey, bayer, are set
according to what happens in the switch
>
>>
>> - /* we cannot output RAW if we do not receive RAW */
>> - if ((bayer) && !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code))
>> + if ((bayer) &&
>> + !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code)) {
>> + v4l2_err(&isc->v4l2_dev, "Cannot output RAW if we do not receive RAW.\n");
>> return -EINVAL;
>> + }
>>
>> - /* we cannot output GREY if we do not receive RAW/GREY */
>> if (grey && !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code) &&
>> - !ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code))
>> + !ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) {
>> + v4l2_err(&isc->v4l2_dev, "Cannot output GREY if we do not receive RAW/GREY.\n");
>> return -EINVAL;
>> + }
>> +
>> + if ((rgb || bayer || yuv) &&
>> + ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) {
>> + v4l2_err(&isc->v4l2_dev, "Cannot convert GREY to another format.\n");
>> + return -EINVAL;
>> + }
>>
>> return ret;
>> }
>> @@ -862,7 +867,7 @@ static void isc_try_fse(struct isc_device *isc,
>> * If we do not know yet which format the subdev is using, we cannot
>> * do anything.
>> */
>> - if (!isc->try_config.sd_format)
>> + if (!isc->config.sd_format)
>> return;
>>
>> fse.code = isc->try_config.sd_format->mbus_code;
>> @@ -883,180 +888,141 @@ static void isc_try_fse(struct isc_device *isc,
>> }
>> }
>>
>> -static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f,
>> - u32 *code)
>> +static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f)
>> {
>> - int i;
>> - struct isc_format *sd_fmt = NULL, *direct_fmt = NULL;
>> struct v4l2_pix_format *pixfmt = &f->fmt.pix;
>> - struct v4l2_subdev_pad_config pad_cfg = {};
>> - struct v4l2_subdev_state pad_state = {
>> - .pads = &pad_cfg
>> - };
>> - struct v4l2_subdev_format format = {
>> - .which = V4L2_SUBDEV_FORMAT_TRY,
>> - };
>> - u32 mbus_code;
>> - int ret;
>> - bool rlp_dma_direct_dump = false;
>> + unsigned int i;
>>
>> if (f->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
>> return -EINVAL;
>>
>> - /* Step 1: find a RAW format that is supported */
>> - for (i = 0; i < isc->num_user_formats; i++) {
>> - if (ISC_IS_FORMAT_RAW(isc->user_formats[i]->mbus_code)) {
>> - sd_fmt = isc->user_formats[i];
>> + isc->try_config.fourcc = isc->user_formats[0]->fourcc;
>> +
>> + /* find if the format requested is supported */
>> + for (i = 0; i < isc->controller_formats_size; i++)
>> + if (isc->controller_formats[i].fourcc == pixfmt->pixelformat) {
>> + isc->try_config.fourcc = pixfmt->pixelformat;
>> break;
>> }
>> - }
>> - /* Step 2: We can continue with this RAW format, or we can look
>> - * for better: maybe sensor supports directly what we need.
>> - */
>> - direct_fmt = find_format_by_fourcc(isc, pixfmt->pixelformat);
>> -
>> - /* Step 3: We have both. We decide given the module parameter which
>> - * one to use.
>> - */
>> - if (direct_fmt && sd_fmt && sensor_preferred)
>> - sd_fmt = direct_fmt;
>> -
>> - /* Step 4: we do not have RAW but we have a direct format. Use it. */
>> - if (direct_fmt && !sd_fmt)
>> - sd_fmt = direct_fmt;
>> -
>> - /* Step 5: if we are using a direct format, we need to package
>> - * everything as 8 bit data and just dump it
>> - */
>> - if (sd_fmt == direct_fmt)
>> - rlp_dma_direct_dump = true;
>> -
>> - /* Step 6: We have no format. This can happen if the userspace
>> - * requests some weird/invalid format.
>> - * In this case, default to whatever we have
>> - */
>> - if (!sd_fmt && !direct_fmt) {
>> - sd_fmt = isc->user_formats[isc->num_user_formats - 1];
>> - v4l2_dbg(1, debug, &isc->v4l2_dev,
>> - "Sensor not supporting %.4s, using %.4s\n",
>> - (char *)&pixfmt->pixelformat, (char *)&sd_fmt->fourcc);
>> - }
>> -
>> - if (!sd_fmt) {
>> - ret = -EINVAL;
>> - goto isc_try_fmt_err;
>> - }
>> -
>> - /* Step 7: Print out what we decided for debugging */
>> - v4l2_dbg(1, debug, &isc->v4l2_dev,
>> - "Preferring to have sensor using format %.4s\n",
>> - (char *)&sd_fmt->fourcc);
>> -
>> - /* Step 8: at this moment we decided which format the subdev will use */
>> - isc->try_config.sd_format = sd_fmt;
>> -
>> - /* Limit to Atmel ISC hardware capabilities */
>> - if (pixfmt->width > isc->max_width)
>> - pixfmt->width = isc->max_width;
>> - if (pixfmt->height > isc->max_height)
>> - pixfmt->height = isc->max_height;
>> -
>> - /*
>> - * The mbus format is the one the subdev outputs.
>> - * The pixels will be transferred in this format Sensor -> ISC
>> - */
>> - mbus_code = sd_fmt->mbus_code;
>> -
>> - /*
>> - * Validate formats. If the required format is not OK, default to raw.
>> - */
>> -
>> - isc->try_config.fourcc = pixfmt->pixelformat;
>> -
>> - if (isc_try_validate_formats(isc)) {
>> - pixfmt->pixelformat = isc->try_config.fourcc = sd_fmt->fourcc;
>> - /* Re-try to validate the new format */
>> - ret = isc_try_validate_formats(isc);
>> - if (ret)
>> - goto isc_try_fmt_err;
>> - }
>> -
>> - ret = isc_try_configure_rlp_dma(isc, rlp_dma_direct_dump);
>> - if (ret)
>> - goto isc_try_fmt_err;
>> -
>> - ret = isc_try_configure_pipeline(isc);
>> - if (ret)
>> - goto isc_try_fmt_err;
>>
>> - /* Obtain frame sizes if possible to have crop requirements ready */
>> - isc_try_fse(isc, &pad_state);
>> -
>> - v4l2_fill_mbus_format(&format.format, pixfmt, mbus_code);
>> - ret = v4l2_subdev_call(isc->current_subdev->sd, pad, set_fmt,
>> - &pad_state, &format);
>> - if (ret < 0)
>> - goto isc_try_fmt_subdev_err;
>> + /* If we did not find the requested format, we will fallback here */
>> + pixfmt->pixelformat = isc->try_config.fourcc;
>> + pixfmt->colorspace = V4L2_COLORSPACE_SRGB;
>> + pixfmt->field = V4L2_FIELD_NONE;
>>
>> - v4l2_fill_pix_format(pixfmt, &format.format);
>> + isc_try_configure_rlp_dma(isc, false);
>>
>> /* Limit to Atmel ISC hardware capabilities */
>> - if (pixfmt->width > isc->max_width)
>> - pixfmt->width = isc->max_width;
>> - if (pixfmt->height > isc->max_height)
>> - pixfmt->height = isc->max_height;
>> + v4l_bound_align_image(&pixfmt->width, 16, isc->max_width, 0,
>> + &pixfmt->height, 16, isc->max_height, 0, 0);
>>
>> pixfmt->field = V4L2_FIELD_NONE;
>> pixfmt->bytesperline = (pixfmt->width * isc->try_config.bpp_v4l2) >> 3;
>> pixfmt->sizeimage = ((pixfmt->width * isc->try_config.bpp) >> 3) *
>> pixfmt->height;
>>
>> - if (code)
>> - *code = mbus_code;
>> + isc->try_fmt = *f;
>>
>> return 0;
>> +}
>>
>> -isc_try_fmt_err:
>> - v4l2_err(&isc->v4l2_dev, "Could not find any possible format for a working pipeline\n");
>> -isc_try_fmt_subdev_err:
>> - memset(&isc->try_config, 0, sizeof(isc->try_config));
>> +static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
>> +{
>> + isc_try_fmt(isc, f);
>>
>> - return ret;
>> + /* make the try configuration active */
>> + isc->config = isc->try_config;
>> + isc->fmt = isc->try_fmt;
>> +
>> + v4l2_dbg(1, debug, &isc->v4l2_dev, "ISC set_fmt to %.4s @%dx%d\n",
>> + (char *)&f->fmt.pix.pixelformat,
>> + f->fmt.pix.width, f->fmt.pix.height);
>> +
>> + return 0;
>> }
>>
>> -static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
>> +static int isc_validate(struct isc_device *isc)
>> {
>> + int ret;
>> + int i;
>> + struct isc_format *sd_fmt = NULL;
>> + struct v4l2_pix_format *pixfmt = &isc->fmt.fmt.pix;
>> struct v4l2_subdev_format format = {
>> .which = V4L2_SUBDEV_FORMAT_ACTIVE,
>> + .pad = isc->remote_pad,
>> + };
>> + struct v4l2_subdev_pad_config pad_cfg = {};
>> + struct v4l2_subdev_state pad_state = {
>> + .pads = &pad_cfg,
>> };
>> - u32 mbus_code = 0;
>> - int ret;
>>
>> - ret = isc_try_fmt(isc, f, &mbus_code);
>> + /* Get current format from subdev */
>> + ret = v4l2_subdev_call(isc->current_subdev->sd, pad, get_fmt, NULL,
>> + &format);
>
> Ah! Haven't you just said we don't care anymore about the subdev
> format ? :)
We don't ! but we can't stream anything if the sensor streams a format
that we don't understand do we ?
>
> Kidding, I might got a bit lost in the logic, but if I look at your
> above isc_try_validate_formats() it seems like an ideal candidate for
> .link_validate() media_entity operation.
>
> Just to make sure we're on the same page, here's how it should ideally
> look like:
> - set format does care about subdev format. It only checks that the
> format required from the user is one of the ISC supported one. Ie.
> no v4l2_subdev_call()
>
> - as s_stream time your top driver calls media_pipeline_start()
>
> - media_pipeline_start() walks all the entities in the pipeline and
> validates the format of connected pads. To validate formats the
> __media_pipeline_start() functions calls link_validate() on each
> entity. You should in your driver set
Is this done automatically ? if this would be the case, where all links
are correctly set, then, in theory, it would not be possible that some
entities have a format set that is not supported by the other side
>
> static const struct media_entity_operations your_media_entity_ops = {
> .link_validate = v4l2_subdev_link_validate,
> };
>
> if you want to use the default link validation procedure, or set the
> callback to your custom validation function, which can behave more
> or less like isc_try_validate_formats()
Does this mean that I can ask the subdev for the format ?
Or how can I obtain the format on the other side of the link ?
link validate can do this for me ?
Thanks for reviewing,
Eugen
>
> Does it match your understanding too ?
>
> Thanks
> j
>
>
>
>> if (ret)
>> return ret;
>>
>> - v4l2_fill_mbus_format(&format.format, &f->fmt.pix, mbus_code);
>> - ret = v4l2_subdev_call(isc->current_subdev->sd, pad,
>> - set_fmt, NULL, &format);
>> - if (ret < 0)
>> - return ret;
>> + /* Identify the subdev's format configuration */
>> + for (i = 0; i < isc->num_user_formats; i++)
>> + if (isc->user_formats[i]->mbus_code == format.format.code) {
>> + sd_fmt = isc->user_formats[i];
>> + break;
>> + }
>> +
>> + /* Check if the format is not supported */
>> + if (!sd_fmt) {
>> + v4l2_err(&isc->v4l2_dev,
>> + "Current subdevice is streaming a media bus code that is not supported 0x%x\n",
>> + format.format.code);
>> + return -EPIPE;
>> + }
>> +
>> + /* At this moment we know which format the subdev will use */
>> + isc->try_config.sd_format = sd_fmt;
>> +
>> + /* If the sensor is not RAW, we can only do a direct dump */
>> + if (!ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code))
>> + isc_try_configure_rlp_dma(isc, true);
>>
>> /* Limit to Atmel ISC hardware capabilities */
>> - if (f->fmt.pix.width > isc->max_width)
>> - f->fmt.pix.width = isc->max_width;
>> - if (f->fmt.pix.height > isc->max_height)
>> - f->fmt.pix.height = isc->max_height;
>> + v4l_bound_align_image(&format.format.width, 16, isc->max_width, 0,
>> + &format.format.height, 16, isc->max_height, 0, 0);
>>
>> - isc->fmt = *f;
>> + /* Check if the frame size is the same. Otherwise we may overflow */
>> + if (pixfmt->height != format.format.height ||
>> + pixfmt->width != format.format.width) {
>> + v4l2_err(&isc->v4l2_dev,
>> + "ISC not configured with the proper frame size: %dx%d\n",
>> + format.format.width, format.format.height);
>> + return -EPIPE;
>> + }
>>
>> + v4l2_dbg(1, debug, &isc->v4l2_dev,
>> + "Identified subdev using format %.4s with %dx%d %d bpp\n",
>> + (char *)&sd_fmt->fourcc, pixfmt->width, pixfmt->height,
>> + isc->try_config.bpp);
>> +
>> + /* Reset and restart AWB if the subdevice changed the format */
>> if (isc->try_config.sd_format && isc->config.sd_format &&
>> isc->try_config.sd_format != isc->config.sd_format) {
>> isc->ctrls.hist_stat = HIST_INIT;
>> isc_reset_awb_ctrls(isc);
>> isc_update_v4l2_ctrls(isc);
>> }
>> - /* make the try configuration active */
>> +
>> + /* Validate formats */
>> + ret = isc_try_validate_formats(isc);
>> + if (ret)
>> + return ret;
>> +
>> + /* Obtain frame sizes if possible to have crop requirements ready */
>> + isc_try_fse(isc, &pad_state);
>> +
>> + /* Configure ISC pipeline for the config */
>> + ret = isc_try_configure_pipeline(isc);
>> + if (ret)
>> + return ret;
>> +
>> isc->config = isc->try_config;
>>
>> v4l2_dbg(1, debug, &isc->v4l2_dev, "New ISC configuration in place\n");
>> @@ -1064,6 +1030,19 @@ static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
>> return 0;
>> }
>>
>> +static int isc_streamon(struct file *file, void *priv, enum v4l2_buf_type bt)
>> +{
>> + struct isc_device *isc = video_drvdata(file);
>> + int ret;
>> +
>> + ret = isc_validate(isc);
>> +
>> + if (ret)
>> + return ret;
>> +
>> + return vb2_ioctl_streamon(file, priv, bt);
>> +}
>> +
>> static int isc_s_fmt_vid_cap(struct file *file, void *priv,
>> struct v4l2_format *f)
>> {
>> @@ -1080,7 +1059,7 @@ static int isc_try_fmt_vid_cap(struct file *file, void *priv,
>> {
>> struct isc_device *isc = video_drvdata(file);
>>
>> - return isc_try_fmt(isc, f, NULL);
>> + return isc_try_fmt(isc, f);
>> }
>>
>> static int isc_enum_input(struct file *file, void *priv,
>> @@ -1176,7 +1155,7 @@ static const struct v4l2_ioctl_ops isc_ioctl_ops = {
>> .vidioc_dqbuf = vb2_ioctl_dqbuf,
>> .vidioc_create_bufs = vb2_ioctl_create_bufs,
>> .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>> - .vidioc_streamon = vb2_ioctl_streamon,
>> + .vidioc_streamon = isc_streamon,
>> .vidioc_streamoff = vb2_ioctl_streamoff,
>>
>> .vidioc_g_parm = isc_g_parm,
>> @@ -1879,7 +1858,7 @@ static int isc_set_default_fmt(struct isc_device *isc)
>> };
>> int ret;
>>
>> - ret = isc_try_fmt(isc, &f, NULL);
>> + ret = isc_try_fmt(isc, &f);
>> if (ret)
>> return ret;
>>
>> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
>> index c2cb805faff3..7081698adddd 100644
>> --- a/drivers/media/platform/atmel/atmel-isc.h
>> +++ b/drivers/media/platform/atmel/atmel-isc.h
>> @@ -297,6 +297,7 @@ struct isc_device {
>> struct completion comp;
>>
>> struct v4l2_format fmt;
>> + struct v4l2_format try_fmt;
>> struct isc_format **user_formats;
>> unsigned int num_user_formats;
>>
>> --
>> 2.25.1
>>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 23/23] media: atmel: atmel-isc: change format propagation to subdev into only verification
2022-01-20 8:43 ` Eugen.Hristev
@ 2022-01-20 8:48 ` Eugen.Hristev
-1 siblings, 0 replies; 94+ messages in thread
From: Eugen.Hristev @ 2022-01-20 8:48 UTC (permalink / raw)
To: jacopo
Cc: linux-media, robh+dt, laurent.pinchart, sakari.ailus, devicetree,
linux-kernel, linux-arm-kernel, Nicolas.Ferre
On 1/20/22 10:43 AM, Eugen Hristev - M18282 wrote:
> On 1/12/22 11:21 AM, Jacopo Mondi wrote:
>> Hi Eugen
>
>
> Hi Jacopo,
>
>>
>> On Mon, Dec 13, 2021 at 03:49:40PM +0200, Eugen Hristev wrote:
>>> As a top MC video driver, the atmel-isc should not propagate the format to the
>>> subdevice.
>>> It should rather check at streamon() time if the subdev is properly configured
>>> with a compatible format.
>>> Removed the whole format finding logic, and reworked the format verification
>>> at streamon time, such that the ISC will return an error if the subdevice
>>> is not properly configured.
>>> With this being done, the module parameter 'sensor_prefered' makes no sense
>>> anymore. The ISC should not decide which format the sensor is using. The
>>> ISC should only cope with the situation and inform userspace if the streaming
>>> is possible in the current configuration.
>>
>> Sounds great!
>>
>>>
>>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>>> ---
>>> Changes in v3:
>>> - clamp to maximum resolution once the frame size from the subdev is found
>>>
>>> drivers/media/platform/atmel/atmel-isc-base.c | 271 ++++++++----------
>>> drivers/media/platform/atmel/atmel-isc.h | 1 +
>>> 2 files changed, 126 insertions(+), 146 deletions(-)
>>>
>>> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
>>> index 31c8e3029eee..00c8c9588a78 100644
>>> --- a/drivers/media/platform/atmel/atmel-isc-base.c
>>> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
>>> @@ -36,11 +36,6 @@ static unsigned int debug;
>>> module_param(debug, int, 0644);
>>> MODULE_PARM_DESC(debug, "debug level (0-2)");
>>>
>>> -static unsigned int sensor_preferred = 1;
>>> -module_param(sensor_preferred, uint, 0644);
>>> -MODULE_PARM_DESC(sensor_preferred,
>>> - "Sensor is preferred to output the specified format (1-on 0-off), default 1");
>>> -
>>> #define ISC_IS_FORMAT_RAW(mbus_code) \
>>> (((mbus_code) & 0xf000) == 0x3000)
>>>
>>> @@ -532,7 +527,7 @@ static int isc_enum_fmt_vid_cap(struct file *file, void *priv,
>>> * convert it to any of the formats that we usually can with a
>>> * RAW sensor. Thus, do not advertise them.
>>> */
>>> - if (!isc->config.sd_format ||
>>> + if (isc->config.sd_format &&
>>
>> Is this change intentional ?
>>
>>> !ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
>>> return -EINVAL;
>>>
>>> @@ -621,20 +616,30 @@ static int isc_try_validate_formats(struct isc_device *isc)
>>> break;
>>> default:
>>> /* any other different formats are not supported */
>>> + v4l2_err(&isc->v4l2_dev, "Requested unsupported format.\n");
>>> ret = -EINVAL;
>>> }
>>> v4l2_dbg(1, debug, &isc->v4l2_dev,
>>> "Format validation, requested rgb=%u, yuv=%u, grey=%u, bayer=%u\n",
>>> rgb, yuv, grey, bayer);
>>
>> Would it make sense to move this before the switch so that the
>> error messages, if any, appear later ?
>
> Actually, no, because the variables rgb, yuv, grey, bayer, are set
> according to what happens in the switch
>>
>>>
>>> - /* we cannot output RAW if we do not receive RAW */
>>> - if ((bayer) && !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code))
>>> + if ((bayer) &&
>>> + !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code)) {
>>> + v4l2_err(&isc->v4l2_dev, "Cannot output RAW if we do not receive RAW.\n");
>>> return -EINVAL;
>>> + }
>>>
>>> - /* we cannot output GREY if we do not receive RAW/GREY */
>>> if (grey && !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code) &&
>>> - !ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code))
>>> + !ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) {
>>> + v4l2_err(&isc->v4l2_dev, "Cannot output GREY if we do not receive RAW/GREY.\n");
>>> return -EINVAL;
>>> + }
>>> +
>>> + if ((rgb || bayer || yuv) &&
>>> + ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) {
>>> + v4l2_err(&isc->v4l2_dev, "Cannot convert GREY to another format.\n");
>>> + return -EINVAL;
>>> + }
>>>
>>> return ret;
>>> }
>>> @@ -862,7 +867,7 @@ static void isc_try_fse(struct isc_device *isc,
>>> * If we do not know yet which format the subdev is using, we cannot
>>> * do anything.
>>> */
>>> - if (!isc->try_config.sd_format)
>>> + if (!isc->config.sd_format)
>>> return;
>>>
>>> fse.code = isc->try_config.sd_format->mbus_code;
>>> @@ -883,180 +888,141 @@ static void isc_try_fse(struct isc_device *isc,
>>> }
>>> }
>>>
>>> -static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f,
>>> - u32 *code)
>>> +static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f)
>>> {
>>> - int i;
>>> - struct isc_format *sd_fmt = NULL, *direct_fmt = NULL;
>>> struct v4l2_pix_format *pixfmt = &f->fmt.pix;
>>> - struct v4l2_subdev_pad_config pad_cfg = {};
>>> - struct v4l2_subdev_state pad_state = {
>>> - .pads = &pad_cfg
>>> - };
>>> - struct v4l2_subdev_format format = {
>>> - .which = V4L2_SUBDEV_FORMAT_TRY,
>>> - };
>>> - u32 mbus_code;
>>> - int ret;
>>> - bool rlp_dma_direct_dump = false;
>>> + unsigned int i;
>>>
>>> if (f->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
>>> return -EINVAL;
>>>
>>> - /* Step 1: find a RAW format that is supported */
>>> - for (i = 0; i < isc->num_user_formats; i++) {
>>> - if (ISC_IS_FORMAT_RAW(isc->user_formats[i]->mbus_code)) {
>>> - sd_fmt = isc->user_formats[i];
>>> + isc->try_config.fourcc = isc->user_formats[0]->fourcc;
>>> +
>>> + /* find if the format requested is supported */
>>> + for (i = 0; i < isc->controller_formats_size; i++)
>>> + if (isc->controller_formats[i].fourcc == pixfmt->pixelformat) {
>>> + isc->try_config.fourcc = pixfmt->pixelformat;
>>> break;
>>> }
>>> - }
>>> - /* Step 2: We can continue with this RAW format, or we can look
>>> - * for better: maybe sensor supports directly what we need.
>>> - */
>>> - direct_fmt = find_format_by_fourcc(isc, pixfmt->pixelformat);
>>> -
>>> - /* Step 3: We have both. We decide given the module parameter which
>>> - * one to use.
>>> - */
>>> - if (direct_fmt && sd_fmt && sensor_preferred)
>>> - sd_fmt = direct_fmt;
>>> -
>>> - /* Step 4: we do not have RAW but we have a direct format. Use it. */
>>> - if (direct_fmt && !sd_fmt)
>>> - sd_fmt = direct_fmt;
>>> -
>>> - /* Step 5: if we are using a direct format, we need to package
>>> - * everything as 8 bit data and just dump it
>>> - */
>>> - if (sd_fmt == direct_fmt)
>>> - rlp_dma_direct_dump = true;
>>> -
>>> - /* Step 6: We have no format. This can happen if the userspace
>>> - * requests some weird/invalid format.
>>> - * In this case, default to whatever we have
>>> - */
>>> - if (!sd_fmt && !direct_fmt) {
>>> - sd_fmt = isc->user_formats[isc->num_user_formats - 1];
>>> - v4l2_dbg(1, debug, &isc->v4l2_dev,
>>> - "Sensor not supporting %.4s, using %.4s\n",
>>> - (char *)&pixfmt->pixelformat, (char *)&sd_fmt->fourcc);
>>> - }
>>> -
>>> - if (!sd_fmt) {
>>> - ret = -EINVAL;
>>> - goto isc_try_fmt_err;
>>> - }
>>> -
>>> - /* Step 7: Print out what we decided for debugging */
>>> - v4l2_dbg(1, debug, &isc->v4l2_dev,
>>> - "Preferring to have sensor using format %.4s\n",
>>> - (char *)&sd_fmt->fourcc);
>>> -
>>> - /* Step 8: at this moment we decided which format the subdev will use */
>>> - isc->try_config.sd_format = sd_fmt;
>>> -
>>> - /* Limit to Atmel ISC hardware capabilities */
>>> - if (pixfmt->width > isc->max_width)
>>> - pixfmt->width = isc->max_width;
>>> - if (pixfmt->height > isc->max_height)
>>> - pixfmt->height = isc->max_height;
>>> -
>>> - /*
>>> - * The mbus format is the one the subdev outputs.
>>> - * The pixels will be transferred in this format Sensor -> ISC
>>> - */
>>> - mbus_code = sd_fmt->mbus_code;
>>> -
>>> - /*
>>> - * Validate formats. If the required format is not OK, default to raw.
>>> - */
>>> -
>>> - isc->try_config.fourcc = pixfmt->pixelformat;
>>> -
>>> - if (isc_try_validate_formats(isc)) {
>>> - pixfmt->pixelformat = isc->try_config.fourcc = sd_fmt->fourcc;
>>> - /* Re-try to validate the new format */
>>> - ret = isc_try_validate_formats(isc);
>>> - if (ret)
>>> - goto isc_try_fmt_err;
>>> - }
>>> -
>>> - ret = isc_try_configure_rlp_dma(isc, rlp_dma_direct_dump);
>>> - if (ret)
>>> - goto isc_try_fmt_err;
>>> -
>>> - ret = isc_try_configure_pipeline(isc);
>>> - if (ret)
>>> - goto isc_try_fmt_err;
>>>
>>> - /* Obtain frame sizes if possible to have crop requirements ready */
>>> - isc_try_fse(isc, &pad_state);
>>> -
>>> - v4l2_fill_mbus_format(&format.format, pixfmt, mbus_code);
>>> - ret = v4l2_subdev_call(isc->current_subdev->sd, pad, set_fmt,
>>> - &pad_state, &format);
>>> - if (ret < 0)
>>> - goto isc_try_fmt_subdev_err;
>>> + /* If we did not find the requested format, we will fallback here */
>>> + pixfmt->pixelformat = isc->try_config.fourcc;
>>> + pixfmt->colorspace = V4L2_COLORSPACE_SRGB;
>>> + pixfmt->field = V4L2_FIELD_NONE;
>>>
>>> - v4l2_fill_pix_format(pixfmt, &format.format);
>>> + isc_try_configure_rlp_dma(isc, false);
>>>
>>> /* Limit to Atmel ISC hardware capabilities */
>>> - if (pixfmt->width > isc->max_width)
>>> - pixfmt->width = isc->max_width;
>>> - if (pixfmt->height > isc->max_height)
>>> - pixfmt->height = isc->max_height;
>>> + v4l_bound_align_image(&pixfmt->width, 16, isc->max_width, 0,
>>> + &pixfmt->height, 16, isc->max_height, 0, 0);
>>>
>>> pixfmt->field = V4L2_FIELD_NONE;
>>> pixfmt->bytesperline = (pixfmt->width * isc->try_config.bpp_v4l2) >> 3;
>>> pixfmt->sizeimage = ((pixfmt->width * isc->try_config.bpp) >> 3) *
>>> pixfmt->height;
>>>
>>> - if (code)
>>> - *code = mbus_code;
>>> + isc->try_fmt = *f;
>>>
>>> return 0;
>>> +}
>>>
>>> -isc_try_fmt_err:
>>> - v4l2_err(&isc->v4l2_dev, "Could not find any possible format for a working pipeline\n");
>>> -isc_try_fmt_subdev_err:
>>> - memset(&isc->try_config, 0, sizeof(isc->try_config));
>>> +static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
>>> +{
>>> + isc_try_fmt(isc, f);
>>>
>>> - return ret;
>>> + /* make the try configuration active */
>>> + isc->config = isc->try_config;
>>> + isc->fmt = isc->try_fmt;
>>> +
>>> + v4l2_dbg(1, debug, &isc->v4l2_dev, "ISC set_fmt to %.4s @%dx%d\n",
>>> + (char *)&f->fmt.pix.pixelformat,
>>> + f->fmt.pix.width, f->fmt.pix.height);
>>> +
>>> + return 0;
>>> }
>>>
>>> -static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
>>> +static int isc_validate(struct isc_device *isc)
>>> {
>>> + int ret;
>>> + int i;
>>> + struct isc_format *sd_fmt = NULL;
>>> + struct v4l2_pix_format *pixfmt = &isc->fmt.fmt.pix;
>>> struct v4l2_subdev_format format = {
>>> .which = V4L2_SUBDEV_FORMAT_ACTIVE,
>>> + .pad = isc->remote_pad,
>>> + };
>>> + struct v4l2_subdev_pad_config pad_cfg = {};
>>> + struct v4l2_subdev_state pad_state = {
>>> + .pads = &pad_cfg,
>>> };
>>> - u32 mbus_code = 0;
>>> - int ret;
>>>
>>> - ret = isc_try_fmt(isc, f, &mbus_code);
>>> + /* Get current format from subdev */
>>> + ret = v4l2_subdev_call(isc->current_subdev->sd, pad, get_fmt, NULL,
>>> + &format);
>>
>> Ah! Haven't you just said we don't care anymore about the subdev
>> format ? :)
>
> We don't ! but we can't stream anything if the sensor streams a format
> that we don't understand do we ?
Actually, thinking about this, we really need to know what format is at
the input for the ISC. According to this format, we need to understand
how to process it, since the ISC could have at input either Raw bayer
and we need to know the rotation, or directly a non-raw format like yuv422 .
Rather than asking the subdev for the streaming format, we could obtain
this information from the pad link ?
>
>>
>> Kidding, I might got a bit lost in the logic, but if I look at your
>> above isc_try_validate_formats() it seems like an ideal candidate for
>> .link_validate() media_entity operation.
>>
>> Just to make sure we're on the same page, here's how it should ideally
>> look like:
>> - set format does care about subdev format. It only checks that the
>> format required from the user is one of the ISC supported one. Ie.
>> no v4l2_subdev_call()
>>
>> - as s_stream time your top driver calls media_pipeline_start()
>>
>> - media_pipeline_start() walks all the entities in the pipeline and
>> validates the format of connected pads. To validate formats the
>> __media_pipeline_start() functions calls link_validate() on each
>> entity. You should in your driver set
>
> Is this done automatically ? if this would be the case, where all links
> are correctly set, then, in theory, it would not be possible that some
> entities have a format set that is not supported by the other side
>
>>
>> static const struct media_entity_operations your_media_entity_ops = {
>> .link_validate = v4l2_subdev_link_validate,
>> };
>>
>> if you want to use the default link validation procedure, or set the
>> callback to your custom validation function, which can behave more
>> or less like isc_try_validate_formats()
>
> Does this mean that I can ask the subdev for the format ?
> Or how can I obtain the format on the other side of the link ?
> link validate can do this for me ?
>
>
> Thanks for reviewing,
> Eugen
>>
>> Does it match your understanding too ?
>>
>> Thanks
>> j
>>
>>
>>
>>> if (ret)
>>> return ret;
>>>
>>> - v4l2_fill_mbus_format(&format.format, &f->fmt.pix, mbus_code);
>>> - ret = v4l2_subdev_call(isc->current_subdev->sd, pad,
>>> - set_fmt, NULL, &format);
>>> - if (ret < 0)
>>> - return ret;
>>> + /* Identify the subdev's format configuration */
>>> + for (i = 0; i < isc->num_user_formats; i++)
>>> + if (isc->user_formats[i]->mbus_code == format.format.code) {
>>> + sd_fmt = isc->user_formats[i];
>>> + break;
>>> + }
>>> +
>>> + /* Check if the format is not supported */
>>> + if (!sd_fmt) {
>>> + v4l2_err(&isc->v4l2_dev,
>>> + "Current subdevice is streaming a media bus code that is not supported 0x%x\n",
>>> + format.format.code);
>>> + return -EPIPE;
>>> + }
>>> +
>>> + /* At this moment we know which format the subdev will use */
>>> + isc->try_config.sd_format = sd_fmt;
>>> +
>>> + /* If the sensor is not RAW, we can only do a direct dump */
>>> + if (!ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code))
>>> + isc_try_configure_rlp_dma(isc, true);
>>>
>>> /* Limit to Atmel ISC hardware capabilities */
>>> - if (f->fmt.pix.width > isc->max_width)
>>> - f->fmt.pix.width = isc->max_width;
>>> - if (f->fmt.pix.height > isc->max_height)
>>> - f->fmt.pix.height = isc->max_height;
>>> + v4l_bound_align_image(&format.format.width, 16, isc->max_width, 0,
>>> + &format.format.height, 16, isc->max_height, 0, 0);
>>>
>>> - isc->fmt = *f;
>>> + /* Check if the frame size is the same. Otherwise we may overflow */
>>> + if (pixfmt->height != format.format.height ||
>>> + pixfmt->width != format.format.width) {
>>> + v4l2_err(&isc->v4l2_dev,
>>> + "ISC not configured with the proper frame size: %dx%d\n",
>>> + format.format.width, format.format.height);
>>> + return -EPIPE;
>>> + }
>>>
>>> + v4l2_dbg(1, debug, &isc->v4l2_dev,
>>> + "Identified subdev using format %.4s with %dx%d %d bpp\n",
>>> + (char *)&sd_fmt->fourcc, pixfmt->width, pixfmt->height,
>>> + isc->try_config.bpp);
>>> +
>>> + /* Reset and restart AWB if the subdevice changed the format */
>>> if (isc->try_config.sd_format && isc->config.sd_format &&
>>> isc->try_config.sd_format != isc->config.sd_format) {
>>> isc->ctrls.hist_stat = HIST_INIT;
>>> isc_reset_awb_ctrls(isc);
>>> isc_update_v4l2_ctrls(isc);
>>> }
>>> - /* make the try configuration active */
>>> +
>>> + /* Validate formats */
>>> + ret = isc_try_validate_formats(isc);
>>> + if (ret)
>>> + return ret;
>>> +
>>> + /* Obtain frame sizes if possible to have crop requirements ready */
>>> + isc_try_fse(isc, &pad_state);
>>> +
>>> + /* Configure ISC pipeline for the config */
>>> + ret = isc_try_configure_pipeline(isc);
>>> + if (ret)
>>> + return ret;
>>> +
>>> isc->config = isc->try_config;
>>>
>>> v4l2_dbg(1, debug, &isc->v4l2_dev, "New ISC configuration in place\n");
>>> @@ -1064,6 +1030,19 @@ static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
>>> return 0;
>>> }
>>>
>>> +static int isc_streamon(struct file *file, void *priv, enum v4l2_buf_type bt)
>>> +{
>>> + struct isc_device *isc = video_drvdata(file);
>>> + int ret;
>>> +
>>> + ret = isc_validate(isc);
>>> +
>>> + if (ret)
>>> + return ret;
>>> +
>>> + return vb2_ioctl_streamon(file, priv, bt);
>>> +}
>>> +
>>> static int isc_s_fmt_vid_cap(struct file *file, void *priv,
>>> struct v4l2_format *f)
>>> {
>>> @@ -1080,7 +1059,7 @@ static int isc_try_fmt_vid_cap(struct file *file, void *priv,
>>> {
>>> struct isc_device *isc = video_drvdata(file);
>>>
>>> - return isc_try_fmt(isc, f, NULL);
>>> + return isc_try_fmt(isc, f);
>>> }
>>>
>>> static int isc_enum_input(struct file *file, void *priv,
>>> @@ -1176,7 +1155,7 @@ static const struct v4l2_ioctl_ops isc_ioctl_ops = {
>>> .vidioc_dqbuf = vb2_ioctl_dqbuf,
>>> .vidioc_create_bufs = vb2_ioctl_create_bufs,
>>> .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>>> - .vidioc_streamon = vb2_ioctl_streamon,
>>> + .vidioc_streamon = isc_streamon,
>>> .vidioc_streamoff = vb2_ioctl_streamoff,
>>>
>>> .vidioc_g_parm = isc_g_parm,
>>> @@ -1879,7 +1858,7 @@ static int isc_set_default_fmt(struct isc_device *isc)
>>> };
>>> int ret;
>>>
>>> - ret = isc_try_fmt(isc, &f, NULL);
>>> + ret = isc_try_fmt(isc, &f);
>>> if (ret)
>>> return ret;
>>>
>>> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
>>> index c2cb805faff3..7081698adddd 100644
>>> --- a/drivers/media/platform/atmel/atmel-isc.h
>>> +++ b/drivers/media/platform/atmel/atmel-isc.h
>>> @@ -297,6 +297,7 @@ struct isc_device {
>>> struct completion comp;
>>>
>>> struct v4l2_format fmt;
>>> + struct v4l2_format try_fmt;
>>> struct isc_format **user_formats;
>>> unsigned int num_user_formats;
>>>
>>> --
>>> 2.25.1
>>>
>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 23/23] media: atmel: atmel-isc: change format propagation to subdev into only verification
@ 2022-01-20 8:48 ` Eugen.Hristev
0 siblings, 0 replies; 94+ messages in thread
From: Eugen.Hristev @ 2022-01-20 8:48 UTC (permalink / raw)
To: jacopo
Cc: devicetree, linux-kernel, robh+dt, sakari.ailus,
laurent.pinchart, linux-arm-kernel, linux-media
On 1/20/22 10:43 AM, Eugen Hristev - M18282 wrote:
> On 1/12/22 11:21 AM, Jacopo Mondi wrote:
>> Hi Eugen
>
>
> Hi Jacopo,
>
>>
>> On Mon, Dec 13, 2021 at 03:49:40PM +0200, Eugen Hristev wrote:
>>> As a top MC video driver, the atmel-isc should not propagate the format to the
>>> subdevice.
>>> It should rather check at streamon() time if the subdev is properly configured
>>> with a compatible format.
>>> Removed the whole format finding logic, and reworked the format verification
>>> at streamon time, such that the ISC will return an error if the subdevice
>>> is not properly configured.
>>> With this being done, the module parameter 'sensor_prefered' makes no sense
>>> anymore. The ISC should not decide which format the sensor is using. The
>>> ISC should only cope with the situation and inform userspace if the streaming
>>> is possible in the current configuration.
>>
>> Sounds great!
>>
>>>
>>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>>> ---
>>> Changes in v3:
>>> - clamp to maximum resolution once the frame size from the subdev is found
>>>
>>> drivers/media/platform/atmel/atmel-isc-base.c | 271 ++++++++----------
>>> drivers/media/platform/atmel/atmel-isc.h | 1 +
>>> 2 files changed, 126 insertions(+), 146 deletions(-)
>>>
>>> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
>>> index 31c8e3029eee..00c8c9588a78 100644
>>> --- a/drivers/media/platform/atmel/atmel-isc-base.c
>>> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
>>> @@ -36,11 +36,6 @@ static unsigned int debug;
>>> module_param(debug, int, 0644);
>>> MODULE_PARM_DESC(debug, "debug level (0-2)");
>>>
>>> -static unsigned int sensor_preferred = 1;
>>> -module_param(sensor_preferred, uint, 0644);
>>> -MODULE_PARM_DESC(sensor_preferred,
>>> - "Sensor is preferred to output the specified format (1-on 0-off), default 1");
>>> -
>>> #define ISC_IS_FORMAT_RAW(mbus_code) \
>>> (((mbus_code) & 0xf000) == 0x3000)
>>>
>>> @@ -532,7 +527,7 @@ static int isc_enum_fmt_vid_cap(struct file *file, void *priv,
>>> * convert it to any of the formats that we usually can with a
>>> * RAW sensor. Thus, do not advertise them.
>>> */
>>> - if (!isc->config.sd_format ||
>>> + if (isc->config.sd_format &&
>>
>> Is this change intentional ?
>>
>>> !ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
>>> return -EINVAL;
>>>
>>> @@ -621,20 +616,30 @@ static int isc_try_validate_formats(struct isc_device *isc)
>>> break;
>>> default:
>>> /* any other different formats are not supported */
>>> + v4l2_err(&isc->v4l2_dev, "Requested unsupported format.\n");
>>> ret = -EINVAL;
>>> }
>>> v4l2_dbg(1, debug, &isc->v4l2_dev,
>>> "Format validation, requested rgb=%u, yuv=%u, grey=%u, bayer=%u\n",
>>> rgb, yuv, grey, bayer);
>>
>> Would it make sense to move this before the switch so that the
>> error messages, if any, appear later ?
>
> Actually, no, because the variables rgb, yuv, grey, bayer, are set
> according to what happens in the switch
>>
>>>
>>> - /* we cannot output RAW if we do not receive RAW */
>>> - if ((bayer) && !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code))
>>> + if ((bayer) &&
>>> + !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code)) {
>>> + v4l2_err(&isc->v4l2_dev, "Cannot output RAW if we do not receive RAW.\n");
>>> return -EINVAL;
>>> + }
>>>
>>> - /* we cannot output GREY if we do not receive RAW/GREY */
>>> if (grey && !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code) &&
>>> - !ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code))
>>> + !ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) {
>>> + v4l2_err(&isc->v4l2_dev, "Cannot output GREY if we do not receive RAW/GREY.\n");
>>> return -EINVAL;
>>> + }
>>> +
>>> + if ((rgb || bayer || yuv) &&
>>> + ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) {
>>> + v4l2_err(&isc->v4l2_dev, "Cannot convert GREY to another format.\n");
>>> + return -EINVAL;
>>> + }
>>>
>>> return ret;
>>> }
>>> @@ -862,7 +867,7 @@ static void isc_try_fse(struct isc_device *isc,
>>> * If we do not know yet which format the subdev is using, we cannot
>>> * do anything.
>>> */
>>> - if (!isc->try_config.sd_format)
>>> + if (!isc->config.sd_format)
>>> return;
>>>
>>> fse.code = isc->try_config.sd_format->mbus_code;
>>> @@ -883,180 +888,141 @@ static void isc_try_fse(struct isc_device *isc,
>>> }
>>> }
>>>
>>> -static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f,
>>> - u32 *code)
>>> +static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f)
>>> {
>>> - int i;
>>> - struct isc_format *sd_fmt = NULL, *direct_fmt = NULL;
>>> struct v4l2_pix_format *pixfmt = &f->fmt.pix;
>>> - struct v4l2_subdev_pad_config pad_cfg = {};
>>> - struct v4l2_subdev_state pad_state = {
>>> - .pads = &pad_cfg
>>> - };
>>> - struct v4l2_subdev_format format = {
>>> - .which = V4L2_SUBDEV_FORMAT_TRY,
>>> - };
>>> - u32 mbus_code;
>>> - int ret;
>>> - bool rlp_dma_direct_dump = false;
>>> + unsigned int i;
>>>
>>> if (f->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
>>> return -EINVAL;
>>>
>>> - /* Step 1: find a RAW format that is supported */
>>> - for (i = 0; i < isc->num_user_formats; i++) {
>>> - if (ISC_IS_FORMAT_RAW(isc->user_formats[i]->mbus_code)) {
>>> - sd_fmt = isc->user_formats[i];
>>> + isc->try_config.fourcc = isc->user_formats[0]->fourcc;
>>> +
>>> + /* find if the format requested is supported */
>>> + for (i = 0; i < isc->controller_formats_size; i++)
>>> + if (isc->controller_formats[i].fourcc == pixfmt->pixelformat) {
>>> + isc->try_config.fourcc = pixfmt->pixelformat;
>>> break;
>>> }
>>> - }
>>> - /* Step 2: We can continue with this RAW format, or we can look
>>> - * for better: maybe sensor supports directly what we need.
>>> - */
>>> - direct_fmt = find_format_by_fourcc(isc, pixfmt->pixelformat);
>>> -
>>> - /* Step 3: We have both. We decide given the module parameter which
>>> - * one to use.
>>> - */
>>> - if (direct_fmt && sd_fmt && sensor_preferred)
>>> - sd_fmt = direct_fmt;
>>> -
>>> - /* Step 4: we do not have RAW but we have a direct format. Use it. */
>>> - if (direct_fmt && !sd_fmt)
>>> - sd_fmt = direct_fmt;
>>> -
>>> - /* Step 5: if we are using a direct format, we need to package
>>> - * everything as 8 bit data and just dump it
>>> - */
>>> - if (sd_fmt == direct_fmt)
>>> - rlp_dma_direct_dump = true;
>>> -
>>> - /* Step 6: We have no format. This can happen if the userspace
>>> - * requests some weird/invalid format.
>>> - * In this case, default to whatever we have
>>> - */
>>> - if (!sd_fmt && !direct_fmt) {
>>> - sd_fmt = isc->user_formats[isc->num_user_formats - 1];
>>> - v4l2_dbg(1, debug, &isc->v4l2_dev,
>>> - "Sensor not supporting %.4s, using %.4s\n",
>>> - (char *)&pixfmt->pixelformat, (char *)&sd_fmt->fourcc);
>>> - }
>>> -
>>> - if (!sd_fmt) {
>>> - ret = -EINVAL;
>>> - goto isc_try_fmt_err;
>>> - }
>>> -
>>> - /* Step 7: Print out what we decided for debugging */
>>> - v4l2_dbg(1, debug, &isc->v4l2_dev,
>>> - "Preferring to have sensor using format %.4s\n",
>>> - (char *)&sd_fmt->fourcc);
>>> -
>>> - /* Step 8: at this moment we decided which format the subdev will use */
>>> - isc->try_config.sd_format = sd_fmt;
>>> -
>>> - /* Limit to Atmel ISC hardware capabilities */
>>> - if (pixfmt->width > isc->max_width)
>>> - pixfmt->width = isc->max_width;
>>> - if (pixfmt->height > isc->max_height)
>>> - pixfmt->height = isc->max_height;
>>> -
>>> - /*
>>> - * The mbus format is the one the subdev outputs.
>>> - * The pixels will be transferred in this format Sensor -> ISC
>>> - */
>>> - mbus_code = sd_fmt->mbus_code;
>>> -
>>> - /*
>>> - * Validate formats. If the required format is not OK, default to raw.
>>> - */
>>> -
>>> - isc->try_config.fourcc = pixfmt->pixelformat;
>>> -
>>> - if (isc_try_validate_formats(isc)) {
>>> - pixfmt->pixelformat = isc->try_config.fourcc = sd_fmt->fourcc;
>>> - /* Re-try to validate the new format */
>>> - ret = isc_try_validate_formats(isc);
>>> - if (ret)
>>> - goto isc_try_fmt_err;
>>> - }
>>> -
>>> - ret = isc_try_configure_rlp_dma(isc, rlp_dma_direct_dump);
>>> - if (ret)
>>> - goto isc_try_fmt_err;
>>> -
>>> - ret = isc_try_configure_pipeline(isc);
>>> - if (ret)
>>> - goto isc_try_fmt_err;
>>>
>>> - /* Obtain frame sizes if possible to have crop requirements ready */
>>> - isc_try_fse(isc, &pad_state);
>>> -
>>> - v4l2_fill_mbus_format(&format.format, pixfmt, mbus_code);
>>> - ret = v4l2_subdev_call(isc->current_subdev->sd, pad, set_fmt,
>>> - &pad_state, &format);
>>> - if (ret < 0)
>>> - goto isc_try_fmt_subdev_err;
>>> + /* If we did not find the requested format, we will fallback here */
>>> + pixfmt->pixelformat = isc->try_config.fourcc;
>>> + pixfmt->colorspace = V4L2_COLORSPACE_SRGB;
>>> + pixfmt->field = V4L2_FIELD_NONE;
>>>
>>> - v4l2_fill_pix_format(pixfmt, &format.format);
>>> + isc_try_configure_rlp_dma(isc, false);
>>>
>>> /* Limit to Atmel ISC hardware capabilities */
>>> - if (pixfmt->width > isc->max_width)
>>> - pixfmt->width = isc->max_width;
>>> - if (pixfmt->height > isc->max_height)
>>> - pixfmt->height = isc->max_height;
>>> + v4l_bound_align_image(&pixfmt->width, 16, isc->max_width, 0,
>>> + &pixfmt->height, 16, isc->max_height, 0, 0);
>>>
>>> pixfmt->field = V4L2_FIELD_NONE;
>>> pixfmt->bytesperline = (pixfmt->width * isc->try_config.bpp_v4l2) >> 3;
>>> pixfmt->sizeimage = ((pixfmt->width * isc->try_config.bpp) >> 3) *
>>> pixfmt->height;
>>>
>>> - if (code)
>>> - *code = mbus_code;
>>> + isc->try_fmt = *f;
>>>
>>> return 0;
>>> +}
>>>
>>> -isc_try_fmt_err:
>>> - v4l2_err(&isc->v4l2_dev, "Could not find any possible format for a working pipeline\n");
>>> -isc_try_fmt_subdev_err:
>>> - memset(&isc->try_config, 0, sizeof(isc->try_config));
>>> +static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
>>> +{
>>> + isc_try_fmt(isc, f);
>>>
>>> - return ret;
>>> + /* make the try configuration active */
>>> + isc->config = isc->try_config;
>>> + isc->fmt = isc->try_fmt;
>>> +
>>> + v4l2_dbg(1, debug, &isc->v4l2_dev, "ISC set_fmt to %.4s @%dx%d\n",
>>> + (char *)&f->fmt.pix.pixelformat,
>>> + f->fmt.pix.width, f->fmt.pix.height);
>>> +
>>> + return 0;
>>> }
>>>
>>> -static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
>>> +static int isc_validate(struct isc_device *isc)
>>> {
>>> + int ret;
>>> + int i;
>>> + struct isc_format *sd_fmt = NULL;
>>> + struct v4l2_pix_format *pixfmt = &isc->fmt.fmt.pix;
>>> struct v4l2_subdev_format format = {
>>> .which = V4L2_SUBDEV_FORMAT_ACTIVE,
>>> + .pad = isc->remote_pad,
>>> + };
>>> + struct v4l2_subdev_pad_config pad_cfg = {};
>>> + struct v4l2_subdev_state pad_state = {
>>> + .pads = &pad_cfg,
>>> };
>>> - u32 mbus_code = 0;
>>> - int ret;
>>>
>>> - ret = isc_try_fmt(isc, f, &mbus_code);
>>> + /* Get current format from subdev */
>>> + ret = v4l2_subdev_call(isc->current_subdev->sd, pad, get_fmt, NULL,
>>> + &format);
>>
>> Ah! Haven't you just said we don't care anymore about the subdev
>> format ? :)
>
> We don't ! but we can't stream anything if the sensor streams a format
> that we don't understand do we ?
Actually, thinking about this, we really need to know what format is at
the input for the ISC. According to this format, we need to understand
how to process it, since the ISC could have at input either Raw bayer
and we need to know the rotation, or directly a non-raw format like yuv422 .
Rather than asking the subdev for the streaming format, we could obtain
this information from the pad link ?
>
>>
>> Kidding, I might got a bit lost in the logic, but if I look at your
>> above isc_try_validate_formats() it seems like an ideal candidate for
>> .link_validate() media_entity operation.
>>
>> Just to make sure we're on the same page, here's how it should ideally
>> look like:
>> - set format does care about subdev format. It only checks that the
>> format required from the user is one of the ISC supported one. Ie.
>> no v4l2_subdev_call()
>>
>> - as s_stream time your top driver calls media_pipeline_start()
>>
>> - media_pipeline_start() walks all the entities in the pipeline and
>> validates the format of connected pads. To validate formats the
>> __media_pipeline_start() functions calls link_validate() on each
>> entity. You should in your driver set
>
> Is this done automatically ? if this would be the case, where all links
> are correctly set, then, in theory, it would not be possible that some
> entities have a format set that is not supported by the other side
>
>>
>> static const struct media_entity_operations your_media_entity_ops = {
>> .link_validate = v4l2_subdev_link_validate,
>> };
>>
>> if you want to use the default link validation procedure, or set the
>> callback to your custom validation function, which can behave more
>> or less like isc_try_validate_formats()
>
> Does this mean that I can ask the subdev for the format ?
> Or how can I obtain the format on the other side of the link ?
> link validate can do this for me ?
>
>
> Thanks for reviewing,
> Eugen
>>
>> Does it match your understanding too ?
>>
>> Thanks
>> j
>>
>>
>>
>>> if (ret)
>>> return ret;
>>>
>>> - v4l2_fill_mbus_format(&format.format, &f->fmt.pix, mbus_code);
>>> - ret = v4l2_subdev_call(isc->current_subdev->sd, pad,
>>> - set_fmt, NULL, &format);
>>> - if (ret < 0)
>>> - return ret;
>>> + /* Identify the subdev's format configuration */
>>> + for (i = 0; i < isc->num_user_formats; i++)
>>> + if (isc->user_formats[i]->mbus_code == format.format.code) {
>>> + sd_fmt = isc->user_formats[i];
>>> + break;
>>> + }
>>> +
>>> + /* Check if the format is not supported */
>>> + if (!sd_fmt) {
>>> + v4l2_err(&isc->v4l2_dev,
>>> + "Current subdevice is streaming a media bus code that is not supported 0x%x\n",
>>> + format.format.code);
>>> + return -EPIPE;
>>> + }
>>> +
>>> + /* At this moment we know which format the subdev will use */
>>> + isc->try_config.sd_format = sd_fmt;
>>> +
>>> + /* If the sensor is not RAW, we can only do a direct dump */
>>> + if (!ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code))
>>> + isc_try_configure_rlp_dma(isc, true);
>>>
>>> /* Limit to Atmel ISC hardware capabilities */
>>> - if (f->fmt.pix.width > isc->max_width)
>>> - f->fmt.pix.width = isc->max_width;
>>> - if (f->fmt.pix.height > isc->max_height)
>>> - f->fmt.pix.height = isc->max_height;
>>> + v4l_bound_align_image(&format.format.width, 16, isc->max_width, 0,
>>> + &format.format.height, 16, isc->max_height, 0, 0);
>>>
>>> - isc->fmt = *f;
>>> + /* Check if the frame size is the same. Otherwise we may overflow */
>>> + if (pixfmt->height != format.format.height ||
>>> + pixfmt->width != format.format.width) {
>>> + v4l2_err(&isc->v4l2_dev,
>>> + "ISC not configured with the proper frame size: %dx%d\n",
>>> + format.format.width, format.format.height);
>>> + return -EPIPE;
>>> + }
>>>
>>> + v4l2_dbg(1, debug, &isc->v4l2_dev,
>>> + "Identified subdev using format %.4s with %dx%d %d bpp\n",
>>> + (char *)&sd_fmt->fourcc, pixfmt->width, pixfmt->height,
>>> + isc->try_config.bpp);
>>> +
>>> + /* Reset and restart AWB if the subdevice changed the format */
>>> if (isc->try_config.sd_format && isc->config.sd_format &&
>>> isc->try_config.sd_format != isc->config.sd_format) {
>>> isc->ctrls.hist_stat = HIST_INIT;
>>> isc_reset_awb_ctrls(isc);
>>> isc_update_v4l2_ctrls(isc);
>>> }
>>> - /* make the try configuration active */
>>> +
>>> + /* Validate formats */
>>> + ret = isc_try_validate_formats(isc);
>>> + if (ret)
>>> + return ret;
>>> +
>>> + /* Obtain frame sizes if possible to have crop requirements ready */
>>> + isc_try_fse(isc, &pad_state);
>>> +
>>> + /* Configure ISC pipeline for the config */
>>> + ret = isc_try_configure_pipeline(isc);
>>> + if (ret)
>>> + return ret;
>>> +
>>> isc->config = isc->try_config;
>>>
>>> v4l2_dbg(1, debug, &isc->v4l2_dev, "New ISC configuration in place\n");
>>> @@ -1064,6 +1030,19 @@ static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
>>> return 0;
>>> }
>>>
>>> +static int isc_streamon(struct file *file, void *priv, enum v4l2_buf_type bt)
>>> +{
>>> + struct isc_device *isc = video_drvdata(file);
>>> + int ret;
>>> +
>>> + ret = isc_validate(isc);
>>> +
>>> + if (ret)
>>> + return ret;
>>> +
>>> + return vb2_ioctl_streamon(file, priv, bt);
>>> +}
>>> +
>>> static int isc_s_fmt_vid_cap(struct file *file, void *priv,
>>> struct v4l2_format *f)
>>> {
>>> @@ -1080,7 +1059,7 @@ static int isc_try_fmt_vid_cap(struct file *file, void *priv,
>>> {
>>> struct isc_device *isc = video_drvdata(file);
>>>
>>> - return isc_try_fmt(isc, f, NULL);
>>> + return isc_try_fmt(isc, f);
>>> }
>>>
>>> static int isc_enum_input(struct file *file, void *priv,
>>> @@ -1176,7 +1155,7 @@ static const struct v4l2_ioctl_ops isc_ioctl_ops = {
>>> .vidioc_dqbuf = vb2_ioctl_dqbuf,
>>> .vidioc_create_bufs = vb2_ioctl_create_bufs,
>>> .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>>> - .vidioc_streamon = vb2_ioctl_streamon,
>>> + .vidioc_streamon = isc_streamon,
>>> .vidioc_streamoff = vb2_ioctl_streamoff,
>>>
>>> .vidioc_g_parm = isc_g_parm,
>>> @@ -1879,7 +1858,7 @@ static int isc_set_default_fmt(struct isc_device *isc)
>>> };
>>> int ret;
>>>
>>> - ret = isc_try_fmt(isc, &f, NULL);
>>> + ret = isc_try_fmt(isc, &f);
>>> if (ret)
>>> return ret;
>>>
>>> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
>>> index c2cb805faff3..7081698adddd 100644
>>> --- a/drivers/media/platform/atmel/atmel-isc.h
>>> +++ b/drivers/media/platform/atmel/atmel-isc.h
>>> @@ -297,6 +297,7 @@ struct isc_device {
>>> struct completion comp;
>>>
>>> struct v4l2_format fmt;
>>> + struct v4l2_format try_fmt;
>>> struct isc_format **user_formats;
>>> unsigned int num_user_formats;
>>>
>>> --
>>> 2.25.1
>>>
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 23/23] media: atmel: atmel-isc: change format propagation to subdev into only verification
2022-01-20 8:43 ` Eugen.Hristev
@ 2022-01-20 8:58 ` Jacopo Mondi
-1 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-20 8:58 UTC (permalink / raw)
To: Eugen.Hristev
Cc: linux-media, robh+dt, laurent.pinchart, sakari.ailus, devicetree,
linux-kernel, linux-arm-kernel, Nicolas.Ferre
Hi Eugen
On Thu, Jan 20, 2022 at 08:43:33AM +0000, Eugen.Hristev@microchip.com wrote:
> On 1/12/22 11:21 AM, Jacopo Mondi wrote:
> > Hi Eugen
>
>
> Hi Jacopo,
>
> >
> > On Mon, Dec 13, 2021 at 03:49:40PM +0200, Eugen Hristev wrote:
> >> As a top MC video driver, the atmel-isc should not propagate the format to the
> >> subdevice.
> >> It should rather check at streamon() time if the subdev is properly configured
> >> with a compatible format.
> >> Removed the whole format finding logic, and reworked the format verification
> >> at streamon time, such that the ISC will return an error if the subdevice
> >> is not properly configured.
> >> With this being done, the module parameter 'sensor_prefered' makes no sense
> >> anymore. The ISC should not decide which format the sensor is using. The
> >> ISC should only cope with the situation and inform userspace if the streaming
> >> is possible in the current configuration.
> >
> > Sounds great!
> >
> >>
> >> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> >> ---
> >> Changes in v3:
> >> - clamp to maximum resolution once the frame size from the subdev is found
> >>
> >> drivers/media/platform/atmel/atmel-isc-base.c | 271 ++++++++----------
> >> drivers/media/platform/atmel/atmel-isc.h | 1 +
> >> 2 files changed, 126 insertions(+), 146 deletions(-)
> >>
> >> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> >> index 31c8e3029eee..00c8c9588a78 100644
> >> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> >> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> >> @@ -36,11 +36,6 @@ static unsigned int debug;
> >> module_param(debug, int, 0644);
> >> MODULE_PARM_DESC(debug, "debug level (0-2)");
> >>
> >> -static unsigned int sensor_preferred = 1;
> >> -module_param(sensor_preferred, uint, 0644);
> >> -MODULE_PARM_DESC(sensor_preferred,
> >> - "Sensor is preferred to output the specified format (1-on 0-off), default 1");
> >> -
> >> #define ISC_IS_FORMAT_RAW(mbus_code) \
> >> (((mbus_code) & 0xf000) == 0x3000)
> >>
> >> @@ -532,7 +527,7 @@ static int isc_enum_fmt_vid_cap(struct file *file, void *priv,
> >> * convert it to any of the formats that we usually can with a
> >> * RAW sensor. Thus, do not advertise them.
> >> */
> >> - if (!isc->config.sd_format ||
> >> + if (isc->config.sd_format &&
> >
> > Is this change intentional ?
> >
> >> !ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
> >> return -EINVAL;
> >>
> >> @@ -621,20 +616,30 @@ static int isc_try_validate_formats(struct isc_device *isc)
> >> break;
> >> default:
> >> /* any other different formats are not supported */
> >> + v4l2_err(&isc->v4l2_dev, "Requested unsupported format.\n");
> >> ret = -EINVAL;
> >> }
> >> v4l2_dbg(1, debug, &isc->v4l2_dev,
> >> "Format validation, requested rgb=%u, yuv=%u, grey=%u, bayer=%u\n",
> >> rgb, yuv, grey, bayer);
> >
> > Would it make sense to move this before the switch so that the
> > error messages, if any, appear later ?
>
> Actually, no, because the variables rgb, yuv, grey, bayer, are set
> according to what happens in the switch
> >
> >>
> >> - /* we cannot output RAW if we do not receive RAW */
> >> - if ((bayer) && !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code))
> >> + if ((bayer) &&
> >> + !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code)) {
> >> + v4l2_err(&isc->v4l2_dev, "Cannot output RAW if we do not receive RAW.\n");
> >> return -EINVAL;
> >> + }
> >>
> >> - /* we cannot output GREY if we do not receive RAW/GREY */
> >> if (grey && !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code) &&
> >> - !ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code))
> >> + !ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) {
> >> + v4l2_err(&isc->v4l2_dev, "Cannot output GREY if we do not receive RAW/GREY.\n");
> >> return -EINVAL;
> >> + }
> >> +
> >> + if ((rgb || bayer || yuv) &&
> >> + ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) {
> >> + v4l2_err(&isc->v4l2_dev, "Cannot convert GREY to another format.\n");
> >> + return -EINVAL;
> >> + }
> >>
> >> return ret;
> >> }
> >> @@ -862,7 +867,7 @@ static void isc_try_fse(struct isc_device *isc,
> >> * If we do not know yet which format the subdev is using, we cannot
> >> * do anything.
> >> */
> >> - if (!isc->try_config.sd_format)
> >> + if (!isc->config.sd_format)
> >> return;
> >>
> >> fse.code = isc->try_config.sd_format->mbus_code;
> >> @@ -883,180 +888,141 @@ static void isc_try_fse(struct isc_device *isc,
> >> }
> >> }
> >>
> >> -static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f,
> >> - u32 *code)
> >> +static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f)
> >> {
> >> - int i;
> >> - struct isc_format *sd_fmt = NULL, *direct_fmt = NULL;
> >> struct v4l2_pix_format *pixfmt = &f->fmt.pix;
> >> - struct v4l2_subdev_pad_config pad_cfg = {};
> >> - struct v4l2_subdev_state pad_state = {
> >> - .pads = &pad_cfg
> >> - };
> >> - struct v4l2_subdev_format format = {
> >> - .which = V4L2_SUBDEV_FORMAT_TRY,
> >> - };
> >> - u32 mbus_code;
> >> - int ret;
> >> - bool rlp_dma_direct_dump = false;
> >> + unsigned int i;
> >>
> >> if (f->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
> >> return -EINVAL;
> >>
> >> - /* Step 1: find a RAW format that is supported */
> >> - for (i = 0; i < isc->num_user_formats; i++) {
> >> - if (ISC_IS_FORMAT_RAW(isc->user_formats[i]->mbus_code)) {
> >> - sd_fmt = isc->user_formats[i];
> >> + isc->try_config.fourcc = isc->user_formats[0]->fourcc;
> >> +
> >> + /* find if the format requested is supported */
> >> + for (i = 0; i < isc->controller_formats_size; i++)
> >> + if (isc->controller_formats[i].fourcc == pixfmt->pixelformat) {
> >> + isc->try_config.fourcc = pixfmt->pixelformat;
> >> break;
> >> }
> >> - }
> >> - /* Step 2: We can continue with this RAW format, or we can look
> >> - * for better: maybe sensor supports directly what we need.
> >> - */
> >> - direct_fmt = find_format_by_fourcc(isc, pixfmt->pixelformat);
> >> -
> >> - /* Step 3: We have both. We decide given the module parameter which
> >> - * one to use.
> >> - */
> >> - if (direct_fmt && sd_fmt && sensor_preferred)
> >> - sd_fmt = direct_fmt;
> >> -
> >> - /* Step 4: we do not have RAW but we have a direct format. Use it. */
> >> - if (direct_fmt && !sd_fmt)
> >> - sd_fmt = direct_fmt;
> >> -
> >> - /* Step 5: if we are using a direct format, we need to package
> >> - * everything as 8 bit data and just dump it
> >> - */
> >> - if (sd_fmt == direct_fmt)
> >> - rlp_dma_direct_dump = true;
> >> -
> >> - /* Step 6: We have no format. This can happen if the userspace
> >> - * requests some weird/invalid format.
> >> - * In this case, default to whatever we have
> >> - */
> >> - if (!sd_fmt && !direct_fmt) {
> >> - sd_fmt = isc->user_formats[isc->num_user_formats - 1];
> >> - v4l2_dbg(1, debug, &isc->v4l2_dev,
> >> - "Sensor not supporting %.4s, using %.4s\n",
> >> - (char *)&pixfmt->pixelformat, (char *)&sd_fmt->fourcc);
> >> - }
> >> -
> >> - if (!sd_fmt) {
> >> - ret = -EINVAL;
> >> - goto isc_try_fmt_err;
> >> - }
> >> -
> >> - /* Step 7: Print out what we decided for debugging */
> >> - v4l2_dbg(1, debug, &isc->v4l2_dev,
> >> - "Preferring to have sensor using format %.4s\n",
> >> - (char *)&sd_fmt->fourcc);
> >> -
> >> - /* Step 8: at this moment we decided which format the subdev will use */
> >> - isc->try_config.sd_format = sd_fmt;
> >> -
> >> - /* Limit to Atmel ISC hardware capabilities */
> >> - if (pixfmt->width > isc->max_width)
> >> - pixfmt->width = isc->max_width;
> >> - if (pixfmt->height > isc->max_height)
> >> - pixfmt->height = isc->max_height;
> >> -
> >> - /*
> >> - * The mbus format is the one the subdev outputs.
> >> - * The pixels will be transferred in this format Sensor -> ISC
> >> - */
> >> - mbus_code = sd_fmt->mbus_code;
> >> -
> >> - /*
> >> - * Validate formats. If the required format is not OK, default to raw.
> >> - */
> >> -
> >> - isc->try_config.fourcc = pixfmt->pixelformat;
> >> -
> >> - if (isc_try_validate_formats(isc)) {
> >> - pixfmt->pixelformat = isc->try_config.fourcc = sd_fmt->fourcc;
> >> - /* Re-try to validate the new format */
> >> - ret = isc_try_validate_formats(isc);
> >> - if (ret)
> >> - goto isc_try_fmt_err;
> >> - }
> >> -
> >> - ret = isc_try_configure_rlp_dma(isc, rlp_dma_direct_dump);
> >> - if (ret)
> >> - goto isc_try_fmt_err;
> >> -
> >> - ret = isc_try_configure_pipeline(isc);
> >> - if (ret)
> >> - goto isc_try_fmt_err;
> >>
> >> - /* Obtain frame sizes if possible to have crop requirements ready */
> >> - isc_try_fse(isc, &pad_state);
> >> -
> >> - v4l2_fill_mbus_format(&format.format, pixfmt, mbus_code);
> >> - ret = v4l2_subdev_call(isc->current_subdev->sd, pad, set_fmt,
> >> - &pad_state, &format);
> >> - if (ret < 0)
> >> - goto isc_try_fmt_subdev_err;
> >> + /* If we did not find the requested format, we will fallback here */
> >> + pixfmt->pixelformat = isc->try_config.fourcc;
> >> + pixfmt->colorspace = V4L2_COLORSPACE_SRGB;
> >> + pixfmt->field = V4L2_FIELD_NONE;
> >>
> >> - v4l2_fill_pix_format(pixfmt, &format.format);
> >> + isc_try_configure_rlp_dma(isc, false);
> >>
> >> /* Limit to Atmel ISC hardware capabilities */
> >> - if (pixfmt->width > isc->max_width)
> >> - pixfmt->width = isc->max_width;
> >> - if (pixfmt->height > isc->max_height)
> >> - pixfmt->height = isc->max_height;
> >> + v4l_bound_align_image(&pixfmt->width, 16, isc->max_width, 0,
> >> + &pixfmt->height, 16, isc->max_height, 0, 0);
> >>
> >> pixfmt->field = V4L2_FIELD_NONE;
> >> pixfmt->bytesperline = (pixfmt->width * isc->try_config.bpp_v4l2) >> 3;
> >> pixfmt->sizeimage = ((pixfmt->width * isc->try_config.bpp) >> 3) *
> >> pixfmt->height;
> >>
> >> - if (code)
> >> - *code = mbus_code;
> >> + isc->try_fmt = *f;
> >>
> >> return 0;
> >> +}
> >>
> >> -isc_try_fmt_err:
> >> - v4l2_err(&isc->v4l2_dev, "Could not find any possible format for a working pipeline\n");
> >> -isc_try_fmt_subdev_err:
> >> - memset(&isc->try_config, 0, sizeof(isc->try_config));
> >> +static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
> >> +{
> >> + isc_try_fmt(isc, f);
> >>
> >> - return ret;
> >> + /* make the try configuration active */
> >> + isc->config = isc->try_config;
> >> + isc->fmt = isc->try_fmt;
> >> +
> >> + v4l2_dbg(1, debug, &isc->v4l2_dev, "ISC set_fmt to %.4s @%dx%d\n",
> >> + (char *)&f->fmt.pix.pixelformat,
> >> + f->fmt.pix.width, f->fmt.pix.height);
> >> +
> >> + return 0;
> >> }
> >>
> >> -static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
> >> +static int isc_validate(struct isc_device *isc)
> >> {
> >> + int ret;
> >> + int i;
> >> + struct isc_format *sd_fmt = NULL;
> >> + struct v4l2_pix_format *pixfmt = &isc->fmt.fmt.pix;
> >> struct v4l2_subdev_format format = {
> >> .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> >> + .pad = isc->remote_pad,
> >> + };
> >> + struct v4l2_subdev_pad_config pad_cfg = {};
> >> + struct v4l2_subdev_state pad_state = {
> >> + .pads = &pad_cfg,
> >> };
> >> - u32 mbus_code = 0;
> >> - int ret;
> >>
> >> - ret = isc_try_fmt(isc, f, &mbus_code);
> >> + /* Get current format from subdev */
> >> + ret = v4l2_subdev_call(isc->current_subdev->sd, pad, get_fmt, NULL,
> >> + &format);
> >
> > Ah! Haven't you just said we don't care anymore about the subdev
> > format ? :)
>
> We don't ! but we can't stream anything if the sensor streams a format
> that we don't understand do we ?
>
> >
> > Kidding, I might got a bit lost in the logic, but if I look at your
> > above isc_try_validate_formats() it seems like an ideal candidate for
> > .link_validate() media_entity operation.
> >
> > Just to make sure we're on the same page, here's how it should ideally
> > look like:
> > - set format does care about subdev format. It only checks that the
> > format required from the user is one of the ISC supported one. Ie.
> > no v4l2_subdev_call()
> >
> > - as s_stream time your top driver calls media_pipeline_start()
> >
> > - media_pipeline_start() walks all the entities in the pipeline and
> > validates the format of connected pads. To validate formats the
> > __media_pipeline_start() functions calls link_validate() on each
> > entity. You should in your driver set
>
> Is this done automatically ? if this would be the case, where all links
Have a look at the media_pipeline_start() function in
drivers/media/mc/mc-entity.c
It walks the entities linked in the graph and calls .link_validate()
on each link it finds
> are correctly set, then, in theory, it would not be possible that some
> entities have a format set that is not supported by the other side
>
Not if validation is done correctly ;)
> >
> > static const struct media_entity_operations your_media_entity_ops = {
> > .link_validate = v4l2_subdev_link_validate,
> > };
> >
> > if you want to use the default link validation procedure, or set the
> > callback to your custom validation function, which can behave more
> > or less like isc_try_validate_formats()
>
> Does this mean that I can ask the subdev for the format ?
> Or how can I obtain the format on the other side of the link ?
> link validate can do this for me ?
If you implement your own validation function then yes.
Have a look at v4l2_subdev_link_validate() in v4l2-subdev.c
It fetches formats from each side of the link and matches them.
As you have some more complex validations, you can define your own
validation function and compare the formats on each side of the link
with the same logic as implemented in isc_try_validate_formats() (if I
remember the function name right)
Thanks
j
>
>
> Thanks for reviewing,
> Eugen
> >
> > Does it match your understanding too ?
> >
> > Thanks
> > j
> >
> >
> >
> >> if (ret)
> >> return ret;
> >>
> >> - v4l2_fill_mbus_format(&format.format, &f->fmt.pix, mbus_code);
> >> - ret = v4l2_subdev_call(isc->current_subdev->sd, pad,
> >> - set_fmt, NULL, &format);
> >> - if (ret < 0)
> >> - return ret;
> >> + /* Identify the subdev's format configuration */
> >> + for (i = 0; i < isc->num_user_formats; i++)
> >> + if (isc->user_formats[i]->mbus_code == format.format.code) {
> >> + sd_fmt = isc->user_formats[i];
> >> + break;
> >> + }
> >> +
> >> + /* Check if the format is not supported */
> >> + if (!sd_fmt) {
> >> + v4l2_err(&isc->v4l2_dev,
> >> + "Current subdevice is streaming a media bus code that is not supported 0x%x\n",
> >> + format.format.code);
> >> + return -EPIPE;
> >> + }
> >> +
> >> + /* At this moment we know which format the subdev will use */
> >> + isc->try_config.sd_format = sd_fmt;
> >> +
> >> + /* If the sensor is not RAW, we can only do a direct dump */
> >> + if (!ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code))
> >> + isc_try_configure_rlp_dma(isc, true);
> >>
> >> /* Limit to Atmel ISC hardware capabilities */
> >> - if (f->fmt.pix.width > isc->max_width)
> >> - f->fmt.pix.width = isc->max_width;
> >> - if (f->fmt.pix.height > isc->max_height)
> >> - f->fmt.pix.height = isc->max_height;
> >> + v4l_bound_align_image(&format.format.width, 16, isc->max_width, 0,
> >> + &format.format.height, 16, isc->max_height, 0, 0);
> >>
> >> - isc->fmt = *f;
> >> + /* Check if the frame size is the same. Otherwise we may overflow */
> >> + if (pixfmt->height != format.format.height ||
> >> + pixfmt->width != format.format.width) {
> >> + v4l2_err(&isc->v4l2_dev,
> >> + "ISC not configured with the proper frame size: %dx%d\n",
> >> + format.format.width, format.format.height);
> >> + return -EPIPE;
> >> + }
> >>
> >> + v4l2_dbg(1, debug, &isc->v4l2_dev,
> >> + "Identified subdev using format %.4s with %dx%d %d bpp\n",
> >> + (char *)&sd_fmt->fourcc, pixfmt->width, pixfmt->height,
> >> + isc->try_config.bpp);
> >> +
> >> + /* Reset and restart AWB if the subdevice changed the format */
> >> if (isc->try_config.sd_format && isc->config.sd_format &&
> >> isc->try_config.sd_format != isc->config.sd_format) {
> >> isc->ctrls.hist_stat = HIST_INIT;
> >> isc_reset_awb_ctrls(isc);
> >> isc_update_v4l2_ctrls(isc);
> >> }
> >> - /* make the try configuration active */
> >> +
> >> + /* Validate formats */
> >> + ret = isc_try_validate_formats(isc);
> >> + if (ret)
> >> + return ret;
> >> +
> >> + /* Obtain frame sizes if possible to have crop requirements ready */
> >> + isc_try_fse(isc, &pad_state);
> >> +
> >> + /* Configure ISC pipeline for the config */
> >> + ret = isc_try_configure_pipeline(isc);
> >> + if (ret)
> >> + return ret;
> >> +
> >> isc->config = isc->try_config;
> >>
> >> v4l2_dbg(1, debug, &isc->v4l2_dev, "New ISC configuration in place\n");
> >> @@ -1064,6 +1030,19 @@ static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
> >> return 0;
> >> }
> >>
> >> +static int isc_streamon(struct file *file, void *priv, enum v4l2_buf_type bt)
> >> +{
> >> + struct isc_device *isc = video_drvdata(file);
> >> + int ret;
> >> +
> >> + ret = isc_validate(isc);
> >> +
> >> + if (ret)
> >> + return ret;
> >> +
> >> + return vb2_ioctl_streamon(file, priv, bt);
> >> +}
> >> +
> >> static int isc_s_fmt_vid_cap(struct file *file, void *priv,
> >> struct v4l2_format *f)
> >> {
> >> @@ -1080,7 +1059,7 @@ static int isc_try_fmt_vid_cap(struct file *file, void *priv,
> >> {
> >> struct isc_device *isc = video_drvdata(file);
> >>
> >> - return isc_try_fmt(isc, f, NULL);
> >> + return isc_try_fmt(isc, f);
> >> }
> >>
> >> static int isc_enum_input(struct file *file, void *priv,
> >> @@ -1176,7 +1155,7 @@ static const struct v4l2_ioctl_ops isc_ioctl_ops = {
> >> .vidioc_dqbuf = vb2_ioctl_dqbuf,
> >> .vidioc_create_bufs = vb2_ioctl_create_bufs,
> >> .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> >> - .vidioc_streamon = vb2_ioctl_streamon,
> >> + .vidioc_streamon = isc_streamon,
> >> .vidioc_streamoff = vb2_ioctl_streamoff,
> >>
> >> .vidioc_g_parm = isc_g_parm,
> >> @@ -1879,7 +1858,7 @@ static int isc_set_default_fmt(struct isc_device *isc)
> >> };
> >> int ret;
> >>
> >> - ret = isc_try_fmt(isc, &f, NULL);
> >> + ret = isc_try_fmt(isc, &f);
> >> if (ret)
> >> return ret;
> >>
> >> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
> >> index c2cb805faff3..7081698adddd 100644
> >> --- a/drivers/media/platform/atmel/atmel-isc.h
> >> +++ b/drivers/media/platform/atmel/atmel-isc.h
> >> @@ -297,6 +297,7 @@ struct isc_device {
> >> struct completion comp;
> >>
> >> struct v4l2_format fmt;
> >> + struct v4l2_format try_fmt;
> >> struct isc_format **user_formats;
> >> unsigned int num_user_formats;
> >>
> >> --
> >> 2.25.1
> >>
>
^ permalink raw reply [flat|nested] 94+ messages in thread
* Re: [PATCH v3 23/23] media: atmel: atmel-isc: change format propagation to subdev into only verification
@ 2022-01-20 8:58 ` Jacopo Mondi
0 siblings, 0 replies; 94+ messages in thread
From: Jacopo Mondi @ 2022-01-20 8:58 UTC (permalink / raw)
To: Eugen.Hristev
Cc: devicetree, linux-kernel, robh+dt, sakari.ailus,
laurent.pinchart, linux-arm-kernel, linux-media
Hi Eugen
On Thu, Jan 20, 2022 at 08:43:33AM +0000, Eugen.Hristev@microchip.com wrote:
> On 1/12/22 11:21 AM, Jacopo Mondi wrote:
> > Hi Eugen
>
>
> Hi Jacopo,
>
> >
> > On Mon, Dec 13, 2021 at 03:49:40PM +0200, Eugen Hristev wrote:
> >> As a top MC video driver, the atmel-isc should not propagate the format to the
> >> subdevice.
> >> It should rather check at streamon() time if the subdev is properly configured
> >> with a compatible format.
> >> Removed the whole format finding logic, and reworked the format verification
> >> at streamon time, such that the ISC will return an error if the subdevice
> >> is not properly configured.
> >> With this being done, the module parameter 'sensor_prefered' makes no sense
> >> anymore. The ISC should not decide which format the sensor is using. The
> >> ISC should only cope with the situation and inform userspace if the streaming
> >> is possible in the current configuration.
> >
> > Sounds great!
> >
> >>
> >> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
> >> ---
> >> Changes in v3:
> >> - clamp to maximum resolution once the frame size from the subdev is found
> >>
> >> drivers/media/platform/atmel/atmel-isc-base.c | 271 ++++++++----------
> >> drivers/media/platform/atmel/atmel-isc.h | 1 +
> >> 2 files changed, 126 insertions(+), 146 deletions(-)
> >>
> >> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
> >> index 31c8e3029eee..00c8c9588a78 100644
> >> --- a/drivers/media/platform/atmel/atmel-isc-base.c
> >> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
> >> @@ -36,11 +36,6 @@ static unsigned int debug;
> >> module_param(debug, int, 0644);
> >> MODULE_PARM_DESC(debug, "debug level (0-2)");
> >>
> >> -static unsigned int sensor_preferred = 1;
> >> -module_param(sensor_preferred, uint, 0644);
> >> -MODULE_PARM_DESC(sensor_preferred,
> >> - "Sensor is preferred to output the specified format (1-on 0-off), default 1");
> >> -
> >> #define ISC_IS_FORMAT_RAW(mbus_code) \
> >> (((mbus_code) & 0xf000) == 0x3000)
> >>
> >> @@ -532,7 +527,7 @@ static int isc_enum_fmt_vid_cap(struct file *file, void *priv,
> >> * convert it to any of the formats that we usually can with a
> >> * RAW sensor. Thus, do not advertise them.
> >> */
> >> - if (!isc->config.sd_format ||
> >> + if (isc->config.sd_format &&
> >
> > Is this change intentional ?
> >
> >> !ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
> >> return -EINVAL;
> >>
> >> @@ -621,20 +616,30 @@ static int isc_try_validate_formats(struct isc_device *isc)
> >> break;
> >> default:
> >> /* any other different formats are not supported */
> >> + v4l2_err(&isc->v4l2_dev, "Requested unsupported format.\n");
> >> ret = -EINVAL;
> >> }
> >> v4l2_dbg(1, debug, &isc->v4l2_dev,
> >> "Format validation, requested rgb=%u, yuv=%u, grey=%u, bayer=%u\n",
> >> rgb, yuv, grey, bayer);
> >
> > Would it make sense to move this before the switch so that the
> > error messages, if any, appear later ?
>
> Actually, no, because the variables rgb, yuv, grey, bayer, are set
> according to what happens in the switch
> >
> >>
> >> - /* we cannot output RAW if we do not receive RAW */
> >> - if ((bayer) && !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code))
> >> + if ((bayer) &&
> >> + !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code)) {
> >> + v4l2_err(&isc->v4l2_dev, "Cannot output RAW if we do not receive RAW.\n");
> >> return -EINVAL;
> >> + }
> >>
> >> - /* we cannot output GREY if we do not receive RAW/GREY */
> >> if (grey && !ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code) &&
> >> - !ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code))
> >> + !ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) {
> >> + v4l2_err(&isc->v4l2_dev, "Cannot output GREY if we do not receive RAW/GREY.\n");
> >> return -EINVAL;
> >> + }
> >> +
> >> + if ((rgb || bayer || yuv) &&
> >> + ISC_IS_FORMAT_GREY(isc->try_config.sd_format->mbus_code)) {
> >> + v4l2_err(&isc->v4l2_dev, "Cannot convert GREY to another format.\n");
> >> + return -EINVAL;
> >> + }
> >>
> >> return ret;
> >> }
> >> @@ -862,7 +867,7 @@ static void isc_try_fse(struct isc_device *isc,
> >> * If we do not know yet which format the subdev is using, we cannot
> >> * do anything.
> >> */
> >> - if (!isc->try_config.sd_format)
> >> + if (!isc->config.sd_format)
> >> return;
> >>
> >> fse.code = isc->try_config.sd_format->mbus_code;
> >> @@ -883,180 +888,141 @@ static void isc_try_fse(struct isc_device *isc,
> >> }
> >> }
> >>
> >> -static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f,
> >> - u32 *code)
> >> +static int isc_try_fmt(struct isc_device *isc, struct v4l2_format *f)
> >> {
> >> - int i;
> >> - struct isc_format *sd_fmt = NULL, *direct_fmt = NULL;
> >> struct v4l2_pix_format *pixfmt = &f->fmt.pix;
> >> - struct v4l2_subdev_pad_config pad_cfg = {};
> >> - struct v4l2_subdev_state pad_state = {
> >> - .pads = &pad_cfg
> >> - };
> >> - struct v4l2_subdev_format format = {
> >> - .which = V4L2_SUBDEV_FORMAT_TRY,
> >> - };
> >> - u32 mbus_code;
> >> - int ret;
> >> - bool rlp_dma_direct_dump = false;
> >> + unsigned int i;
> >>
> >> if (f->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
> >> return -EINVAL;
> >>
> >> - /* Step 1: find a RAW format that is supported */
> >> - for (i = 0; i < isc->num_user_formats; i++) {
> >> - if (ISC_IS_FORMAT_RAW(isc->user_formats[i]->mbus_code)) {
> >> - sd_fmt = isc->user_formats[i];
> >> + isc->try_config.fourcc = isc->user_formats[0]->fourcc;
> >> +
> >> + /* find if the format requested is supported */
> >> + for (i = 0; i < isc->controller_formats_size; i++)
> >> + if (isc->controller_formats[i].fourcc == pixfmt->pixelformat) {
> >> + isc->try_config.fourcc = pixfmt->pixelformat;
> >> break;
> >> }
> >> - }
> >> - /* Step 2: We can continue with this RAW format, or we can look
> >> - * for better: maybe sensor supports directly what we need.
> >> - */
> >> - direct_fmt = find_format_by_fourcc(isc, pixfmt->pixelformat);
> >> -
> >> - /* Step 3: We have both. We decide given the module parameter which
> >> - * one to use.
> >> - */
> >> - if (direct_fmt && sd_fmt && sensor_preferred)
> >> - sd_fmt = direct_fmt;
> >> -
> >> - /* Step 4: we do not have RAW but we have a direct format. Use it. */
> >> - if (direct_fmt && !sd_fmt)
> >> - sd_fmt = direct_fmt;
> >> -
> >> - /* Step 5: if we are using a direct format, we need to package
> >> - * everything as 8 bit data and just dump it
> >> - */
> >> - if (sd_fmt == direct_fmt)
> >> - rlp_dma_direct_dump = true;
> >> -
> >> - /* Step 6: We have no format. This can happen if the userspace
> >> - * requests some weird/invalid format.
> >> - * In this case, default to whatever we have
> >> - */
> >> - if (!sd_fmt && !direct_fmt) {
> >> - sd_fmt = isc->user_formats[isc->num_user_formats - 1];
> >> - v4l2_dbg(1, debug, &isc->v4l2_dev,
> >> - "Sensor not supporting %.4s, using %.4s\n",
> >> - (char *)&pixfmt->pixelformat, (char *)&sd_fmt->fourcc);
> >> - }
> >> -
> >> - if (!sd_fmt) {
> >> - ret = -EINVAL;
> >> - goto isc_try_fmt_err;
> >> - }
> >> -
> >> - /* Step 7: Print out what we decided for debugging */
> >> - v4l2_dbg(1, debug, &isc->v4l2_dev,
> >> - "Preferring to have sensor using format %.4s\n",
> >> - (char *)&sd_fmt->fourcc);
> >> -
> >> - /* Step 8: at this moment we decided which format the subdev will use */
> >> - isc->try_config.sd_format = sd_fmt;
> >> -
> >> - /* Limit to Atmel ISC hardware capabilities */
> >> - if (pixfmt->width > isc->max_width)
> >> - pixfmt->width = isc->max_width;
> >> - if (pixfmt->height > isc->max_height)
> >> - pixfmt->height = isc->max_height;
> >> -
> >> - /*
> >> - * The mbus format is the one the subdev outputs.
> >> - * The pixels will be transferred in this format Sensor -> ISC
> >> - */
> >> - mbus_code = sd_fmt->mbus_code;
> >> -
> >> - /*
> >> - * Validate formats. If the required format is not OK, default to raw.
> >> - */
> >> -
> >> - isc->try_config.fourcc = pixfmt->pixelformat;
> >> -
> >> - if (isc_try_validate_formats(isc)) {
> >> - pixfmt->pixelformat = isc->try_config.fourcc = sd_fmt->fourcc;
> >> - /* Re-try to validate the new format */
> >> - ret = isc_try_validate_formats(isc);
> >> - if (ret)
> >> - goto isc_try_fmt_err;
> >> - }
> >> -
> >> - ret = isc_try_configure_rlp_dma(isc, rlp_dma_direct_dump);
> >> - if (ret)
> >> - goto isc_try_fmt_err;
> >> -
> >> - ret = isc_try_configure_pipeline(isc);
> >> - if (ret)
> >> - goto isc_try_fmt_err;
> >>
> >> - /* Obtain frame sizes if possible to have crop requirements ready */
> >> - isc_try_fse(isc, &pad_state);
> >> -
> >> - v4l2_fill_mbus_format(&format.format, pixfmt, mbus_code);
> >> - ret = v4l2_subdev_call(isc->current_subdev->sd, pad, set_fmt,
> >> - &pad_state, &format);
> >> - if (ret < 0)
> >> - goto isc_try_fmt_subdev_err;
> >> + /* If we did not find the requested format, we will fallback here */
> >> + pixfmt->pixelformat = isc->try_config.fourcc;
> >> + pixfmt->colorspace = V4L2_COLORSPACE_SRGB;
> >> + pixfmt->field = V4L2_FIELD_NONE;
> >>
> >> - v4l2_fill_pix_format(pixfmt, &format.format);
> >> + isc_try_configure_rlp_dma(isc, false);
> >>
> >> /* Limit to Atmel ISC hardware capabilities */
> >> - if (pixfmt->width > isc->max_width)
> >> - pixfmt->width = isc->max_width;
> >> - if (pixfmt->height > isc->max_height)
> >> - pixfmt->height = isc->max_height;
> >> + v4l_bound_align_image(&pixfmt->width, 16, isc->max_width, 0,
> >> + &pixfmt->height, 16, isc->max_height, 0, 0);
> >>
> >> pixfmt->field = V4L2_FIELD_NONE;
> >> pixfmt->bytesperline = (pixfmt->width * isc->try_config.bpp_v4l2) >> 3;
> >> pixfmt->sizeimage = ((pixfmt->width * isc->try_config.bpp) >> 3) *
> >> pixfmt->height;
> >>
> >> - if (code)
> >> - *code = mbus_code;
> >> + isc->try_fmt = *f;
> >>
> >> return 0;
> >> +}
> >>
> >> -isc_try_fmt_err:
> >> - v4l2_err(&isc->v4l2_dev, "Could not find any possible format for a working pipeline\n");
> >> -isc_try_fmt_subdev_err:
> >> - memset(&isc->try_config, 0, sizeof(isc->try_config));
> >> +static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
> >> +{
> >> + isc_try_fmt(isc, f);
> >>
> >> - return ret;
> >> + /* make the try configuration active */
> >> + isc->config = isc->try_config;
> >> + isc->fmt = isc->try_fmt;
> >> +
> >> + v4l2_dbg(1, debug, &isc->v4l2_dev, "ISC set_fmt to %.4s @%dx%d\n",
> >> + (char *)&f->fmt.pix.pixelformat,
> >> + f->fmt.pix.width, f->fmt.pix.height);
> >> +
> >> + return 0;
> >> }
> >>
> >> -static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
> >> +static int isc_validate(struct isc_device *isc)
> >> {
> >> + int ret;
> >> + int i;
> >> + struct isc_format *sd_fmt = NULL;
> >> + struct v4l2_pix_format *pixfmt = &isc->fmt.fmt.pix;
> >> struct v4l2_subdev_format format = {
> >> .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> >> + .pad = isc->remote_pad,
> >> + };
> >> + struct v4l2_subdev_pad_config pad_cfg = {};
> >> + struct v4l2_subdev_state pad_state = {
> >> + .pads = &pad_cfg,
> >> };
> >> - u32 mbus_code = 0;
> >> - int ret;
> >>
> >> - ret = isc_try_fmt(isc, f, &mbus_code);
> >> + /* Get current format from subdev */
> >> + ret = v4l2_subdev_call(isc->current_subdev->sd, pad, get_fmt, NULL,
> >> + &format);
> >
> > Ah! Haven't you just said we don't care anymore about the subdev
> > format ? :)
>
> We don't ! but we can't stream anything if the sensor streams a format
> that we don't understand do we ?
>
> >
> > Kidding, I might got a bit lost in the logic, but if I look at your
> > above isc_try_validate_formats() it seems like an ideal candidate for
> > .link_validate() media_entity operation.
> >
> > Just to make sure we're on the same page, here's how it should ideally
> > look like:
> > - set format does care about subdev format. It only checks that the
> > format required from the user is one of the ISC supported one. Ie.
> > no v4l2_subdev_call()
> >
> > - as s_stream time your top driver calls media_pipeline_start()
> >
> > - media_pipeline_start() walks all the entities in the pipeline and
> > validates the format of connected pads. To validate formats the
> > __media_pipeline_start() functions calls link_validate() on each
> > entity. You should in your driver set
>
> Is this done automatically ? if this would be the case, where all links
Have a look at the media_pipeline_start() function in
drivers/media/mc/mc-entity.c
It walks the entities linked in the graph and calls .link_validate()
on each link it finds
> are correctly set, then, in theory, it would not be possible that some
> entities have a format set that is not supported by the other side
>
Not if validation is done correctly ;)
> >
> > static const struct media_entity_operations your_media_entity_ops = {
> > .link_validate = v4l2_subdev_link_validate,
> > };
> >
> > if you want to use the default link validation procedure, or set the
> > callback to your custom validation function, which can behave more
> > or less like isc_try_validate_formats()
>
> Does this mean that I can ask the subdev for the format ?
> Or how can I obtain the format on the other side of the link ?
> link validate can do this for me ?
If you implement your own validation function then yes.
Have a look at v4l2_subdev_link_validate() in v4l2-subdev.c
It fetches formats from each side of the link and matches them.
As you have some more complex validations, you can define your own
validation function and compare the formats on each side of the link
with the same logic as implemented in isc_try_validate_formats() (if I
remember the function name right)
Thanks
j
>
>
> Thanks for reviewing,
> Eugen
> >
> > Does it match your understanding too ?
> >
> > Thanks
> > j
> >
> >
> >
> >> if (ret)
> >> return ret;
> >>
> >> - v4l2_fill_mbus_format(&format.format, &f->fmt.pix, mbus_code);
> >> - ret = v4l2_subdev_call(isc->current_subdev->sd, pad,
> >> - set_fmt, NULL, &format);
> >> - if (ret < 0)
> >> - return ret;
> >> + /* Identify the subdev's format configuration */
> >> + for (i = 0; i < isc->num_user_formats; i++)
> >> + if (isc->user_formats[i]->mbus_code == format.format.code) {
> >> + sd_fmt = isc->user_formats[i];
> >> + break;
> >> + }
> >> +
> >> + /* Check if the format is not supported */
> >> + if (!sd_fmt) {
> >> + v4l2_err(&isc->v4l2_dev,
> >> + "Current subdevice is streaming a media bus code that is not supported 0x%x\n",
> >> + format.format.code);
> >> + return -EPIPE;
> >> + }
> >> +
> >> + /* At this moment we know which format the subdev will use */
> >> + isc->try_config.sd_format = sd_fmt;
> >> +
> >> + /* If the sensor is not RAW, we can only do a direct dump */
> >> + if (!ISC_IS_FORMAT_RAW(isc->try_config.sd_format->mbus_code))
> >> + isc_try_configure_rlp_dma(isc, true);
> >>
> >> /* Limit to Atmel ISC hardware capabilities */
> >> - if (f->fmt.pix.width > isc->max_width)
> >> - f->fmt.pix.width = isc->max_width;
> >> - if (f->fmt.pix.height > isc->max_height)
> >> - f->fmt.pix.height = isc->max_height;
> >> + v4l_bound_align_image(&format.format.width, 16, isc->max_width, 0,
> >> + &format.format.height, 16, isc->max_height, 0, 0);
> >>
> >> - isc->fmt = *f;
> >> + /* Check if the frame size is the same. Otherwise we may overflow */
> >> + if (pixfmt->height != format.format.height ||
> >> + pixfmt->width != format.format.width) {
> >> + v4l2_err(&isc->v4l2_dev,
> >> + "ISC not configured with the proper frame size: %dx%d\n",
> >> + format.format.width, format.format.height);
> >> + return -EPIPE;
> >> + }
> >>
> >> + v4l2_dbg(1, debug, &isc->v4l2_dev,
> >> + "Identified subdev using format %.4s with %dx%d %d bpp\n",
> >> + (char *)&sd_fmt->fourcc, pixfmt->width, pixfmt->height,
> >> + isc->try_config.bpp);
> >> +
> >> + /* Reset and restart AWB if the subdevice changed the format */
> >> if (isc->try_config.sd_format && isc->config.sd_format &&
> >> isc->try_config.sd_format != isc->config.sd_format) {
> >> isc->ctrls.hist_stat = HIST_INIT;
> >> isc_reset_awb_ctrls(isc);
> >> isc_update_v4l2_ctrls(isc);
> >> }
> >> - /* make the try configuration active */
> >> +
> >> + /* Validate formats */
> >> + ret = isc_try_validate_formats(isc);
> >> + if (ret)
> >> + return ret;
> >> +
> >> + /* Obtain frame sizes if possible to have crop requirements ready */
> >> + isc_try_fse(isc, &pad_state);
> >> +
> >> + /* Configure ISC pipeline for the config */
> >> + ret = isc_try_configure_pipeline(isc);
> >> + if (ret)
> >> + return ret;
> >> +
> >> isc->config = isc->try_config;
> >>
> >> v4l2_dbg(1, debug, &isc->v4l2_dev, "New ISC configuration in place\n");
> >> @@ -1064,6 +1030,19 @@ static int isc_set_fmt(struct isc_device *isc, struct v4l2_format *f)
> >> return 0;
> >> }
> >>
> >> +static int isc_streamon(struct file *file, void *priv, enum v4l2_buf_type bt)
> >> +{
> >> + struct isc_device *isc = video_drvdata(file);
> >> + int ret;
> >> +
> >> + ret = isc_validate(isc);
> >> +
> >> + if (ret)
> >> + return ret;
> >> +
> >> + return vb2_ioctl_streamon(file, priv, bt);
> >> +}
> >> +
> >> static int isc_s_fmt_vid_cap(struct file *file, void *priv,
> >> struct v4l2_format *f)
> >> {
> >> @@ -1080,7 +1059,7 @@ static int isc_try_fmt_vid_cap(struct file *file, void *priv,
> >> {
> >> struct isc_device *isc = video_drvdata(file);
> >>
> >> - return isc_try_fmt(isc, f, NULL);
> >> + return isc_try_fmt(isc, f);
> >> }
> >>
> >> static int isc_enum_input(struct file *file, void *priv,
> >> @@ -1176,7 +1155,7 @@ static const struct v4l2_ioctl_ops isc_ioctl_ops = {
> >> .vidioc_dqbuf = vb2_ioctl_dqbuf,
> >> .vidioc_create_bufs = vb2_ioctl_create_bufs,
> >> .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> >> - .vidioc_streamon = vb2_ioctl_streamon,
> >> + .vidioc_streamon = isc_streamon,
> >> .vidioc_streamoff = vb2_ioctl_streamoff,
> >>
> >> .vidioc_g_parm = isc_g_parm,
> >> @@ -1879,7 +1858,7 @@ static int isc_set_default_fmt(struct isc_device *isc)
> >> };
> >> int ret;
> >>
> >> - ret = isc_try_fmt(isc, &f, NULL);
> >> + ret = isc_try_fmt(isc, &f);
> >> if (ret)
> >> return ret;
> >>
> >> diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h
> >> index c2cb805faff3..7081698adddd 100644
> >> --- a/drivers/media/platform/atmel/atmel-isc.h
> >> +++ b/drivers/media/platform/atmel/atmel-isc.h
> >> @@ -297,6 +297,7 @@ struct isc_device {
> >> struct completion comp;
> >>
> >> struct v4l2_format fmt;
> >> + struct v4l2_format try_fmt;
> >> struct isc_format **user_formats;
> >> unsigned int num_user_formats;
> >>
> >> --
> >> 2.25.1
> >>
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 94+ messages in thread
end of thread, other threads:[~2022-01-20 8:59 UTC | newest]
Thread overview: 94+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-12-13 13:49 [PATCH v3 00/23] media: atmel: atmel-isc: implement media controller Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 01/23] MAINTAINERS: add microchip csi2dc Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 02/23] dt-bindings: media: atmel: csi2dc: add bindings for " Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2021-12-15 20:16 ` Rob Herring
2021-12-15 20:16 ` Rob Herring
2021-12-13 13:49 ` [PATCH v3 03/23] media: atmel: introduce microchip csi2dc driver Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2022-01-11 15:43 ` Jacopo Mondi
2022-01-11 15:43 ` Jacopo Mondi
2021-12-13 13:49 ` [PATCH v3 04/23] media: atmel: atmel-isc: split the clock code into separate source file Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2022-01-11 15:43 ` Jacopo Mondi
2022-01-11 15:43 ` Jacopo Mondi
2021-12-13 13:49 ` [PATCH v3 05/23] media: atmel: atmel-isc: replace video device name with module name Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 06/23] media: atmel: atmel-sama7g5-isc: fix ispck leftover Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 07/23] media: atmel: atmel-isc-base: use streaming status when queueing buffers Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2022-01-11 15:53 ` Hans Verkuil
2022-01-11 15:53 ` Hans Verkuil
2022-01-11 16:38 ` Eugen.Hristev
2022-01-11 16:38 ` Eugen.Hristev
2022-01-12 9:52 ` Hans Verkuil
2022-01-12 9:52 ` Hans Verkuil
2021-12-13 13:49 ` [PATCH v3 08/23] media: atmel: atmel-isc-base: remove frameintervals VIDIOC Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 09/23] media: atmel: atmel-isc-base: report frame sizes as full supported range Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 10/23] media: atmel: atmel-isc-base: implement mbus_code support in enumfmt Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 11/23] media: atmel: atmel-isc-base: fix bytesperline value for planar formats Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 12/23] media: atmel: atmel-isc: implement media controller Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2022-01-11 16:06 ` Jacopo Mondi
2022-01-11 16:06 ` Jacopo Mondi
2022-01-11 16:30 ` Eugen.Hristev
2022-01-11 16:30 ` Eugen.Hristev
2022-01-11 16:51 ` Jacopo Mondi
2022-01-11 16:51 ` Jacopo Mondi
2022-01-11 17:00 ` Eugen.Hristev
2022-01-11 17:00 ` Eugen.Hristev
2021-12-13 13:49 ` [PATCH v3 13/23] ARM: dts: at91: sama7g5: add nodes for video capture Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 14/23] ARM: configs: at91: sama7: add xisc and csi2dc Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 15/23] ARM: multi_v7_defconfig: add atmel video pipeline modules Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 16/23] media: atmel: atmel-sama5d2-isc: fix wrong mask in YUYV format check Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 17/23] media: atmel: atmel-isc-base: use mutex to lock awb workqueue from streaming Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2022-01-12 8:58 ` Jacopo Mondi
2022-01-12 8:58 ` Jacopo Mondi
2022-01-19 15:40 ` Eugen.Hristev
2022-01-19 15:40 ` Eugen.Hristev
2021-12-13 13:49 ` [PATCH v3 18/23] media: atmel: atmel-isc-base: add wb debug messages Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2022-01-12 9:00 ` Jacopo Mondi
2022-01-12 9:00 ` Jacopo Mondi
2021-12-13 13:49 ` [PATCH v3 19/23] media: atmel: atmel-isc-base: clamp wb gain coefficients Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2022-01-12 9:04 ` Jacopo Mondi
2022-01-12 9:04 ` Jacopo Mondi
2021-12-13 13:49 ` [PATCH v3 20/23] media: atmel: atmel-sama7g5-isc: fix UYVY input format mbus_code typo Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2022-01-12 9:04 ` Jacopo Mondi
2022-01-12 9:04 ` Jacopo Mondi
2021-12-13 13:49 ` [PATCH v3 21/23] media: atmel: atmel-isc: add raw Bayer 8bit 10bit output formats Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2022-01-12 9:05 ` Jacopo Mondi
2022-01-12 9:05 ` Jacopo Mondi
2021-12-13 13:49 ` [PATCH v3 22/23] media: atmel: atmel-isc: compact the controller formats list Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2022-01-12 9:06 ` Jacopo Mondi
2022-01-12 9:06 ` Jacopo Mondi
2021-12-13 13:49 ` [PATCH v3 23/23] media: atmel: atmel-isc: change format propagation to subdev into only verification Eugen Hristev
2021-12-13 13:49 ` Eugen Hristev
2022-01-12 9:21 ` Jacopo Mondi
2022-01-12 9:21 ` Jacopo Mondi
2022-01-20 8:43 ` Eugen.Hristev
2022-01-20 8:43 ` Eugen.Hristev
2022-01-20 8:48 ` Eugen.Hristev
2022-01-20 8:48 ` Eugen.Hristev
2022-01-20 8:58 ` Jacopo Mondi
2022-01-20 8:58 ` Jacopo Mondi
2022-01-12 12:07 ` [PATCH v3 00/23] media: atmel: atmel-isc: implement media controller Hans Verkuil
2022-01-12 12:07 ` Hans Verkuil
2022-01-12 12:46 ` Eugen.Hristev
2022-01-12 12:46 ` Eugen.Hristev
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.