Linux-Media Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH v3 00/31] v4l: add support for multiplexed streams
@ 2019-03-05 18:51 Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 01/31] media: entity: Use pad as a starting point for graph walk Jacopo Mondi
                   ` (31 more replies)
  0 siblings, 32 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

Hello,
   third version of multiplexed stream support patch series.

V2 sent by Niklas is available at:
https://patchwork.kernel.org/cover/10573817/

As per v2, most of the core patches are work from Sakari and Laurent, with
Niklas' support on top for adv748x and rcar-csi2.

The use case of the series remains the same: support for virtual channel
selection implemented on R-Car Gen3 and adv748x. Quoting the v2 cover letter:

-------------------------------------------------------------------------------
I have added driver support for the devices used on the Renesas Gen3
platforms, a ADV7482 connected to the R-Car CSI-2 receiver. With these
changes I can control which of the analog inputs of the ADV7482 the
video source is captured from and on which CSI-2 virtual channel the
video is transmitted on to the R-Car CSI-2 receiver.

The series adds two new subdev IOCTLs [GS]_ROUTING which allows
user-space to get and set routes inside a subdevice. I have added RFC
support for these to v4l-utils [2] which can be used to test this
series, example:

    Check the internal routing of the adv748x csi-2 transmitter:
    v4l2-ctl -d /dev/v4l-subdev24 --get-routing
    0/0 -> 1/0 [ENABLED]
    0/0 -> 1/1 []
    0/0 -> 1/2 []
    0/0 -> 1/3 []


    Select that video should be outputed on VC 2 and check the result:
    $ v4l2-ctl -d /dev/v4l-subdev24 --set-routing '0/0 -> 1/2 [1]'

    $ v4l2-ctl -d /dev/v4l-subdev24 --get-routing
    0/0 -> 1/0 []
    0/0 -> 1/1 []
    0/0 -> 1/2 [ENABLED]
    0/0 -> 1/3 []
-------------------------------------------------------------------------------

Below is reported the media graph of the system used for testing [1].

v4l2-ctl patches to handle the newly introduced IOCTLs are available from
Niklas' repository at:
git://git.ragnatech.se/v4l-utils routing

The series is now based on latest media-master branch (v5.0-rc6) and I've
addressed most of the comments received on v2. More precisely, the most notable
changes from v2 are:

1) Added IOCTLs documentation, and a section to describe multiplexed media links
to the v4l2-subdevice interface chapter and expanded functions documentation
2) Re-worked the IOCTLs so that they don't need a compat version
3) Sort pads to be passed to has_route() by their index values
4) Drop indirect routes support from this initial version. Might be considered
later to simplify subdevice drivers implementations
5) Add has_route() operations to adv748x and rcar-csi2 subdevice drivers

Based on this series, I plan to implement support for CSI-2 data lane
negotiation for ADV748x and R-Car CSI-2 receiver.

Tested on R-Car Gen3 Salvator-M3W capturing images transmitted on different
virtual channels.

Series available at:
git://jmondi.org/linux v4l2-mux/media-master-v5.0/v3

Thanks
   j

Jacopo Mondi (4):
  media: entity: Add iterator helper for entity pads
  media: Documentation: Add GS_ROUTING documentation
  adv748x: afe: Implement has_route()
  media: rcar-csi2: Implement has_route()

Laurent Pinchart (4):
  media: entity: Add has_route entity operation
  media: entity: Add media_has_route() function
  media: entity: Use routing information during graph traversal
  v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations

Niklas Söderlund (7):
  adv748x: csi2: add translation from pixelcode to CSI-2 datatype
  adv748x: csi2: only allow formats on sink pads
  adv748x: csi2: describe the multiplexed stream
  adv748x: csi2: add internal routing configuration
  adv748x: afe: add routing support
  rcar-csi2: use frame description information to configure CSI-2 bus
  rcar-csi2: expose the subdevice internal routing

Sakari Ailus (16):
  media: entity: Use pad as a starting point for graph walk
  media: entity: Use pads instead of entities in the media graph walk
    stack
  media: entity: Walk the graph based on pads
  v4l: mc: Start walk from a specific pad in use count calculation
  media: entity: Move the pipeline from entity to pads
  media: entity: Use pad as the starting point for a pipeline
  media: entity: Skip link validation for pads to which there is no
    route to
  media: entity: Add an iterator helper for connected pads
  media: entity: Add only connected pads to the pipeline
  media: entity: Add debug information in graph walk route check
  v4l: Add bus type to frame descriptors
  v4l: Add CSI-2 bus configuration to frame descriptors
  v4l: Add stream to frame descriptor
  v4l: subdev: Take routing information into account in link validation
  v4l: subdev: Improve link format validation debug messages
  v4l: mc: Add an S_ROUTING helper function for power state changes

 Documentation/media/kapi/mc-core.rst          |  15 +-
 Documentation/media/uapi/v4l/dev-subdev.rst   |  90 ++++++
 Documentation/media/uapi/v4l/user-func.rst    |   1 +
 .../uapi/v4l/vidioc-subdev-g-routing.rst      | 142 +++++++++
 drivers/media/i2c/adv748x/adv748x-afe.c       |  91 ++++++
 drivers/media/i2c/adv748x/adv748x-csi2.c      | 136 +++++++-
 drivers/media/i2c/adv748x/adv748x.h           |   1 +
 drivers/media/media-device.c                  |  13 +-
 drivers/media/media-entity.c                  | 232 ++++++++------
 drivers/media/pci/intel/ipu3/ipu3-cio2.c      |   6 +-
 .../media/platform/exynos4-is/fimc-capture.c  |   8 +-
 .../platform/exynos4-is/fimc-isp-video.c      |   8 +-
 drivers/media/platform/exynos4-is/fimc-isp.c  |   2 +-
 drivers/media/platform/exynos4-is/fimc-lite.c |  10 +-
 drivers/media/platform/exynos4-is/media-dev.c |  20 +-
 drivers/media/platform/omap3isp/isp.c         |   2 +-
 drivers/media/platform/omap3isp/ispvideo.c    |  25 +-
 drivers/media/platform/omap3isp/ispvideo.h    |   2 +-
 .../media/platform/qcom/camss/camss-video.c   |   6 +-
 drivers/media/platform/rcar-vin/rcar-core.c   |  16 +-
 drivers/media/platform/rcar-vin/rcar-csi2.c   | 222 ++++++++++---
 drivers/media/platform/rcar-vin/rcar-dma.c    |   8 +-
 .../media/platform/s3c-camif/camif-capture.c  |   6 +-
 drivers/media/platform/vimc/vimc-capture.c    |   5 +-
 drivers/media/platform/vsp1/vsp1_video.c      |  18 +-
 drivers/media/platform/xilinx/xilinx-dma.c    |  20 +-
 drivers/media/platform/xilinx/xilinx-dma.h    |   2 +-
 drivers/media/usb/au0828/au0828-core.c        |   4 +-
 drivers/media/v4l2-core/v4l2-ioctl.c          |  25 +-
 drivers/media/v4l2-core/v4l2-mc.c             |  76 +++--
 drivers/media/v4l2-core/v4l2-subdev.c         | 296 ++++++++++++++++--
 .../staging/media/davinci_vpfe/vpfe_video.c   |  47 +--
 drivers/staging/media/imx/imx-media-utils.c   |   8 +-
 drivers/staging/media/omap4iss/iss.c          |   2 +-
 drivers/staging/media/omap4iss/iss_video.c    |  38 +--
 drivers/staging/media/omap4iss/iss_video.h    |   2 +-
 include/media/media-entity.h                  | 143 ++++++---
 include/media/v4l2-mc.h                       |  22 ++
 include/media/v4l2-subdev.h                   |  49 +++
 include/uapi/linux/v4l2-subdev.h              |  40 +++
 40 files changed, 1492 insertions(+), 367 deletions(-)
 create mode 100644 Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst

[1]
-------------------------------------------------------------------------------
Device topology
- entity 1: rcar_csi2 feaa0000.csi2 (5 pads, 15 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
	pad0: Sink
		<- "adv748x 4-0070 txa":1 [ENABLED,IMMUTABLE]
	pad1: Source
		[fmt:RGB888_1X24/1280x720 field:none]
		-> "VIN0 output":0 []
		-> "VIN1 output":0 []
		-> "VIN2 output":0 []
		-> "VIN4 output":0 [ENABLED]
		-> "VIN5 output":0 []
		-> "VIN6 output":0 []
	pad2: Source
		[fmt:unknown/0x0]
		-> "VIN1 output":0 []
		-> "VIN3 output":0 []
		-> "VIN5 output":0 []
		-> "VIN7 output":0 []
	pad3: Source
		[fmt:unknown/0x0]
		-> "VIN2 output":0 []
		-> "VIN6 output":0 []
	pad4: Source
		[fmt:unknown/0x0]
		-> "VIN3 output":0 []
		-> "VIN7 output":0 []

- entity 7: adv748x 4-0070 txa (2 pads, 3 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev3
	pad0: Sink
		[fmt:RGB888_1X24/1280x720 field:none]
		<- "adv748x 4-0070 afe":8 []
		<- "adv748x 4-0070 hdmi":1 [ENABLED]
	pad1: Source
		-> "rcar_csi2 feaa0000.csi2":0 [ENABLED,IMMUTABLE]

- entity 10: adv748x 4-0070 afe (9 pads, 2 links)
             type V4L2 subdev subtype Decoder flags 0
             device node name /dev/v4l-subdev1
	pad0: Sink
	pad1: Sink
	pad2: Sink
	pad3: Sink
	pad4: Sink
	pad5: Sink
	pad6: Sink
	pad7: Sink
	pad8: Source
		[fmt:UYVY8_2X8/720x240 field:alternate colorspace:smpte170m]
		-> "adv748x 4-0070 txa":0 []
		-> "adv748x 4-0070 txb":0 []

- entity 22: adv748x 4-0070 hdmi (2 pads, 1 link)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev2
	pad0: Sink
		[dv.caps:BT.656/1120 min:640x480@13000000 max:1920x1200@162000000 stds:CEA-861,DMT caps:progressive]
	pad1: Source
		[fmt:RGB888_1X24/1280x720 field:none colorspace:srgb]
		[dv.caps:BT.656/1120 min:640x480@13000000 max:1920x1200@162000000 stds:CEA-861,DMT caps:progressive]
		[dv.detect:BT.656/1120 1280x720p50 (1980x750) stds:CEA-861 flags:CE-video,has-cea861-vic]
		[dv.current:BT.656/1120 1280x720p50 (1980x750) stds:CEA-861 flags:CE-video,has-cea861-vic]
		-> "adv748x 4-0070 txa":0 [ENABLED]

- entity 29: rcar_csi2 fea80000.csi2 (5 pads, 15 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev4
	pad0: Sink
		<- "adv748x 4-0070 txb":1 [ENABLED,IMMUTABLE]
	pad1: Source
		[fmt:unknown/0x0]
		-> "VIN0 output":0 []
		-> "VIN1 output":0 []
		-> "VIN2 output":0 []
		-> "VIN4 output":0 []
		-> "VIN5 output":0 []
		-> "VIN6 output":0 []
	pad2: Source
		[fmt:unknown/0x0]
		-> "VIN1 output":0 []
		-> "VIN3 output":0 []
		-> "VIN5 output":0 []
		-> "VIN7 output":0 []
	pad3: Source
		[fmt:unknown/0x0]
		-> "VIN2 output":0 []
		-> "VIN6 output":0 []
	pad4: Source
		[fmt:unknown/0x0]
		-> "VIN3 output":0 []
		-> "VIN7 output":0 []

- entity 35: adv748x 4-0070 txb (2 pads, 2 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev5
	pad0: Sink
		[fmt:unknown/0x0]
		<- "adv748x 4-0070 afe":8 []
	pad1: Source
		-> "rcar_csi2 fea80000.csi2":0 [ENABLED,IMMUTABLE]

- entity 54: VIN0 output (1 pad, 2 links)
             type Node subtype V4L flags 0
             device node name /dev/video0
	pad0: Sink
		<- "rcar_csi2 feaa0000.csi2":1 []
		<- "rcar_csi2 fea80000.csi2":1 []

- entity 58: VIN1 output (1 pad, 4 links)
             type Node subtype V4L flags 0
             device node name /dev/video1
	pad0: Sink
		<- "rcar_csi2 feaa0000.csi2":1 []
		<- "rcar_csi2 feaa0000.csi2":2 []
		<- "rcar_csi2 fea80000.csi2":1 []
		<- "rcar_csi2 fea80000.csi2":2 []

- entity 62: VIN2 output (1 pad, 4 links)
             type Node subtype V4L flags 0
             device node name /dev/video2
	pad0: Sink
		<- "rcar_csi2 feaa0000.csi2":1 []
		<- "rcar_csi2 feaa0000.csi2":3 []
		<- "rcar_csi2 fea80000.csi2":1 []
		<- "rcar_csi2 fea80000.csi2":3 []

- entity 66: VIN3 output (1 pad, 4 links)
             type Node subtype V4L flags 0
             device node name /dev/video3
	pad0: Sink
		<- "rcar_csi2 feaa0000.csi2":2 []
		<- "rcar_csi2 feaa0000.csi2":4 []
		<- "rcar_csi2 fea80000.csi2":2 []
		<- "rcar_csi2 fea80000.csi2":4 []

- entity 70: VIN4 output (1 pad, 2 links)
             type Node subtype V4L flags 0
             device node name /dev/video4
	pad0: Sink
		<- "rcar_csi2 feaa0000.csi2":1 [ENABLED]
		<- "rcar_csi2 fea80000.csi2":1 []

- entity 74: VIN5 output (1 pad, 4 links)
             type Node subtype V4L flags 0
             device node name /dev/video5
	pad0: Sink
		<- "rcar_csi2 feaa0000.csi2":1 []
		<- "rcar_csi2 feaa0000.csi2":2 []
		<- "rcar_csi2 fea80000.csi2":1 []
		<- "rcar_csi2 fea80000.csi2":2 []

- entity 78: VIN6 output (1 pad, 4 links)
             type Node subtype V4L flags 0
             device node name /dev/video6
	pad0: Sink
		<- "rcar_csi2 feaa0000.csi2":1 []
		<- "rcar_csi2 feaa0000.csi2":3 []
		<- "rcar_csi2 fea80000.csi2":1 []
		<- "rcar_csi2 fea80000.csi2":3 []

- entity 82: VIN7 output (1 pad, 4 links)
             type Node subtype V4L flags 0
             device node name /dev/video7
	pad0: Sink
		<- "rcar_csi2 feaa0000.csi2":2 []
		<- "rcar_csi2 feaa0000.csi2":4 []
		<- "rcar_csi2 fea80000.csi2":2 []
		<- "rcar_csi2 fea80000.csi2":4 []


--
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 01/31] media: entity: Use pad as a starting point for graph walk
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 02/31] media: entity: Use pads instead of entities in the media graph walk stack Jacopo Mondi
                   ` (30 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Sakari Ailus <sakari.ailus@linux.intel.com>

With the upcoming use of the recently added has_route() media entity op, all
the pads in an entity will no longer be considered interconnected. This has
an effect where the media graph is traversed: the starting pad does make a
difference.

Prepare for this change by using pad instead of the entity as an argument
for the graph walk operations. The actual graph traversal algorithm change
is in further patches.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 Documentation/media/kapi/mc-core.rst            |  2 +-
 drivers/media/media-entity.c                    | 17 ++++++++---------
 drivers/media/platform/exynos4-is/media-dev.c   |  4 ++--
 drivers/media/platform/omap3isp/ispvideo.c      |  2 +-
 drivers/media/platform/vsp1/vsp1_video.c        |  2 +-
 drivers/media/platform/xilinx/xilinx-dma.c      |  2 +-
 drivers/media/v4l2-core/v4l2-mc.c               |  6 +++---
 drivers/staging/media/davinci_vpfe/vpfe_video.c |  6 +++---
 drivers/staging/media/omap4iss/iss_video.c      |  4 ++--
 include/media/media-entity.h                    | 10 ++++------
 10 files changed, 26 insertions(+), 29 deletions(-)

diff --git a/Documentation/media/kapi/mc-core.rst b/Documentation/media/kapi/mc-core.rst
index f930725e0d6b..a0cecc5e4453 100644
--- a/Documentation/media/kapi/mc-core.rst
+++ b/Documentation/media/kapi/mc-core.rst
@@ -167,7 +167,7 @@ Drivers initiate a graph traversal by calling
 :c:func:`media_graph_walk_start()`
 
 The graph structure, provided by the caller, is initialized to start graph
-traversal at the given entity.
+traversal at the given pad in an entity.
 
 Drivers can then retrieve the next entity by calling
 :c:func:`media_graph_walk_next()`
diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
index 0b1cb3559140..2bbc07de71aa 100644
--- a/drivers/media/media-entity.c
+++ b/drivers/media/media-entity.c
@@ -300,17 +300,16 @@ void media_graph_walk_cleanup(struct media_graph *graph)
 }
 EXPORT_SYMBOL_GPL(media_graph_walk_cleanup);
 
-void media_graph_walk_start(struct media_graph *graph,
-			    struct media_entity *entity)
+void media_graph_walk_start(struct media_graph *graph, struct media_pad *pad)
 {
 	media_entity_enum_zero(&graph->ent_enum);
-	media_entity_enum_set(&graph->ent_enum, entity);
+	media_entity_enum_set(&graph->ent_enum, pad->entity);
 
 	graph->top = 0;
 	graph->stack[graph->top].entity = NULL;
-	stack_push(graph, entity);
-	dev_dbg(entity->graph_obj.mdev->dev,
-		"begin graph walk at '%s'\n", entity->name);
+	stack_push(graph, pad->entity);
+	dev_dbg(pad->graph_obj.mdev->dev,
+		"begin graph walk at '%s':%u\n", pad->entity->name, pad->index);
 }
 EXPORT_SYMBOL_GPL(media_graph_walk_start);
 
@@ -428,7 +427,7 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 			goto error_graph_walk_start;
 	}
 
-	media_graph_walk_start(&pipe->graph, entity);
+	media_graph_walk_start(&pipe->graph, entity->pads);
 
 	while ((entity = media_graph_walk_next(graph))) {
 		DECLARE_BITMAP(active, MEDIA_ENTITY_MAX_PADS);
@@ -509,7 +508,7 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 	 * Link validation on graph failed. We revert what we did and
 	 * return the error.
 	 */
-	media_graph_walk_start(graph, entity_err);
+	media_graph_walk_start(graph, entity_err->pads);
 
 	while ((entity_err = media_graph_walk_next(graph))) {
 		/* Sanity check for negative stream_count */
@@ -560,7 +559,7 @@ void __media_pipeline_stop(struct media_entity *entity)
 	if (WARN_ON(!pipe))
 		return;
 
-	media_graph_walk_start(graph, entity);
+	media_graph_walk_start(graph, entity->pads);
 
 	while ((entity = media_graph_walk_next(graph))) {
 		/* Sanity check for negative stream_count */
diff --git a/drivers/media/platform/exynos4-is/media-dev.c b/drivers/media/platform/exynos4-is/media-dev.c
index 463f2d84553e..9f72f127a2c6 100644
--- a/drivers/media/platform/exynos4-is/media-dev.c
+++ b/drivers/media/platform/exynos4-is/media-dev.c
@@ -1144,7 +1144,7 @@ static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
 	 * through active links. This is needed as we cannot power on/off the
 	 * subdevs in random order.
 	 */
-	media_graph_walk_start(graph, entity);
+	media_graph_walk_start(graph, entity->pads);
 
 	while ((entity = media_graph_walk_next(graph))) {
 		if (!is_media_entity_v4l2_video_device(entity))
@@ -1159,7 +1159,7 @@ static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
 	return 0;
 
 err:
-	media_graph_walk_start(graph, entity_err);
+	media_graph_walk_start(graph, entity_err->pads);
 
 	while ((entity_err = media_graph_walk_next(graph))) {
 		if (!is_media_entity_v4l2_video_device(entity_err))
diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
index 078d64114b24..8577b646642f 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -238,7 +238,7 @@ static int isp_video_get_graph_data(struct isp_video *video,
 		return ret;
 	}
 
-	media_graph_walk_start(&graph, entity);
+	media_graph_walk_start(&graph, entity->pads);
 
 	while ((entity = media_graph_walk_next(&graph))) {
 		struct isp_video *__video;
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index 7ceaf3222145..2a8453ca0087 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -580,7 +580,7 @@ static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
 	if (ret)
 		return ret;
 
-	media_graph_walk_start(&graph, entity);
+	media_graph_walk_start(&graph, entity->pads);
 
 	while ((entity = media_graph_walk_next(&graph))) {
 		struct v4l2_subdev *subdev;
diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index c9d5fdb2d407..a85862b469d9 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -190,7 +190,7 @@ static int xvip_pipeline_validate(struct xvip_pipeline *pipe,
 		return ret;
 	}
 
-	media_graph_walk_start(&graph, entity);
+	media_graph_walk_start(&graph, entity->pads);
 
 	while ((entity = media_graph_walk_next(&graph))) {
 		struct xvip_dma *dma;
diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
index 014a2a97cadd..9ed480fe5b6e 100644
--- a/drivers/media/v4l2-core/v4l2-mc.c
+++ b/drivers/media/v4l2-core/v4l2-mc.c
@@ -341,7 +341,7 @@ static int pipeline_pm_use_count(struct media_entity *entity,
 {
 	int use = 0;
 
-	media_graph_walk_start(graph, entity);
+	media_graph_walk_start(graph, entity->pads);
 
 	while ((entity = media_graph_walk_next(graph))) {
 		if (is_media_entity_v4l2_video_device(entity))
@@ -404,7 +404,7 @@ static int pipeline_pm_power(struct media_entity *entity, int change,
 	if (!change)
 		return 0;
 
-	media_graph_walk_start(graph, entity);
+	media_graph_walk_start(graph, entity->pads);
 
 	while (!ret && (entity = media_graph_walk_next(graph)))
 		if (is_media_entity_v4l2_subdev(entity))
@@ -413,7 +413,7 @@ static int pipeline_pm_power(struct media_entity *entity, int change,
 	if (!ret)
 		return ret;
 
-	media_graph_walk_start(graph, first);
+	media_graph_walk_start(graph, first->pads);
 
 	while ((first = media_graph_walk_next(graph))
 	       && first != entity)
diff --git a/drivers/staging/media/davinci_vpfe/vpfe_video.c b/drivers/staging/media/davinci_vpfe/vpfe_video.c
index 510202a3b091..58ead9a30754 100644
--- a/drivers/staging/media/davinci_vpfe/vpfe_video.c
+++ b/drivers/staging/media/davinci_vpfe/vpfe_video.c
@@ -147,7 +147,7 @@ static int vpfe_prepare_pipeline(struct vpfe_video_device *video)
 		mutex_unlock(&mdev->graph_mutex);
 		return -ENOMEM;
 	}
-	media_graph_walk_start(&graph, entity);
+	media_graph_walk_start(&graph, entity->pads);
 	while ((entity = media_graph_walk_next(&graph))) {
 		if (entity == &video->video_dev.entity)
 			continue;
@@ -300,7 +300,7 @@ static int vpfe_pipeline_enable(struct vpfe_pipeline *pipe)
 	ret = media_graph_walk_init(&pipe->graph, mdev);
 	if (ret)
 		goto out;
-	media_graph_walk_start(&pipe->graph, entity);
+	media_graph_walk_start(&pipe->graph, entity->pads);
 	while ((entity = media_graph_walk_next(&pipe->graph))) {
 
 		if (!is_media_entity_v4l2_subdev(entity))
@@ -342,7 +342,7 @@ static int vpfe_pipeline_disable(struct vpfe_pipeline *pipe)
 
 	mdev = entity->graph_obj.mdev;
 	mutex_lock(&mdev->graph_mutex);
-	media_graph_walk_start(&pipe->graph, entity);
+	media_graph_walk_start(&pipe->graph, entity->pads);
 
 	while ((entity = media_graph_walk_next(&pipe->graph))) {
 
diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
index c2c5a9cd8642..ba3036e5a369 100644
--- a/drivers/staging/media/omap4iss/iss_video.c
+++ b/drivers/staging/media/omap4iss/iss_video.c
@@ -217,7 +217,7 @@ iss_video_far_end(struct iss_video *video)
 		return NULL;
 	}
 
-	media_graph_walk_start(&graph, entity);
+	media_graph_walk_start(&graph, entity->pads);
 
 	while ((entity = media_graph_walk_next(&graph))) {
 		if (entity == &video->video.entity)
@@ -898,7 +898,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 	if (ret < 0)
 		goto err_media_pipeline_start;
 
-	media_graph_walk_start(&graph, entity);
+	media_graph_walk_start(&graph, entity->pads);
 	while ((entity = media_graph_walk_next(&graph)))
 		media_entity_enum_set(&pipe->ent_enum, entity);
 
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index e5f6960d92f6..07ab141e739e 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -928,22 +928,20 @@ void media_graph_walk_cleanup(struct media_graph *graph);
 void media_entity_put(struct media_entity *entity);
 
 /**
- * media_graph_walk_start - Start walking the media graph at a
- *	given entity
+ * media_graph_walk_start - Start walking the media graph at a given pad
  *
  * @graph: Media graph structure that will be used to walk the graph
- * @entity: Starting entity
+ * @pad: Starting pad
  *
  * Before using this function, media_graph_walk_init() must be
  * used to allocate resources used for walking the graph. This
  * function initializes the graph traversal structure to walk the
- * entities graph starting at the given entity. The traversal
+ * entities graph starting at the given pad. The traversal
  * structure must not be modified by the caller during graph
  * traversal. After the graph walk, the resources must be released
  * using media_graph_walk_cleanup().
  */
-void media_graph_walk_start(struct media_graph *graph,
-			    struct media_entity *entity);
+void media_graph_walk_start(struct media_graph *graph, struct media_pad *pad);
 
 /**
  * media_graph_walk_next - Get the next entity in the graph
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 02/31] media: entity: Use pads instead of entities in the media graph walk stack
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 01/31] media: entity: Use pad as a starting point for graph walk Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 03/31] media: entity: Walk the graph based on pads Jacopo Mondi
                   ` (29 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Sakari Ailus <sakari.ailus@linux.intel.com>

Change the media graph walk stack structure to use media pads instead of
using media entities. In addition to the entity, the pad contains the
information which pad in the entity are being dealt with.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/media-entity.c | 53 ++++++++++++++++++------------------
 include/media/media-entity.h |  8 +++---
 2 files changed, 30 insertions(+), 31 deletions(-)

diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
index 2bbc07de71aa..bddfcd3b2dd9 100644
--- a/drivers/media/media-entity.c
+++ b/drivers/media/media-entity.c
@@ -237,40 +237,39 @@ EXPORT_SYMBOL_GPL(media_entity_pads_init);
  * Graph traversal
  */
 
-static struct media_entity *
-media_entity_other(struct media_entity *entity, struct media_link *link)
+static struct media_pad *
+media_pad_other(struct media_pad *pad, struct media_link *link)
 {
-	if (link->source->entity == entity)
-		return link->sink->entity;
+	if (link->source == pad)
+		return link->sink;
 	else
-		return link->source->entity;
+		return link->source;
 }
 
 /* push an entity to traversal stack */
-static void stack_push(struct media_graph *graph,
-		       struct media_entity *entity)
+static void stack_push(struct media_graph *graph, struct media_pad *pad)
 {
 	if (graph->top == MEDIA_ENTITY_ENUM_MAX_DEPTH - 1) {
 		WARN_ON(1);
 		return;
 	}
 	graph->top++;
-	graph->stack[graph->top].link = entity->links.next;
-	graph->stack[graph->top].entity = entity;
+	graph->stack[graph->top].link = pad->entity->links.next;
+	graph->stack[graph->top].pad = pad;
 }
 
-static struct media_entity *stack_pop(struct media_graph *graph)
+static struct media_pad *stack_pop(struct media_graph *graph)
 {
-	struct media_entity *entity;
+	struct media_pad *pad;
 
-	entity = graph->stack[graph->top].entity;
+	pad = graph->stack[graph->top].pad;
 	graph->top--;
 
-	return entity;
+	return pad;
 }
 
 #define link_top(en)	((en)->stack[(en)->top].link)
-#define stack_top(en)	((en)->stack[(en)->top].entity)
+#define stack_top(en)	((en)->stack[(en)->top].pad)
 
 /**
  * media_graph_walk_init - Allocate resources for graph walk
@@ -306,8 +305,8 @@ void media_graph_walk_start(struct media_graph *graph, struct media_pad *pad)
 	media_entity_enum_set(&graph->ent_enum, pad->entity);
 
 	graph->top = 0;
-	graph->stack[graph->top].entity = NULL;
-	stack_push(graph, pad->entity);
+	graph->stack[graph->top].pad = NULL;
+	stack_push(graph, pad);
 	dev_dbg(pad->graph_obj.mdev->dev,
 		"begin graph walk at '%s':%u\n", pad->entity->name, pad->index);
 }
@@ -315,16 +314,16 @@ EXPORT_SYMBOL_GPL(media_graph_walk_start);
 
 static void media_graph_walk_iter(struct media_graph *graph)
 {
-	struct media_entity *entity = stack_top(graph);
+	struct media_pad *pad = stack_top(graph);
 	struct media_link *link;
-	struct media_entity *next;
+	struct media_pad *next;
 
 	link = list_entry(link_top(graph), typeof(*link), list);
 
 	/* The link is not enabled so we do not follow. */
 	if (!(link->flags & MEDIA_LNK_FL_ENABLED)) {
 		link_top(graph) = link_top(graph)->next;
-		dev_dbg(entity->graph_obj.mdev->dev,
+		dev_dbg(pad->graph_obj.mdev->dev,
 			"walk: skipping disabled link '%s':%u -> '%s':%u\n",
 			link->source->entity->name, link->source->index,
 			link->sink->entity->name, link->sink->index);
@@ -332,22 +331,22 @@ static void media_graph_walk_iter(struct media_graph *graph)
 	}
 
 	/* Get the entity in the other end of the link . */
-	next = media_entity_other(entity, link);
+	next = media_pad_other(pad, link);
 
 	/* Has the entity already been visited? */
-	if (media_entity_enum_test_and_set(&graph->ent_enum, next)) {
+	if (media_entity_enum_test_and_set(&graph->ent_enum, next->entity)) {
 		link_top(graph) = link_top(graph)->next;
-		dev_dbg(entity->graph_obj.mdev->dev,
+		dev_dbg(pad->graph_obj.mdev->dev,
 			"walk: skipping entity '%s' (already seen)\n",
-			next->name);
+			next->entity->name);
 		return;
 	}
 
 	/* Push the new entity to stack and start over. */
 	link_top(graph) = link_top(graph)->next;
 	stack_push(graph, next);
-	dev_dbg(entity->graph_obj.mdev->dev, "walk: pushing '%s' on stack\n",
-		next->name);
+	dev_dbg(next->graph_obj.mdev->dev, "walk: pushing '%s':%u on stack\n",
+		next->entity->name, next->index);
 }
 
 struct media_entity *media_graph_walk_next(struct media_graph *graph)
@@ -362,10 +361,10 @@ struct media_entity *media_graph_walk_next(struct media_graph *graph)
 	 * top of the stack until no more entities on the level can be
 	 * found.
 	 */
-	while (link_top(graph) != &stack_top(graph)->links)
+	while (link_top(graph) != &stack_top(graph)->entity->links)
 		media_graph_walk_iter(graph);
 
-	entity = stack_pop(graph);
+	entity = stack_pop(graph)->entity;
 	dev_dbg(entity->graph_obj.mdev->dev,
 		"walk: returning entity '%s'\n", entity->name);
 
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 07ab141e739e..e50972d44fb5 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -86,16 +86,16 @@ struct media_entity_enum {
  * struct media_graph - Media graph traversal state
  *
  * @stack:		Graph traversal stack; the stack contains information
- *			on the path the media entities to be walked and the
- *			links through which they were reached.
- * @stack.entity:	pointer to &struct media_entity at the graph.
+ *			on the media pads to be walked and the links through
+ *			which they were reached.
+ * @stack.pad:		pointer to &struct media_pad at the graph.
  * @stack.link:		pointer to &struct list_head.
  * @ent_enum:		Visited entities
  * @top:		The top of the stack
  */
 struct media_graph {
 	struct {
-		struct media_entity *entity;
+		struct media_pad *pad;
 		struct list_head *link;
 	} stack[MEDIA_ENTITY_ENUM_MAX_DEPTH];
 
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 03/31] media: entity: Walk the graph based on pads
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 01/31] media: entity: Use pad as a starting point for graph walk Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 02/31] media: entity: Use pads instead of entities in the media graph walk stack Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 04/31] v4l: mc: Start walk from a specific pad in use count calculation Jacopo Mondi
                   ` (28 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Sakari Ailus <sakari.ailus@linux.intel.com>

Instead of iterating over graph entities during the walk, iterate the pads
through which the entity was first reached. This is required in order to
make the entity pipeline pad-based rather than entity based.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 Documentation/media/kapi/mc-core.rst          |  7 ++-
 drivers/media/media-entity.c                  | 46 ++++++++++--------
 drivers/media/platform/exynos4-is/media-dev.c | 20 ++++----
 drivers/media/platform/omap3isp/ispvideo.c    | 17 +++----
 drivers/media/platform/vsp1/vsp1_video.c      | 12 ++---
 drivers/media/platform/xilinx/xilinx-dma.c    | 12 ++---
 drivers/media/v4l2-core/v4l2-mc.c             | 25 +++++-----
 .../staging/media/davinci_vpfe/vpfe_video.c   | 47 ++++++++++---------
 drivers/staging/media/omap4iss/iss_video.c    | 34 +++++++-------
 include/media/media-entity.h                  |  7 +--
 10 files changed, 122 insertions(+), 105 deletions(-)

diff --git a/Documentation/media/kapi/mc-core.rst b/Documentation/media/kapi/mc-core.rst
index a0cecc5e4453..a516563b358a 100644
--- a/Documentation/media/kapi/mc-core.rst
+++ b/Documentation/media/kapi/mc-core.rst
@@ -169,8 +169,11 @@ Drivers initiate a graph traversal by calling
 The graph structure, provided by the caller, is initialized to start graph
 traversal at the given pad in an entity.
 
-Drivers can then retrieve the next entity by calling
-:c:func:`media_graph_walk_next()`
+Drivers can then retrieve the next pad by calling
+:c:func:`media_graph_walk_next()`. Only the pad through which the entity
+is first reached is returned. If the caller is interested in knowing which
+further pads would be connected, the :c:func:`media_entity_has_route()`
+function can be used for that.
 
 When the graph traversal is complete the function will return ``NULL``.
 
diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
index bddfcd3b2dd9..04e987681ab5 100644
--- a/drivers/media/media-entity.c
+++ b/drivers/media/media-entity.c
@@ -349,9 +349,9 @@ static void media_graph_walk_iter(struct media_graph *graph)
 		next->entity->name, next->index);
 }
 
-struct media_entity *media_graph_walk_next(struct media_graph *graph)
+struct media_pad *media_graph_walk_next(struct media_graph *graph)
 {
-	struct media_entity *entity;
+	struct media_pad *pad;
 
 	if (stack_top(graph) == NULL)
 		return NULL;
@@ -364,11 +364,11 @@ struct media_entity *media_graph_walk_next(struct media_graph *graph)
 	while (link_top(graph) != &stack_top(graph)->entity->links)
 		media_graph_walk_iter(graph);
 
-	entity = stack_pop(graph)->entity;
-	dev_dbg(entity->graph_obj.mdev->dev,
-		"walk: returning entity '%s'\n", entity->name);
+	pad = stack_pop(graph);
+	dev_dbg(pad->graph_obj.mdev->dev,
+		"walk: returning pad '%s':%u\n", pad->entity->name, pad->index);
 
-	return entity;
+	return pad;
 }
 EXPORT_SYMBOL_GPL(media_graph_walk_next);
 
@@ -416,7 +416,8 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 {
 	struct media_device *mdev = entity->graph_obj.mdev;
 	struct media_graph *graph = &pipe->graph;
-	struct media_entity *entity_err = entity;
+	struct media_pad *pad = entity->pads;
+	struct media_pad *pad_err = pad;
 	struct media_link *link;
 	int ret;
 
@@ -426,9 +427,11 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 			goto error_graph_walk_start;
 	}
 
-	media_graph_walk_start(&pipe->graph, entity->pads);
+	media_graph_walk_start(&pipe->graph, pad);
+
+	while ((pad = media_graph_walk_next(graph))) {
+		struct media_entity *entity = pad->entity;
 
-	while ((entity = media_graph_walk_next(graph))) {
 		DECLARE_BITMAP(active, MEDIA_ENTITY_MAX_PADS);
 		DECLARE_BITMAP(has_no_links, MEDIA_ENTITY_MAX_PADS);
 
@@ -452,11 +455,11 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 		bitmap_fill(has_no_links, entity->num_pads);
 
 		list_for_each_entry(link, &entity->links, list) {
-			struct media_pad *pad = link->sink->entity == entity
-						? link->sink : link->source;
+			struct media_pad *other_pad = link->sink->entity == entity
+				? link->sink : link->source;
 
 			/* Mark that a pad is connected by a link. */
-			bitmap_clear(has_no_links, pad->index, 1);
+			bitmap_clear(has_no_links, other_pad->index, 1);
 
 			/*
 			 * Pads that either do not need to connect or
@@ -465,13 +468,13 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 			 */
 			if (!(pad->flags & MEDIA_PAD_FL_MUST_CONNECT) ||
 			    link->flags & MEDIA_LNK_FL_ENABLED)
-				bitmap_set(active, pad->index, 1);
+				bitmap_set(active, other_pad->index, 1);
 
 			/*
 			 * Link validation will only take place for
 			 * sink ends of the link that are enabled.
 			 */
-			if (link->sink != pad ||
+			if (link->sink != other_pad ||
 			    !(link->flags & MEDIA_LNK_FL_ENABLED))
 				continue;
 
@@ -507,9 +510,11 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 	 * Link validation on graph failed. We revert what we did and
 	 * return the error.
 	 */
-	media_graph_walk_start(graph, entity_err->pads);
+	media_graph_walk_start(graph, pad_err);
+
+	while ((pad_err = media_graph_walk_next(graph))) {
+		struct media_entity *entity_err = pad_err->entity;
 
-	while ((entity_err = media_graph_walk_next(graph))) {
 		/* Sanity check for negative stream_count */
 		if (!WARN_ON_ONCE(entity_err->stream_count <= 0)) {
 			entity_err->stream_count--;
@@ -521,7 +526,7 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 		 * We haven't increased stream_count further than this
 		 * so we quit here.
 		 */
-		if (entity_err == entity)
+		if (pad_err == pad)
 			break;
 	}
 
@@ -548,8 +553,9 @@ EXPORT_SYMBOL_GPL(media_pipeline_start);
 
 void __media_pipeline_stop(struct media_entity *entity)
 {
-	struct media_graph *graph = &entity->pipe->graph;
 	struct media_pipeline *pipe = entity->pipe;
+	struct media_graph *graph = &pipe->graph;
+	struct media_pad *pad;
 
 	/*
 	 * If the following check fails, the driver has performed an
@@ -560,7 +566,9 @@ void __media_pipeline_stop(struct media_entity *entity)
 
 	media_graph_walk_start(graph, entity->pads);
 
-	while ((entity = media_graph_walk_next(graph))) {
+	while ((pad = media_graph_walk_next(graph))) {
+		struct media_entity *entity = pad->entity;
+
 		/* Sanity check for negative stream_count */
 		if (!WARN_ON_ONCE(entity->stream_count <= 0)) {
 			entity->stream_count--;
diff --git a/drivers/media/platform/exynos4-is/media-dev.c b/drivers/media/platform/exynos4-is/media-dev.c
index 9f72f127a2c6..b9df07798ec1 100644
--- a/drivers/media/platform/exynos4-is/media-dev.c
+++ b/drivers/media/platform/exynos4-is/media-dev.c
@@ -1135,7 +1135,7 @@ static int __fimc_md_modify_pipeline(struct media_entity *entity, bool enable)
 static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
 				      struct media_graph *graph)
 {
-	struct media_entity *entity_err = entity;
+	struct media_pad *pad, *pad_err = entity->pads;
 	int ret;
 
 	/*
@@ -1144,13 +1144,13 @@ static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
 	 * through active links. This is needed as we cannot power on/off the
 	 * subdevs in random order.
 	 */
-	media_graph_walk_start(graph, entity->pads);
+	media_graph_walk_start(graph, pad_err);
 
-	while ((entity = media_graph_walk_next(graph))) {
-		if (!is_media_entity_v4l2_video_device(entity))
+	while ((pad = media_graph_walk_next(graph))) {
+		if (!is_media_entity_v4l2_video_device(pad->entity))
 			continue;
 
-		ret  = __fimc_md_modify_pipeline(entity, enable);
+		ret  = __fimc_md_modify_pipeline(pad->entity, enable);
 
 		if (ret < 0)
 			goto err;
@@ -1159,15 +1159,15 @@ static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
 	return 0;
 
 err:
-	media_graph_walk_start(graph, entity_err->pads);
+	media_graph_walk_start(graph, pad_err);
 
-	while ((entity_err = media_graph_walk_next(graph))) {
-		if (!is_media_entity_v4l2_video_device(entity_err))
+	while ((pad_err = media_graph_walk_next(graph))) {
+		if (!is_media_entity_v4l2_video_device(pad_err->entity))
 			continue;
 
-		__fimc_md_modify_pipeline(entity_err, !enable);
+		__fimc_md_modify_pipeline(pad_err->entity, !enable);
 
-		if (entity_err == entity)
+		if (pad_err == pad)
 			break;
 	}
 
diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
index 8577b646642f..be364eb64e40 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -226,8 +226,8 @@ static int isp_video_get_graph_data(struct isp_video *video,
 				    struct isp_pipeline *pipe)
 {
 	struct media_graph graph;
-	struct media_entity *entity = &video->video.entity;
-	struct media_device *mdev = entity->graph_obj.mdev;
+	struct media_pad *pad = video->video.entity.pads;
+	struct media_device *mdev = pad->entity->graph_obj.mdev;
 	struct isp_video *far_end = NULL;
 	int ret;
 
@@ -238,23 +238,24 @@ static int isp_video_get_graph_data(struct isp_video *video,
 		return ret;
 	}
 
-	media_graph_walk_start(&graph, entity->pads);
+	media_graph_walk_start(&graph, pad);
 
-	while ((entity = media_graph_walk_next(&graph))) {
+	while ((pad = media_graph_walk_next(&graph))) {
 		struct isp_video *__video;
 
-		media_entity_enum_set(&pipe->ent_enum, entity);
+		media_entity_enum_set(&pipe->ent_enum, pad->entity);
 
 		if (far_end != NULL)
 			continue;
 
-		if (entity == &video->video.entity)
+		if (pad == video->video.entity.pads)
 			continue;
 
-		if (!is_media_entity_v4l2_video_device(entity))
+		if (!is_media_entity_v4l2_video_device(pad->entity))
 			continue;
 
-		__video = to_isp_video(media_entity_to_video_device(entity));
+		__video = to_isp_video(media_entity_to_video_device(
+					       pad->entity));
 		if (__video->type != video->type)
 			far_end = __video;
 	}
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index 2a8453ca0087..bf444bb254d1 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -570,8 +570,8 @@ static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
 				     struct vsp1_video *video)
 {
 	struct media_graph graph;
-	struct media_entity *entity = &video->video.entity;
-	struct media_device *mdev = entity->graph_obj.mdev;
+	struct media_pad *pad = video->video.entity.pads;
+	struct media_device *mdev = pad->entity->graph_obj.mdev;
 	unsigned int i;
 	int ret;
 
@@ -580,17 +580,17 @@ static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
 	if (ret)
 		return ret;
 
-	media_graph_walk_start(&graph, entity->pads);
+	media_graph_walk_start(&graph, pad);
 
-	while ((entity = media_graph_walk_next(&graph))) {
+	while ((pad = media_graph_walk_next(&graph))) {
 		struct v4l2_subdev *subdev;
 		struct vsp1_rwpf *rwpf;
 		struct vsp1_entity *e;
 
-		if (!is_media_entity_v4l2_subdev(entity))
+		if (!is_media_entity_v4l2_subdev(pad->entity))
 			continue;
 
-		subdev = media_entity_to_v4l2_subdev(entity);
+		subdev = media_entity_to_v4l2_subdev(pad->entity);
 		e = to_vsp1_entity(subdev);
 		list_add_tail(&e->list_pipe, &pipe->entities);
 		e->pipe = pipe;
diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index a85862b469d9..66063ccbf84d 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -175,8 +175,8 @@ static int xvip_pipeline_validate(struct xvip_pipeline *pipe,
 				  struct xvip_dma *start)
 {
 	struct media_graph graph;
-	struct media_entity *entity = &start->video.entity;
-	struct media_device *mdev = entity->graph_obj.mdev;
+	struct media_pad *pad = start->video.entity.pads;
+	struct media_device *mdev = pad->entity->graph_obj.mdev;
 	unsigned int num_inputs = 0;
 	unsigned int num_outputs = 0;
 	int ret;
@@ -190,15 +190,15 @@ static int xvip_pipeline_validate(struct xvip_pipeline *pipe,
 		return ret;
 	}
 
-	media_graph_walk_start(&graph, entity->pads);
+	media_graph_walk_start(&graph, pad);
 
-	while ((entity = media_graph_walk_next(&graph))) {
+	while ((pad = media_graph_walk_next(&graph))) {
 		struct xvip_dma *dma;
 
-		if (entity->function != MEDIA_ENT_F_IO_V4L)
+		if (pad->entity->function != MEDIA_ENT_F_IO_V4L)
 			continue;
 
-		dma = to_xvip_dma(media_entity_to_video_device(entity));
+		dma = to_xvip_dma(media_entity_to_video_device(pad->entity));
 
 		if (dma->pad.flags & MEDIA_PAD_FL_SINK) {
 			pipe->output = dma;
diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
index 9ed480fe5b6e..98edd47b2f0a 100644
--- a/drivers/media/v4l2-core/v4l2-mc.c
+++ b/drivers/media/v4l2-core/v4l2-mc.c
@@ -339,13 +339,14 @@ EXPORT_SYMBOL_GPL(v4l_vb2q_enable_media_source);
 static int pipeline_pm_use_count(struct media_entity *entity,
 	struct media_graph *graph)
 {
+	struct media_pad *pad;
 	int use = 0;
 
 	media_graph_walk_start(graph, entity->pads);
 
-	while ((entity = media_graph_walk_next(graph))) {
-		if (is_media_entity_v4l2_video_device(entity))
-			use += entity->use_count;
+	while ((pad = media_graph_walk_next(graph))) {
+		if (is_media_entity_v4l2_video_device(pad->entity))
+			use += pad->entity->use_count;
 	}
 
 	return use;
@@ -398,7 +399,7 @@ static int pipeline_pm_power_one(struct media_entity *entity, int change)
 static int pipeline_pm_power(struct media_entity *entity, int change,
 	struct media_graph *graph)
 {
-	struct media_entity *first = entity;
+	struct media_pad *tmp_pad, *pad;
 	int ret = 0;
 
 	if (!change)
@@ -406,19 +407,19 @@ static int pipeline_pm_power(struct media_entity *entity, int change,
 
 	media_graph_walk_start(graph, entity->pads);
 
-	while (!ret && (entity = media_graph_walk_next(graph)))
-		if (is_media_entity_v4l2_subdev(entity))
-			ret = pipeline_pm_power_one(entity, change);
+	while (!ret && (pad = media_graph_walk_next(graph)))
+		if (is_media_entity_v4l2_subdev(pad->entity))
+			ret = pipeline_pm_power_one(pad->entity, change);
 
 	if (!ret)
 		return ret;
 
-	media_graph_walk_start(graph, first->pads);
+	media_graph_walk_start(graph, entity->pads);
 
-	while ((first = media_graph_walk_next(graph))
-	       && first != entity)
-		if (is_media_entity_v4l2_subdev(first))
-			pipeline_pm_power_one(first, -change);
+	while ((tmp_pad = media_graph_walk_next(graph))
+	       && tmp_pad != pad)
+		if (is_media_entity_v4l2_subdev(tmp_pad->entity))
+			pipeline_pm_power_one(tmp_pad->entity, -change);
 
 	return ret;
 }
diff --git a/drivers/staging/media/davinci_vpfe/vpfe_video.c b/drivers/staging/media/davinci_vpfe/vpfe_video.c
index 58ead9a30754..6b4507c7e208 100644
--- a/drivers/staging/media/davinci_vpfe/vpfe_video.c
+++ b/drivers/staging/media/davinci_vpfe/vpfe_video.c
@@ -127,8 +127,8 @@ __vpfe_video_get_format(struct vpfe_video_device *video,
 static int vpfe_prepare_pipeline(struct vpfe_video_device *video)
 {
 	struct media_graph graph;
-	struct media_entity *entity = &video->video_dev.entity;
-	struct media_device *mdev = entity->graph_obj.mdev;
+	struct media_pad *pad = video->video_dev.entity.pads;
+	struct media_device *mdev = pad->entity->graph_obj.mdev;
 	struct vpfe_pipeline *pipe = &video->pipe;
 	struct vpfe_video_device *far_end = NULL;
 	int ret;
@@ -147,13 +147,14 @@ static int vpfe_prepare_pipeline(struct vpfe_video_device *video)
 		mutex_unlock(&mdev->graph_mutex);
 		return -ENOMEM;
 	}
-	media_graph_walk_start(&graph, entity->pads);
-	while ((entity = media_graph_walk_next(&graph))) {
-		if (entity == &video->video_dev.entity)
+	media_graph_walk_start(&graph, pad);
+	while ((pad = media_graph_walk_next(&graph))) {
+		if (pad->entity == &video->video_dev.entity)
 			continue;
-		if (!is_media_entity_v4l2_video_device(entity))
+		if (!is_media_entity_v4l2_video_device(pad->entity))
 			continue;
-		far_end = to_vpfe_video(media_entity_to_video_device(entity));
+		far_end = to_vpfe_video(media_entity_to_video_device(
+						pad->entity));
 		if (far_end->type == V4L2_BUF_TYPE_VIDEO_OUTPUT)
 			pipe->inputs[pipe->input_num++] = far_end;
 		else
@@ -285,27 +286,27 @@ static int vpfe_video_validate_pipeline(struct vpfe_pipeline *pipe)
  */
 static int vpfe_pipeline_enable(struct vpfe_pipeline *pipe)
 {
-	struct media_entity *entity;
+	struct media_pad *pad;
 	struct v4l2_subdev *subdev;
 	struct media_device *mdev;
 	int ret;
 
 	if (pipe->state == VPFE_PIPELINE_STREAM_CONTINUOUS)
-		entity = vpfe_get_input_entity(pipe->outputs[0]);
+		pad = vpfe_get_input_entity(pipe->outputs[0])->pads;
 	else
-		entity = &pipe->inputs[0]->video_dev.entity;
+		pad = pipe->inputs[0]->video_dev.entity.pads;
 
-	mdev = entity->graph_obj.mdev;
+	mdev = pad->graph_obj.mdev;
 	mutex_lock(&mdev->graph_mutex);
 	ret = media_graph_walk_init(&pipe->graph, mdev);
 	if (ret)
 		goto out;
-	media_graph_walk_start(&pipe->graph, entity->pads);
-	while ((entity = media_graph_walk_next(&pipe->graph))) {
+	media_graph_walk_start(&pipe->graph, pad);
+	while ((pad = media_graph_walk_next(&pipe->graph))) {
 
-		if (!is_media_entity_v4l2_subdev(entity))
+		if (!is_media_entity_v4l2_subdev(pad->entity))
 			continue;
-		subdev = media_entity_to_v4l2_subdev(entity);
+		subdev = media_entity_to_v4l2_subdev(pad->entity);
 		ret = v4l2_subdev_call(subdev, video, s_stream, 1);
 		if (ret < 0 && ret != -ENOIOCTLCMD)
 			break;
@@ -330,25 +331,25 @@ static int vpfe_pipeline_enable(struct vpfe_pipeline *pipe)
  */
 static int vpfe_pipeline_disable(struct vpfe_pipeline *pipe)
 {
-	struct media_entity *entity;
+	struct media_pad *pad;
 	struct v4l2_subdev *subdev;
 	struct media_device *mdev;
 	int ret = 0;
 
 	if (pipe->state == VPFE_PIPELINE_STREAM_CONTINUOUS)
-		entity = vpfe_get_input_entity(pipe->outputs[0]);
+		pad = vpfe_get_input_entity(pipe->outputs[0])->pads;
 	else
-		entity = &pipe->inputs[0]->video_dev.entity;
+		pad = pipe->inputs[0]->video_dev.entity.pads;
 
-	mdev = entity->graph_obj.mdev;
+	mdev = pad->graph_obj.mdev;
 	mutex_lock(&mdev->graph_mutex);
-	media_graph_walk_start(&pipe->graph, entity->pads);
+	media_graph_walk_start(&pipe->graph, pad);
 
-	while ((entity = media_graph_walk_next(&pipe->graph))) {
+	while ((pad = media_graph_walk_next(&pipe->graph))) {
 
-		if (!is_media_entity_v4l2_subdev(entity))
+		if (!is_media_entity_v4l2_subdev(pad->entity))
 			continue;
-		subdev = media_entity_to_v4l2_subdev(entity);
+		subdev = media_entity_to_v4l2_subdev(pad->entity);
 		ret = v4l2_subdev_call(subdev, video, s_stream, 0);
 		if (ret < 0 && ret != -ENOIOCTLCMD)
 			break;
diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
index ba3036e5a369..82095f627d65 100644
--- a/drivers/staging/media/omap4iss/iss_video.c
+++ b/drivers/staging/media/omap4iss/iss_video.c
@@ -206,8 +206,8 @@ static struct iss_video *
 iss_video_far_end(struct iss_video *video)
 {
 	struct media_graph graph;
-	struct media_entity *entity = &video->video.entity;
-	struct media_device *mdev = entity->graph_obj.mdev;
+	struct media_pad *pad = video->video.entity.pads;
+	struct media_device *mdev = pad->entity->graph_obj.mdev;
 	struct iss_video *far_end = NULL;
 
 	mutex_lock(&mdev->graph_mutex);
@@ -217,16 +217,17 @@ iss_video_far_end(struct iss_video *video)
 		return NULL;
 	}
 
-	media_graph_walk_start(&graph, entity->pads);
+	media_graph_walk_start(&graph, pad);
 
-	while ((entity = media_graph_walk_next(&graph))) {
-		if (entity == &video->video.entity)
+	while ((pad = media_graph_walk_next(&graph))) {
+		if (pad->entity == &video->video.entity)
 			continue;
 
-		if (!is_media_entity_v4l2_video_device(entity))
+		if (!is_media_entity_v4l2_video_device(pad->entity))
 			continue;
 
-		far_end = to_iss_video(media_entity_to_video_device(entity));
+		far_end = to_iss_video(media_entity_to_video_device(
+						pad->entity));
 		if (far_end->type != video->type)
 			break;
 
@@ -861,7 +862,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 	struct iss_video_fh *vfh = to_iss_video_fh(fh);
 	struct iss_video *video = video_drvdata(file);
 	struct media_graph graph;
-	struct media_entity *entity = &video->video.entity;
+	struct media_pad *pad = video->video.entity.pads;
 	enum iss_pipeline_state state;
 	struct iss_pipeline *pipe;
 	struct iss_video *far_end;
@@ -877,30 +878,31 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 	 * Start streaming on the pipeline. No link touching an entity in the
 	 * pipeline can be activated or deactivated once streaming is started.
 	 */
-	pipe = entity->pipe
-	     ? to_iss_pipeline(entity) : &video->pipe;
+	pipe = pad->entity->pipe
+	     ? to_iss_pipeline(pad->entity) : &video->pipe;
 	pipe->external = NULL;
 	pipe->external_rate = 0;
 	pipe->external_bpp = 0;
 
-	ret = media_entity_enum_init(&pipe->ent_enum, entity->graph_obj.mdev);
+	ret = media_entity_enum_init(&pipe->ent_enum,
+				     pad->entity->graph_obj.mdev);
 	if (ret)
 		goto err_graph_walk_init;
 
-	ret = media_graph_walk_init(&graph, entity->graph_obj.mdev);
+	ret = media_graph_walk_init(&graph, pad->entity->graph_obj.mdev);
 	if (ret)
 		goto err_graph_walk_init;
 
 	if (video->iss->pdata->set_constraints)
 		video->iss->pdata->set_constraints(video->iss, true);
 
-	ret = media_pipeline_start(entity, &pipe->pipe);
+	ret = media_pipeline_start(pad->entity, &pipe->pipe);
 	if (ret < 0)
 		goto err_media_pipeline_start;
 
-	media_graph_walk_start(&graph, entity->pads);
-	while ((entity = media_graph_walk_next(&graph)))
-		media_entity_enum_set(&pipe->ent_enum, entity);
+	media_graph_walk_start(&graph, pad);
+	while ((pad = media_graph_walk_next(&graph)))
+		media_entity_enum_set(&pipe->ent_enum, pad->entity);
 
 	/*
 	 * Verify that the currently configured format matches the output of
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index e50972d44fb5..662ebc105093 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -952,10 +952,11 @@ void media_graph_walk_start(struct media_graph *graph, struct media_pad *pad);
  * The graph structure must have been previously initialized with a call to
  * media_graph_walk_start().
  *
- * Return: returns the next entity in the graph or %NULL if the whole graph
- * have been traversed.
+ * Return: returns the next pad in the graph or %NULL if the whole
+ * graph have been traversed. The pad which is returned is the pad
+ * through which a new entity is reached when parsing the graph.
  */
-struct media_entity *media_graph_walk_next(struct media_graph *graph);
+struct media_pad *media_graph_walk_next(struct media_graph *graph);
 
 /**
  * media_pipeline_start - Mark a pipeline as streaming
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 04/31] v4l: mc: Start walk from a specific pad in use count calculation
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (2 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 03/31] media: entity: Walk the graph based on pads Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 05/31] media: entity: Add iterator helper for entity pads Jacopo Mondi
                   ` (27 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Sakari Ailus <sakari.ailus@linux.intel.com>

With the addition of the recent has_route() media entity op, the pads of a
media entity are no longer all interconnected. This has to be taken into
account in power management.

Prepare for the addition of a helper function supporting S_ROUTING.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/v4l2-core/v4l2-mc.c | 25 ++++++++++++-------------
 1 file changed, 12 insertions(+), 13 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
index 98edd47b2f0a..558dec225838 100644
--- a/drivers/media/v4l2-core/v4l2-mc.c
+++ b/drivers/media/v4l2-core/v4l2-mc.c
@@ -332,17 +332,16 @@ EXPORT_SYMBOL_GPL(v4l_vb2q_enable_media_source);
 
 /*
  * pipeline_pm_use_count - Count the number of users of a pipeline
- * @entity: The entity
+ * @pad: Any pad along the pipeline
  *
  * Return the total number of users of all video device nodes in the pipeline.
  */
-static int pipeline_pm_use_count(struct media_entity *entity,
-	struct media_graph *graph)
+static int pipeline_pm_use_count(struct media_pad *pad,
+				 struct media_graph *graph)
 {
-	struct media_pad *pad;
 	int use = 0;
 
-	media_graph_walk_start(graph, entity->pads);
+	media_graph_walk_start(graph, pad);
 
 	while ((pad = media_graph_walk_next(graph))) {
 		if (is_media_entity_v4l2_video_device(pad->entity))
@@ -388,7 +387,7 @@ static int pipeline_pm_power_one(struct media_entity *entity, int change)
 
 /*
  * pipeline_pm_power - Apply power change to all entities in a pipeline
- * @entity: The entity
+ * @pad: Any pad along the pipeline
  * @change: Use count change
  *
  * Walk the pipeline to update the use count and the power state of all non-node
@@ -396,16 +395,16 @@ static int pipeline_pm_power_one(struct media_entity *entity, int change)
  *
  * Return 0 on success or a negative error code on failure.
  */
-static int pipeline_pm_power(struct media_entity *entity, int change,
+static int pipeline_pm_power(struct media_pad *pad, int change,
 	struct media_graph *graph)
 {
-	struct media_pad *tmp_pad, *pad;
+	struct media_pad *tmp_pad, *first = pad;
 	int ret = 0;
 
 	if (!change)
 		return 0;
 
-	media_graph_walk_start(graph, entity->pads);
+	media_graph_walk_start(graph, pad);
 
 	while (!ret && (pad = media_graph_walk_next(graph)))
 		if (is_media_entity_v4l2_subdev(pad->entity))
@@ -414,7 +413,7 @@ static int pipeline_pm_power(struct media_entity *entity, int change,
 	if (!ret)
 		return ret;
 
-	media_graph_walk_start(graph, entity->pads);
+	media_graph_walk_start(graph, first);
 
 	while ((tmp_pad = media_graph_walk_next(graph))
 	       && tmp_pad != pad)
@@ -437,7 +436,7 @@ int v4l2_pipeline_pm_use(struct media_entity *entity, int use)
 	WARN_ON(entity->use_count < 0);
 
 	/* Apply power change to connected non-nodes. */
-	ret = pipeline_pm_power(entity, change, &mdev->pm_count_walk);
+	ret = pipeline_pm_power(entity->pads, change, &mdev->pm_count_walk);
 	if (ret < 0)
 		entity->use_count -= change;
 
@@ -451,8 +450,8 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 			      unsigned int notification)
 {
 	struct media_graph *graph = &link->graph_obj.mdev->pm_count_walk;
-	struct media_entity *source = link->source->entity;
-	struct media_entity *sink = link->sink->entity;
+	struct media_pad *source = link->source;
+	struct media_pad *sink = link->sink;
 	int source_use;
 	int sink_use;
 	int ret = 0;
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 05/31] media: entity: Add iterator helper for entity pads
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (3 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 04/31] v4l: mc: Start walk from a specific pad in use count calculation Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 06/31] media: entity: Move the pipeline from entity to pads Jacopo Mondi
                   ` (26 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

Add an iterator helper to easily cycle through all pads in an entity and
use it in media-entity and media-device code where appropriate.

Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/media-device.c | 13 ++++++-------
 drivers/media/media-entity.c | 11 ++++++-----
 include/media/media-entity.h | 12 ++++++++++++
 3 files changed, 24 insertions(+), 12 deletions(-)

diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
index b8ec88612df7..cddf5a0857b9 100644
--- a/drivers/media/media-device.c
+++ b/drivers/media/media-device.c
@@ -584,7 +584,7 @@ int __must_check media_device_register_entity(struct media_device *mdev,
 					      struct media_entity *entity)
 {
 	struct media_entity_notify *notify, *next;
-	unsigned int i;
+	struct media_pad *iter;
 	int ret;
 
 	if (entity->function == MEDIA_ENT_F_V4L2_SUBDEV_UNKNOWN ||
@@ -613,9 +613,8 @@ int __must_check media_device_register_entity(struct media_device *mdev,
 	media_gobj_create(mdev, MEDIA_GRAPH_ENTITY, &entity->graph_obj);
 
 	/* Initialize objects at the pads */
-	for (i = 0; i < entity->num_pads; i++)
-		media_gobj_create(mdev, MEDIA_GRAPH_PAD,
-			       &entity->pads[i].graph_obj);
+	media_entity_for_each_pad(entity, iter)
+		media_gobj_create(mdev, MEDIA_GRAPH_PAD, &iter->graph_obj);
 
 	/* invoke entity_notify callbacks */
 	list_for_each_entry_safe(notify, next, &mdev->entity_notify, list)
@@ -649,7 +648,7 @@ static void __media_device_unregister_entity(struct media_entity *entity)
 	struct media_device *mdev = entity->graph_obj.mdev;
 	struct media_link *link, *tmp;
 	struct media_interface *intf;
-	unsigned int i;
+	struct media_pad *iter;
 
 	ida_free(&mdev->entity_internal_idx, entity->internal_idx);
 
@@ -665,8 +664,8 @@ static void __media_device_unregister_entity(struct media_entity *entity)
 	__media_entity_remove_links(entity);
 
 	/* Remove all pads that belong to this entity */
-	for (i = 0; i < entity->num_pads; i++)
-		media_gobj_destroy(&entity->pads[i].graph_obj);
+	media_entity_for_each_pad(entity, iter)
+		media_gobj_destroy(&iter->graph_obj);
 
 	/* Remove the entity */
 	media_gobj_destroy(&entity->graph_obj);
diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
index 04e987681ab5..5ac7ea2ae15a 100644
--- a/drivers/media/media-entity.c
+++ b/drivers/media/media-entity.c
@@ -207,7 +207,8 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads,
 			   struct media_pad *pads)
 {
 	struct media_device *mdev = entity->graph_obj.mdev;
-	unsigned int i;
+	struct media_pad *iter;
+	unsigned int i = 0;
 
 	if (num_pads >= MEDIA_ENTITY_MAX_PADS)
 		return -E2BIG;
@@ -218,12 +219,12 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads,
 	if (mdev)
 		mutex_lock(&mdev->graph_mutex);
 
-	for (i = 0; i < num_pads; i++) {
-		pads[i].entity = entity;
-		pads[i].index = i;
+	media_entity_for_each_pad(entity, iter) {
+		iter->entity = entity;
+		iter->index = i++;
 		if (mdev)
 			media_gobj_create(mdev, MEDIA_GRAPH_PAD,
-					&entity->pads[i].graph_obj);
+					&iter->graph_obj);
 	}
 
 	if (mdev)
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 662ebc105093..16505a249c59 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -1133,3 +1133,15 @@ void media_remove_intf_links(struct media_interface *intf);
 	 (entity)->ops->operation((entity) , ##args) : -ENOIOCTLCMD)
 
 #endif
+
+/**
+ * media_entity_for_each_pad - Iterate on all pads in an entity
+ * @entity: The entity the pads belong to
+ * @iter: The iterator pad
+ *
+ * Iterate on all pads in a media entity.
+ */
+#define media_entity_for_each_pad(entity, iter)			\
+	for (iter = (entity)->pads;				\
+	     iter < &(entity)->pads[(entity)->num_pads];	\
+	     ++iter)
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 06/31] media: entity: Move the pipeline from entity to pads
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (4 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 05/31] media: entity: Add iterator helper for entity pads Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 07/31] media: entity: Use pad as the starting point for a pipeline Jacopo Mondi
                   ` (25 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Sakari Ailus <sakari.ailus@linux.intel.com>

This moves the pipe and stream_count fields from struct media_entity to
struct media_pad. Effectively streams become pad-specific rather than
being entity specific, allowing several independent streams to traverse a
single entity and an entity to be part of several streams.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

- Update documentation to use 'pads'
- Use the media pad iterator in media_entity.c
- Update rcar-dma.c to use the new per-pad stream count
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/media-entity.c                  | 62 +++++++++++--------
 drivers/media/platform/exynos4-is/fimc-isp.c  |  2 +-
 drivers/media/platform/exynos4-is/fimc-lite.c |  2 +-
 drivers/media/platform/omap3isp/isp.c         |  2 +-
 drivers/media/platform/omap3isp/ispvideo.c    |  2 +-
 drivers/media/platform/omap3isp/ispvideo.h    |  2 +-
 drivers/media/platform/rcar-vin/rcar-core.c   | 16 +++--
 drivers/media/platform/rcar-vin/rcar-dma.c    |  2 +-
 drivers/media/platform/xilinx/xilinx-dma.c    |  2 +-
 drivers/media/platform/xilinx/xilinx-dma.h    |  2 +-
 drivers/staging/media/imx/imx-media-utils.c   |  2 +-
 drivers/staging/media/omap4iss/iss.c          |  2 +-
 drivers/staging/media/omap4iss/iss_video.c    |  2 +-
 drivers/staging/media/omap4iss/iss_video.h    |  2 +-
 include/media/media-entity.h                  | 21 ++++---
 15 files changed, 70 insertions(+), 53 deletions(-)

diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
index 5ac7ea2ae15a..e74f206377da 100644
--- a/drivers/media/media-entity.c
+++ b/drivers/media/media-entity.c
@@ -432,21 +432,25 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 
 	while ((pad = media_graph_walk_next(graph))) {
 		struct media_entity *entity = pad->entity;
+		bool skip_validation = pad->pipe;
+		struct media_pad *iter;
 
 		DECLARE_BITMAP(active, MEDIA_ENTITY_MAX_PADS);
 		DECLARE_BITMAP(has_no_links, MEDIA_ENTITY_MAX_PADS);
 
-		entity->stream_count++;
-
-		if (WARN_ON(entity->pipe && entity->pipe != pipe)) {
-			ret = -EBUSY;
-			goto error;
+		media_entity_for_each_pad(entity, iter) {
+			if (iter->pipe && WARN_ON(iter->pipe != pipe))
+				ret = -EBUSY;
+			else
+				iter->pipe = pipe;
+			iter->stream_count++;
 		}
 
-		entity->pipe = pipe;
+		if (ret)
+			goto error;
 
-		/* Already streaming --- no need to check. */
-		if (entity->stream_count > 1)
+		/* Already part of the pipeline, skip validation. */
+		if (skip_validation)
 			continue;
 
 		if (!entity->ops || !entity->ops->link_validate)
@@ -514,20 +518,23 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 	media_graph_walk_start(graph, pad_err);
 
 	while ((pad_err = media_graph_walk_next(graph))) {
-		struct media_entity *entity_err = pad_err->entity;
-
-		/* Sanity check for negative stream_count */
-		if (!WARN_ON_ONCE(entity_err->stream_count <= 0)) {
-			entity_err->stream_count--;
-			if (entity_err->stream_count == 0)
-				entity_err->pipe = NULL;
+		struct media_entity *entity = pad->entity;
+		struct media_pad *iter;
+
+		media_entity_for_each_pad(entity, iter) {
+			/* Sanity check for negative stream_count */
+			if (!WARN_ON_ONCE(iter->stream_count <= 0)) {
+				--iter->stream_count;
+				if (iter->stream_count == 0)
+					iter->pipe = NULL;
+			}
 		}
 
 		/*
 		 * We haven't increased stream_count further than this
 		 * so we quit here.
 		 */
-		if (pad_err == pad)
+		if (pad_err->entity == pad->entity)
 			break;
 	}
 
@@ -554,7 +561,7 @@ EXPORT_SYMBOL_GPL(media_pipeline_start);
 
 void __media_pipeline_stop(struct media_entity *entity)
 {
-	struct media_pipeline *pipe = entity->pipe;
+	struct media_pipeline *pipe = entity->pads->pipe;
 	struct media_graph *graph = &pipe->graph;
 	struct media_pad *pad;
 
@@ -569,12 +576,15 @@ void __media_pipeline_stop(struct media_entity *entity)
 
 	while ((pad = media_graph_walk_next(graph))) {
 		struct media_entity *entity = pad->entity;
-
-		/* Sanity check for negative stream_count */
-		if (!WARN_ON_ONCE(entity->stream_count <= 0)) {
-			entity->stream_count--;
-			if (entity->stream_count == 0)
-				entity->pipe = NULL;
+		struct media_pad *iter;
+
+		media_entity_for_each_pad(entity, iter) {
+			/* Sanity check for negative stream_count */
+			if (!WARN_ON_ONCE(iter->stream_count <= 0)) {
+				iter->stream_count--;
+				if (iter->stream_count == 0)
+					iter->pipe = NULL;
+			}
 		}
 	}
 
@@ -866,7 +876,7 @@ int __media_entity_setup_link(struct media_link *link, u32 flags)
 {
 	const u32 mask = MEDIA_LNK_FL_ENABLED;
 	struct media_device *mdev;
-	struct media_entity *source, *sink;
+	struct media_pad *source, *sink;
 	int ret = -EBUSY;
 
 	if (link == NULL)
@@ -882,8 +892,8 @@ int __media_entity_setup_link(struct media_link *link, u32 flags)
 	if (link->flags == flags)
 		return 0;
 
-	source = link->source->entity;
-	sink = link->sink->entity;
+	source = link->source;
+	sink = link->sink;
 
 	if (!(link->flags & MEDIA_LNK_FL_DYNAMIC) &&
 	    (source->stream_count || sink->stream_count))
diff --git a/drivers/media/platform/exynos4-is/fimc-isp.c b/drivers/media/platform/exynos4-is/fimc-isp.c
index 9a48c0f69320..79d128a57e87 100644
--- a/drivers/media/platform/exynos4-is/fimc-isp.c
+++ b/drivers/media/platform/exynos4-is/fimc-isp.c
@@ -229,7 +229,7 @@ static int fimc_isp_subdev_set_fmt(struct v4l2_subdev *sd,
 			}
 		}
 	} else {
-		if (sd->entity.stream_count == 0) {
+		if (sd->entity.pads->stream_count == 0) {
 			if (fmt->pad == FIMC_ISP_SD_PAD_SINK) {
 				struct v4l2_subdev_format format = *fmt;
 
diff --git a/drivers/media/platform/exynos4-is/fimc-lite.c b/drivers/media/platform/exynos4-is/fimc-lite.c
index 96f0a8a0dcae..dbadcba6739a 100644
--- a/drivers/media/platform/exynos4-is/fimc-lite.c
+++ b/drivers/media/platform/exynos4-is/fimc-lite.c
@@ -1096,7 +1096,7 @@ static int fimc_lite_subdev_set_fmt(struct v4l2_subdev *sd,
 	mutex_lock(&fimc->lock);
 
 	if ((atomic_read(&fimc->out_path) == FIMC_IO_ISP &&
-	    sd->entity.stream_count > 0) ||
+	    sd->entity.pads->stream_count > 0) ||
 	    (atomic_read(&fimc->out_path) == FIMC_IO_DMA &&
 	    vb2_is_busy(&fimc->vb_queue))) {
 		mutex_unlock(&fimc->lock);
diff --git a/drivers/media/platform/omap3isp/isp.c b/drivers/media/platform/omap3isp/isp.c
index bd57174d81a7..322a7ebdaf0d 100644
--- a/drivers/media/platform/omap3isp/isp.c
+++ b/drivers/media/platform/omap3isp/isp.c
@@ -927,7 +927,7 @@ static int isp_pipeline_is_last(struct media_entity *me)
 	struct isp_pipeline *pipe;
 	struct media_pad *pad;
 
-	if (!me->pipe)
+	if (!me->pads->pipe)
 		return 0;
 	pipe = to_isp_pipeline(me);
 	if (pipe->stream_state == ISP_PIPELINE_STREAM_STOPPED)
diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
index be364eb64e40..aed6c0a08284 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -1102,7 +1102,7 @@ isp_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 	/* Start streaming on the pipeline. No link touching an entity in the
 	 * pipeline can be activated or deactivated once streaming is started.
 	 */
-	pipe = video->video.entity.pipe
+	pipe = video->video.entity.pads->pipe
 	     ? to_isp_pipeline(&video->video.entity) : &video->pipe;
 
 	ret = media_entity_enum_init(&pipe->ent_enum, &video->isp->media_dev);
diff --git a/drivers/media/platform/omap3isp/ispvideo.h b/drivers/media/platform/omap3isp/ispvideo.h
index f6a2082b4a0a..8f4146c25a1b 100644
--- a/drivers/media/platform/omap3isp/ispvideo.h
+++ b/drivers/media/platform/omap3isp/ispvideo.h
@@ -103,7 +103,7 @@ struct isp_pipeline {
 };
 
 #define to_isp_pipeline(__e) \
-	container_of((__e)->pipe, struct isp_pipeline, pipe)
+	container_of((__e)->pads->pipe, struct isp_pipeline, pipe)
 
 static inline int isp_pipeline_ready(struct isp_pipeline *pipe)
 {
diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index 594d80434004..d4eae6fddb46 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -132,13 +132,17 @@ static int rvin_group_link_notify(struct media_link *link, u32 flags,
 		return 0;
 
 	/*
-	 * Don't allow link changes if any entity in the graph is
-	 * streaming, modifying the CHSEL register fields can disrupt
-	 * running streams.
+	 * Don't allow link changes if any stream in the graph is active as
+	 * modifying the CHSEL register fields can disrupt running streams.
 	 */
-	media_device_for_each_entity(entity, &group->mdev)
-		if (entity->stream_count)
-			return -EBUSY;
+	media_device_for_each_entity(entity, &group->mdev) {
+		struct media_pad *iter;
+
+		media_entity_for_each_pad(entity, iter) {
+			if (iter->stream_count)
+				return -EBUSY;
+		}
+	}
 
 	mutex_lock(&group->lock);
 
diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index 2207a31d355e..c8437810ebf8 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -1128,7 +1128,7 @@ static int rvin_set_stream(struct rvin_dev *vin, int on)
 	 */
 	mdev = vin->vdev.entity.graph_obj.mdev;
 	mutex_lock(&mdev->graph_mutex);
-	pipe = sd->entity.pipe ? sd->entity.pipe : &vin->vdev.pipe;
+	pipe = sd->entity.pads->pipe ? sd->entity.pads->pipe : &vin->vdev.pipe;
 	ret = __media_pipeline_start(&vin->vdev.entity, pipe);
 	mutex_unlock(&mdev->graph_mutex);
 	if (ret)
diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index 66063ccbf84d..08ff171d7aac 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -403,7 +403,7 @@ static int xvip_dma_start_streaming(struct vb2_queue *vq, unsigned int count)
 	 * Use the pipeline object embedded in the first DMA object that starts
 	 * streaming.
 	 */
-	pipe = dma->video.entity.pipe
+	pipe = dma->video.entity.pads->pipe
 	     ? to_xvip_pipeline(&dma->video.entity) : &dma->pipe;
 
 	ret = media_pipeline_start(&dma->video.entity, &pipe->pipe);
diff --git a/drivers/media/platform/xilinx/xilinx-dma.h b/drivers/media/platform/xilinx/xilinx-dma.h
index 5aec4d17eb21..45b815e83d69 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.h
+++ b/drivers/media/platform/xilinx/xilinx-dma.h
@@ -47,7 +47,7 @@ struct xvip_pipeline {
 
 static inline struct xvip_pipeline *to_xvip_pipeline(struct media_entity *e)
 {
-	return container_of(e->pipe, struct xvip_pipeline, pipe);
+	return container_of(e->pads->pipe, struct xvip_pipeline, pipe);
 }
 
 /**
diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
index 1c63a2765a81..cc10f2d5b51b 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -936,7 +936,7 @@ int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
 			__media_pipeline_stop(entity);
 	} else {
 		v4l2_subdev_call(sd, video, s_stream, 0);
-		if (entity->pipe)
+		if (entity->pads->pipe)
 			__media_pipeline_stop(entity);
 	}
 
diff --git a/drivers/staging/media/omap4iss/iss.c b/drivers/staging/media/omap4iss/iss.c
index c8be1db532ab..030808b222cf 100644
--- a/drivers/staging/media/omap4iss/iss.c
+++ b/drivers/staging/media/omap4iss/iss.c
@@ -543,7 +543,7 @@ static int iss_pipeline_is_last(struct media_entity *me)
 	struct iss_pipeline *pipe;
 	struct media_pad *pad;
 
-	if (!me->pipe)
+	if (!me->pads->pipe)
 		return 0;
 	pipe = to_iss_pipeline(me);
 	if (pipe->stream_state == ISS_PIPELINE_STREAM_STOPPED)
diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
index 82095f627d65..2a2f7a7983db 100644
--- a/drivers/staging/media/omap4iss/iss_video.c
+++ b/drivers/staging/media/omap4iss/iss_video.c
@@ -878,7 +878,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 	 * Start streaming on the pipeline. No link touching an entity in the
 	 * pipeline can be activated or deactivated once streaming is started.
 	 */
-	pipe = pad->entity->pipe
+	pipe = pad->pipe
 	     ? to_iss_pipeline(pad->entity) : &video->pipe;
 	pipe->external = NULL;
 	pipe->external_rate = 0;
diff --git a/drivers/staging/media/omap4iss/iss_video.h b/drivers/staging/media/omap4iss/iss_video.h
index f22489edb562..cdea8543b3f9 100644
--- a/drivers/staging/media/omap4iss/iss_video.h
+++ b/drivers/staging/media/omap4iss/iss_video.h
@@ -94,7 +94,7 @@ struct iss_pipeline {
 };
 
 #define to_iss_pipeline(__e) \
-	container_of((__e)->pipe, struct iss_pipeline, pipe)
+	container_of((__e)->pads->pipe, struct iss_pipeline, pipe)
 
 static inline int iss_pipeline_ready(struct iss_pipeline *pipe)
 {
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 16505a249c59..fcde7dd247ed 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -188,15 +188,24 @@ enum media_pad_signal_type {
  *
  * @graph_obj:	Embedded structure containing the media object common data
  * @entity:	Entity this pad belongs to
+ * @pipe:	Pipeline this pad belongs to.
+ * @stream_count: Stream count for the pad.
  * @index:	Pad index in the entity pads array, numbered from 0 to n
  * @sig_type:	Type of the signal inside a media pad
  * @flags:	Pad flags, as defined in
  *		:ref:`include/uapi/linux/media.h <media_header>`
  *		(seek for ``MEDIA_PAD_FL_*``)
+ * .. note::
+ *
+ *    @stream_count reference count must never be negative, but is a signed
+ *    integer on purpose: a simple ``WARN_ON(<0)`` check can be used to
+ *    detect reference count bugs that would make them negative.
  */
 struct media_pad {
 	struct media_gobj graph_obj;	/* must be first field in struct */
 	struct media_entity *entity;
+	struct media_pipeline *pipe;
+	int stream_count;
 	u16 index;
 	enum media_pad_signal_type sig_type;
 	unsigned long flags;
@@ -274,9 +283,7 @@ enum media_entity_type {
  * @pads:	Pads array with the size defined by @num_pads.
  * @links:	List of data links.
  * @ops:	Entity operations.
- * @stream_count: Stream count for the entity.
  * @use_count:	Use count for the entity.
- * @pipe:	Pipeline this entity belongs to.
  * @info:	Union with devnode information.  Kept just for backward
  *		compatibility.
  * @info.dev:	Contains device major and minor info.
@@ -289,10 +296,9 @@ enum media_entity_type {
  *
  * .. note::
  *
- *    @stream_count and @use_count reference counts must never be
- *    negative, but are signed integers on purpose: a simple ``WARN_ON(<0)``
- *    check can be used to detect reference count bugs that would make them
- *    negative.
+ *    @use_count reference count must never be negative, but is a signed
+ *    integer on purpose: a simple ``WARN_ON(<0)`` check can be used to
+ *    detect reference count bugs that would make them negative.
  */
 struct media_entity {
 	struct media_gobj graph_obj;	/* must be first field in struct */
@@ -311,11 +317,8 @@ struct media_entity {
 
 	const struct media_entity_operations *ops;
 
-	int stream_count;
 	int use_count;
 
-	struct media_pipeline *pipe;
-
 	union {
 		struct {
 			u32 major;
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 07/31] media: entity: Use pad as the starting point for a pipeline
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (5 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 06/31] media: entity: Move the pipeline from entity to pads Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 08/31] media: entity: Add has_route entity operation Jacopo Mondi
                   ` (24 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Sakari Ailus <sakari.ailus@linux.intel.com>

The pipeline has been moved from the entity to the pads; reflect this in
the media pipeline function API.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 Documentation/media/kapi/mc-core.rst          |  6 ++--
 drivers/media/media-entity.c                  | 24 ++++++-------
 drivers/media/pci/intel/ipu3/ipu3-cio2.c      |  6 ++--
 .../media/platform/exynos4-is/fimc-capture.c  |  8 ++---
 .../platform/exynos4-is/fimc-isp-video.c      |  8 ++---
 drivers/media/platform/exynos4-is/fimc-lite.c |  8 ++---
 drivers/media/platform/omap3isp/ispvideo.c    |  6 ++--
 .../media/platform/qcom/camss/camss-video.c   |  6 ++--
 drivers/media/platform/rcar-vin/rcar-dma.c    |  6 ++--
 .../media/platform/s3c-camif/camif-capture.c  |  6 ++--
 drivers/media/platform/vimc/vimc-capture.c    |  5 +--
 drivers/media/platform/vsp1/vsp1_video.c      |  6 ++--
 drivers/media/platform/xilinx/xilinx-dma.c    |  6 ++--
 drivers/media/usb/au0828/au0828-core.c        |  4 +--
 drivers/staging/media/imx/imx-media-utils.c   |  6 ++--
 drivers/staging/media/omap4iss/iss_video.c    |  6 ++--
 include/media/media-entity.h                  | 34 +++++++++----------
 17 files changed, 75 insertions(+), 76 deletions(-)

diff --git a/Documentation/media/kapi/mc-core.rst b/Documentation/media/kapi/mc-core.rst
index a516563b358a..d7498df18c29 100644
--- a/Documentation/media/kapi/mc-core.rst
+++ b/Documentation/media/kapi/mc-core.rst
@@ -213,11 +213,11 @@ When starting streaming, drivers must notify all entities in the pipeline to
 prevent link states from being modified during streaming by calling
 :c:func:`media_pipeline_start()`.
 
-The function will mark all entities connected to the given entity through
-enabled links, either directly or indirectly, as streaming.
+The function will mark all entities connected to the given pad through
+enabled routes and links, either directly or indirectly, as streaming.
 
 The struct :c:type:`media_pipeline` instance pointed to by
-the pipe argument will be stored in every entity in the pipeline.
+the pipe argument will be stored in every pad in the pipeline.
 Drivers should embed the struct :c:type:`media_pipeline`
 in higher-level pipeline structures and can then access the
 pipeline through the struct :c:type:`media_entity`
diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
index e74f206377da..6f5196d05894 100644
--- a/drivers/media/media-entity.c
+++ b/drivers/media/media-entity.c
@@ -412,12 +412,11 @@ EXPORT_SYMBOL_GPL(media_entity_get_fwnode_pad);
  * Pipeline management
  */
 
-__must_check int __media_pipeline_start(struct media_entity *entity,
+__must_check int __media_pipeline_start(struct media_pad *pad,
 					struct media_pipeline *pipe)
 {
-	struct media_device *mdev = entity->graph_obj.mdev;
+	struct media_device *mdev = pad->graph_obj.mdev;
 	struct media_graph *graph = &pipe->graph;
-	struct media_pad *pad = entity->pads;
 	struct media_pad *pad_err = pad;
 	struct media_link *link;
 	int ret;
@@ -546,24 +545,23 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 }
 EXPORT_SYMBOL_GPL(__media_pipeline_start);
 
-__must_check int media_pipeline_start(struct media_entity *entity,
+__must_check int media_pipeline_start(struct media_pad *pad,
 				      struct media_pipeline *pipe)
 {
-	struct media_device *mdev = entity->graph_obj.mdev;
+	struct media_device *mdev = pad->graph_obj.mdev;
 	int ret;
 
 	mutex_lock(&mdev->graph_mutex);
-	ret = __media_pipeline_start(entity, pipe);
+	ret = __media_pipeline_start(pad, pipe);
 	mutex_unlock(&mdev->graph_mutex);
 	return ret;
 }
 EXPORT_SYMBOL_GPL(media_pipeline_start);
 
-void __media_pipeline_stop(struct media_entity *entity)
+void __media_pipeline_stop(struct media_pad *pad)
 {
-	struct media_pipeline *pipe = entity->pads->pipe;
+	struct media_pipeline *pipe = pad->pipe;
 	struct media_graph *graph = &pipe->graph;
-	struct media_pad *pad;
 
 	/*
 	 * If the following check fails, the driver has performed an
@@ -572,7 +570,7 @@ void __media_pipeline_stop(struct media_entity *entity)
 	if (WARN_ON(!pipe))
 		return;
 
-	media_graph_walk_start(graph, entity->pads);
+	media_graph_walk_start(graph, pad);
 
 	while ((pad = media_graph_walk_next(graph))) {
 		struct media_entity *entity = pad->entity;
@@ -594,12 +592,12 @@ void __media_pipeline_stop(struct media_entity *entity)
 }
 EXPORT_SYMBOL_GPL(__media_pipeline_stop);
 
-void media_pipeline_stop(struct media_entity *entity)
+void media_pipeline_stop(struct media_pad *pad)
 {
-	struct media_device *mdev = entity->graph_obj.mdev;
+	struct media_device *mdev = pad->graph_obj.mdev;
 
 	mutex_lock(&mdev->graph_mutex);
-	__media_pipeline_stop(entity);
+	__media_pipeline_stop(pad);
 	mutex_unlock(&mdev->graph_mutex);
 }
 EXPORT_SYMBOL_GPL(media_pipeline_stop);
diff --git a/drivers/media/pci/intel/ipu3/ipu3-cio2.c b/drivers/media/pci/intel/ipu3/ipu3-cio2.c
index 617fb2e944dc..94ba6d7c3fd7 100644
--- a/drivers/media/pci/intel/ipu3/ipu3-cio2.c
+++ b/drivers/media/pci/intel/ipu3/ipu3-cio2.c
@@ -1000,7 +1000,7 @@ static int cio2_vb2_start_streaming(struct vb2_queue *vq, unsigned int count)
 		return r;
 	}
 
-	r = media_pipeline_start(&q->vdev.entity, &q->pipe);
+	r = media_pipeline_start(q->vdev.entity.pads, &q->pipe);
 	if (r)
 		goto fail_pipeline;
 
@@ -1020,7 +1020,7 @@ static int cio2_vb2_start_streaming(struct vb2_queue *vq, unsigned int count)
 fail_csi2_subdev:
 	cio2_hw_exit(cio2, q);
 fail_hw:
-	media_pipeline_stop(&q->vdev.entity);
+	media_pipeline_stop(q->vdev.entity.pads);
 fail_pipeline:
 	dev_dbg(&cio2->pci_dev->dev, "failed to start streaming (%d)\n", r);
 	cio2_vb2_return_all_buffers(q, VB2_BUF_STATE_QUEUED);
@@ -1041,7 +1041,7 @@ static void cio2_vb2_stop_streaming(struct vb2_queue *vq)
 	cio2_hw_exit(cio2, q);
 	synchronize_irq(cio2->pci_dev->irq);
 	cio2_vb2_return_all_buffers(q, VB2_BUF_STATE_ERROR);
-	media_pipeline_stop(&q->vdev.entity);
+	media_pipeline_stop(q->vdev.entity.pads);
 	pm_runtime_put(&cio2->pci_dev->dev);
 	cio2->streaming = false;
 }
diff --git a/drivers/media/platform/exynos4-is/fimc-capture.c b/drivers/media/platform/exynos4-is/fimc-capture.c
index 3e9fcf4f8a13..f803877a512d 100644
--- a/drivers/media/platform/exynos4-is/fimc-capture.c
+++ b/drivers/media/platform/exynos4-is/fimc-capture.c
@@ -537,7 +537,7 @@ static int fimc_capture_release(struct file *file)
 	mutex_lock(&fimc->lock);
 
 	if (close && vc->streaming) {
-		media_pipeline_stop(&vc->ve.vdev.entity);
+		media_pipeline_stop(vc->ve.vdev.entity.pads);
 		vc->streaming = false;
 	}
 
@@ -1201,7 +1201,7 @@ static int fimc_cap_streamon(struct file *file, void *priv,
 	if (fimc_capture_active(fimc))
 		return -EBUSY;
 
-	ret = media_pipeline_start(entity, &vc->ve.pipe->mp);
+	ret = media_pipeline_start(entity->pads, &vc->ve.pipe->mp);
 	if (ret < 0)
 		return ret;
 
@@ -1235,7 +1235,7 @@ static int fimc_cap_streamon(struct file *file, void *priv,
 	}
 
 err_p_stop:
-	media_pipeline_stop(entity);
+	media_pipeline_stop(entity->pads);
 	return ret;
 }
 
@@ -1250,7 +1250,7 @@ static int fimc_cap_streamoff(struct file *file, void *priv,
 	if (ret < 0)
 		return ret;
 
-	media_pipeline_stop(&vc->ve.vdev.entity);
+	media_pipeline_stop(vc->ve.vdev.entity.pads);
 	vc->streaming = false;
 	return 0;
 }
diff --git a/drivers/media/platform/exynos4-is/fimc-isp-video.c b/drivers/media/platform/exynos4-is/fimc-isp-video.c
index bb35a2017f21..5904931c4385 100644
--- a/drivers/media/platform/exynos4-is/fimc-isp-video.c
+++ b/drivers/media/platform/exynos4-is/fimc-isp-video.c
@@ -312,7 +312,7 @@ static int isp_video_release(struct file *file)
 	mutex_lock(&isp->video_lock);
 
 	if (v4l2_fh_is_singular_file(file) && ivc->streaming) {
-		media_pipeline_stop(entity);
+		media_pipeline_stop(entity->pads);
 		ivc->streaming = 0;
 	}
 
@@ -494,7 +494,7 @@ static int isp_video_streamon(struct file *file, void *priv,
 	struct media_entity *me = &ve->vdev.entity;
 	int ret;
 
-	ret = media_pipeline_start(me, &ve->pipe->mp);
+	ret = media_pipeline_start(me->pads, &ve->pipe->mp);
 	if (ret < 0)
 		return ret;
 
@@ -509,7 +509,7 @@ static int isp_video_streamon(struct file *file, void *priv,
 	isp->video_capture.streaming = 1;
 	return 0;
 p_stop:
-	media_pipeline_stop(me);
+	media_pipeline_stop(me->pads);
 	return ret;
 }
 
@@ -524,7 +524,7 @@ static int isp_video_streamoff(struct file *file, void *priv,
 	if (ret < 0)
 		return ret;
 
-	media_pipeline_stop(&video->ve.vdev.entity);
+	media_pipeline_stop(video->ve.vdev.entity.pads);
 	video->streaming = 0;
 	return 0;
 }
diff --git a/drivers/media/platform/exynos4-is/fimc-lite.c b/drivers/media/platform/exynos4-is/fimc-lite.c
index dbadcba6739a..1d6858e5abb0 100644
--- a/drivers/media/platform/exynos4-is/fimc-lite.c
+++ b/drivers/media/platform/exynos4-is/fimc-lite.c
@@ -524,7 +524,7 @@ static int fimc_lite_release(struct file *file)
 	if (v4l2_fh_is_singular_file(file) &&
 	    atomic_read(&fimc->out_path) == FIMC_IO_DMA) {
 		if (fimc->streaming) {
-			media_pipeline_stop(entity);
+			media_pipeline_stop(entity->pads);
 			fimc->streaming = false;
 		}
 		fimc_lite_stop_capture(fimc, false);
@@ -832,7 +832,7 @@ static int fimc_lite_streamon(struct file *file, void *priv,
 	if (fimc_lite_active(fimc))
 		return -EBUSY;
 
-	ret = media_pipeline_start(entity, &fimc->ve.pipe->mp);
+	ret = media_pipeline_start(entity->pads, &fimc->ve.pipe->mp);
 	if (ret < 0)
 		return ret;
 
@@ -849,7 +849,7 @@ static int fimc_lite_streamon(struct file *file, void *priv,
 	}
 
 err_p_stop:
-	media_pipeline_stop(entity);
+	media_pipeline_stop(entity->pads);
 	return 0;
 }
 
@@ -863,7 +863,7 @@ static int fimc_lite_streamoff(struct file *file, void *priv,
 	if (ret < 0)
 		return ret;
 
-	media_pipeline_stop(&fimc->ve.vdev.entity);
+	media_pipeline_stop(fimc->ve.vdev.entity.pads);
 	fimc->streaming = false;
 	return 0;
 }
diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
index aed6c0a08284..9bca57c0e5c7 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -1113,7 +1113,7 @@ isp_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 	pipe->l3_ick = clk_get_rate(video->isp->clock[ISP_CLK_L3_ICK]);
 	pipe->max_rate = pipe->l3_ick;
 
-	ret = media_pipeline_start(&video->video.entity, &pipe->pipe);
+	ret = media_pipeline_start(video->video.entity.pads, &pipe->pipe);
 	if (ret < 0)
 		goto err_pipeline_start;
 
@@ -1170,7 +1170,7 @@ isp_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 	return 0;
 
 err_check_format:
-	media_pipeline_stop(&video->video.entity);
+	media_pipeline_stop(video->video.entity.pads);
 err_pipeline_start:
 	/* TODO: Implement PM QoS */
 	/* The DMA queue must be emptied here, otherwise CCDC interrupts that
@@ -1237,7 +1237,7 @@ isp_video_streamoff(struct file *file, void *fh, enum v4l2_buf_type type)
 	video->error = false;
 
 	/* TODO: Implement PM QoS */
-	media_pipeline_stop(&video->video.entity);
+	media_pipeline_stop(video->video.entity.pads);
 
 	media_entity_enum_cleanup(&pipe->ent_enum);
 
diff --git a/drivers/media/platform/qcom/camss/camss-video.c b/drivers/media/platform/qcom/camss/camss-video.c
index 58aebe7114cd..3ec24286e38d 100644
--- a/drivers/media/platform/qcom/camss/camss-video.c
+++ b/drivers/media/platform/qcom/camss/camss-video.c
@@ -436,7 +436,7 @@ static int video_start_streaming(struct vb2_queue *q, unsigned int count)
 	struct v4l2_subdev *subdev;
 	int ret;
 
-	ret = media_pipeline_start(&vdev->entity, &video->pipe);
+	ret = media_pipeline_start(vdev->entity.pads, &video->pipe);
 	if (ret < 0)
 		return ret;
 
@@ -465,7 +465,7 @@ static int video_start_streaming(struct vb2_queue *q, unsigned int count)
 	return 0;
 
 error:
-	media_pipeline_stop(&vdev->entity);
+	media_pipeline_stop(vdev->entity.pads);
 
 	video->ops->flush_buffers(video, VB2_BUF_STATE_QUEUED);
 
@@ -496,7 +496,7 @@ static void video_stop_streaming(struct vb2_queue *q)
 		v4l2_subdev_call(subdev, video, s_stream, 0);
 	}
 
-	media_pipeline_stop(&vdev->entity);
+	media_pipeline_stop(vdev->entity.pads);
 
 	video->ops->flush_buffers(video, VB2_BUF_STATE_ERROR);
 }
diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index c8437810ebf8..ba4bbe8ed6e3 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -1112,7 +1112,7 @@ static int rvin_set_stream(struct rvin_dev *vin, int on)
 	sd = media_entity_to_v4l2_subdev(pad->entity);
 
 	if (!on) {
-		media_pipeline_stop(&vin->vdev.entity);
+		media_pipeline_stop(vin->vdev.entity.pads);
 		return v4l2_subdev_call(sd, video, s_stream, 0);
 	}
 
@@ -1129,7 +1129,7 @@ static int rvin_set_stream(struct rvin_dev *vin, int on)
 	mdev = vin->vdev.entity.graph_obj.mdev;
 	mutex_lock(&mdev->graph_mutex);
 	pipe = sd->entity.pads->pipe ? sd->entity.pads->pipe : &vin->vdev.pipe;
-	ret = __media_pipeline_start(&vin->vdev.entity, pipe);
+	ret = __media_pipeline_start(vin->vdev.entity.pads, pipe);
 	mutex_unlock(&mdev->graph_mutex);
 	if (ret)
 		return ret;
@@ -1138,7 +1138,7 @@ static int rvin_set_stream(struct rvin_dev *vin, int on)
 	if (ret == -ENOIOCTLCMD)
 		ret = 0;
 	if (ret)
-		media_pipeline_stop(&vin->vdev.entity);
+		media_pipeline_stop(vin->vdev.entity.pads);
 
 	return ret;
 }
diff --git a/drivers/media/platform/s3c-camif/camif-capture.c b/drivers/media/platform/s3c-camif/camif-capture.c
index c3fc94ef251e..f84240fa5c7b 100644
--- a/drivers/media/platform/s3c-camif/camif-capture.c
+++ b/drivers/media/platform/s3c-camif/camif-capture.c
@@ -858,13 +858,13 @@ static int s3c_camif_streamon(struct file *file, void *priv,
 	if (s3c_vp_active(vp))
 		return 0;
 
-	ret = media_pipeline_start(sensor, camif->m_pipeline);
+	ret = media_pipeline_start(sensor->pads, camif->m_pipeline);
 	if (ret < 0)
 		return ret;
 
 	ret = camif_pipeline_validate(camif);
 	if (ret < 0) {
-		media_pipeline_stop(sensor);
+		media_pipeline_stop(sensor->pads);
 		return ret;
 	}
 
@@ -888,7 +888,7 @@ static int s3c_camif_streamoff(struct file *file, void *priv,
 
 	ret = vb2_streamoff(&vp->vb_queue, type);
 	if (ret == 0)
-		media_pipeline_stop(&camif->sensor.sd->entity);
+		media_pipeline_stop(camif->sensor.sd->entity.pads);
 	return ret;
 }
 
diff --git a/drivers/media/platform/vimc/vimc-capture.c b/drivers/media/platform/vimc/vimc-capture.c
index ea869631a3f6..4f88f41f7785 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -248,6 +248,7 @@ static int vimc_cap_start_streaming(struct vb2_queue *vq, unsigned int count)
 
 	/* Start the media pipeline */
 	ret = media_pipeline_start(entity, &vcap->stream.pipe);
+	ret = media_pipeline_start(entity->pads, &vcap->stream.pipe);
 	if (ret) {
 		vimc_cap_return_all_buffers(vcap, VB2_BUF_STATE_QUEUED);
 		return ret;
@@ -255,7 +256,7 @@ static int vimc_cap_start_streaming(struct vb2_queue *vq, unsigned int count)
 
 	ret = vimc_streamer_s_stream(&vcap->stream, &vcap->ved, 1);
 	if (ret) {
-		media_pipeline_stop(entity);
+		media_pipeline_stop(entity->pads);
 		vimc_cap_return_all_buffers(vcap, VB2_BUF_STATE_QUEUED);
 		return ret;
 	}
@@ -274,7 +275,7 @@ static void vimc_cap_stop_streaming(struct vb2_queue *vq)
 	vimc_streamer_s_stream(&vcap->stream, &vcap->ved, 0);
 
 	/* Stop the media pipeline */
-	media_pipeline_stop(&vcap->vdev.entity);
+	media_pipeline_stop(vcap->vdev.entity.pads);
 
 	/* Release all active buffers */
 	vimc_cap_return_all_buffers(vcap, VB2_BUF_STATE_ERROR);
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index bf444bb254d1..a8c1f86e41e1 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -937,7 +937,7 @@ static void vsp1_video_stop_streaming(struct vb2_queue *vq)
 	}
 	mutex_unlock(&pipe->lock);
 
-	media_pipeline_stop(&video->video.entity);
+	media_pipeline_stop(video->video.entity.pads);
 	vsp1_video_release_buffers(video);
 	vsp1_video_pipeline_put(pipe);
 }
@@ -1064,7 +1064,7 @@ vsp1_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 		return PTR_ERR(pipe);
 	}
 
-	ret = __media_pipeline_start(&video->video.entity, &pipe->pipe);
+	ret = __media_pipeline_start(video->video.entity.pads, &pipe->pipe);
 	if (ret < 0) {
 		mutex_unlock(&mdev->graph_mutex);
 		goto err_pipe;
@@ -1088,7 +1088,7 @@ vsp1_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 	return 0;
 
 err_stop:
-	media_pipeline_stop(&video->video.entity);
+	media_pipeline_stop(video->video.entity.pads);
 err_pipe:
 	vsp1_video_pipeline_put(pipe);
 	return ret;
diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index 08ff171d7aac..c8503a3eb703 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -406,7 +406,7 @@ static int xvip_dma_start_streaming(struct vb2_queue *vq, unsigned int count)
 	pipe = dma->video.entity.pads->pipe
 	     ? to_xvip_pipeline(&dma->video.entity) : &dma->pipe;
 
-	ret = media_pipeline_start(&dma->video.entity, &pipe->pipe);
+	ret = media_pipeline_start(dma->video.entity.pads, &pipe->pipe);
 	if (ret < 0)
 		goto error;
 
@@ -432,7 +432,7 @@ static int xvip_dma_start_streaming(struct vb2_queue *vq, unsigned int count)
 	return 0;
 
 error_stop:
-	media_pipeline_stop(&dma->video.entity);
+	media_pipeline_stop(dma->video.entity.pads);
 
 error:
 	/* Give back all queued buffers to videobuf2. */
@@ -460,7 +460,7 @@ static void xvip_dma_stop_streaming(struct vb2_queue *vq)
 
 	/* Cleanup the pipeline and mark it as being stopped. */
 	xvip_pipeline_cleanup(pipe);
-	media_pipeline_stop(&dma->video.entity);
+	media_pipeline_stop(dma->video.entity.pads);
 
 	/* Give back all queued buffers to videobuf2. */
 	spin_lock_irq(&dma->queued_lock);
diff --git a/drivers/media/usb/au0828/au0828-core.c b/drivers/media/usb/au0828/au0828-core.c
index 3f8c92a70116..7fc5e8b6676e 100644
--- a/drivers/media/usb/au0828/au0828-core.c
+++ b/drivers/media/usb/au0828/au0828-core.c
@@ -396,7 +396,7 @@ static int au0828_enable_source(struct media_entity *entity,
 		goto end;
 	}
 
-	ret = __media_pipeline_start(entity, pipe);
+	ret = __media_pipeline_start(entity->pads, pipe);
 	if (ret) {
 		pr_err("Start Pipeline: %s->%s Error %d\n",
 			source->name, entity->name, ret);
@@ -447,7 +447,7 @@ static void au0828_disable_source(struct media_entity *entity)
 		*/
 		if (dev->active_link_owner != entity)
 			return;
-		__media_pipeline_stop(entity);
+		__media_pipeline_stop(entity->pads);
 		ret = __media_entity_setup_link(dev->active_link, 0);
 		if (ret)
 			pr_err("Deactivate link Error %d\n", ret);
diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
index cc10f2d5b51b..5cc5a88db1bf 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -928,16 +928,16 @@ int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
 	mutex_lock(&imxmd->md.graph_mutex);
 
 	if (on) {
-		ret = __media_pipeline_start(entity, &imxmd->pipe);
+		ret = __media_pipeline_start(entity->pads, &imxmd->pipe);
 		if (ret)
 			goto out;
 		ret = v4l2_subdev_call(sd, video, s_stream, 1);
 		if (ret)
-			__media_pipeline_stop(entity);
+			__media_pipeline_stop(entity->pads);
 	} else {
 		v4l2_subdev_call(sd, video, s_stream, 0);
 		if (entity->pads->pipe)
-			__media_pipeline_stop(entity);
+			__media_pipeline_stop(entity->pads);
 	}
 
 out:
diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
index 2a2f7a7983db..0ae8698f8de0 100644
--- a/drivers/staging/media/omap4iss/iss_video.c
+++ b/drivers/staging/media/omap4iss/iss_video.c
@@ -896,7 +896,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 	if (video->iss->pdata->set_constraints)
 		video->iss->pdata->set_constraints(video->iss, true);
 
-	ret = media_pipeline_start(pad->entity, &pipe->pipe);
+	ret = media_pipeline_start(pad, &pipe->pipe);
 	if (ret < 0)
 		goto err_media_pipeline_start;
 
@@ -985,7 +985,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 err_omap4iss_set_stream:
 	vb2_streamoff(&vfh->queue, type);
 err_iss_video_check_format:
-	media_pipeline_stop(&video->video.entity);
+	media_pipeline_stop(video->video.entity.pads);
 err_media_pipeline_start:
 	if (video->iss->pdata->set_constraints)
 		video->iss->pdata->set_constraints(video->iss, false);
@@ -1039,7 +1039,7 @@ iss_video_streamoff(struct file *file, void *fh, enum v4l2_buf_type type)
 
 	if (video->iss->pdata->set_constraints)
 		video->iss->pdata->set_constraints(video->iss, false);
-	media_pipeline_stop(&video->video.entity);
+	media_pipeline_stop(video->video.entity.pads);
 
 done:
 	mutex_unlock(&video->stream_lock);
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index fcde7dd247ed..e806356b1512 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -963,53 +963,53 @@ struct media_pad *media_graph_walk_next(struct media_graph *graph);
 
 /**
  * media_pipeline_start - Mark a pipeline as streaming
- * @entity: Starting entity
- * @pipe: Media pipeline to be assigned to all entities in the pipeline.
+ * @pad: Starting pad
+ * @pipe: Media pipeline to be assigned to all pads in the pipeline.
  *
- * Mark all entities connected to a given entity through enabled links, either
- * directly or indirectly, as streaming. The given pipeline object is assigned
- * to every entity in the pipeline and stored in the media_entity pipe field.
+ * Mark all pads connected to a given pad through enabled routes or links,
+ * either directly or indirectly, as streaming. The given pipeline object is
+ * assigned to every pad in the pipeline and stored in the media_pad pipe
+ * field.
  *
  * Calls to this function can be nested, in which case the same number of
  * media_pipeline_stop() calls will be required to stop streaming. The
  * pipeline pointer must be identical for all nested calls to
  * media_pipeline_start().
  */
-__must_check int media_pipeline_start(struct media_entity *entity,
+__must_check int media_pipeline_start(struct media_pad *pad,
 				      struct media_pipeline *pipe);
 /**
  * __media_pipeline_start - Mark a pipeline as streaming
  *
- * @entity: Starting entity
- * @pipe: Media pipeline to be assigned to all entities in the pipeline.
+ * @pad: Starting pad
+ * @pipe: Media pipeline to be assigned to all pads in the pipeline.
  *
  * ..note:: This is the non-locking version of media_pipeline_start()
  */
-__must_check int __media_pipeline_start(struct media_entity *entity,
+__must_check int __media_pipeline_start(struct media_pad *pad,
 					struct media_pipeline *pipe);
 
 /**
  * media_pipeline_stop - Mark a pipeline as not streaming
- * @entity: Starting entity
+ * @pad: Starting pad
  *
- * Mark all entities connected to a given entity through enabled links, either
- * directly or indirectly, as not streaming. The media_entity pipe field is
- * reset to %NULL.
+ * Mark all pads connected to a given pad through enabled routes or links,
+ * either directly or indirectly, as not streaming.
  *
  * If multiple calls to media_pipeline_start() have been made, the same
  * number of calls to this function are required to mark the pipeline as not
- * streaming.
+ * streaming and reset the media_pad pipe field to %NULL.
  */
-void media_pipeline_stop(struct media_entity *entity);
+void media_pipeline_stop(struct media_pad *pad);
 
 /**
  * __media_pipeline_stop - Mark a pipeline as not streaming
  *
- * @entity: Starting entity
+ * @pad: Starting pad
  *
  * .. note:: This is the non-locking version of media_pipeline_stop()
  */
-void __media_pipeline_stop(struct media_entity *entity);
+void __media_pipeline_stop(struct media_pad *pad);
 
 /**
  * media_devnode_create() - creates and initializes a device node interface
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 08/31] media: entity: Add has_route entity operation
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (6 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 07/31] media: entity: Use pad as the starting point for a pipeline Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 09/31] media: entity: Add media_has_route() function Jacopo Mondi
                   ` (23 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc, Michal Simek

From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

The optional operation can be used by entities to report whether two
pads are internally connected.

While at there, fix a Sphinx compiler warning in a comment block a few
lines above.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 include/media/media-entity.h | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index e806356b1512..675bc27b8b3c 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -195,6 +195,7 @@ enum media_pad_signal_type {
  * @flags:	Pad flags, as defined in
  *		:ref:`include/uapi/linux/media.h <media_header>`
  *		(seek for ``MEDIA_PAD_FL_*``)
+ *
  * .. note::
  *
  *    @stream_count reference count must never be negative, but is a signed
@@ -222,6 +223,10 @@ struct media_pad {
  * @link_validate:	Return whether a link is valid from the entity point of
  *			view. The media_pipeline_start() function
  *			validates all links by calling this operation. Optional.
+ * @has_route:		Return whether a route exists inside the entity between
+ *			two given pads. Pads are passed to the operation ordered
+ *			by index. Optional: If the operation isn't implemented
+ *			all pads will be considered as connected.
  *
  * .. note::
  *
@@ -234,6 +239,8 @@ struct media_entity_operations {
 			  const struct media_pad *local,
 			  const struct media_pad *remote, u32 flags);
 	int (*link_validate)(struct media_link *link);
+	bool (*has_route)(struct media_entity *entity, unsigned int pad0,
+			  unsigned int pad1);
 };
 
 /**
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 09/31] media: entity: Add media_has_route() function
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (7 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 08/31] media: entity: Add has_route entity operation Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-14 14:45   ` Luca Ceresoli
  2019-03-05 18:51 ` [PATCH v3 10/31] media: entity: Use routing information during graph traversal Jacopo Mondi
                   ` (22 subsequent siblings)
  31 siblings, 1 reply; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc, Michal Simek

From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

This is a wrapper around the media entity has_route operation.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/media-entity.c | 19 +++++++++++++++++++
 include/media/media-entity.h | 17 +++++++++++++++++
 2 files changed, 36 insertions(+)

diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
index 6f5196d05894..8e0ca8b1cfa2 100644
--- a/drivers/media/media-entity.c
+++ b/drivers/media/media-entity.c
@@ -238,6 +238,25 @@ EXPORT_SYMBOL_GPL(media_entity_pads_init);
  * Graph traversal
  */
 
+bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
+			    unsigned int pad1)
+{
+	if (pad0 >= entity->num_pads || pad1 >= entity->num_pads)
+		return false;
+
+	if (pad0 == pad1)
+		return true;
+
+	if (!entity->ops || !entity->ops->has_route)
+		return true;
+
+	if (entity->pads[pad1].index < entity->pads[pad0].index)
+		swap(pad0, pad1);
+
+	return entity->ops->has_route(entity, pad0, pad1);
+}
+EXPORT_SYMBOL_GPL(media_entity_has_route);
+
 static struct media_pad *
 media_pad_other(struct media_pad *pad, struct media_link *link)
 {
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 675bc27b8b3c..205561545d7e 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -919,6 +919,23 @@ int media_entity_get_fwnode_pad(struct media_entity *entity,
 __must_check int media_graph_walk_init(
 	struct media_graph *graph, struct media_device *mdev);
 
+/**
+ * media_entity_has_route - Check if two entity pads are connected internally
+ *
+ * @entity: The entity
+ * @pad0: The first pad index
+ * @pad1: The second pad index
+ *
+ * This function can be used to check whether two pads of an entity are
+ * connected internally in the entity.
+ *
+ * The caller must hold entity->graph_obj.mdev->mutex.
+ *
+ * Return: true if the pads are connected internally and false otherwise.
+ */
+bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
+			    unsigned int pad1);
+
 /**
  * media_graph_walk_cleanup - Release resources used by graph walk.
  *
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 10/31] media: entity: Use routing information during graph traversal
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (8 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 09/31] media: entity: Add media_has_route() function Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 11/31] media: entity: Skip link validation for pads to which there is no route to Jacopo Mondi
                   ` (21 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc, Michal Simek

From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Take internal routing information as reported by the entity has_route
operation into account during graph traversal to avoid following
unrelated links.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/media-entity.c | 44 ++++++++++++++++++++++--------------
 1 file changed, 27 insertions(+), 17 deletions(-)

diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
index 8e0ca8b1cfa2..3304e76bbafa 100644
--- a/drivers/media/media-entity.c
+++ b/drivers/media/media-entity.c
@@ -257,15 +257,6 @@ bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
 }
 EXPORT_SYMBOL_GPL(media_entity_has_route);
 
-static struct media_pad *
-media_pad_other(struct media_pad *pad, struct media_link *link)
-{
-	if (link->source == pad)
-		return link->sink;
-	else
-		return link->source;
-}
-
 /* push an entity to traversal stack */
 static void stack_push(struct media_graph *graph, struct media_pad *pad)
 {
@@ -336,7 +327,8 @@ static void media_graph_walk_iter(struct media_graph *graph)
 {
 	struct media_pad *pad = stack_top(graph);
 	struct media_link *link;
-	struct media_pad *next;
+	struct media_pad *remote;
+	struct media_pad *local;
 
 	link = list_entry(link_top(graph), typeof(*link), list);
 
@@ -350,23 +342,41 @@ static void media_graph_walk_iter(struct media_graph *graph)
 		return;
 	}
 
-	/* Get the entity in the other end of the link . */
-	next = media_pad_other(pad, link);
+	/*
+	 * Get the local pad, the remote pad and the entity at the other
+	 * end of the link.
+	 */
+	if (link->source->entity == pad->entity) {
+		remote = link->sink;
+		local = link->source;
+	} else {
+		remote = link->source;
+		local = link->sink;
+	}
+
+	/*
+	 * Are the local pad and the pad we came from connected
+	 * internally in the entity ?
+	 */
+	if (!media_entity_has_route(pad->entity, pad->index, local->index)) {
+		link_top(graph) = link_top(graph)->next;
+		return;
+	}
 
 	/* Has the entity already been visited? */
-	if (media_entity_enum_test_and_set(&graph->ent_enum, next->entity)) {
+	if (media_entity_enum_test_and_set(&graph->ent_enum, remote->entity)) {
 		link_top(graph) = link_top(graph)->next;
 		dev_dbg(pad->graph_obj.mdev->dev,
 			"walk: skipping entity '%s' (already seen)\n",
-			next->entity->name);
+			remote->entity->name);
 		return;
 	}
 
 	/* Push the new entity to stack and start over. */
 	link_top(graph) = link_top(graph)->next;
-	stack_push(graph, next);
-	dev_dbg(next->graph_obj.mdev->dev, "walk: pushing '%s':%u on stack\n",
-		next->entity->name, next->index);
+	stack_push(graph, remote);
+	dev_dbg(remote->graph_obj.mdev->dev, "walk: pushing '%s':%u on stack\n",
+		remote->entity->name, remote->index);
 }
 
 struct media_pad *media_graph_walk_next(struct media_graph *graph)
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 11/31] media: entity: Skip link validation for pads to which there is no route to
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (9 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 10/31] media: entity: Use routing information during graph traversal Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 12/31] media: entity: Add an iterator helper for connected pads Jacopo Mondi
                   ` (20 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Sakari Ailus <sakari.ailus@linux.intel.com>

Links are validated along the pipeline which is about to start streaming.
Not all the pads in entities that are traversed along that pipeline are
part of the pipeline, however. Skip the link validation for such pads,
and while at there rename "other_pad" to "local_pad" to convey the fact
the route to be checked is internal to the entity.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/media-entity.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
index 3304e76bbafa..5d21ecaaaf4b 100644
--- a/drivers/media/media-entity.c
+++ b/drivers/media/media-entity.c
@@ -488,11 +488,16 @@ __must_check int __media_pipeline_start(struct media_pad *pad,
 		bitmap_fill(has_no_links, entity->num_pads);
 
 		list_for_each_entry(link, &entity->links, list) {
-			struct media_pad *other_pad = link->sink->entity == entity
+			struct media_pad *local_pad = link->sink->entity == entity
 				? link->sink : link->source;
 
+			/* Ignore pads to which there is no route. */
+			if (!media_entity_has_route(entity, pad->index,
+						    local_pad->index))
+				continue;
+
 			/* Mark that a pad is connected by a link. */
-			bitmap_clear(has_no_links, other_pad->index, 1);
+			bitmap_clear(has_no_links, local_pad->index, 1);
 
 			/*
 			 * Pads that either do not need to connect or
@@ -501,13 +506,13 @@ __must_check int __media_pipeline_start(struct media_pad *pad,
 			 */
 			if (!(pad->flags & MEDIA_PAD_FL_MUST_CONNECT) ||
 			    link->flags & MEDIA_LNK_FL_ENABLED)
-				bitmap_set(active, other_pad->index, 1);
+				bitmap_set(active, local_pad->index, 1);
 
 			/*
 			 * Link validation will only take place for
 			 * sink ends of the link that are enabled.
 			 */
-			if (link->sink != other_pad ||
+			if (link->sink != local_pad ||
 			    !(link->flags & MEDIA_LNK_FL_ENABLED))
 				continue;
 
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 12/31] media: entity: Add an iterator helper for connected pads
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (10 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 11/31] media: entity: Skip link validation for pads to which there is no route to Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-07 10:09   ` Sakari Ailus
  2019-03-05 18:51 ` [PATCH v3 13/31] media: entity: Add only connected pads to the pipeline Jacopo Mondi
                   ` (19 subsequent siblings)
  31 siblings, 1 reply; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Sakari Ailus <sakari.ailus@linux.intel.com>

Add a helper macro for iterating over pads that are connected through
enabled routes. This can be used to find all the connected pads within an
entity, for instance starting from the pad which has been obtained during
the graph walk.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

- Make __media_entity_next_routed_pad() return NULL and adjust the
  iterator to handle that
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 include/media/media-entity.h | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 205561545d7e..82f0bdf2a6d1 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -936,6 +936,33 @@ __must_check int media_graph_walk_init(
 bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
 			    unsigned int pad1);
 
+static inline struct media_pad *__media_entity_next_routed_pad(
+	struct media_pad *start, struct media_pad *iter)
+{
+	struct media_entity *entity = start->entity;
+
+	while (iter < &entity->pads[entity->num_pads] &&
+	       !media_entity_has_route(entity, start->index, iter->index))
+		iter++;
+
+	return iter == &entity->pads[entity->num_pads] ? NULL : iter;
+}
+
+/**
+ * media_entity_for_each_routed_pad - Iterate over entity pads connected by routes
+ *
+ * @start: The stating pad
+ * @iter: The iterator pad
+ *
+ * Iterate over all pads connected through routes from a given pad
+ * within an entity. The iteration will include the starting pad itself.
+ */
+#define media_entity_for_each_routed_pad(start, iter)			\
+	for (iter = __media_entity_next_routed_pad(			\
+		     start, (start)->entity->pads);			\
+	     iter != NULL;						\
+	     iter = __media_entity_next_routed_pad(start, iter + 1))
+
 /**
  * media_graph_walk_cleanup - Release resources used by graph walk.
  *
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 13/31] media: entity: Add only connected pads to the pipeline
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (11 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 12/31] media: entity: Add an iterator helper for connected pads Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 14/31] media: entity: Add debug information in graph walk route check Jacopo Mondi
                   ` (18 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Sakari Ailus <sakari.ailus@linux.intel.com>

A single entity may contain multiple pipelines. Only add pads that were
connected to the pad through which the entity was reached to the pipeline.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/media-entity.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
index 5d21ecaaaf4b..b48dd1e7c805 100644
--- a/drivers/media/media-entity.c
+++ b/drivers/media/media-entity.c
@@ -466,7 +466,7 @@ __must_check int __media_pipeline_start(struct media_pad *pad,
 		DECLARE_BITMAP(active, MEDIA_ENTITY_MAX_PADS);
 		DECLARE_BITMAP(has_no_links, MEDIA_ENTITY_MAX_PADS);
 
-		media_entity_for_each_pad(entity, iter) {
+		media_entity_for_each_routed_pad(pad, iter) {
 			if (iter->pipe && WARN_ON(iter->pipe != pipe))
 				ret = -EBUSY;
 			else
@@ -551,10 +551,9 @@ __must_check int __media_pipeline_start(struct media_pad *pad,
 	media_graph_walk_start(graph, pad_err);
 
 	while ((pad_err = media_graph_walk_next(graph))) {
-		struct media_entity *entity = pad->entity;
 		struct media_pad *iter;
 
-		media_entity_for_each_pad(entity, iter) {
+		media_entity_for_each_routed_pad(pad, iter) {
 			/* Sanity check for negative stream_count */
 			if (!WARN_ON_ONCE(iter->stream_count <= 0)) {
 				--iter->stream_count;
@@ -607,10 +606,9 @@ void __media_pipeline_stop(struct media_pad *pad)
 	media_graph_walk_start(graph, pad);
 
 	while ((pad = media_graph_walk_next(graph))) {
-		struct media_entity *entity = pad->entity;
 		struct media_pad *iter;
 
-		media_entity_for_each_pad(entity, iter) {
+		media_entity_for_each_routed_pad(pad, iter) {
 			/* Sanity check for negative stream_count */
 			if (!WARN_ON_ONCE(iter->stream_count <= 0)) {
 				iter->stream_count--;
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 14/31] media: entity: Add debug information in graph walk route check
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (12 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 13/31] media: entity: Add only connected pads to the pipeline Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 15/31] v4l: Add bus type to frame descriptors Jacopo Mondi
                   ` (17 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Sakari Ailus <sakari.ailus@linux.intel.com>

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/media-entity.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
index b48dd1e7c805..3556daae48d5 100644
--- a/drivers/media/media-entity.c
+++ b/drivers/media/media-entity.c
@@ -360,6 +360,9 @@ static void media_graph_walk_iter(struct media_graph *graph)
 	 */
 	if (!media_entity_has_route(pad->entity, pad->index, local->index)) {
 		link_top(graph) = link_top(graph)->next;
+		dev_dbg(pad->graph_obj.mdev->dev,
+			"walk: skipping \"%s\":%u -> %u (no route)\n",
+			pad->entity->name, pad->index, local->index);
 		return;
 	}
 
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 15/31] v4l: Add bus type to frame descriptors
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (13 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 14/31] media: entity: Add debug information in graph walk route check Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 16/31] v4l: Add CSI-2 bus configuration " Jacopo Mondi
                   ` (16 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Sakari Ailus <sakari.ailus@linux.intel.com>

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

- Make the bus type a named enum
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 include/media/v4l2-subdev.h | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 349e1c18cf48..7a84df83bd21 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -349,12 +349,21 @@ struct v4l2_mbus_frame_desc_entry {
 
 #define V4L2_FRAME_DESC_ENTRY_MAX	4
 
+enum v4l2_mbus_frame_desc_type {
+	V4L2_MBUS_FRAME_DESC_TYPE_PLATFORM,
+	V4L2_MBUS_FRAME_DESC_TYPE_PARALLEL,
+	V4L2_MBUS_FRAME_DESC_TYPE_CCP2,
+	V4L2_MBUS_FRAME_DESC_TYPE_CSI2,
+};
+
 /**
  * struct v4l2_mbus_frame_desc - media bus data frame description
+ * @type: type of the bus (enum v4l2_mbus_frame_desc_type)
  * @entry: frame descriptors array
  * @num_entries: number of entries in @entry array
  */
 struct v4l2_mbus_frame_desc {
+	enum v4l2_mbus_frame_desc_type type;
 	struct v4l2_mbus_frame_desc_entry entry[V4L2_FRAME_DESC_ENTRY_MAX];
 	unsigned short num_entries;
 };
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 16/31] v4l: Add CSI-2 bus configuration to frame descriptors
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (14 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 15/31] v4l: Add bus type to frame descriptors Jacopo Mondi
@ 2019-03-05 18:51 ` " Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 17/31] v4l: Add stream to frame descriptor Jacopo Mondi
                   ` (15 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Sakari Ailus <sakari.ailus@linux.intel.com>

Add CSI-2 bus specific configuration to the frame descriptors. This allows
obtaining the virtual channel and data type information for each stream
the transmitter is sending.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 include/media/v4l2-subdev.h | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 7a84df83bd21..0d2d7836b5a2 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -317,6 +317,17 @@ struct v4l2_subdev_audio_ops {
 	int (*s_stream)(struct v4l2_subdev *sd, int enable);
 };
 
+/**
+ * struct v4l2_mbus_frame_desc_entry_csi2
+ *
+ * @channel: CSI-2 virtual channel
+ * @data_type: CSI-2 data type ID
+ */
+struct v4l2_mbus_frame_desc_entry_csi2 {
+	u8 channel;
+	u8 data_type;
+};
+
 /**
  * enum v4l2_mbus_frame_desc_entry - media bus frame description flags
  *
@@ -340,11 +351,16 @@ enum v4l2_mbus_frame_desc_flags {
  *		%FRAME_DESC_FL_BLOB is not set.
  * @length:	number of octets per frame, valid if @flags
  *		%V4L2_MBUS_FRAME_DESC_FL_LEN_MAX is set.
+ * @bus:	Bus specific frame descriptor parameters
+ * @bus.csi2:	CSI-2 specific bus configuration
  */
 struct v4l2_mbus_frame_desc_entry {
 	enum v4l2_mbus_frame_desc_flags flags;
 	u32 pixelcode;
 	u32 length;
+	union {
+		struct v4l2_mbus_frame_desc_entry_csi2 csi2;
+	} bus;
 };
 
 #define V4L2_FRAME_DESC_ENTRY_MAX	4
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 17/31] v4l: Add stream to frame descriptor
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (15 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 16/31] v4l: Add CSI-2 bus configuration " Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 18/31] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations Jacopo Mondi
                   ` (14 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Sakari Ailus <sakari.ailus@linux.intel.com>

The stream field identifies the stream this frame descriptor applies to in
routing configuration across a multiplexed link.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 include/media/v4l2-subdev.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 0d2d7836b5a2..03707d16d668 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -347,6 +347,7 @@ enum v4l2_mbus_frame_desc_flags {
  * struct v4l2_mbus_frame_desc_entry - media bus frame description structure
  *
  * @flags:	bitmask flags, as defined by &enum v4l2_mbus_frame_desc_flags.
+ * @stream:	stream in routing configuration
  * @pixelcode:	media bus pixel code, valid if @flags
  *		%FRAME_DESC_FL_BLOB is not set.
  * @length:	number of octets per frame, valid if @flags
@@ -356,6 +357,7 @@ enum v4l2_mbus_frame_desc_flags {
  */
 struct v4l2_mbus_frame_desc_entry {
 	enum v4l2_mbus_frame_desc_flags flags;
+	u32 stream;
 	u32 pixelcode;
 	u32 length;
 	union {
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 18/31] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (16 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 17/31] v4l: Add stream to frame descriptor Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 19/31] media: Documentation: Add GS_ROUTING documentation Jacopo Mondi
                   ` (13 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc, Michal Simek

From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>

- Add sink and source streams for multiplexed links
- Copy the argument back in case of an error. This is needed to let the
  caller know the number of routes.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

- Expand and refine documentation.
- Make the 'routes' pointer a __u64 __user pointer so that a compat32
  version of the ioctl is not required.
- Add struct v4l2_subdev_krouting to be used for subdevice operations.

Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/v4l2-core/v4l2-ioctl.c  | 25 ++++++++++++++++-
 drivers/media/v4l2-core/v4l2-subdev.c | 36 ++++++++++++++++++++++++
 include/media/v4l2-subdev.h           | 22 +++++++++++++++
 include/uapi/linux/v4l2-subdev.h      | 40 +++++++++++++++++++++++++++
 4 files changed, 122 insertions(+), 1 deletion(-)

diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index f6d663934648..255f8aff5db8 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -19,6 +19,7 @@
 #include <linux/kernel.h>
 #include <linux/version.h>
 
+#include <linux/v4l2-subdev.h>
 #include <linux/videodev2.h>
 
 #include <media/v4l2-common.h>
@@ -2967,6 +2968,21 @@ static int check_array_args(unsigned int cmd, void *parg, size_t *array_size,
 		}
 		break;
 	}
+
+	case VIDIOC_SUBDEV_G_ROUTING:
+	case VIDIOC_SUBDEV_S_ROUTING: {
+		struct v4l2_subdev_routing *route = parg;
+
+		if (route->num_routes > 256)
+			return -EINVAL;
+
+		*user_ptr = (void __user *)route->routes;
+		*kernel_ptr = (void *)&route->routes;
+		*array_size = sizeof(struct v4l2_subdev_route)
+			    * route->num_routes;
+		ret = 1;
+		break;
+	}
 	}
 
 	return ret;
@@ -3075,8 +3091,15 @@ video_usercopy(struct file *file, unsigned int cmd, unsigned long arg,
 	/*
 	 * Some ioctls can return an error, but still have valid
 	 * results that must be returned.
+	 *
+	 * FIXME: subdev IOCTLS are partially handled here and partially in
+	 * v4l2-subdev.c and the 'always_copy' flag can only be set for IOCTLS
+	 * defined here as part of the 'v4l2_ioctls' array. As
+	 * VIDIOC_SUBDEV_G_ROUTING needs to return results to applications even
+	 * in case of failure, but it is not defined here as part of the
+	 * 'v4l2_ioctls' array, insert an ad-hoc check to address that.
 	 */
-	if (err < 0 && !always_copy)
+	if (err < 0 && !always_copy && cmd != VIDIOC_SUBDEV_G_ROUTING)
 		goto out;
 
 out_array_args:
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index f5f0d71ec745..f845b0658913 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -519,6 +519,42 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg)
 
 	case VIDIOC_SUBDEV_QUERYSTD:
 		return v4l2_subdev_call(sd, video, querystd, arg);
+
+	case VIDIOC_SUBDEV_G_ROUTING: {
+		struct v4l2_subdev_routing *routing = arg;
+		struct v4l2_subdev_krouting krouting = {
+			.num_routes = routing->num_routes,
+			.routes = (struct v4l2_subdev_route *)routing->routes,
+		};
+		int ret;
+
+		ret = v4l2_subdev_call(sd, pad, get_routing, &krouting);
+		if (ret)
+			return ret;
+
+		routing->num_routes = krouting.num_routes;
+
+		return 0;
+	}
+
+	case VIDIOC_SUBDEV_S_ROUTING: {
+		struct v4l2_subdev_routing *routing = arg;
+		struct v4l2_subdev_route *route = (struct v4l2_subdev_route *)
+						  routing->routes;
+		struct v4l2_subdev_krouting krouting = {};
+		unsigned int i;
+
+		for (i = 0; i < routing->num_routes; ++i) {
+			if (route[i].sink_pad >= sd->entity.num_pads ||
+			    route[i].source_pad >= sd->entity.num_pads)
+				return -EINVAL;
+		}
+
+		krouting.num_routes = routing->num_routes;
+		krouting.routes = route;
+
+		return v4l2_subdev_call(sd, pad, set_routing, &krouting);
+	}
 #endif
 	default:
 		return v4l2_subdev_call(sd, core, ioctl, cmd, arg);
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 03707d16d668..6311f670de3c 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -669,6 +669,20 @@ struct v4l2_subdev_pad_config {
 	struct v4l2_rect try_compose;
 };
 
+/**
+ * struct v4l2_subdev_krouting - subdev routing table
+ *
+ * @routes: &struct v4l2_subdev_route
+ * @num_routes: number of routes
+ *
+ * This structure is used to translate argument received from
+ * VIDIOC_SUBDEV_G/S_ROUTING() ioctl to sudev device drivers operations.
+ */
+struct v4l2_subdev_krouting {
+	struct v4l2_subdev_route *routes;
+	unsigned int num_routes;
+};
+
 /**
  * struct v4l2_subdev_pad_ops - v4l2-subdev pad level operations
  *
@@ -706,6 +720,10 @@ struct v4l2_subdev_pad_config {
  *
  * @set_frame_desc: set the low level media bus frame parameters, @fd array
  *                  may be adjusted by the subdev driver to device capabilities.
+ *
+ * @get_routing: get the subdevice routing table.
+ * @set_routing: enable or disable data connection routes described in the
+ *		 subdevice routing table.
  */
 struct v4l2_subdev_pad_ops {
 	int (*init_cfg)(struct v4l2_subdev *sd,
@@ -746,6 +764,10 @@ struct v4l2_subdev_pad_ops {
 			      struct v4l2_mbus_frame_desc *fd);
 	int (*set_frame_desc)(struct v4l2_subdev *sd, unsigned int pad,
 			      struct v4l2_mbus_frame_desc *fd);
+	int (*get_routing)(struct v4l2_subdev *sd,
+			   struct v4l2_subdev_krouting *route);
+	int (*set_routing)(struct v4l2_subdev *sd,
+			   struct v4l2_subdev_krouting *route);
 };
 
 /**
diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h
index 03970ce30741..b8924a6a029c 100644
--- a/include/uapi/linux/v4l2-subdev.h
+++ b/include/uapi/linux/v4l2-subdev.h
@@ -155,6 +155,44 @@ struct v4l2_subdev_selection {
 	__u32 reserved[8];
 };
 
+#define V4L2_SUBDEV_ROUTE_FL_ACTIVE	(1 << 0)
+#define V4L2_SUBDEV_ROUTE_FL_IMMUTABLE	(1 << 1)
+
+/**
+ * struct v4l2_subdev_route - A route inside a subdev
+ * @sink_pad: the sink pad index
+ * @sink_stream: the sink stream identifier
+ * @source_pad: the source pad index
+ * @source_stream: the source stream identifier
+ * @flags: route flags:
+ *
+ *	V4L2_SUBDEV_ROUTE_FL_ACTIVE: Is the route active? An active
+ *	route will start when streaming is enabled on a video node.
+ *	Set by the user.
+ *
+ *	V4L2_SUBDEV_ROUTE_FL_IMMUTABLE: Is the route immutable, i.e.
+ *	can it be activated and inactivated? Set by the driver.
+ */
+struct v4l2_subdev_route {
+	__u32 sink_pad;
+	__u32 sink_stream;
+	__u32 source_pad;
+	__u32 source_stream;
+	__u32 flags;
+	__u32 reserved[5];
+};
+
+/**
+ * struct v4l2_subdev_routing - Subdev routing information
+ * @routes: pointer to the routes array
+ * @num_routes: the total number of routes in the routes array
+ */
+struct v4l2_subdev_routing {
+	__u64 routes;
+	__u32 num_routes;
+	__u32 reserved[5];
+};
+
 /* Backwards compatibility define --- to be removed */
 #define v4l2_subdev_edid v4l2_edid
 
@@ -181,5 +219,7 @@ struct v4l2_subdev_selection {
 #define VIDIOC_SUBDEV_ENUM_DV_TIMINGS		_IOWR('V', 98, struct v4l2_enum_dv_timings)
 #define VIDIOC_SUBDEV_QUERY_DV_TIMINGS		_IOR('V', 99, struct v4l2_dv_timings)
 #define VIDIOC_SUBDEV_DV_TIMINGS_CAP		_IOWR('V', 100, struct v4l2_dv_timings_cap)
+#define VIDIOC_SUBDEV_G_ROUTING			_IOWR('V', 38, struct v4l2_subdev_routing)
+#define VIDIOC_SUBDEV_S_ROUTING			_IOWR('V', 39, struct v4l2_subdev_routing)
 
 #endif
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 19/31] media: Documentation: Add GS_ROUTING documentation
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (17 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 18/31] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-07 15:19   ` Sakari Ailus
                     ` (2 more replies)
  2019-03-05 18:51 ` [PATCH v3 20/31] v4l: subdev: Take routing information into account in link validation Jacopo Mondi
                   ` (12 subsequent siblings)
  31 siblings, 3 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

Add documentation for VIDIOC_SUBDEV_G/S_ROUTING ioctl and add
description of multiplexed media pads and internal routing to the
V4L2-subdev documentation section.

Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 Documentation/media/uapi/v4l/dev-subdev.rst   |  90 +++++++++++
 Documentation/media/uapi/v4l/user-func.rst    |   1 +
 .../uapi/v4l/vidioc-subdev-g-routing.rst      | 142 ++++++++++++++++++
 3 files changed, 233 insertions(+)
 create mode 100644 Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst

diff --git a/Documentation/media/uapi/v4l/dev-subdev.rst b/Documentation/media/uapi/v4l/dev-subdev.rst
index 2c2768c7343b..b9fbb5d2caec 100644
--- a/Documentation/media/uapi/v4l/dev-subdev.rst
+++ b/Documentation/media/uapi/v4l/dev-subdev.rst
@@ -36,6 +36,8 @@ will feature a character device node on which ioctls can be called to
 
 -  negotiate image formats on individual pads
 
+-  inspect and modify internal data routing between pads of the same entity
+
 Sub-device character device nodes, conventionally named
 ``/dev/v4l-subdev*``, use major number 81.
 
@@ -461,3 +463,91 @@ source pads.
     :maxdepth: 1
 
     subdev-formats
+
+
+Multiplexed media pads and internal routing
+-------------------------------------------
+
+Subdevice drivers might expose the internal connections between media pads of an
+entity by providing a routing table that applications can inspect and manipulate
+to change the internal routing between sink and source pads' data connection
+endpoints. A routing table is described by a struct
+:c:type:`v4l2_subdev_routing`, which contains ``num_routes`` route entries, each
+one represented by a struct :c:type:`v4l2_subdev_route`.
+
+Data routes do not just connect one pad to another in an entity, but they refer
+instead to the ``streams`` a media pad provides. Streams are data connection
+endpoints in a media pad. Multiplexed media pads expose multiple ``streams``
+which represent, when the underlying hardware technology allows that, logical
+data streams transported over a single physical media bus.
+
+One of the most notable examples of logical stream multiplexing techniques is
+represented by the data interleaving mechanism implemented by mean of Virtual
+Channels identifiers as defined by the MIPI CSI-2 media bus specifications. A
+subdevice that implements support for Virtual Channel data interleaving might
+expose up to 4 data ``streams``, one for each available Virtual Channel, on the
+source media pad that represents a CSI-2 connection endpoint.
+
+Routes are defined as potential data connections between a ``(sink_pad,
+sink_stream)`` pair and a ``(source_pad, source_stream)`` one, where
+``sink_pad`` and ``source_pad`` are the indexes of two media pads part of the
+same media entity, and ``sink_stream`` and ``source_stream`` are the identifiers
+of the data streams to be connected in the media pads. Media pads that do not
+support data multiplexing expose a single stream, usually with identifier 0.
+
+Routes are reported to applications in a routing table which can be
+inspected and manipulated using the :ref:`routing <VIDIOC_SUBDEV_G_ROUTING>`
+ioctls.
+
+Routes can be activated and deactivated by setting or clearing the
+V4L2_SUBDEV_ROUTE_FL_ACTIVE flag in the ``flags`` field of struct
+:c:type:`v4l2_subdev_route`.
+
+Subdev driver might create routes that cannot be modified by applications. Such
+routes are identified by the presence of the V4L2_SUBDEV_ROUTE_FL_IMMUTABLE
+flag in the ``flags`` field of struct :c:type:`v4l2_subdev_route`.
+
+As an example, the routing table of a source pad which supports two logical
+video streams and can be connected to two sink pads is here below described.
+
+.. flat-table::
+    :widths:       1 2 1
+
+    * - Pad Index
+      - Function
+      - Number of streams
+    * - 0
+      - SINK
+      - 1
+    * - 1
+      - SINK
+      - 1
+    * - 2
+      - SOURCE
+      - 2
+
+In such an example, the source media pad will report a routing table with 4
+entries, one entry for each possible ``(sink_pad, sink_stream) - (source_pad,
+source_stream)`` combination.
+
+.. flat-table:: routing table
+    :widths:       2 1 2
+
+    * - Sink Pad/Sink Stream
+      - ->
+      - Source Pad/Source Stream
+    * - 0/0
+      - ->
+      - 2/0
+    * - 0/0
+      - ->
+      - 2/1
+    * - 1/0
+      - ->
+      - 2/0
+    * - 1/0
+      - ->
+      - 2/1
+
+Subdev drivers are free to decide how many routes an application can enable on
+a media pad at the same time, and refuse to enable or disable specific routes.
diff --git a/Documentation/media/uapi/v4l/user-func.rst b/Documentation/media/uapi/v4l/user-func.rst
index ca0ef21d77fe..0166446f4ab4 100644
--- a/Documentation/media/uapi/v4l/user-func.rst
+++ b/Documentation/media/uapi/v4l/user-func.rst
@@ -77,6 +77,7 @@ Function Reference
     vidioc-subdev-g-crop
     vidioc-subdev-g-fmt
     vidioc-subdev-g-frame-interval
+    vidioc-subdev-g-routing
     vidioc-subdev-g-selection
     vidioc-subscribe-event
     func-mmap
diff --git a/Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst b/Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst
new file mode 100644
index 000000000000..8b592722c477
--- /dev/null
+++ b/Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst
@@ -0,0 +1,142 @@
+.. Permission is granted to copy, distribute and/or modify this
+.. document under the terms of the GNU Free Documentation License,
+.. Version 1.1 or any later version published by the Free Software
+.. Foundation, with no Invariant Sections, no Front-Cover Texts
+.. and no Back-Cover Texts. A copy of the license is included at
+.. Documentation/media/uapi/fdl-appendix.rst.
+..
+.. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections
+
+.. _VIDIOC_SUBDEV_G_ROUTING:
+
+******************************************************
+ioctl VIDIOC_SUBDEV_G_ROUTING, VIDIOC_SUBDEV_S_ROUTING
+******************************************************
+
+Name
+====
+
+VIDIOC_SUBDEV_G_ROUTING - VIDIOC_SUBDEV_S_ROUTING - Get or set routing between streams of media pads in a media entity.
+
+
+Synopsis
+========
+
+.. c:function:: int ioctl( int fd, VIDIOC_SUBDEV_G_ROUTING, struct v4l2_subdev_routing *argp )
+    :name: VIDIOC_SUBDEV_G_ROUTING
+
+.. c:function:: int ioctl( int fd, VIDIOC_SUBDEV_S_ROUTING, struct v4l2_subdev_routing *argp )
+    :name: VIDIOC_SUBDEV_S_ROUTING
+
+
+Arguments
+=========
+
+``fd``
+    File descriptor returned by :ref:`open() <func-open>`.
+
+``argp``
+    Pointer to struct :c:type:`v4l2_subdev_routing`.
+
+
+Description
+===========
+
+These ioctls are used to get and set the routing informations associated to
+media pads in a media entity. Routing informations are used to enable or disable
+data connections between stream endpoints of multiplexed media pads.
+
+Drivers report their routing tables using VIDIOC_SUBDEV_G_ROUTING and
+application use the information there reported to enable or disable data
+connections between streams in a pad, by setting or clearing the
+V4L2_SUBDEV_ROUTE_FL_ACTIVE flag of ``flags`` field of a struct
+:c:type:`v4l2_subdev_route`.
+
+When inspecting routes through VIDIOC_SUBDEV_G_ROUTING and the application
+provided ``num_routes`` is not big enough to contain all the available routes
+the subdevice exposes, drivers return the ENOSPC error code and adjust the
+``num_routes`` value. Application should then reserve enough memory for all the
+route entries and call VIDIOC_SUBDEV_G_ROUTING again.
+
+.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}|
+
+.. c:type:: v4l2_subdev_routing
+
+.. flat-table:: struct v4l2_subdev_routing
+    :header-rows:  0
+    :stub-columns: 0
+    :widths:       1 1 2
+
+    * - struct :c:type:`v4l2_subdev_route`
+      - ``routes[]``
+      - Array of struct :c:type:`v4l2_subdev_route` entries
+    * - __u32
+      - ``num_routes``
+      - Number of entries of the routes array
+    * - __u32
+      - ``reserved``\ [5]
+      - Reserved for future extensions. Applications and drivers must set
+	the array to zero.
+
+.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}|
+
+.. c:type:: v4l2_subdev_route
+
+.. flat-table:: struct v4l2_subdev_route
+    :header-rows:  0
+    :stub-columns: 0
+    :widths:       1 1 2
+
+    * - __u32
+      - ``sink_pad``
+      - Sink pad number
+    * - __u32
+      - ``sink_stream``
+      - Sink pad stream number
+    * - __u32
+      - ``source_stream``
+      - Source pad stream number
+    * - __u32
+      - ``sink_stream``
+      - Sink pad stream number
+    * - __u32
+      - ``flags``
+      - Route enable/disable flags
+	:ref:`v4l2_subdev_routing_flags <v4l2-subdev-routing-flags>`.
+    * - __u32
+      - ``reserved``\ [5]
+      - Reserved for future extensions. Applications and drivers must set
+	the array to zero.
+
+.. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
+
+.. _v4l2-subdev-routing-flags:
+
+.. flat-table:: enum v4l2_subdev_routing_flags
+    :header-rows:  0
+    :stub-columns: 0
+    :widths:       3 1 4
+
+    * - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+      - 0
+      - The route is enabled. Set by applications.
+    * - V4L2_SUBDEV_ROUTE_FL_IMMUTABLE
+      - 1
+      - The route is immutable. Set by the driver.
+
+Return Value
+============
+
+On success 0 is returned, on error -1 and the ``errno`` variable is set
+appropriately. The generic error codes are described at the
+:ref:`Generic Error Codes <gen-errors>` chapter.
+
+ENOSPC
+   The number of provided route entries is less than the available ones.
+
+EINVAL
+   The sink or source pad identifiers reference a non-existing pad, or refernce
+   pads of different types (ie. the sink_pad identifiers refers to a source pad)
+   The sink or source stream identifiers reference a non-existing stream
+   in the sink or source pad.
+
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 20/31] v4l: subdev: Take routing information into account in link validation
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (18 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 19/31] media: Documentation: Add GS_ROUTING documentation Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 21/31] v4l: subdev: Improve link format validation debug messages Jacopo Mondi
                   ` (11 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Sakari Ailus <sakari.ailus@linux.intel.com>

The routing information is essential in link validation for multiplexed
links: the pads at the ends of a multiplexed link have no single format
defined for them. Instead, the format is accessible in the sink (or
source) pads of the sub-devices at both ends of that link.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Tested-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 220 ++++++++++++++++++++++++--
 1 file changed, 206 insertions(+), 14 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index f845b0658913..9b4f3528a4da 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -651,12 +651,17 @@ static int
 v4l2_subdev_link_validate_get_format(struct media_pad *pad,
 				     struct v4l2_subdev_format *fmt)
 {
+	dev_dbg(pad->entity->graph_obj.mdev->dev,
+		"obtaining format on \"%s\":%u\n", pad->entity->name,
+		pad->index);
+
 	if (is_media_entity_v4l2_subdev(pad->entity)) {
 		struct v4l2_subdev *sd =
 			media_entity_to_v4l2_subdev(pad->entity);
 
 		fmt->which = V4L2_SUBDEV_FORMAT_ACTIVE;
 		fmt->pad = pad->index;
+
 		return v4l2_subdev_call(sd, pad, get_fmt, NULL, fmt);
 	}
 
@@ -667,31 +672,218 @@ v4l2_subdev_link_validate_get_format(struct media_pad *pad,
 	return -EINVAL;
 }
 
-int v4l2_subdev_link_validate(struct media_link *link)
+static int v4l2_subdev_link_validate_one(struct media_link *link,
+					 struct v4l2_subdev_format *source_fmt,
+					 struct v4l2_subdev_format *sink_fmt)
 {
 	struct v4l2_subdev *sink;
-	struct v4l2_subdev_format sink_fmt, source_fmt;
 	int rval;
 
-	rval = v4l2_subdev_link_validate_get_format(
-		link->source, &source_fmt);
-	if (rval < 0)
-		return 0;
-
-	rval = v4l2_subdev_link_validate_get_format(
-		link->sink, &sink_fmt);
-	if (rval < 0)
-		return 0;
-
 	sink = media_entity_to_v4l2_subdev(link->sink->entity);
 
 	rval = v4l2_subdev_call(sink, pad, link_validate, link,
-				&source_fmt, &sink_fmt);
+				source_fmt, sink_fmt);
 	if (rval != -ENOIOCTLCMD)
 		return rval;
 
 	return v4l2_subdev_link_validate_default(
-		sink, link, &source_fmt, &sink_fmt);
+		sink, link, source_fmt, sink_fmt);
+}
+
+/* How many routes to assume there can be per a sub-device? */
+#define LINK_VALIDATE_ROUTES	8
+
+#define R_SRC	0
+#define R_SINK	1
+#define NR_R	2
+
+int v4l2_subdev_link_validate(struct media_link *link)
+{
+	struct v4l2_subdev *sink;
+	struct route_info {
+		struct v4l2_subdev_route routes[LINK_VALIDATE_ROUTES];
+		struct v4l2_subdev_krouting routing;
+		bool has_route;
+		struct media_pad *pad;
+		/* Format for a non-multiplexed pad. */
+		struct v4l2_subdev_format fmt;
+	} r[NR_R] = {
+		{
+			/* Source end of the link */
+			.routing = {
+				.routes = r[R_SRC].routes,
+				.num_routes = ARRAY_SIZE(r[R_SRC].routes),
+			},
+			.pad = link->source,
+		},
+		{
+			/* Sink end of the link */
+			.routing = {
+				.routes = r[R_SINK].routes,
+				.num_routes = ARRAY_SIZE(r[R_SINK].routes),
+			},
+			.pad = link->sink,
+		},
+	};
+	unsigned int i, j;
+	int rval;
+
+	sink = media_entity_to_v4l2_subdev(link->sink->entity);
+
+	dev_dbg(sink->entity.graph_obj.mdev->dev,
+		"validating link \"%s\":%u -> \"%s\":%u\n",
+		link->source->entity->name, link->source->index,
+		sink->entity.name, link->sink->index);
+
+	for (i = 0; i < NR_R; i++) {
+		struct route_info *ri = &r[i];
+
+		ri->has_route = true;
+
+		rval = v4l2_subdev_call(
+			media_entity_to_v4l2_subdev(ri->pad->entity),
+			pad, get_routing, &ri->routing);
+
+		switch (rval) {
+		case 0:
+			break;
+		case -ENOIOCTLCMD:
+			dev_dbg(sink->entity.graph_obj.mdev->dev,
+				"no routing information on \"%s\":%u\n",
+				ri->pad->entity->name, ri->pad->index);
+			ri->has_route = false;
+			break;
+		default:
+			dev_dbg(sink->entity.graph_obj.mdev->dev,
+				"error %d in get_routing() on \"%s\":%u\n",
+				rval, ri->pad->entity->name, ri->pad->index);
+			return rval;
+		}
+
+		rval = v4l2_subdev_link_validate_get_format(ri->pad,
+							    &ri->fmt);
+
+		if (!rval) {
+			dev_dbg(sink->entity.graph_obj.mdev->dev,
+				"format information available on \"%s\":%u\n",
+				ri->pad->entity->name, ri->pad->index);
+			ri->has_route = false;
+		}
+
+		if (!ri->has_route) {
+			ri->routing.num_routes = 1;
+		} else {
+			dev_dbg(sink->entity.graph_obj.mdev->dev,
+				"routes on \"%s\":%u:\n",
+				ri->pad->entity->name, ri->pad->index);
+			for (j = 0; j < ri->routing.num_routes; j++)
+				dev_dbg(sink->entity.graph_obj.mdev->dev,
+					"\t%u: %u/%u -> %u/%u [%s]\n", j,
+					ri->routing.routes[j].sink_pad,
+					ri->routing.routes[j].sink_stream,
+					ri->routing.routes[j].source_pad,
+					ri->routing.routes[j].source_stream,
+					ri->routing.routes[j].flags &
+					V4L2_SUBDEV_ROUTE_FL_ACTIVE ? "1" : "0"
+					);
+		}
+	}
+
+	for (i = 0, j = 0;
+	     i < r[R_SRC].routing.num_routes &&
+		     j < r[R_SINK].routing.num_routes; ) {
+		unsigned int *ro[] = { &i, &j };
+		unsigned int k;
+
+		/* Get the first active route for the sink pad. */
+		if (r[R_SINK].has_route &&
+		    (r[R_SINK].routes[j].sink_pad != link->sink->index ||
+		     !(r[R_SINK].routes[j].flags
+		       & V4L2_SUBDEV_ROUTE_FL_ACTIVE))) {
+			dev_dbg(sink->entity.graph_obj.mdev->dev,
+				"skipping route %u/%u -> %u/%u[%u]\n",
+				r[R_SINK].routes[j].sink_pad,
+				r[R_SINK].routes[j].sink_stream,
+				r[R_SINK].routes[j].source_pad,
+				r[R_SINK].routes[j].source_stream,
+				(bool)(r[R_SINK].routes[j].flags
+				       & V4L2_SUBDEV_ROUTE_FL_ACTIVE));
+			j++;
+			continue;
+		}
+
+		/*
+		 * Get the corresponding route for the source pad.
+		 * It's ok for the source pad to have routes active
+		 * where the sink pad does not, but the routes that
+		 * are active on the sink pad have to be active on
+		 * the source pad as well.
+		 */
+		if (r[R_SRC].has_route &&
+		    (r[R_SRC].routes[i].source_pad != link->source->index ||
+		     r[R_SRC].routes[i].source_stream
+		     != r[R_SINK].routes[j].sink_stream)) {
+			dev_dbg(sink->entity.graph_obj.mdev->dev,
+				"skipping source route %u/%u -> %u/%u\n",
+				r[R_SRC].routes[i].sink_pad,
+				r[R_SRC].routes[i].sink_stream,
+				r[R_SRC].routes[i].source_pad,
+				r[R_SRC].routes[i].source_stream);
+			i++;
+			continue;
+		}
+
+		/* The source route must be active. */
+		if (r[R_SRC].has_route &&
+		    !(r[R_SRC].routes[i].flags
+		      & V4L2_SUBDEV_ROUTE_FL_ACTIVE)) {
+			dev_dbg(sink->entity.graph_obj.mdev->dev,
+				"source route not active\n");
+			return -EINVAL;
+		}
+
+		for (k = 0; k < NR_R; k++) {
+			if (r[k].has_route) {
+				dev_dbg(sink->entity.graph_obj.mdev->dev,
+					"validating %s route \"%s\": %u/%u -> %u/%u\n",
+					k == R_SINK ? "sink" : "source",
+					r[k].pad->entity->name,
+					r[k].routes[*ro[k]].sink_pad,
+					r[k].routes[*ro[k]].sink_stream,
+					r[k].routes[*ro[k]].source_pad,
+					r[k].routes[*ro[k]].source_stream);
+				rval = v4l2_subdev_link_validate_get_format(
+					&r[k].pad->entity->pads[
+						k == R_SINK
+						? r[k].routes[*ro[k]].source_pad
+						: r[k].routes[*ro[k]].sink_pad],
+					&r[k].fmt);
+			} else {
+				dev_dbg(sink->entity.graph_obj.mdev->dev,
+					"routing not supported by \"%s\":%u",
+					r[k].pad->entity->name,
+					r[k].pad->index);
+			}
+		}
+
+		rval = v4l2_subdev_link_validate_one(
+			link, &r[R_SRC].fmt, &r[R_SINK].fmt);
+		if (rval) {
+			dev_dbg(sink->entity.graph_obj.mdev->dev,
+				"error %d in link validation\n", rval);
+			return rval;
+		}
+
+		i++, j++;
+	}
+
+	if (j < r[R_SINK].routing.num_routes) {
+		dev_dbg(sink->entity.graph_obj.mdev->dev,
+			"not all sink routes verified; out of source routes\n");
+		return -EINVAL;
+	}
+
+	return 0;
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_link_validate);
 
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 21/31] v4l: subdev: Improve link format validation debug messages
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (19 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 20/31] v4l: subdev: Take routing information into account in link validation Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 22/31] v4l: mc: Add an S_ROUTING helper function for power state changes Jacopo Mondi
                   ` (10 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Sakari Ailus <sakari.ailus@linux.intel.com>

The existing link format validation failure debug message in media-entity.c
helped to poinpoint the point of failure but provided no additional
information what's wrong. Tell the user exactly why the validation failed.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 40 ++++++++++++++++++++++-----
 1 file changed, 33 insertions(+), 7 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 9b4f3528a4da..6c96d6f4bdfc 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -629,21 +629,47 @@ int v4l2_subdev_link_validate_default(struct v4l2_subdev *sd,
 				      struct v4l2_subdev_format *source_fmt,
 				      struct v4l2_subdev_format *sink_fmt)
 {
+	bool pass = true;
+
 	/* The width, height and code must match. */
-	if (source_fmt->format.width != sink_fmt->format.width
-	    || source_fmt->format.height != sink_fmt->format.height
-	    || source_fmt->format.code != sink_fmt->format.code)
-		return -EPIPE;
+	if (source_fmt->format.width != sink_fmt->format.width) {
+		dev_dbg(sd->entity.graph_obj.mdev->dev,
+			"%s: width does not match (source %u, sink %u)\n",
+			__func__,
+			source_fmt->format.width, sink_fmt->format.width);
+		pass = false;
+	}
+
+	if (source_fmt->format.height != sink_fmt->format.height) {
+		dev_dbg(sd->entity.graph_obj.mdev->dev,
+			"%s: height does not match (source %u, sink %u)\n",
+			__func__,
+			source_fmt->format.height, sink_fmt->format.height);
+		pass = false;
+	}
+
+	if (source_fmt->format.code != sink_fmt->format.code) {
+		dev_dbg(sd->entity.graph_obj.mdev->dev,
+			"%s: media bus code does not match (source 0x%8.8x, sink 0x%8.8x)\n",
+			__func__,
+			source_fmt->format.code, sink_fmt->format.code);
+		pass = false;
+	}
 
 	/* The field order must match, or the sink field order must be NONE
 	 * to support interlaced hardware connected to bridges that support
 	 * progressive formats only.
 	 */
 	if (source_fmt->format.field != sink_fmt->format.field &&
-	    sink_fmt->format.field != V4L2_FIELD_NONE)
-		return -EPIPE;
+	    sink_fmt->format.field != V4L2_FIELD_NONE) {
+		dev_dbg(sd->entity.graph_obj.mdev->dev,
+			"%s: field does not match (source %u, sink %u)\n",
+			__func__,
+			source_fmt->format.field, sink_fmt->format.field);
+		pass = false;
+	}
 
-	return 0;
+	return pass ? 0 : -EPIPE;
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_link_validate_default);
 
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 22/31] v4l: mc: Add an S_ROUTING helper function for power state changes
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (20 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 21/31] v4l: subdev: Improve link format validation debug messages Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 23/31] adv748x: csi2: add translation from pixelcode to CSI-2 datatype Jacopo Mondi
                   ` (9 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Sakari Ailus <sakari.ailus@linux.intel.com>

With the addition of the has_route() media entity operation, all pads of an
entity are no longer interconnected. The S_ROUTING IOCTL for sub-devices can
be used to enable and disable routes for an entity. The consequence is that
the routing information has to be taken into account in use count
calculation: disabling a route has a very similar effect on use counts as
has disabling a link.

Add a helper function for drivers implementing VIDIOC_SUBDEV_S_ROUTING
IOCTL to take the change into account.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/v4l2-core/v4l2-mc.c | 34 +++++++++++++++++++++++++++++++
 include/media/v4l2-mc.h           | 22 ++++++++++++++++++++
 2 files changed, 56 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
index 558dec225838..7a2536d3c75e 100644
--- a/drivers/media/v4l2-core/v4l2-mc.c
+++ b/drivers/media/v4l2-core/v4l2-mc.c
@@ -482,3 +482,37 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 	return ret;
 }
 EXPORT_SYMBOL_GPL(v4l2_pipeline_link_notify);
+
+int v4l2_subdev_routing_pm_use(struct media_entity *entity,
+			       struct v4l2_subdev_route *route)
+{
+	struct media_graph *graph =
+		&entity->graph_obj.mdev->pm_count_walk;
+	struct media_pad *source = &entity->pads[route->source_pad];
+	struct media_pad *sink = &entity->pads[route->sink_pad];
+	int source_use;
+	int sink_use;
+	int ret;
+
+	source_use = pipeline_pm_use_count(source, graph);
+	sink_use = pipeline_pm_use_count(sink, graph);
+
+	if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE)) {
+		/* Route disabled. */
+		pipeline_pm_power(source, -sink_use, graph);
+		pipeline_pm_power(sink, -source_use, graph);
+		return 0;
+	}
+
+	/* Route enabled. */
+	ret = pipeline_pm_power(source, sink_use, graph);
+	if (ret < 0)
+		return ret;
+
+	ret = pipeline_pm_power(sink, source_use, graph);
+	if (ret < 0)
+		pipeline_pm_power(source, -sink_use, graph);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_routing_pm_use);
diff --git a/include/media/v4l2-mc.h b/include/media/v4l2-mc.h
index bf5043c1ab6b..730922636579 100644
--- a/include/media/v4l2-mc.h
+++ b/include/media/v4l2-mc.h
@@ -26,6 +26,7 @@
 /* We don't need to include pci.h or usb.h here */
 struct pci_dev;
 struct usb_device;
+struct v4l2_subdev_route;
 
 #ifdef CONFIG_MEDIA_CONTROLLER
 /**
@@ -132,6 +133,22 @@ int v4l2_pipeline_pm_use(struct media_entity *entity, int use);
 int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 			      unsigned int notification);
 
+/**
+ * v4l2_subdev_routing_pm_use - Handle power state changes due to S_ROUTING
+ * @entity: The entity
+ * @route: The new state of the route
+ *
+ * Propagate the use count across a route in a pipeline whenever the
+ * route is enabled or disabled. The function is called before
+ * changing the route state when enabling a route, and after changing
+ * the route state when disabling a route.
+ *
+ * Return 0 on success or a negative error code on failure. Powering entities
+ * off is assumed to never fail. This function will not fail for disconnection
+ * events.
+ */
+int v4l2_subdev_routing_pm_use(struct media_entity *entity,
+			       struct v4l2_subdev_route *route);
 #else /* CONFIG_MEDIA_CONTROLLER */
 
 static inline int v4l2_mc_create_media_graph(struct media_device *mdev)
@@ -164,5 +181,10 @@ static inline int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 	return 0;
 }
 
+static inline int v4l2_subdev_routing_pm_use(struct media_entity *entity,
+					     struct v4l2_subdev_route *route)
+{
+	return 0;
+}
 #endif /* CONFIG_MEDIA_CONTROLLER */
 #endif /* _V4L2_MC_H */
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 23/31] adv748x: csi2: add translation from pixelcode to CSI-2 datatype
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (21 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 22/31] v4l: mc: Add an S_ROUTING helper function for power state changes Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 24/31] adv748x: csi2: only allow formats on sink pads Jacopo Mondi
                   ` (8 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

Prepare to implement frame descriptors to support multiplexed streams by
adding a function to map pixelcode to CSI-2 datatype. This is needed to
properly be able to fill out the struct v4l2_mbus_frame_desc.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/i2c/adv748x/adv748x-csi2.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
index 2091cda50935..25798d723174 100644
--- a/drivers/media/i2c/adv748x/adv748x-csi2.c
+++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
@@ -14,6 +14,28 @@
 
 #include "adv748x.h"
 
+struct adv748x_csi2_format {
+	unsigned int code;
+	unsigned int datatype;
+};
+
+static const struct adv748x_csi2_format adv748x_csi2_formats[] = {
+	{ .code = MEDIA_BUS_FMT_RGB888_1X24,    .datatype = 0x24, },
+	{ .code = MEDIA_BUS_FMT_UYVY8_1X16,     .datatype = 0x1e, },
+	{ .code = MEDIA_BUS_FMT_UYVY8_2X8,      .datatype = 0x1e, },
+	{ .code = MEDIA_BUS_FMT_YUYV10_2X10,    .datatype = 0x1e, },
+};
+
+static unsigned int adv748x_csi2_code_to_datatype(unsigned int code)
+{
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(adv748x_csi2_formats); i++)
+		if (adv748x_csi2_formats[i].code == code)
+			return adv748x_csi2_formats[i].datatype;
+	return 0;
+}
+
 static int adv748x_csi2_set_virtual_channel(struct adv748x_csi2 *tx,
 					    unsigned int vc)
 {
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 24/31] adv748x: csi2: only allow formats on sink pads
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (22 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 23/31] adv748x: csi2: add translation from pixelcode to CSI-2 datatype Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 25/31] adv748x: csi2: describe the multiplexed stream Jacopo Mondi
                   ` (7 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

Once the CSI-2 subdevice of the ADV748X becomes aware of multiplexed
streams the format of the source pad is of no value as it carries
multiple streams. Prepare for this by explicitly denying setting a
format on anything but the sink pad.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/i2c/adv748x/adv748x-csi2.c | 22 ++++++----------------
 1 file changed, 6 insertions(+), 16 deletions(-)

diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
index 25798d723174..1abe34183d7d 100644
--- a/drivers/media/i2c/adv748x/adv748x-csi2.c
+++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
@@ -183,6 +183,9 @@ static int adv748x_csi2_get_format(struct v4l2_subdev *sd,
 	struct adv748x_state *state = tx->state;
 	struct v4l2_mbus_framefmt *mbusformat;
 
+	if (sdformat->pad != ADV748X_CSI2_SINK)
+		return -EINVAL;
+
 	mbusformat = adv748x_csi2_get_pad_format(sd, cfg, sdformat->pad,
 						 sdformat->which);
 	if (!mbusformat)
@@ -206,6 +209,9 @@ static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
 	struct v4l2_mbus_framefmt *mbusformat;
 	int ret = 0;
 
+	if (sdformat->pad != ADV748X_CSI2_SINK)
+		return -EINVAL;
+
 	mbusformat = adv748x_csi2_get_pad_format(sd, cfg, sdformat->pad,
 						 sdformat->which);
 	if (!mbusformat)
@@ -213,24 +219,8 @@ static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
 
 	mutex_lock(&state->mutex);
 
-	if (sdformat->pad == ADV748X_CSI2_SOURCE) {
-		const struct v4l2_mbus_framefmt *sink_fmt;
-
-		sink_fmt = adv748x_csi2_get_pad_format(sd, cfg,
-						       ADV748X_CSI2_SINK,
-						       sdformat->which);
-
-		if (!sink_fmt) {
-			ret = -EINVAL;
-			goto unlock;
-		}
-
-		sdformat->format = *sink_fmt;
-	}
-
 	*mbusformat = sdformat->format;
 
-unlock:
 	mutex_unlock(&state->mutex);
 
 	return ret;
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 25/31] adv748x: csi2: describe the multiplexed stream
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (23 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 24/31] adv748x: csi2: only allow formats on sink pads Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 26/31] adv748x: csi2: add internal routing configuration Jacopo Mondi
                   ` (6 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

The adv748x CSI-2 transmitter can only transmit one stream over the
CSI-2 link, however it can choose which virtual channel is used. This
choice effects the CSI-2 receiver and needs to be captured in the frame
descriptor information, solve this by implementing .get_frame_desc().

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/i2c/adv748x/adv748x-csi2.c | 31 +++++++++++++++++++++++-
 drivers/media/i2c/adv748x/adv748x.h      |  1 +
 2 files changed, 31 insertions(+), 1 deletion(-)

diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
index 1abe34183d7d..d8f7cbee86e7 100644
--- a/drivers/media/i2c/adv748x/adv748x-csi2.c
+++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
@@ -226,9 +226,37 @@ static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
 	return ret;
 }
 
+static int adv748x_csi2_get_frame_desc(struct v4l2_subdev *sd, unsigned int pad,
+				       struct v4l2_mbus_frame_desc *fd)
+{
+	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
+	struct v4l2_mbus_framefmt *mbusformat;
+
+	memset(fd, 0, sizeof(*fd));
+
+	if (pad != ADV748X_CSI2_SOURCE)
+		return -EINVAL;
+
+	mbusformat = adv748x_csi2_get_pad_format(sd, NULL, ADV748X_CSI2_SINK,
+						 V4L2_SUBDEV_FORMAT_ACTIVE);
+	if (!mbusformat)
+		return -EINVAL;
+
+	fd->entry->stream = tx->vc;
+	fd->entry->bus.csi2.channel = tx->vc;
+	fd->entry->bus.csi2.data_type =
+		adv748x_csi2_code_to_datatype(mbusformat->code);
+
+	fd->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
+	fd->num_entries = 1;
+
+	return 0;
+}
+
 static const struct v4l2_subdev_pad_ops adv748x_csi2_pad_ops = {
 	.get_fmt = adv748x_csi2_get_format,
 	.set_fmt = adv748x_csi2_set_format,
+	.get_frame_desc = adv748x_csi2_get_frame_desc,
 };
 
 /* -----------------------------------------------------------------------------
@@ -295,7 +323,8 @@ int adv748x_csi2_init(struct adv748x_state *state, struct adv748x_csi2 *tx)
 		return 0;
 
 	/* Initialise the virtual channel */
-	adv748x_csi2_set_virtual_channel(tx, 0);
+	tx->vc = 0;
+	adv748x_csi2_set_virtual_channel(tx, tx->vc);
 
 	adv748x_subdev_init(&tx->sd, state, &adv748x_csi2_ops,
 			    MEDIA_ENT_F_VID_IF_BRIDGE,
diff --git a/drivers/media/i2c/adv748x/adv748x.h b/drivers/media/i2c/adv748x/adv748x.h
index 5042f9e94aee..4a5a6445604f 100644
--- a/drivers/media/i2c/adv748x/adv748x.h
+++ b/drivers/media/i2c/adv748x/adv748x.h
@@ -75,6 +75,7 @@ enum adv748x_csi2_pads {
 
 struct adv748x_csi2 {
 	struct adv748x_state *state;
+	unsigned int vc;
 	struct v4l2_mbus_framefmt format;
 	unsigned int page;
 	unsigned int port;
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 26/31] adv748x: csi2: add internal routing configuration
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (24 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 25/31] adv748x: csi2: describe the multiplexed stream Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-14 14:45   ` Luca Ceresoli
  2019-03-05 18:51 ` [PATCH v3 27/31] adv748x: afe: add routing support Jacopo Mondi
                   ` (5 subsequent siblings)
  31 siblings, 1 reply; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

Add support to get and set the internal routing between the adv748x
CSI-2 transmitters sink pad and its multiplexed source pad. This routing
includes which stream of the multiplexed pad to use, allowing the user
to select which CSI-2 virtual channel to use when transmitting the
stream.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/i2c/adv748x/adv748x-csi2.c | 65 ++++++++++++++++++++++++
 1 file changed, 65 insertions(+)

diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
index d8f7cbee86e7..13454af72c6e 100644
--- a/drivers/media/i2c/adv748x/adv748x-csi2.c
+++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
@@ -14,6 +14,8 @@
 
 #include "adv748x.h"
 
+#define ADV748X_CSI2_ROUTES_MAX 4
+
 struct adv748x_csi2_format {
 	unsigned int code;
 	unsigned int datatype;
@@ -253,10 +255,73 @@ static int adv748x_csi2_get_frame_desc(struct v4l2_subdev *sd, unsigned int pad,
 	return 0;
 }
 
+static int adv748x_csi2_get_routing(struct v4l2_subdev *sd,
+				    struct v4l2_subdev_krouting *routing)
+{
+	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
+	struct v4l2_subdev_route *r = routing->routes;
+	unsigned int vc;
+
+	if (routing->num_routes < ADV748X_CSI2_ROUTES_MAX) {
+		routing->num_routes = ADV748X_CSI2_ROUTES_MAX;
+		return -ENOSPC;
+	}
+
+	routing->num_routes = ADV748X_CSI2_ROUTES_MAX;
+
+	for (vc = 0; vc < ADV748X_CSI2_ROUTES_MAX; vc++) {
+		r->sink_pad = ADV748X_CSI2_SINK;
+		r->sink_stream = 0;
+		r->source_pad = ADV748X_CSI2_SOURCE;
+		r->source_stream = vc;
+		r->flags = vc == tx->vc ? V4L2_SUBDEV_ROUTE_FL_ACTIVE : 0;
+		r++;
+	}
+
+	return 0;
+}
+
+static int adv748x_csi2_set_routing(struct v4l2_subdev *sd,
+				    struct v4l2_subdev_krouting *routing)
+{
+	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
+	struct v4l2_subdev_route *r = routing->routes;
+	unsigned int i;
+	int vc = -1;
+
+	if (routing->num_routes > ADV748X_CSI2_ROUTES_MAX)
+		return -ENOSPC;
+
+	for (i = 0; i < routing->num_routes; i++) {
+		if (r->sink_pad != ADV748X_CSI2_SINK ||
+		    r->sink_stream != 0 ||
+		    r->source_pad != ADV748X_CSI2_SOURCE ||
+		    r->source_stream >= ADV748X_CSI2_ROUTES_MAX)
+			return -EINVAL;
+
+		if (r->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE) {
+			if (vc != -1)
+				return -EMLINK;
+
+			vc = r->source_stream;
+		}
+		r++;
+	}
+
+	if (vc != -1)
+		tx->vc = vc;
+
+	adv748x_csi2_set_virtual_channel(tx, tx->vc);
+
+	return 0;
+}
+
 static const struct v4l2_subdev_pad_ops adv748x_csi2_pad_ops = {
 	.get_fmt = adv748x_csi2_get_format,
 	.set_fmt = adv748x_csi2_set_format,
 	.get_frame_desc = adv748x_csi2_get_frame_desc,
+	.get_routing = adv748x_csi2_get_routing,
+	.set_routing = adv748x_csi2_set_routing,
 };
 
 /* -----------------------------------------------------------------------------
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 27/31] adv748x: afe: add routing support
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (25 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 26/31] adv748x: csi2: add internal routing configuration Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 28/31] adv748x: afe: Implement has_route() Jacopo Mondi
                   ` (4 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc, Jacopo Mondi

From: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

The adv748x afe has eight analog sink pads, currently one of them is
chosen to be the active route based on device tree configuration. With
the new routing API it's possible to expose and change which of the
eight sink pads are routed to the source pad.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
---
 drivers/media/i2c/adv748x/adv748x-afe.c | 65 +++++++++++++++++++++++++
 1 file changed, 65 insertions(+)

diff --git a/drivers/media/i2c/adv748x/adv748x-afe.c b/drivers/media/i2c/adv748x/adv748x-afe.c
index dbbb1e4d6363..3f770f71413f 100644
--- a/drivers/media/i2c/adv748x/adv748x-afe.c
+++ b/drivers/media/i2c/adv748x/adv748x-afe.c
@@ -39,6 +39,9 @@
 #define ADV748X_AFE_STD_PAL_SECAM			0xe
 #define ADV748X_AFE_STD_PAL_SECAM_PED			0xf
 
+#define ADV748X_AFE_ROUTES_MAX ((ADV748X_AFE_SINK_AIN7 - \
+				ADV748X_AFE_SINK_AIN0) + 1)
+
 static int adv748x_afe_read_ro_map(struct adv748x_state *state, u8 reg)
 {
 	int ret;
@@ -383,10 +386,72 @@ static int adv748x_afe_set_format(struct v4l2_subdev *sd,
 	return 0;
 }
 
+static int adv748x_afe_get_routing(struct v4l2_subdev *sd,
+				   struct v4l2_subdev_krouting *routing)
+{
+	struct adv748x_afe *afe = adv748x_sd_to_afe(sd);
+	struct v4l2_subdev_route *r = routing->routes;
+	unsigned int i;
+
+	/* There is one possible route from each sink */
+	if (routing->num_routes < ADV748X_AFE_ROUTES_MAX) {
+		routing->num_routes = ADV748X_AFE_ROUTES_MAX;
+		return -ENOSPC;
+	}
+
+	routing->num_routes = ADV748X_AFE_ROUTES_MAX;
+
+	for (i = ADV748X_AFE_SINK_AIN0; i <= ADV748X_AFE_SINK_AIN7; i++) {
+		r->sink_pad = i;
+		r->sink_stream = 0;
+		r->source_pad = ADV748X_AFE_SOURCE;
+		r->source_stream = 0;
+		r->flags = afe->input == i ? V4L2_SUBDEV_ROUTE_FL_ACTIVE : 0;
+		r++;
+	}
+
+	return 0;
+}
+
+static int adv748x_afe_set_routing(struct v4l2_subdev *sd,
+				   struct v4l2_subdev_krouting *routing)
+{
+	struct adv748x_afe *afe = adv748x_sd_to_afe(sd);
+	struct v4l2_subdev_route *r = routing->routes;
+	int input = -1;
+	unsigned int i;
+
+	if (routing->num_routes > ADV748X_AFE_ROUTES_MAX)
+		return -ENOSPC;
+
+	for (i = 0; i < routing->num_routes; i++) {
+		if (r->sink_pad > ADV748X_AFE_SINK_AIN7 ||
+		    r->sink_stream != 0 ||
+		    r->source_pad != ADV748X_AFE_SOURCE ||
+		    r->source_stream != 0)
+			return -EINVAL;
+
+		if (r->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE) {
+			if (input != -1)
+				return -EMLINK;
+
+			input = r->sink_pad;
+		}
+		r++;
+	}
+
+	if (input != -1)
+		afe->input = input;
+
+	return 0;
+}
+
 static const struct v4l2_subdev_pad_ops adv748x_afe_pad_ops = {
 	.enum_mbus_code = adv748x_afe_enum_mbus_code,
 	.set_fmt = adv748x_afe_set_format,
 	.get_fmt = adv748x_afe_get_format,
+	.get_routing = adv748x_afe_get_routing,
+	.set_routing = adv748x_afe_set_routing,
 };
 
 /* -----------------------------------------------------------------------------
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 28/31] adv748x: afe: Implement has_route()
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (26 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 27/31] adv748x: afe: add routing support Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-07 12:49   ` Sakari Ailus
  2019-03-14 14:45   ` Luca Ceresoli
  2019-03-05 18:51 ` [PATCH v3 29/31] rcar-csi2: use frame description information to configure CSI-2 bus Jacopo Mondi
                   ` (3 subsequent siblings)
  31 siblings, 2 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

Now that the adv748x subdevice supports internal routing, add an
has_route() operation used during media graph traversal.

Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/i2c/adv748x/adv748x-afe.c | 26 +++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/drivers/media/i2c/adv748x/adv748x-afe.c b/drivers/media/i2c/adv748x/adv748x-afe.c
index 3f770f71413f..39ac55f0adbb 100644
--- a/drivers/media/i2c/adv748x/adv748x-afe.c
+++ b/drivers/media/i2c/adv748x/adv748x-afe.c
@@ -463,6 +463,30 @@ static const struct v4l2_subdev_ops adv748x_afe_ops = {
 	.pad = &adv748x_afe_pad_ops,
 };
 
+/* -----------------------------------------------------------------------------
+ * media_entity_operations
+ */
+
+static bool adv748x_afe_has_route(struct media_entity *entity,
+				  unsigned int pad0, unsigned int pad1)
+{
+	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+	struct adv748x_afe *afe = adv748x_sd_to_afe(sd);
+
+	/* Only consider direct sink->source routes. */
+	if (pad0 > ADV748X_AFE_SINK_AIN7 ||
+	    pad1 != ADV748X_AFE_SOURCE)
+		return false;
+
+	if (pad0 != afe->input)
+		return false;
+
+	return true;
+}
+
+static const struct media_entity_operations adv748x_afe_entity_ops = {
+	.has_route = adv748x_afe_has_route,
+};
 /* -----------------------------------------------------------------------------
  * Controls
  */
@@ -595,6 +619,8 @@ int adv748x_afe_init(struct adv748x_afe *afe)
 
 	afe->pads[ADV748X_AFE_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
 
+	afe->sd.entity.ops = &adv748x_afe_entity_ops;
+
 	ret = media_entity_pads_init(&afe->sd.entity, ADV748X_AFE_NR_PADS,
 			afe->pads);
 	if (ret)
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 29/31] rcar-csi2: use frame description information to configure CSI-2 bus
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (27 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 28/31] adv748x: afe: Implement has_route() Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 30/31] rcar-csi2: expose the subdevice internal routing Jacopo Mondi
                   ` (2 subsequent siblings)
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

The driver can now access frame descriptor information, use it when
configuring the CSI-2 bus. Only enable the virtual channels which are
described in the frame descriptor and calculate the link based on all
enabled streams.

With multiplexed stream supported it's now meaningful to have different
formats on the different source pads. Make source formats independent
off each other and disallowing a format on the multiplexed sink pad.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/platform/rcar-vin/rcar-csi2.c | 134 ++++++++++++++------
 1 file changed, 96 insertions(+), 38 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-csi2.c b/drivers/media/platform/rcar-vin/rcar-csi2.c
index f64528d2be3c..f9cc99ba00bc 100644
--- a/drivers/media/platform/rcar-vin/rcar-csi2.c
+++ b/drivers/media/platform/rcar-vin/rcar-csi2.c
@@ -304,25 +304,21 @@ static const struct rcsi2_mbps_reg hsfreqrange_m3w_h3es1[] = {
 #define CSI0CLKFREQRANGE(n)		((n & 0x3f) << 16)
 
 struct rcar_csi2_format {
-	u32 code;
 	unsigned int datatype;
 	unsigned int bpp;
 };
 
 static const struct rcar_csi2_format rcar_csi2_formats[] = {
-	{ .code = MEDIA_BUS_FMT_RGB888_1X24,	.datatype = 0x24, .bpp = 24 },
-	{ .code = MEDIA_BUS_FMT_UYVY8_1X16,	.datatype = 0x1e, .bpp = 16 },
-	{ .code = MEDIA_BUS_FMT_YUYV8_1X16,	.datatype = 0x1e, .bpp = 16 },
-	{ .code = MEDIA_BUS_FMT_UYVY8_2X8,	.datatype = 0x1e, .bpp = 16 },
-	{ .code = MEDIA_BUS_FMT_YUYV10_2X10,	.datatype = 0x1e, .bpp = 20 },
+	{ .datatype = 0x1e, .bpp = 16 },
+	{ .datatype = 0x24, .bpp = 24 },
 };
 
-static const struct rcar_csi2_format *rcsi2_code_to_fmt(unsigned int code)
+static const struct rcar_csi2_format *rcsi2_datatype_to_fmt(unsigned int dt)
 {
 	unsigned int i;
 
 	for (i = 0; i < ARRAY_SIZE(rcar_csi2_formats); i++)
-		if (rcar_csi2_formats[i].code == code)
+		if (rcar_csi2_formats[i].datatype == dt)
 			return &rcar_csi2_formats[i];
 
 	return NULL;
@@ -337,6 +333,14 @@ enum rcar_csi2_pads {
 	NR_OF_RCAR_CSI2_PAD,
 };
 
+static int rcsi2_pad_to_vc(unsigned int pad)
+{
+	if (pad < RCAR_CSI2_SOURCE_VC0 || pad > RCAR_CSI2_SOURCE_VC3)
+		return -EINVAL;
+
+	return pad - RCAR_CSI2_SOURCE_VC0;
+}
+
 struct rcar_csi2_info {
 	int (*init_phtw)(struct rcar_csi2 *priv, unsigned int mbps);
 	int (*confirm_start)(struct rcar_csi2 *priv);
@@ -358,7 +362,7 @@ struct rcar_csi2 {
 	struct v4l2_async_subdev asd;
 	struct v4l2_subdev *remote;
 
-	struct v4l2_mbus_framefmt mf;
+	struct v4l2_mbus_framefmt mf[4];
 
 	struct mutex lock;
 	int stream_count;
@@ -394,6 +398,32 @@ static void rcsi2_reset(struct rcar_csi2 *priv)
 	rcsi2_write(priv, SRST_REG, 0);
 }
 
+static int rcsi2_get_remote_frame_desc(struct rcar_csi2 *priv,
+				       struct v4l2_mbus_frame_desc *fd)
+{
+	struct media_pad *pad;
+	int ret;
+
+	if (!priv->remote)
+		return -ENODEV;
+
+	pad = media_entity_remote_pad(&priv->pads[RCAR_CSI2_SINK]);
+	if (!pad)
+		return -ENODEV;
+
+	ret = v4l2_subdev_call(priv->remote, pad, get_frame_desc,
+			       pad->index, fd);
+	if (ret)
+		return -ENODEV;
+
+	if (fd->type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2) {
+		dev_err(priv->dev, "Frame desc do not describe CSI-2 link");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
 static int rcsi2_wait_phy_start(struct rcar_csi2 *priv)
 {
 	unsigned int timeout;
@@ -432,10 +462,12 @@ static int rcsi2_set_phypll(struct rcar_csi2 *priv, unsigned int mbps)
 	return 0;
 }
 
-static int rcsi2_calc_mbps(struct rcar_csi2 *priv, unsigned int bpp)
+static int rcsi2_calc_mbps(struct rcar_csi2 *priv,
+			   struct v4l2_mbus_frame_desc *fd)
 {
 	struct v4l2_subdev *source;
 	struct v4l2_ctrl *ctrl;
+	unsigned int i, bpp = 0;
 	u64 mbps;
 
 	if (!priv->remote)
@@ -451,6 +483,20 @@ static int rcsi2_calc_mbps(struct rcar_csi2 *priv, unsigned int bpp)
 		return -EINVAL;
 	}
 
+	/* Calculate total bpp */
+	for (i = 0; i < fd->num_entries; i++) {
+		const struct rcar_csi2_format *format;
+
+		format = rcsi2_datatype_to_fmt(fd->entry[i].bus.csi2.data_type);
+		if (!format) {
+			dev_err(priv->dev, "Unknown data type: %d\n",
+				fd->entry[i].bus.csi2.data_type);
+			return -EINVAL;
+		}
+
+		bpp += format->bpp;
+	}
+
 	/*
 	 * Calculate the phypll in mbps.
 	 * link_freq = (pixel_rate * bits_per_sample) / (2 * nr_of_lanes)
@@ -464,43 +510,40 @@ static int rcsi2_calc_mbps(struct rcar_csi2 *priv, unsigned int bpp)
 
 static int rcsi2_start(struct rcar_csi2 *priv)
 {
-	const struct rcar_csi2_format *format;
 	u32 phycnt, vcdt = 0, vcdt2 = 0;
+	struct v4l2_mbus_frame_desc fd;
 	unsigned int i;
 	int mbps, ret;
 
-	dev_dbg(priv->dev, "Input size (%ux%u%c)\n",
-		priv->mf.width, priv->mf.height,
-		priv->mf.field == V4L2_FIELD_NONE ? 'p' : 'i');
-
-	/* Code is validated in set_fmt. */
-	format = rcsi2_code_to_fmt(priv->mf.code);
+	/* Get information about multiplexed link. */
+	ret = rcsi2_get_remote_frame_desc(priv, &fd);
+	if (ret)
+		return ret;
 
-	/*
-	 * Enable all supported CSI-2 channels with virtual channel and
-	 * data type matching.
-	 *
-	 * NOTE: It's not possible to get individual datatype for each
-	 *       source virtual channel. Once this is possible in V4L2
-	 *       it should be used here.
-	 */
-	for (i = 0; i < priv->info->num_channels; i++) {
+	for (i = 0; i < fd.num_entries; i++) {
+		struct v4l2_mbus_frame_desc_entry *entry = &fd.entry[i];
 		u32 vcdt_part;
 
-		vcdt_part = VCDT_SEL_VC(i) | VCDT_VCDTN_EN | VCDT_SEL_DTN_ON |
-			VCDT_SEL_DT(format->datatype);
+		vcdt_part = VCDT_SEL_VC(entry->bus.csi2.channel) |
+			VCDT_VCDTN_EN | VCDT_SEL_DTN_ON |
+			VCDT_SEL_DT(entry->bus.csi2.data_type);
 
 		/* Store in correct reg and offset. */
-		if (i < 2)
-			vcdt |= vcdt_part << ((i % 2) * 16);
+		if (entry->bus.csi2.channel < 2)
+			vcdt |= vcdt_part <<
+				((entry->bus.csi2.channel % 2) * 16);
 		else
-			vcdt2 |= vcdt_part << ((i % 2) * 16);
+			vcdt2 |= vcdt_part <<
+				((entry->bus.csi2.channel % 2) * 16);
+
+		dev_dbg(priv->dev, "VC%d datatype: 0x%x\n",
+			entry->bus.csi2.channel, entry->bus.csi2.data_type);
 	}
 
 	phycnt = PHYCNT_ENABLECLK;
 	phycnt |= (1 << priv->lanes) - 1;
 
-	mbps = rcsi2_calc_mbps(priv, format->bpp);
+	mbps = rcsi2_calc_mbps(priv, &fd);
 	if (mbps < 0)
 		return mbps;
 
@@ -622,14 +665,16 @@ static int rcsi2_set_pad_format(struct v4l2_subdev *sd,
 {
 	struct rcar_csi2 *priv = sd_to_csi2(sd);
 	struct v4l2_mbus_framefmt *framefmt;
+	int vc;
 
-	if (!rcsi2_code_to_fmt(format->format.code))
-		format->format.code = rcar_csi2_formats[0].code;
+	vc = rcsi2_pad_to_vc(format->pad);
+	if (vc < 0)
+		return vc;
 
 	if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
-		priv->mf = format->format;
+		priv->mf[vc] = format->format;
 	} else {
-		framefmt = v4l2_subdev_get_try_format(sd, cfg, 0);
+		framefmt = v4l2_subdev_get_try_format(sd, cfg, format->pad);
 		*framefmt = format->format;
 	}
 
@@ -641,11 +686,17 @@ static int rcsi2_get_pad_format(struct v4l2_subdev *sd,
 				struct v4l2_subdev_format *format)
 {
 	struct rcar_csi2 *priv = sd_to_csi2(sd);
+	int vc;
+
+	vc = rcsi2_pad_to_vc(format->pad);
+	if (vc < 0)
+		return vc;
 
 	if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE)
-		format->format = priv->mf;
+		format->format = priv->mf[vc];
 	else
-		format->format = *v4l2_subdev_get_try_format(sd, cfg, 0);
+		format->format = *v4l2_subdev_get_try_format(sd, cfg,
+							     format->pad);
 
 	return 0;
 }
@@ -675,6 +726,13 @@ static int rcsi2_notify_bound(struct v4l2_async_notifier *notifier,
 	struct rcar_csi2 *priv = notifier_to_csi2(notifier);
 	int pad;
 
+	if (!v4l2_subdev_has_op(subdev, pad, get_frame_desc)) {
+		dev_err(priv->dev,
+			"Failed as '%s' do not support frame descriptors\n",
+			subdev->name);
+		return -EINVAL;
+	}
+
 	pad = media_entity_get_fwnode_pad(&subdev->entity, asd->match.fwnode,
 					  MEDIA_PAD_FL_SOURCE);
 	if (pad < 0) {
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 30/31] rcar-csi2: expose the subdevice internal routing
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (28 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 29/31] rcar-csi2: use frame description information to configure CSI-2 bus Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-05 18:51 ` [PATCH v3 31/31] media: rcar-csi2: Implement has_route() Jacopo Mondi
  2019-03-07  9:47 ` [PATCH v3 00/31] v4l: add support for multiplexed streams Sakari Ailus
  31 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

From: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

Expose the subdevice internal routing from the single multiplexed sink
pad to its source pads by implementing .get_routing(). This information
is used to do link validation at stream start and allows user-space to
view the route configuration.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/platform/rcar-vin/rcar-csi2.c | 53 +++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/drivers/media/platform/rcar-vin/rcar-csi2.c b/drivers/media/platform/rcar-vin/rcar-csi2.c
index f9cc99ba00bc..cc7077b40f18 100644
--- a/drivers/media/platform/rcar-vin/rcar-csi2.c
+++ b/drivers/media/platform/rcar-vin/rcar-csi2.c
@@ -341,6 +341,14 @@ static int rcsi2_pad_to_vc(unsigned int pad)
 	return pad - RCAR_CSI2_SOURCE_VC0;
 }
 
+static int rcsi2_vc_to_pad(unsigned int vc)
+{
+	if (vc > 3)
+		return -EINVAL;
+
+	return vc + RCAR_CSI2_SOURCE_VC0;
+}
+
 struct rcar_csi2_info {
 	int (*init_phtw)(struct rcar_csi2 *priv, unsigned int mbps);
 	int (*confirm_start)(struct rcar_csi2 *priv);
@@ -705,9 +713,54 @@ static const struct v4l2_subdev_video_ops rcar_csi2_video_ops = {
 	.s_stream = rcsi2_s_stream,
 };
 
+static int rcsi2_get_routing(struct v4l2_subdev *sd,
+			     struct v4l2_subdev_krouting *routing)
+{
+	struct v4l2_subdev_route *r = routing->routes;
+	struct rcar_csi2 *priv = sd_to_csi2(sd);
+	struct v4l2_mbus_frame_desc fd;
+	unsigned int i;
+	int ret;
+
+	/* Get information about multiplexed link */
+	ret = rcsi2_get_remote_frame_desc(priv, &fd);
+	if (ret)
+		return ret;
+
+	if (routing->num_routes < fd.num_entries) {
+		routing->num_routes = fd.num_entries;
+		return -ENOSPC;
+	}
+
+	routing->num_routes = fd.num_entries;
+
+	for (i = 0; i < fd.num_entries; i++) {
+		struct v4l2_mbus_frame_desc_entry *entry = &fd.entry[i];
+		int source_pad;
+
+		source_pad = rcsi2_vc_to_pad(entry->bus.csi2.channel);
+		if (source_pad < 0) {
+			dev_err(priv->dev, "Virtual Channel out of range: %u\n",
+				entry->bus.csi2.channel);
+			return -EINVAL;
+		}
+
+		r->sink_pad = RCAR_CSI2_SINK;
+		r->sink_stream = entry->stream;
+		r->source_pad = source_pad;
+		r->source_stream = 0;
+		r->flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE |
+			   V4L2_SUBDEV_ROUTE_FL_IMMUTABLE;
+		r++;
+	}
+
+	return 0;
+}
+
 static const struct v4l2_subdev_pad_ops rcar_csi2_pad_ops = {
 	.set_fmt = rcsi2_set_pad_format,
 	.get_fmt = rcsi2_get_pad_format,
+	.get_routing = rcsi2_get_routing,
 };
 
 static const struct v4l2_subdev_ops rcar_csi2_subdev_ops = {
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* [PATCH v3 31/31] media: rcar-csi2: Implement has_route()
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (29 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 30/31] rcar-csi2: expose the subdevice internal routing Jacopo Mondi
@ 2019-03-05 18:51 ` Jacopo Mondi
  2019-03-07 12:56   ` Sakari Ailus
  2019-03-07  9:47 ` [PATCH v3 00/31] v4l: add support for multiplexed streams Sakari Ailus
  31 siblings, 1 reply; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-05 18:51 UTC (permalink / raw)
  To: sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: Jacopo Mondi, linux-media, linux-renesas-soc

Now that the rcar-csi2 subdevice supports internal routing, add an
has_route() operation used during graph traversal.

The internal routing between the sink and the source pads depends on the
virtual channel used to transmit the video stream from the remote
subdevice to the R-Car CSI-2 receiver.

Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/platform/rcar-vin/rcar-csi2.c | 35 +++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/drivers/media/platform/rcar-vin/rcar-csi2.c b/drivers/media/platform/rcar-vin/rcar-csi2.c
index cc7077b40f18..6c46bcc0ee83 100644
--- a/drivers/media/platform/rcar-vin/rcar-csi2.c
+++ b/drivers/media/platform/rcar-vin/rcar-csi2.c
@@ -1028,7 +1028,42 @@ static int rcsi2_confirm_start_v3m_e3(struct rcar_csi2 *priv)
  * Platform Device Driver.
  */
 
+static bool rcar_csi2_has_route(struct media_entity *entity,
+				unsigned int pad0, unsigned int pad1)
+{
+	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+	struct rcar_csi2 *priv = sd_to_csi2(sd);
+	struct v4l2_mbus_frame_desc fd;
+	unsigned int i;
+	int ret;
+
+	/* Support only direct sink->source routes. */
+	if (pad0 != RCAR_CSI2_SINK)
+		return false;
+
+	/* Get the frame description: from CSI-2 VC to source pad number. */
+	ret = rcsi2_get_remote_frame_desc(priv, &fd);
+	if (ret)
+		return ret;
+
+	for (i = 0; i < fd.num_entries; i++) {
+		struct v4l2_mbus_frame_desc_entry *entry = &fd.entry[i];
+		int source_pad = rcsi2_vc_to_pad(entry->bus.csi2.channel);
+		if (source_pad < 0) {
+			dev_err(priv->dev, "Virtual Channel out of range: %u\n",
+				entry->bus.csi2.channel);
+			return -EINVAL;
+		}
+
+		if (source_pad == pad1)
+			return true;
+	}
+
+	return false;
+}
+
 static const struct media_entity_operations rcar_csi2_entity_ops = {
+	.has_route = rcar_csi2_has_route,
 	.link_validate = v4l2_subdev_link_validate,
 };
 
-- 
2.20.1


^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 00/31] v4l: add support for multiplexed streams
  2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
                   ` (30 preceding siblings ...)
  2019-03-05 18:51 ` [PATCH v3 31/31] media: rcar-csi2: Implement has_route() Jacopo Mondi
@ 2019-03-07  9:47 ` Sakari Ailus
  2019-03-08 13:19   ` Jacopo Mondi
  31 siblings, 1 reply; 59+ messages in thread
From: Sakari Ailus @ 2019-03-07  9:47 UTC (permalink / raw)
  To: Jacopo Mondi
  Cc: laurent.pinchart, niklas.soderlund+renesas, linux-media,
	linux-renesas-soc

Hi Jacopo,

On Tue, Mar 05, 2019 at 07:51:19PM +0100, Jacopo Mondi wrote:
> Hello,
>    third version of multiplexed stream support patch series.
> 
> V2 sent by Niklas is available at:
> https://patchwork.kernel.org/cover/10573817/
> 
> As per v2, most of the core patches are work from Sakari and Laurent, with
> Niklas' support on top for adv748x and rcar-csi2.
> 
> The use case of the series remains the same: support for virtual channel
> selection implemented on R-Car Gen3 and adv748x. Quoting the v2 cover letter:
> 
> -------------------------------------------------------------------------------
> I have added driver support for the devices used on the Renesas Gen3
> platforms, a ADV7482 connected to the R-Car CSI-2 receiver. With these
> changes I can control which of the analog inputs of the ADV7482 the
> video source is captured from and on which CSI-2 virtual channel the
> video is transmitted on to the R-Car CSI-2 receiver.
> 
> The series adds two new subdev IOCTLs [GS]_ROUTING which allows
> user-space to get and set routes inside a subdevice. I have added RFC
> support for these to v4l-utils [2] which can be used to test this
> series, example:
> 
>     Check the internal routing of the adv748x csi-2 transmitter:
>     v4l2-ctl -d /dev/v4l-subdev24 --get-routing
>     0/0 -> 1/0 [ENABLED]
>     0/0 -> 1/1 []
>     0/0 -> 1/2 []
>     0/0 -> 1/3 []
> 
> 
>     Select that video should be outputed on VC 2 and check the result:
>     $ v4l2-ctl -d /dev/v4l-subdev24 --set-routing '0/0 -> 1/2 [1]'
> 
>     $ v4l2-ctl -d /dev/v4l-subdev24 --get-routing
>     0/0 -> 1/0 []
>     0/0 -> 1/1 []
>     0/0 -> 1/2 [ENABLED]
>     0/0 -> 1/3 []
> -------------------------------------------------------------------------------
> 
> Below is reported the media graph of the system used for testing [1].
> 
> v4l2-ctl patches to handle the newly introduced IOCTLs are available from
> Niklas' repository at:
> git://git.ragnatech.se/v4l-utils routing

Could you send the v4l2-ctl patches out as well, please?

-- 
Sakari Ailus
sakari.ailus@linux.intel.com

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 12/31] media: entity: Add an iterator helper for connected pads
  2019-03-05 18:51 ` [PATCH v3 12/31] media: entity: Add an iterator helper for connected pads Jacopo Mondi
@ 2019-03-07 10:09   ` Sakari Ailus
  2019-03-07 10:27     ` Ian Arkver
  0 siblings, 1 reply; 59+ messages in thread
From: Sakari Ailus @ 2019-03-07 10:09 UTC (permalink / raw)
  To: Jacopo Mondi
  Cc: laurent.pinchart, niklas.soderlund+renesas, linux-media,
	linux-renesas-soc

Hi Jacopo,

On Tue, Mar 05, 2019 at 07:51:31PM +0100, Jacopo Mondi wrote:
> From: Sakari Ailus <sakari.ailus@linux.intel.com>
> 
> Add a helper macro for iterating over pads that are connected through
> enabled routes. This can be used to find all the connected pads within an
> entity, for instance starting from the pad which has been obtained during
> the graph walk.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> 
> - Make __media_entity_next_routed_pad() return NULL and adjust the
>   iterator to handle that
> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> ---
>  include/media/media-entity.h | 27 +++++++++++++++++++++++++++
>  1 file changed, 27 insertions(+)
> 
> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> index 205561545d7e..82f0bdf2a6d1 100644
> --- a/include/media/media-entity.h
> +++ b/include/media/media-entity.h
> @@ -936,6 +936,33 @@ __must_check int media_graph_walk_init(
>  bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
>  			    unsigned int pad1);
>  
> +static inline struct media_pad *__media_entity_next_routed_pad(
> +	struct media_pad *start, struct media_pad *iter)
> +{
> +	struct media_entity *entity = start->entity;
> +
> +	while (iter < &entity->pads[entity->num_pads] &&
> +	       !media_entity_has_route(entity, start->index, iter->index))
> +		iter++;
> +
> +	return iter == &entity->pads[entity->num_pads] ? NULL : iter;

Could you use iter <= ...?

It doesn't seem to matter here, but it'd seem safer to change the check.

> +}
> +
> +/**
> + * media_entity_for_each_routed_pad - Iterate over entity pads connected by routes
> + *
> + * @start: The stating pad
> + * @iter: The iterator pad
> + *
> + * Iterate over all pads connected through routes from a given pad
> + * within an entity. The iteration will include the starting pad itself.
> + */
> +#define media_entity_for_each_routed_pad(start, iter)			\
> +	for (iter = __media_entity_next_routed_pad(			\
> +		     start, (start)->entity->pads);			\
> +	     iter != NULL;						\
> +	     iter = __media_entity_next_routed_pad(start, iter + 1))
> +
>  /**
>   * media_graph_walk_cleanup - Release resources used by graph walk.
>   *

-- 
Kind regards,

Sakari Ailus
sakari.ailus@linux.intel.com

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 12/31] media: entity: Add an iterator helper for connected pads
  2019-03-07 10:09   ` Sakari Ailus
@ 2019-03-07 10:27     ` Ian Arkver
  2019-03-07 12:38       ` Sakari Ailus
  2019-03-07 20:18       ` Jacopo Mondi
  0 siblings, 2 replies; 59+ messages in thread
From: Ian Arkver @ 2019-03-07 10:27 UTC (permalink / raw)
  To: Sakari Ailus, Jacopo Mondi
  Cc: laurent.pinchart, niklas.soderlund+renesas, linux-media,
	linux-renesas-soc

On 07/03/2019 10:09, Sakari Ailus wrote:
> Hi Jacopo,
> 
> On Tue, Mar 05, 2019 at 07:51:31PM +0100, Jacopo Mondi wrote:
>> From: Sakari Ailus <sakari.ailus@linux.intel.com>
>>
>> Add a helper macro for iterating over pads that are connected through
>> enabled routes. This can be used to find all the connected pads within an
>> entity, for instance starting from the pad which has been obtained during
>> the graph walk.
>>
>> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
>> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
>>
>> - Make __media_entity_next_routed_pad() return NULL and adjust the
>>    iterator to handle that
>> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
>> ---
>>   include/media/media-entity.h | 27 +++++++++++++++++++++++++++
>>   1 file changed, 27 insertions(+)
>>
>> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
>> index 205561545d7e..82f0bdf2a6d1 100644
>> --- a/include/media/media-entity.h
>> +++ b/include/media/media-entity.h
>> @@ -936,6 +936,33 @@ __must_check int media_graph_walk_init(
>>   bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
>>   			    unsigned int pad1);
>>   
>> +static inline struct media_pad *__media_entity_next_routed_pad(
>> +	struct media_pad *start, struct media_pad *iter)
>> +{
>> +	struct media_entity *entity = start->entity;
>> +
>> +	while (iter < &entity->pads[entity->num_pads] &&
>> +	       !media_entity_has_route(entity, start->index, iter->index))
>> +		iter++;
>> +
>> +	return iter == &entity->pads[entity->num_pads] ? NULL : iter;
> 
> Could you use iter <= ...?
> 
> It doesn't seem to matter here, but it'd seem safer to change the check.
> 

How about something like...

for (; iter < &entity->pads[entity->num_pads]; iter++)
     if (media_entity_has_route(entity, start->index, iter->index))
         return iter;

return NULL;

Regards,
Ian

>> +}
>> +
>> +/**
>> + * media_entity_for_each_routed_pad - Iterate over entity pads connected by routes
>> + *
>> + * @start: The stating pad
>> + * @iter: The iterator pad
>> + *
>> + * Iterate over all pads connected through routes from a given pad
>> + * within an entity. The iteration will include the starting pad itself.
>> + */
>> +#define media_entity_for_each_routed_pad(start, iter)			\
>> +	for (iter = __media_entity_next_routed_pad(			\
>> +		     start, (start)->entity->pads);			\
>> +	     iter != NULL;						\
>> +	     iter = __media_entity_next_routed_pad(start, iter + 1))
>> +
>>   /**
>>    * media_graph_walk_cleanup - Release resources used by graph walk.
>>    *
> 

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 12/31] media: entity: Add an iterator helper for connected pads
  2019-03-07 10:27     ` Ian Arkver
@ 2019-03-07 12:38       ` Sakari Ailus
  2019-03-07 20:18       ` Jacopo Mondi
  1 sibling, 0 replies; 59+ messages in thread
From: Sakari Ailus @ 2019-03-07 12:38 UTC (permalink / raw)
  To: Ian Arkver
  Cc: Jacopo Mondi, laurent.pinchart, niklas.soderlund+renesas,
	linux-media, linux-renesas-soc

On Thu, Mar 07, 2019 at 10:27:36AM +0000, Ian Arkver wrote:
> On 07/03/2019 10:09, Sakari Ailus wrote:
> > Hi Jacopo,
> > 
> > On Tue, Mar 05, 2019 at 07:51:31PM +0100, Jacopo Mondi wrote:
> > > From: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > 
> > > Add a helper macro for iterating over pads that are connected through
> > > enabled routes. This can be used to find all the connected pads within an
> > > entity, for instance starting from the pad which has been obtained during
> > > the graph walk.
> > > 
> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > > 
> > > - Make __media_entity_next_routed_pad() return NULL and adjust the
> > >    iterator to handle that
> > > Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> > > ---
> > >   include/media/media-entity.h | 27 +++++++++++++++++++++++++++
> > >   1 file changed, 27 insertions(+)
> > > 
> > > diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> > > index 205561545d7e..82f0bdf2a6d1 100644
> > > --- a/include/media/media-entity.h
> > > +++ b/include/media/media-entity.h
> > > @@ -936,6 +936,33 @@ __must_check int media_graph_walk_init(
> > >   bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
> > >   			    unsigned int pad1);
> > > +static inline struct media_pad *__media_entity_next_routed_pad(
> > > +	struct media_pad *start, struct media_pad *iter)
> > > +{
> > > +	struct media_entity *entity = start->entity;
> > > +
> > > +	while (iter < &entity->pads[entity->num_pads] &&
> > > +	       !media_entity_has_route(entity, start->index, iter->index))
> > > +		iter++;
> > > +
> > > +	return iter == &entity->pads[entity->num_pads] ? NULL : iter;
> > 
> > Could you use iter <= ...?
> > 
> > It doesn't seem to matter here, but it'd seem safer to change the check.
> > 
> 
> How about something like...
> 
> for (; iter < &entity->pads[entity->num_pads]; iter++)
>     if (media_entity_has_route(entity, start->index, iter->index))
>         return iter;
> 
> return NULL;

Even better!

-- 
Sakari Ailus
sakari.ailus@linux.intel.com

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 28/31] adv748x: afe: Implement has_route()
  2019-03-05 18:51 ` [PATCH v3 28/31] adv748x: afe: Implement has_route() Jacopo Mondi
@ 2019-03-07 12:49   ` Sakari Ailus
  2019-03-14 14:45   ` Luca Ceresoli
  1 sibling, 0 replies; 59+ messages in thread
From: Sakari Ailus @ 2019-03-07 12:49 UTC (permalink / raw)
  To: Jacopo Mondi
  Cc: laurent.pinchart, niklas.soderlund+renesas, linux-media,
	linux-renesas-soc

On Tue, Mar 05, 2019 at 07:51:47PM +0100, Jacopo Mondi wrote:
> Now that the adv748x subdevice supports internal routing, add an
> has_route() operation used during media graph traversal.
> 
> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> ---
>  drivers/media/i2c/adv748x/adv748x-afe.c | 26 +++++++++++++++++++++++++
>  1 file changed, 26 insertions(+)
> 
> diff --git a/drivers/media/i2c/adv748x/adv748x-afe.c b/drivers/media/i2c/adv748x/adv748x-afe.c
> index 3f770f71413f..39ac55f0adbb 100644
> --- a/drivers/media/i2c/adv748x/adv748x-afe.c
> +++ b/drivers/media/i2c/adv748x/adv748x-afe.c
> @@ -463,6 +463,30 @@ static const struct v4l2_subdev_ops adv748x_afe_ops = {
>  	.pad = &adv748x_afe_pad_ops,
>  };
>  
> +/* -----------------------------------------------------------------------------
> + * media_entity_operations
> + */
> +
> +static bool adv748x_afe_has_route(struct media_entity *entity,
> +				  unsigned int pad0, unsigned int pad1)
> +{
> +	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
> +	struct adv748x_afe *afe = adv748x_sd_to_afe(sd);
> +
> +	/* Only consider direct sink->source routes. */
> +	if (pad0 > ADV748X_AFE_SINK_AIN7 ||
> +	    pad1 != ADV748X_AFE_SOURCE)

Fits on a single line.

> +		return false;
> +
> +	if (pad0 != afe->input)
> +		return false;
> +
> +	return true;
> +}
> +
> +static const struct media_entity_operations adv748x_afe_entity_ops = {
> +	.has_route = adv748x_afe_has_route,
> +};
>  /* -----------------------------------------------------------------------------
>   * Controls
>   */
> @@ -595,6 +619,8 @@ int adv748x_afe_init(struct adv748x_afe *afe)
>  
>  	afe->pads[ADV748X_AFE_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
>  
> +	afe->sd.entity.ops = &adv748x_afe_entity_ops;
> +
>  	ret = media_entity_pads_init(&afe->sd.entity, ADV748X_AFE_NR_PADS,
>  			afe->pads);
>  	if (ret)
> -- 
> 2.20.1
> 

-- 
Sakari Ailus
sakari.ailus@linux.intel.com

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 31/31] media: rcar-csi2: Implement has_route()
  2019-03-05 18:51 ` [PATCH v3 31/31] media: rcar-csi2: Implement has_route() Jacopo Mondi
@ 2019-03-07 12:56   ` Sakari Ailus
  2019-03-07 22:28     ` Jacopo Mondi
  0 siblings, 1 reply; 59+ messages in thread
From: Sakari Ailus @ 2019-03-07 12:56 UTC (permalink / raw)
  To: Jacopo Mondi
  Cc: laurent.pinchart, niklas.soderlund+renesas, linux-media,
	linux-renesas-soc

Hi Jacopo,

On Tue, Mar 05, 2019 at 07:51:50PM +0100, Jacopo Mondi wrote:
> Now that the rcar-csi2 subdevice supports internal routing, add an
> has_route() operation used during graph traversal.
> 
> The internal routing between the sink and the source pads depends on the
> virtual channel used to transmit the video stream from the remote
> subdevice to the R-Car CSI-2 receiver.
> 
> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> ---
>  drivers/media/platform/rcar-vin/rcar-csi2.c | 35 +++++++++++++++++++++
>  1 file changed, 35 insertions(+)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-csi2.c b/drivers/media/platform/rcar-vin/rcar-csi2.c
> index cc7077b40f18..6c46bcc0ee83 100644
> --- a/drivers/media/platform/rcar-vin/rcar-csi2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-csi2.c
> @@ -1028,7 +1028,42 @@ static int rcsi2_confirm_start_v3m_e3(struct rcar_csi2 *priv)
>   * Platform Device Driver.
>   */
>  
> +static bool rcar_csi2_has_route(struct media_entity *entity,
> +				unsigned int pad0, unsigned int pad1)
> +{
> +	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
> +	struct rcar_csi2 *priv = sd_to_csi2(sd);
> +	struct v4l2_mbus_frame_desc fd;
> +	unsigned int i;
> +	int ret;
> +
> +	/* Support only direct sink->source routes. */
> +	if (pad0 != RCAR_CSI2_SINK)
> +		return false;
> +
> +	/* Get the frame description: from CSI-2 VC to source pad number. */
> +	ret = rcsi2_get_remote_frame_desc(priv, &fd);
> +	if (ret)
> +		return ret;
> +
> +	for (i = 0; i < fd.num_entries; i++) {
> +		struct v4l2_mbus_frame_desc_entry *entry = &fd.entry[i];
> +		int source_pad = rcsi2_vc_to_pad(entry->bus.csi2.channel);

A newline here would make this prettier IMO.

> +		if (source_pad < 0) {
> +			dev_err(priv->dev, "Virtual Channel out of range: %u\n",
> +				entry->bus.csi2.channel);
> +			return -EINVAL;

-EINVAL will be cast as true... same above.

This op wasn't really intended to fail. That should instead happen in e.g.
link or route configuration.

I think that if the two endpoints are in an agreement on the fundamental
CSI-2 bus parameters, I'd expect this just to work. Or is your CSI-2
receiver restricted to fewer virtual channels?

Alternatively we could make the frame descriptors settable as well so that
the receiver driver could use them to configure the transmitter. That'd
probably not be trivial to implement though.

> +		}
> +
> +		if (source_pad == pad1)
> +			return true;
> +	}
> +
> +	return false;
> +}
> +
>  static const struct media_entity_operations rcar_csi2_entity_ops = {
> +	.has_route = rcar_csi2_has_route,
>  	.link_validate = v4l2_subdev_link_validate,
>  };
>  

-- 
Regards,

Sakari Ailus
sakari.ailus@linux.intel.com

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 19/31] media: Documentation: Add GS_ROUTING documentation
  2019-03-05 18:51 ` [PATCH v3 19/31] media: Documentation: Add GS_ROUTING documentation Jacopo Mondi
@ 2019-03-07 15:19   ` Sakari Ailus
  2019-03-08 13:31     ` Jacopo Mondi
  2019-03-14 14:44     ` Luca Ceresoli
  2019-03-14 14:43   ` Luca Ceresoli
  2020-02-13 13:36   ` Hans Verkuil
  2 siblings, 2 replies; 59+ messages in thread
From: Sakari Ailus @ 2019-03-07 15:19 UTC (permalink / raw)
  To: Jacopo Mondi
  Cc: laurent.pinchart, niklas.soderlund+renesas, linux-media,
	linux-renesas-soc

Hi Jacopo,

Thanks for writing the documentation for this!

The text is nice and informative; I have a few suggestions below.

On Tue, Mar 05, 2019 at 07:51:38PM +0100, Jacopo Mondi wrote:
> Add documentation for VIDIOC_SUBDEV_G/S_ROUTING ioctl and add
> description of multiplexed media pads and internal routing to the
> V4L2-subdev documentation section.
> 
> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> ---
>  Documentation/media/uapi/v4l/dev-subdev.rst   |  90 +++++++++++
>  Documentation/media/uapi/v4l/user-func.rst    |   1 +
>  .../uapi/v4l/vidioc-subdev-g-routing.rst      | 142 ++++++++++++++++++
>  3 files changed, 233 insertions(+)
>  create mode 100644 Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst
> 
> diff --git a/Documentation/media/uapi/v4l/dev-subdev.rst b/Documentation/media/uapi/v4l/dev-subdev.rst
> index 2c2768c7343b..b9fbb5d2caec 100644
> --- a/Documentation/media/uapi/v4l/dev-subdev.rst
> +++ b/Documentation/media/uapi/v4l/dev-subdev.rst
> @@ -36,6 +36,8 @@ will feature a character device node on which ioctls can be called to
>  
>  -  negotiate image formats on individual pads
>  
> +-  inspect and modify internal data routing between pads of the same entity
> +
>  Sub-device character device nodes, conventionally named
>  ``/dev/v4l-subdev*``, use major number 81.
>  
> @@ -461,3 +463,91 @@ source pads.
>      :maxdepth: 1
>  
>      subdev-formats
> +
> +
> +Multiplexed media pads and internal routing
> +-------------------------------------------
> +
> +Subdevice drivers might expose the internal connections between media pads of an

s/might/may/

> +entity by providing a routing table that applications can inspect and manipulate

s/providing/exposing/

> +to change the internal routing between sink and source pads' data connection
> +endpoints. A routing table is described by a struct
> +:c:type:`v4l2_subdev_routing`, which contains ``num_routes`` route entries, each
> +one represented by a struct :c:type:`v4l2_subdev_route`.
> +
> +Data routes do not just connect one pad to another in an entity, but they refer
> +instead to the ``streams`` a media pad provides. Streams are data connection
> +endpoints in a media pad. Multiplexed media pads expose multiple ``streams``
> +which represent, when the underlying hardware technology allows that, logical
> +data streams transported over a single physical media bus.

How about s/streams/flows/ for this instance?

> +
> +One of the most notable examples of logical stream multiplexing techniques is

s/One of the most notable examples/A noteworthy example/

> +represented by the data interleaving mechanism implemented by mean of Virtual
> +Channels identifiers as defined by the MIPI CSI-2 media bus specifications. A

s/identifiers //

> +subdevice that implements support for Virtual Channel data interleaving might
> +expose up to 4 data ``streams``, one for each available Virtual Channel, on the
> +source media pad that represents a CSI-2 connection endpoint.
> +
> +Routes are defined as potential data connections between a ``(sink_pad,
> +sink_stream)`` pair and a ``(source_pad, source_stream)`` one, where
> +``sink_pad`` and ``source_pad`` are the indexes of two media pads part of the
> +same media entity, and ``sink_stream`` and ``source_stream`` are the identifiers
> +of the data streams to be connected in the media pads. Media pads that do not
> +support data multiplexing expose a single stream, usually with identifier 0.
> +
> +Routes are reported to applications in a routing table which can be
> +inspected and manipulated using the :ref:`routing <VIDIOC_SUBDEV_G_ROUTING>`
> +ioctls.
> +
> +Routes can be activated and deactivated by setting or clearing the
> +V4L2_SUBDEV_ROUTE_FL_ACTIVE flag in the ``flags`` field of struct
> +:c:type:`v4l2_subdev_route`.
> +
> +Subdev driver might create routes that cannot be modified by applications. Such

s/S/A s/

> +routes are identified by the presence of the V4L2_SUBDEV_ROUTE_FL_IMMUTABLE
> +flag in the ``flags`` field of struct :c:type:`v4l2_subdev_route`.
> +
> +As an example, the routing table of a source pad which supports two logical
> +video streams and can be connected to two sink pads is here below described.
> +
> +.. flat-table::
> +    :widths:       1 2 1
> +
> +    * - Pad Index
> +      - Function
> +      - Number of streams
> +    * - 0
> +      - SINK
> +      - 1
> +    * - 1
> +      - SINK
> +      - 1
> +    * - 2
> +      - SOURCE
> +      - 2
> +
> +In such an example, the source media pad will report a routing table with 4
> +entries, one entry for each possible ``(sink_pad, sink_stream) - (source_pad,
> +source_stream)`` combination.
> +
> +.. flat-table:: routing table
> +    :widths:       2 1 2
> +
> +    * - Sink Pad/Sink Stream
> +      - ->
> +      - Source Pad/Source Stream
> +    * - 0/0
> +      - ->
> +      - 2/0
> +    * - 0/0
> +      - ->
> +      - 2/1
> +    * - 1/0
> +      - ->
> +      - 2/0
> +    * - 1/0
> +      - ->
> +      - 2/1
> +
> +Subdev drivers are free to decide how many routes an application can enable on
> +a media pad at the same time, and refuse to enable or disable specific routes.
> diff --git a/Documentation/media/uapi/v4l/user-func.rst b/Documentation/media/uapi/v4l/user-func.rst
> index ca0ef21d77fe..0166446f4ab4 100644
> --- a/Documentation/media/uapi/v4l/user-func.rst
> +++ b/Documentation/media/uapi/v4l/user-func.rst
> @@ -77,6 +77,7 @@ Function Reference
>      vidioc-subdev-g-crop
>      vidioc-subdev-g-fmt
>      vidioc-subdev-g-frame-interval
> +    vidioc-subdev-g-routing
>      vidioc-subdev-g-selection
>      vidioc-subscribe-event
>      func-mmap
> diff --git a/Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst b/Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst
> new file mode 100644
> index 000000000000..8b592722c477
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst
> @@ -0,0 +1,142 @@
> +.. Permission is granted to copy, distribute and/or modify this
> +.. document under the terms of the GNU Free Documentation License,
> +.. Version 1.1 or any later version published by the Free Software
> +.. Foundation, with no Invariant Sections, no Front-Cover Texts
> +.. and no Back-Cover Texts. A copy of the license is included at
> +.. Documentation/media/uapi/fdl-appendix.rst.
> +..
> +.. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections
> +
> +.. _VIDIOC_SUBDEV_G_ROUTING:
> +
> +******************************************************
> +ioctl VIDIOC_SUBDEV_G_ROUTING, VIDIOC_SUBDEV_S_ROUTING
> +******************************************************
> +
> +Name
> +====
> +
> +VIDIOC_SUBDEV_G_ROUTING - VIDIOC_SUBDEV_S_ROUTING - Get or set routing between streams of media pads in a media entity.
> +
> +
> +Synopsis
> +========
> +
> +.. c:function:: int ioctl( int fd, VIDIOC_SUBDEV_G_ROUTING, struct v4l2_subdev_routing *argp )
> +    :name: VIDIOC_SUBDEV_G_ROUTING
> +
> +.. c:function:: int ioctl( int fd, VIDIOC_SUBDEV_S_ROUTING, struct v4l2_subdev_routing *argp )
> +    :name: VIDIOC_SUBDEV_S_ROUTING
> +
> +
> +Arguments
> +=========
> +
> +``fd``
> +    File descriptor returned by :ref:`open() <func-open>`.
> +
> +``argp``
> +    Pointer to struct :c:type:`v4l2_subdev_routing`.
> +
> +
> +Description
> +===========
> +
> +These ioctls are used to get and set the routing informations associated to
> +media pads in a media entity. Routing informations are used to enable or disable

The routing is a property of an entity. How about

s/the routing informations associated to media pads in/routing/

s/R[^\.]+(?=\.)/The routing configuration determines the flows of data
inside an entity/

> +data connections between stream endpoints of multiplexed media pads.
> +
> +Drivers report their routing tables using VIDIOC_SUBDEV_G_ROUTING and
> +application use the information there reported to enable or disable data
> +connections between streams in a pad, by setting or clearing the

How about:

s/applications\K.*a pad/may enable or disable routes with the
VIDIOC_SUBDEV_S_ROUTING IOCTL,

> +V4L2_SUBDEV_ROUTE_FL_ACTIVE flag of ``flags`` field of a struct
> +:c:type:`v4l2_subdev_route`.
> +
> +When inspecting routes through VIDIOC_SUBDEV_G_ROUTING and the application
> +provided ``num_routes`` is not big enough to contain all the available routes
> +the subdevice exposes, drivers return the ENOSPC error code and adjust the
> +``num_routes`` value. Application should then reserve enough memory for all the

s/the\K$/value of the/
s/value\K\./field/

> +route entries and call VIDIOC_SUBDEV_G_ROUTING again.
> +
> +.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}|
> +
> +.. c:type:: v4l2_subdev_routing
> +
> +.. flat-table:: struct v4l2_subdev_routing
> +    :header-rows:  0
> +    :stub-columns: 0
> +    :widths:       1 1 2
> +
> +    * - struct :c:type:`v4l2_subdev_route`
> +      - ``routes[]``
> +      - Array of struct :c:type:`v4l2_subdev_route` entries
> +    * - __u32
> +      - ``num_routes``
> +      - Number of entries of the routes array
> +    * - __u32
> +      - ``reserved``\ [5]
> +      - Reserved for future extensions. Applications and drivers must set
> +	the array to zero.
> +
> +.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}|
> +
> +.. c:type:: v4l2_subdev_route
> +
> +.. flat-table:: struct v4l2_subdev_route
> +    :header-rows:  0
> +    :stub-columns: 0
> +    :widths:       1 1 2
> +
> +    * - __u32
> +      - ``sink_pad``
> +      - Sink pad number
> +    * - __u32
> +      - ``sink_stream``
> +      - Sink pad stream number
> +    * - __u32
> +      - ``source_stream``
> +      - Source pad stream number
> +    * - __u32
> +      - ``sink_stream``
> +      - Sink pad stream number
> +    * - __u32
> +      - ``flags``
> +      - Route enable/disable flags
> +	:ref:`v4l2_subdev_routing_flags <v4l2-subdev-routing-flags>`.
> +    * - __u32
> +      - ``reserved``\ [5]
> +      - Reserved for future extensions. Applications and drivers must set
> +	the array to zero.
> +
> +.. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
> +
> +.. _v4l2-subdev-routing-flags:
> +
> +.. flat-table:: enum v4l2_subdev_routing_flags
> +    :header-rows:  0
> +    :stub-columns: 0
> +    :widths:       3 1 4
> +
> +    * - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> +      - 0
> +      - The route is enabled. Set by applications.
> +    * - V4L2_SUBDEV_ROUTE_FL_IMMUTABLE
> +      - 1
> +      - The route is immutable. Set by the driver.
> +
> +Return Value
> +============
> +
> +On success 0 is returned, on error -1 and the ``errno`` variable is set
> +appropriately. The generic error codes are described at the
> +:ref:`Generic Error Codes <gen-errors>` chapter.
> +
> +ENOSPC
> +   The number of provided route entries is less than the available ones.
> +
> +EINVAL
> +   The sink or source pad identifiers reference a non-existing pad, or refernce
> +   pads of different types (ie. the sink_pad identifiers refers to a source pad)
> +   The sink or source stream identifiers reference a non-existing stream

s/The/or the/

> +   in the sink or source pad.

s/i/o/

> +

-- 
Kind regards,

Sakari Ailus
sakari.ailus@linux.intel.com

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 12/31] media: entity: Add an iterator helper for connected pads
  2019-03-07 10:27     ` Ian Arkver
  2019-03-07 12:38       ` Sakari Ailus
@ 2019-03-07 20:18       ` Jacopo Mondi
  1 sibling, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-07 20:18 UTC (permalink / raw)
  To: Ian Arkver
  Cc: Sakari Ailus, Jacopo Mondi, laurent.pinchart,
	niklas.soderlund+renesas, linux-media, linux-renesas-soc

[-- Attachment #1: Type: text/plain, Size: 3016 bytes --]

Hi Ian, Sakari,

On Thu, Mar 07, 2019 at 10:27:36AM +0000, Ian Arkver wrote:
> On 07/03/2019 10:09, Sakari Ailus wrote:
> > Hi Jacopo,
> >
> > On Tue, Mar 05, 2019 at 07:51:31PM +0100, Jacopo Mondi wrote:
> > > From: Sakari Ailus <sakari.ailus@linux.intel.com>
> > >
> > > Add a helper macro for iterating over pads that are connected through
> > > enabled routes. This can be used to find all the connected pads within an
> > > entity, for instance starting from the pad which has been obtained during
> > > the graph walk.
> > >
> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > >
> > > - Make __media_entity_next_routed_pad() return NULL and adjust the
> > >    iterator to handle that
> > > Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> > > ---
> > >   include/media/media-entity.h | 27 +++++++++++++++++++++++++++
> > >   1 file changed, 27 insertions(+)
> > >
> > > diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> > > index 205561545d7e..82f0bdf2a6d1 100644
> > > --- a/include/media/media-entity.h
> > > +++ b/include/media/media-entity.h
> > > @@ -936,6 +936,33 @@ __must_check int media_graph_walk_init(
> > >   bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
> > >   			    unsigned int pad1);
> > > +static inline struct media_pad *__media_entity_next_routed_pad(
> > > +	struct media_pad *start, struct media_pad *iter)
> > > +{
> > > +	struct media_entity *entity = start->entity;
> > > +
> > > +	while (iter < &entity->pads[entity->num_pads] &&
> > > +	       !media_entity_has_route(entity, start->index, iter->index))
> > > +		iter++;
> > > +
> > > +	return iter == &entity->pads[entity->num_pads] ? NULL : iter;
> >
> > Could you use iter <= ...?
> >
> > It doesn't seem to matter here, but it'd seem safer to change the check.
> >
>
> How about something like...
>
> for (; iter < &entity->pads[entity->num_pads]; iter++)
>     if (media_entity_has_route(entity, start->index, iter->index))
>         return iter;
>
> return NULL;
>

Great, thank you both, I'll take this in!


> Regards,
> Ian
>
> > > +}
> > > +
> > > +/**
> > > + * media_entity_for_each_routed_pad - Iterate over entity pads connected by routes
> > > + *
> > > + * @start: The stating pad
> > > + * @iter: The iterator pad
> > > + *
> > > + * Iterate over all pads connected through routes from a given pad
> > > + * within an entity. The iteration will include the starting pad itself.
> > > + */
> > > +#define media_entity_for_each_routed_pad(start, iter)			\
> > > +	for (iter = __media_entity_next_routed_pad(			\
> > > +		     start, (start)->entity->pads);			\
> > > +	     iter != NULL;						\
> > > +	     iter = __media_entity_next_routed_pad(start, iter + 1))
> > > +
> > >   /**
> > >    * media_graph_walk_cleanup - Release resources used by graph walk.
> > >    *
> >

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 31/31] media: rcar-csi2: Implement has_route()
  2019-03-07 12:56   ` Sakari Ailus
@ 2019-03-07 22:28     ` Jacopo Mondi
  0 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-07 22:28 UTC (permalink / raw)
  To: Sakari Ailus
  Cc: Jacopo Mondi, laurent.pinchart, niklas.soderlund+renesas,
	linux-media, linux-renesas-soc

[-- Attachment #1: Type: text/plain, Size: 4812 bytes --]

Hi Sakari,

On Thu, Mar 07, 2019 at 02:56:10PM +0200, Sakari Ailus wrote:
> Hi Jacopo,
>
> On Tue, Mar 05, 2019 at 07:51:50PM +0100, Jacopo Mondi wrote:
> > Now that the rcar-csi2 subdevice supports internal routing, add an
> > has_route() operation used during graph traversal.
> >
> > The internal routing between the sink and the source pads depends on the
> > virtual channel used to transmit the video stream from the remote
> > subdevice to the R-Car CSI-2 receiver.
> >
> > Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> > ---
> >  drivers/media/platform/rcar-vin/rcar-csi2.c | 35 +++++++++++++++++++++
> >  1 file changed, 35 insertions(+)
> >
> > diff --git a/drivers/media/platform/rcar-vin/rcar-csi2.c b/drivers/media/platform/rcar-vin/rcar-csi2.c
> > index cc7077b40f18..6c46bcc0ee83 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-csi2.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-csi2.c
> > @@ -1028,7 +1028,42 @@ static int rcsi2_confirm_start_v3m_e3(struct rcar_csi2 *priv)
> >   * Platform Device Driver.
> >   */
> >
> > +static bool rcar_csi2_has_route(struct media_entity *entity,
> > +				unsigned int pad0, unsigned int pad1)
> > +{
> > +	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
> > +	struct rcar_csi2 *priv = sd_to_csi2(sd);
> > +	struct v4l2_mbus_frame_desc fd;
> > +	unsigned int i;
> > +	int ret;
> > +
> > +	/* Support only direct sink->source routes. */
> > +	if (pad0 != RCAR_CSI2_SINK)
> > +		return false;
> > +
> > +	/* Get the frame description: from CSI-2 VC to source pad number. */
> > +	ret = rcsi2_get_remote_frame_desc(priv, &fd);
> > +	if (ret)
> > +		return ret;
> > +
> > +	for (i = 0; i < fd.num_entries; i++) {
> > +		struct v4l2_mbus_frame_desc_entry *entry = &fd.entry[i];
> > +		int source_pad = rcsi2_vc_to_pad(entry->bus.csi2.channel);
>
> A newline here would make this prettier IMO.
>

Yes, it's probably nicer (and actually reported by checkpatch...)

> > +		if (source_pad < 0) {
> > +			dev_err(priv->dev, "Virtual Channel out of range: %u\n",
> > +				entry->bus.csi2.channel);
> > +			return -EINVAL;
>
> -EINVAL will be cast as true... same above.
>

Argh, dumb! Thank you

> This op wasn't really intended to fail. That should instead happen in e.g.
> link or route configuration.
>
> I think that if the two endpoints are in an agreement on the fundamental
> CSI-2 bus parameters, I'd expect this just to work. Or is your CSI-2
> receiver restricted to fewer virtual channels?

Not exactly. The CSI-2 receivers are connected to the VIN instances
through 4 data channels and there is a number of valid
"(CSI-2/VC) -> VIN" routes which are captured in the per-SoC
"rvin_group_route" structure.

Each source pad of the CSI-2 receiver represent an output channel, and the
media links to the VINs represent a possible valid route. So far, without
VC support, all 4 outputs where enabled, and the data was output to
all the connected VINs.

With the introduction of virtual channels support for the R-Car CSI-2
receiver in Niklas' [29/31], the video stream to VINs is now only output
to the channel corresponding to the VC reported by the remote's frame
descriptor (in this setup, the one reported on the adv748x source pad).
This work as streaming VC(i) on Channel(i) (which corresponds to pad(i+1))
is always a safe combination as long as one receiver is use (*) and
this is reflected by this operation, that reports that the stream received
on the sink pad is now internally routed only to the source pad that
corresponds to the enabled output channel.

>
> Alternatively we could make the frame descriptors settable as well so that
> the receiver driver could use them to configure the transmitter. That'd
> probably not be trivial to implement though.
>

I think that might be something to be considered in future, as
controlling the VC from the transmitter makes it very hard for VINs to
validate combinations when multiple receivers are active at the same
time, while setting routes on the VINs and propagating them backward
would centralize the validation of the VIN/CSI-2/VC combinations.

That's something quite far-fetched though, and I might got lost a bit,
so I think what is here at the moment is fine :)

Thanks
  j

*) Niklas: am I wrong or things might get interesting when
more than one CSI-2 receiver is enabled simultaneously and if I got
this right, this might lead to, ie. having both CSI40/VC0 and
CSI20/VC0 sent to the same VIN0 only?

> > +		}
> > +
> > +		if (source_pad == pad1)
> > +			return true;
> > +	}
> > +
> > +	return false;
> > +}
> > +
> >  static const struct media_entity_operations rcar_csi2_entity_ops = {
> > +	.has_route = rcar_csi2_has_route,
> >  	.link_validate = v4l2_subdev_link_validate,
> >  };
> >
>
> --
> Regards,
>
> Sakari Ailus
> sakari.ailus@linux.intel.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 00/31] v4l: add support for multiplexed streams
  2019-03-07  9:47 ` [PATCH v3 00/31] v4l: add support for multiplexed streams Sakari Ailus
@ 2019-03-08 13:19   ` Jacopo Mondi
  2019-03-08 14:12     ` Sakari Ailus
  0 siblings, 1 reply; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-08 13:19 UTC (permalink / raw)
  To: Sakari Ailus, niklas soderlund
  Cc: Jacopo Mondi, laurent.pinchart, niklas.soderlund+renesas,
	linux-media, linux-renesas-soc

[-- Attachment #1: Type: text/plain, Size: 2409 bytes --]

Hi Sakari and Niklas,

On Thu, Mar 07, 2019 at 11:47:26AM +0200, Sakari Ailus wrote:
> Hi Jacopo,
>
> On Tue, Mar 05, 2019 at 07:51:19PM +0100, Jacopo Mondi wrote:
> > Hello,
> >    third version of multiplexed stream support patch series.
> >
> > V2 sent by Niklas is available at:
> > https://patchwork.kernel.org/cover/10573817/
> >
> > As per v2, most of the core patches are work from Sakari and Laurent, with
> > Niklas' support on top for adv748x and rcar-csi2.
> >
> > The use case of the series remains the same: support for virtual channel
> > selection implemented on R-Car Gen3 and adv748x. Quoting the v2 cover letter:
> >
> > -------------------------------------------------------------------------------
> > I have added driver support for the devices used on the Renesas Gen3
> > platforms, a ADV7482 connected to the R-Car CSI-2 receiver. With these
> > changes I can control which of the analog inputs of the ADV7482 the
> > video source is captured from and on which CSI-2 virtual channel the
> > video is transmitted on to the R-Car CSI-2 receiver.
> >
> > The series adds two new subdev IOCTLs [GS]_ROUTING which allows
> > user-space to get and set routes inside a subdevice. I have added RFC
> > support for these to v4l-utils [2] which can be used to test this
> > series, example:
> >
> >     Check the internal routing of the adv748x csi-2 transmitter:
> >     v4l2-ctl -d /dev/v4l-subdev24 --get-routing
> >     0/0 -> 1/0 [ENABLED]
> >     0/0 -> 1/1 []
> >     0/0 -> 1/2 []
> >     0/0 -> 1/3 []
> >
> >
> >     Select that video should be outputed on VC 2 and check the result:
> >     $ v4l2-ctl -d /dev/v4l-subdev24 --set-routing '0/0 -> 1/2 [1]'
> >
> >     $ v4l2-ctl -d /dev/v4l-subdev24 --get-routing
> >     0/0 -> 1/0 []
> >     0/0 -> 1/1 []
> >     0/0 -> 1/2 [ENABLED]
> >     0/0 -> 1/3 []
> > -------------------------------------------------------------------------------
> >
> > Below is reported the media graph of the system used for testing [1].
> >
> > v4l2-ctl patches to handle the newly introduced IOCTLs are available from
> > Niklas' repository at:
> > git://git.ragnatech.se/v4l-utils routing
>
> Could you send the v4l2-ctl patches out as well, please?
>

Niklas sent them on late 2017... time flies :)
https://patchwork.kernel.org/patch/10113189/

Would you like to have them re-sent?

Thanks
   j

> --
> Sakari Ailus
> sakari.ailus@linux.intel.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 19/31] media: Documentation: Add GS_ROUTING documentation
  2019-03-07 15:19   ` Sakari Ailus
@ 2019-03-08 13:31     ` Jacopo Mondi
  2019-03-08 14:14       ` Sakari Ailus
  2019-03-14 14:44     ` Luca Ceresoli
  1 sibling, 1 reply; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-08 13:31 UTC (permalink / raw)
  To: Sakari Ailus
  Cc: Jacopo Mondi, laurent.pinchart, niklas.soderlund+renesas,
	linux-media, linux-renesas-soc

[-- Attachment #1: Type: text/plain, Size: 12176 bytes --]

Hi Sakari,
   thanks for the review

On Thu, Mar 07, 2019 at 05:19:29PM +0200, Sakari Ailus wrote:
> Hi Jacopo,
>
> Thanks for writing the documentation for this!
>
> The text is nice and informative; I have a few suggestions below.
>
> On Tue, Mar 05, 2019 at 07:51:38PM +0100, Jacopo Mondi wrote:
> > Add documentation for VIDIOC_SUBDEV_G/S_ROUTING ioctl and add
> > description of multiplexed media pads and internal routing to the
> > V4L2-subdev documentation section.
> >
> > Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> > ---
> >  Documentation/media/uapi/v4l/dev-subdev.rst   |  90 +++++++++++
> >  Documentation/media/uapi/v4l/user-func.rst    |   1 +
> >  .../uapi/v4l/vidioc-subdev-g-routing.rst      | 142 ++++++++++++++++++
> >  3 files changed, 233 insertions(+)
> >  create mode 100644 Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst
> >
> > diff --git a/Documentation/media/uapi/v4l/dev-subdev.rst b/Documentation/media/uapi/v4l/dev-subdev.rst
> > index 2c2768c7343b..b9fbb5d2caec 100644
> > --- a/Documentation/media/uapi/v4l/dev-subdev.rst
> > +++ b/Documentation/media/uapi/v4l/dev-subdev.rst
> > @@ -36,6 +36,8 @@ will feature a character device node on which ioctls can be called to
> >
> >  -  negotiate image formats on individual pads
> >
> > +-  inspect and modify internal data routing between pads of the same entity
> > +
> >  Sub-device character device nodes, conventionally named
> >  ``/dev/v4l-subdev*``, use major number 81.
> >
> > @@ -461,3 +463,91 @@ source pads.
> >      :maxdepth: 1
> >
> >      subdev-formats
> > +
> > +
> > +Multiplexed media pads and internal routing
> > +-------------------------------------------
> > +
> > +Subdevice drivers might expose the internal connections between media pads of an
>
> s/might/may/
>
> > +entity by providing a routing table that applications can inspect and manipulate
>
> s/providing/exposing/
>

Ack on both the above suggestions

> > +to change the internal routing between sink and source pads' data connection
> > +endpoints. A routing table is described by a struct
> > +:c:type:`v4l2_subdev_routing`, which contains ``num_routes`` route entries, each
> > +one represented by a struct :c:type:`v4l2_subdev_route`.
> > +
> > +Data routes do not just connect one pad to another in an entity, but they refer
> > +instead to the ``streams`` a media pad provides. Streams are data connection
> > +endpoints in a media pad. Multiplexed media pads expose multiple ``streams``
> > +which represent, when the underlying hardware technology allows that, logical
> > +data streams transported over a single physical media bus.
>
> How about s/streams/flows/ for this instance?
>

Agreed, there are too many "streams" already in that paragraph

> > +
> > +One of the most notable examples of logical stream multiplexing techniques is
>
> s/One of the most notable examples/A noteworthy example/
>
> > +represented by the data interleaving mechanism implemented by mean of Virtual
> > +Channels identifiers as defined by the MIPI CSI-2 media bus specifications. A
>
> s/identifiers //
>

Ack on both the above suggestions

> > +subdevice that implements support for Virtual Channel data interleaving might
> > +expose up to 4 data ``streams``, one for each available Virtual Channel, on the
> > +source media pad that represents a CSI-2 connection endpoint.
> > +
> > +Routes are defined as potential data connections between a ``(sink_pad,
> > +sink_stream)`` pair and a ``(source_pad, source_stream)`` one, where
> > +``sink_pad`` and ``source_pad`` are the indexes of two media pads part of the
> > +same media entity, and ``sink_stream`` and ``source_stream`` are the identifiers
> > +of the data streams to be connected in the media pads. Media pads that do not
> > +support data multiplexing expose a single stream, usually with identifier 0.
> > +
> > +Routes are reported to applications in a routing table which can be
> > +inspected and manipulated using the :ref:`routing <VIDIOC_SUBDEV_G_ROUTING>`
> > +ioctls.
> > +
> > +Routes can be activated and deactivated by setting or clearing the
> > +V4L2_SUBDEV_ROUTE_FL_ACTIVE flag in the ``flags`` field of struct
> > +:c:type:`v4l2_subdev_route`.
> > +
> > +Subdev driver might create routes that cannot be modified by applications. Such
>
> s/S/A s/

Or "Subdevice drivers" ?

>
> > +routes are identified by the presence of the V4L2_SUBDEV_ROUTE_FL_IMMUTABLE
> > +flag in the ``flags`` field of struct :c:type:`v4l2_subdev_route`.
> > +
> > +As an example, the routing table of a source pad which supports two logical
> > +video streams and can be connected to two sink pads is here below described.
> > +
> > +.. flat-table::
> > +    :widths:       1 2 1
> > +
> > +    * - Pad Index
> > +      - Function
> > +      - Number of streams
> > +    * - 0
> > +      - SINK
> > +      - 1
> > +    * - 1
> > +      - SINK
> > +      - 1
> > +    * - 2
> > +      - SOURCE
> > +      - 2
> > +
> > +In such an example, the source media pad will report a routing table with 4
> > +entries, one entry for each possible ``(sink_pad, sink_stream) - (source_pad,
> > +source_stream)`` combination.
> > +
> > +.. flat-table:: routing table
> > +    :widths:       2 1 2
> > +
> > +    * - Sink Pad/Sink Stream
> > +      - ->
> > +      - Source Pad/Source Stream
> > +    * - 0/0
> > +      - ->
> > +      - 2/0
> > +    * - 0/0
> > +      - ->
> > +      - 2/1
> > +    * - 1/0
> > +      - ->
> > +      - 2/0
> > +    * - 1/0
> > +      - ->
> > +      - 2/1
> > +
> > +Subdev drivers are free to decide how many routes an application can enable on
> > +a media pad at the same time, and refuse to enable or disable specific routes.
> > diff --git a/Documentation/media/uapi/v4l/user-func.rst b/Documentation/media/uapi/v4l/user-func.rst
> > index ca0ef21d77fe..0166446f4ab4 100644
> > --- a/Documentation/media/uapi/v4l/user-func.rst
> > +++ b/Documentation/media/uapi/v4l/user-func.rst
> > @@ -77,6 +77,7 @@ Function Reference
> >      vidioc-subdev-g-crop
> >      vidioc-subdev-g-fmt
> >      vidioc-subdev-g-frame-interval
> > +    vidioc-subdev-g-routing
> >      vidioc-subdev-g-selection
> >      vidioc-subscribe-event
> >      func-mmap
> > diff --git a/Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst b/Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst
> > new file mode 100644
> > index 000000000000..8b592722c477
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst
> > @@ -0,0 +1,142 @@
> > +.. Permission is granted to copy, distribute and/or modify this
> > +.. document under the terms of the GNU Free Documentation License,
> > +.. Version 1.1 or any later version published by the Free Software
> > +.. Foundation, with no Invariant Sections, no Front-Cover Texts
> > +.. and no Back-Cover Texts. A copy of the license is included at
> > +.. Documentation/media/uapi/fdl-appendix.rst.
> > +..
> > +.. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections
> > +
> > +.. _VIDIOC_SUBDEV_G_ROUTING:
> > +
> > +******************************************************
> > +ioctl VIDIOC_SUBDEV_G_ROUTING, VIDIOC_SUBDEV_S_ROUTING
> > +******************************************************
> > +
> > +Name
> > +====
> > +
> > +VIDIOC_SUBDEV_G_ROUTING - VIDIOC_SUBDEV_S_ROUTING - Get or set routing between streams of media pads in a media entity.
> > +
> > +
> > +Synopsis
> > +========
> > +
> > +.. c:function:: int ioctl( int fd, VIDIOC_SUBDEV_G_ROUTING, struct v4l2_subdev_routing *argp )
> > +    :name: VIDIOC_SUBDEV_G_ROUTING
> > +
> > +.. c:function:: int ioctl( int fd, VIDIOC_SUBDEV_S_ROUTING, struct v4l2_subdev_routing *argp )
> > +    :name: VIDIOC_SUBDEV_S_ROUTING
> > +
> > +
> > +Arguments
> > +=========
> > +
> > +``fd``
> > +    File descriptor returned by :ref:`open() <func-open>`.
> > +
> > +``argp``
> > +    Pointer to struct :c:type:`v4l2_subdev_routing`.
> > +
> > +
> > +Description
> > +===========
> > +
> > +These ioctls are used to get and set the routing informations associated to
> > +media pads in a media entity. Routing informations are used to enable or disable
>
> The routing is a property of an entity. How about
>
> s/the routing informations associated to media pads in/routing/
>
> s/R[^\.]+(?=\.)/The routing configuration determines the flows of data
> inside an entity/

Thanks, you made me discover regexr.com here
>
> > +data connections between stream endpoints of multiplexed media pads.
> > +
> > +Drivers report their routing tables using VIDIOC_SUBDEV_G_ROUTING and
> > +application use the information there reported to enable or disable data
> > +connections between streams in a pad, by setting or clearing the
>
> How about:
>
> s/applications\K.*a pad/may enable or disable routes with the
> VIDIOC_SUBDEV_S_ROUTING IOCTL,
>

Yes, reads better

> > +V4L2_SUBDEV_ROUTE_FL_ACTIVE flag of ``flags`` field of a struct
> > +:c:type:`v4l2_subdev_route`.
> > +
> > +When inspecting routes through VIDIOC_SUBDEV_G_ROUTING and the application
> > +provided ``num_routes`` is not big enough to contain all the available routes
> > +the subdevice exposes, drivers return the ENOSPC error code and adjust the
> > +``num_routes`` value. Application should then reserve enough memory for all the
>
> s/the\K$/value of the/
> s/value\K\./field/
>

Will update

> > +route entries and call VIDIOC_SUBDEV_G_ROUTING again.
> > +
> > +.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}|
> > +
> > +.. c:type:: v4l2_subdev_routing
> > +
> > +.. flat-table:: struct v4l2_subdev_routing
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +    :widths:       1 1 2
> > +
> > +    * - struct :c:type:`v4l2_subdev_route`
> > +      - ``routes[]``
> > +      - Array of struct :c:type:`v4l2_subdev_route` entries
> > +    * - __u32
> > +      - ``num_routes``
> > +      - Number of entries of the routes array
> > +    * - __u32
> > +      - ``reserved``\ [5]
> > +      - Reserved for future extensions. Applications and drivers must set
> > +	the array to zero.
> > +
> > +.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}|
> > +
> > +.. c:type:: v4l2_subdev_route
> > +
> > +.. flat-table:: struct v4l2_subdev_route
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +    :widths:       1 1 2
> > +
> > +    * - __u32
> > +      - ``sink_pad``
> > +      - Sink pad number
> > +    * - __u32
> > +      - ``sink_stream``
> > +      - Sink pad stream number
> > +    * - __u32
> > +      - ``source_stream``
> > +      - Source pad stream number
> > +    * - __u32
> > +      - ``sink_stream``
> > +      - Sink pad stream number
> > +    * - __u32
> > +      - ``flags``
> > +      - Route enable/disable flags
> > +	:ref:`v4l2_subdev_routing_flags <v4l2-subdev-routing-flags>`.
> > +    * - __u32
> > +      - ``reserved``\ [5]
> > +      - Reserved for future extensions. Applications and drivers must set
> > +	the array to zero.
> > +
> > +.. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
> > +
> > +.. _v4l2-subdev-routing-flags:
> > +
> > +.. flat-table:: enum v4l2_subdev_routing_flags
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +    :widths:       3 1 4
> > +
> > +    * - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> > +      - 0
> > +      - The route is enabled. Set by applications.
> > +    * - V4L2_SUBDEV_ROUTE_FL_IMMUTABLE
> > +      - 1
> > +      - The route is immutable. Set by the driver.
> > +
> > +Return Value
> > +============
> > +
> > +On success 0 is returned, on error -1 and the ``errno`` variable is set
> > +appropriately. The generic error codes are described at the
> > +:ref:`Generic Error Codes <gen-errors>` chapter.
> > +
> > +ENOSPC
> > +   The number of provided route entries is less than the available ones.
> > +
> > +EINVAL
> > +   The sink or source pad identifiers reference a non-existing pad, or refernce
> > +   pads of different types (ie. the sink_pad identifiers refers to a source pad)
> > +   The sink or source stream identifiers reference a non-existing stream
>
> s/The/or the/
>
> > +   in the sink or source pad.
>
> s/i/o/

Thanks, I will update all of these in v4.

>
> > +
>
> --
> Kind regards,
>
> Sakari Ailus
> sakari.ailus@linux.intel.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 00/31] v4l: add support for multiplexed streams
  2019-03-08 13:19   ` Jacopo Mondi
@ 2019-03-08 14:12     ` Sakari Ailus
  0 siblings, 0 replies; 59+ messages in thread
From: Sakari Ailus @ 2019-03-08 14:12 UTC (permalink / raw)
  To: Jacopo Mondi
  Cc: niklas soderlund, Jacopo Mondi, laurent.pinchart,
	niklas.soderlund+renesas, linux-media, linux-renesas-soc

On Fri, Mar 08, 2019 at 02:19:03PM +0100, Jacopo Mondi wrote:
> Hi Sakari and Niklas,
> 
> On Thu, Mar 07, 2019 at 11:47:26AM +0200, Sakari Ailus wrote:
> > Hi Jacopo,
> >
> > On Tue, Mar 05, 2019 at 07:51:19PM +0100, Jacopo Mondi wrote:
> > > Hello,
> > >    third version of multiplexed stream support patch series.
> > >
> > > V2 sent by Niklas is available at:
> > > https://patchwork.kernel.org/cover/10573817/
> > >
> > > As per v2, most of the core patches are work from Sakari and Laurent, with
> > > Niklas' support on top for adv748x and rcar-csi2.
> > >
> > > The use case of the series remains the same: support for virtual channel
> > > selection implemented on R-Car Gen3 and adv748x. Quoting the v2 cover letter:
> > >
> > > -------------------------------------------------------------------------------
> > > I have added driver support for the devices used on the Renesas Gen3
> > > platforms, a ADV7482 connected to the R-Car CSI-2 receiver. With these
> > > changes I can control which of the analog inputs of the ADV7482 the
> > > video source is captured from and on which CSI-2 virtual channel the
> > > video is transmitted on to the R-Car CSI-2 receiver.
> > >
> > > The series adds two new subdev IOCTLs [GS]_ROUTING which allows
> > > user-space to get and set routes inside a subdevice. I have added RFC
> > > support for these to v4l-utils [2] which can be used to test this
> > > series, example:
> > >
> > >     Check the internal routing of the adv748x csi-2 transmitter:
> > >     v4l2-ctl -d /dev/v4l-subdev24 --get-routing
> > >     0/0 -> 1/0 [ENABLED]
> > >     0/0 -> 1/1 []
> > >     0/0 -> 1/2 []
> > >     0/0 -> 1/3 []
> > >
> > >
> > >     Select that video should be outputed on VC 2 and check the result:
> > >     $ v4l2-ctl -d /dev/v4l-subdev24 --set-routing '0/0 -> 1/2 [1]'
> > >
> > >     $ v4l2-ctl -d /dev/v4l-subdev24 --get-routing
> > >     0/0 -> 1/0 []
> > >     0/0 -> 1/1 []
> > >     0/0 -> 1/2 [ENABLED]
> > >     0/0 -> 1/3 []
> > > -------------------------------------------------------------------------------
> > >
> > > Below is reported the media graph of the system used for testing [1].
> > >
> > > v4l2-ctl patches to handle the newly introduced IOCTLs are available from
> > > Niklas' repository at:
> > > git://git.ragnatech.se/v4l-utils routing
> >
> > Could you send the v4l2-ctl patches out as well, please?
> >
> 
> Niklas sent them on late 2017... time flies :)
> https://patchwork.kernel.org/patch/10113189/
> 
> Would you like to have them re-sent?

Oh, well, sure, if there are no changes, no need to resend. Thanks for the
pointer.

-- 
Sakari Ailus
sakari.ailus@linux.intel.com

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 19/31] media: Documentation: Add GS_ROUTING documentation
  2019-03-08 13:31     ` Jacopo Mondi
@ 2019-03-08 14:14       ` Sakari Ailus
  0 siblings, 0 replies; 59+ messages in thread
From: Sakari Ailus @ 2019-03-08 14:14 UTC (permalink / raw)
  To: Jacopo Mondi
  Cc: Jacopo Mondi, laurent.pinchart, niklas.soderlund+renesas,
	linux-media, linux-renesas-soc

Hi Jacopo,

On Fri, Mar 08, 2019 at 02:31:33PM +0100, Jacopo Mondi wrote:
> Hi Sakari,
>    thanks for the review
> 
> On Thu, Mar 07, 2019 at 05:19:29PM +0200, Sakari Ailus wrote:
> > Hi Jacopo,
> >
> > Thanks for writing the documentation for this!
> >
> > The text is nice and informative; I have a few suggestions below.
> >
> > On Tue, Mar 05, 2019 at 07:51:38PM +0100, Jacopo Mondi wrote:
> > > Add documentation for VIDIOC_SUBDEV_G/S_ROUTING ioctl and add
> > > description of multiplexed media pads and internal routing to the
> > > V4L2-subdev documentation section.
> > >
> > > Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> > > ---
> > >  Documentation/media/uapi/v4l/dev-subdev.rst   |  90 +++++++++++
> > >  Documentation/media/uapi/v4l/user-func.rst    |   1 +
> > >  .../uapi/v4l/vidioc-subdev-g-routing.rst      | 142 ++++++++++++++++++
> > >  3 files changed, 233 insertions(+)
> > >  create mode 100644 Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst
> > >
> > > diff --git a/Documentation/media/uapi/v4l/dev-subdev.rst b/Documentation/media/uapi/v4l/dev-subdev.rst
> > > index 2c2768c7343b..b9fbb5d2caec 100644
> > > --- a/Documentation/media/uapi/v4l/dev-subdev.rst
> > > +++ b/Documentation/media/uapi/v4l/dev-subdev.rst
> > > @@ -36,6 +36,8 @@ will feature a character device node on which ioctls can be called to
> > >
> > >  -  negotiate image formats on individual pads
> > >
> > > +-  inspect and modify internal data routing between pads of the same entity
> > > +
> > >  Sub-device character device nodes, conventionally named
> > >  ``/dev/v4l-subdev*``, use major number 81.
> > >
> > > @@ -461,3 +463,91 @@ source pads.
> > >      :maxdepth: 1
> > >
> > >      subdev-formats
> > > +
> > > +
> > > +Multiplexed media pads and internal routing
> > > +-------------------------------------------
> > > +
> > > +Subdevice drivers might expose the internal connections between media pads of an
> >
> > s/might/may/
> >
> > > +entity by providing a routing table that applications can inspect and manipulate
> >
> > s/providing/exposing/
> >
> 
> Ack on both the above suggestions
> 
> > > +to change the internal routing between sink and source pads' data connection
> > > +endpoints. A routing table is described by a struct
> > > +:c:type:`v4l2_subdev_routing`, which contains ``num_routes`` route entries, each
> > > +one represented by a struct :c:type:`v4l2_subdev_route`.
> > > +
> > > +Data routes do not just connect one pad to another in an entity, but they refer
> > > +instead to the ``streams`` a media pad provides. Streams are data connection
> > > +endpoints in a media pad. Multiplexed media pads expose multiple ``streams``
> > > +which represent, when the underlying hardware technology allows that, logical
> > > +data streams transported over a single physical media bus.
> >
> > How about s/streams/flows/ for this instance?
> >
> 
> Agreed, there are too many "streams" already in that paragraph
> 
> > > +
> > > +One of the most notable examples of logical stream multiplexing techniques is
> >
> > s/One of the most notable examples/A noteworthy example/
> >
> > > +represented by the data interleaving mechanism implemented by mean of Virtual
> > > +Channels identifiers as defined by the MIPI CSI-2 media bus specifications. A
> >
> > s/identifiers //
> >
> 
> Ack on both the above suggestions
> 
> > > +subdevice that implements support for Virtual Channel data interleaving might
> > > +expose up to 4 data ``streams``, one for each available Virtual Channel, on the
> > > +source media pad that represents a CSI-2 connection endpoint.
> > > +
> > > +Routes are defined as potential data connections between a ``(sink_pad,
> > > +sink_stream)`` pair and a ``(source_pad, source_stream)`` one, where
> > > +``sink_pad`` and ``source_pad`` are the indexes of two media pads part of the
> > > +same media entity, and ``sink_stream`` and ``source_stream`` are the identifiers
> > > +of the data streams to be connected in the media pads. Media pads that do not
> > > +support data multiplexing expose a single stream, usually with identifier 0.
> > > +
> > > +Routes are reported to applications in a routing table which can be
> > > +inspected and manipulated using the :ref:`routing <VIDIOC_SUBDEV_G_ROUTING>`
> > > +ioctls.
> > > +
> > > +Routes can be activated and deactivated by setting or clearing the
> > > +V4L2_SUBDEV_ROUTE_FL_ACTIVE flag in the ``flags`` field of struct
> > > +:c:type:`v4l2_subdev_route`.
> > > +
> > > +Subdev driver might create routes that cannot be modified by applications. Such
> >
> > s/S/A s/
> 
> Or "Subdevice drivers" ?

I think singular is fine. And s/might/may/.

> 
> >
> > > +routes are identified by the presence of the V4L2_SUBDEV_ROUTE_FL_IMMUTABLE
> > > +flag in the ``flags`` field of struct :c:type:`v4l2_subdev_route`.
> > > +
> > > +As an example, the routing table of a source pad which supports two logical
> > > +video streams and can be connected to two sink pads is here below described.
> > > +
> > > +.. flat-table::
> > > +    :widths:       1 2 1
> > > +
> > > +    * - Pad Index
> > > +      - Function
> > > +      - Number of streams
> > > +    * - 0
> > > +      - SINK
> > > +      - 1
> > > +    * - 1
> > > +      - SINK
> > > +      - 1
> > > +    * - 2
> > > +      - SOURCE
> > > +      - 2
> > > +
> > > +In such an example, the source media pad will report a routing table with 4
> > > +entries, one entry for each possible ``(sink_pad, sink_stream) - (source_pad,
> > > +source_stream)`` combination.
> > > +
> > > +.. flat-table:: routing table
> > > +    :widths:       2 1 2
> > > +
> > > +    * - Sink Pad/Sink Stream
> > > +      - ->
> > > +      - Source Pad/Source Stream
> > > +    * - 0/0
> > > +      - ->
> > > +      - 2/0
> > > +    * - 0/0
> > > +      - ->
> > > +      - 2/1
> > > +    * - 1/0
> > > +      - ->
> > > +      - 2/0
> > > +    * - 1/0
> > > +      - ->
> > > +      - 2/1
> > > +
> > > +Subdev drivers are free to decide how many routes an application can enable on
> > > +a media pad at the same time, and refuse to enable or disable specific routes.
> > > diff --git a/Documentation/media/uapi/v4l/user-func.rst b/Documentation/media/uapi/v4l/user-func.rst
> > > index ca0ef21d77fe..0166446f4ab4 100644
> > > --- a/Documentation/media/uapi/v4l/user-func.rst
> > > +++ b/Documentation/media/uapi/v4l/user-func.rst
> > > @@ -77,6 +77,7 @@ Function Reference
> > >      vidioc-subdev-g-crop
> > >      vidioc-subdev-g-fmt
> > >      vidioc-subdev-g-frame-interval
> > > +    vidioc-subdev-g-routing
> > >      vidioc-subdev-g-selection
> > >      vidioc-subscribe-event
> > >      func-mmap
> > > diff --git a/Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst b/Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst
> > > new file mode 100644
> > > index 000000000000..8b592722c477
> > > --- /dev/null
> > > +++ b/Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst
> > > @@ -0,0 +1,142 @@
> > > +.. Permission is granted to copy, distribute and/or modify this
> > > +.. document under the terms of the GNU Free Documentation License,
> > > +.. Version 1.1 or any later version published by the Free Software
> > > +.. Foundation, with no Invariant Sections, no Front-Cover Texts
> > > +.. and no Back-Cover Texts. A copy of the license is included at
> > > +.. Documentation/media/uapi/fdl-appendix.rst.
> > > +..
> > > +.. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections
> > > +
> > > +.. _VIDIOC_SUBDEV_G_ROUTING:
> > > +
> > > +******************************************************
> > > +ioctl VIDIOC_SUBDEV_G_ROUTING, VIDIOC_SUBDEV_S_ROUTING
> > > +******************************************************
> > > +
> > > +Name
> > > +====
> > > +
> > > +VIDIOC_SUBDEV_G_ROUTING - VIDIOC_SUBDEV_S_ROUTING - Get or set routing between streams of media pads in a media entity.
> > > +
> > > +
> > > +Synopsis
> > > +========
> > > +
> > > +.. c:function:: int ioctl( int fd, VIDIOC_SUBDEV_G_ROUTING, struct v4l2_subdev_routing *argp )
> > > +    :name: VIDIOC_SUBDEV_G_ROUTING
> > > +
> > > +.. c:function:: int ioctl( int fd, VIDIOC_SUBDEV_S_ROUTING, struct v4l2_subdev_routing *argp )
> > > +    :name: VIDIOC_SUBDEV_S_ROUTING
> > > +
> > > +
> > > +Arguments
> > > +=========
> > > +
> > > +``fd``
> > > +    File descriptor returned by :ref:`open() <func-open>`.
> > > +
> > > +``argp``
> > > +    Pointer to struct :c:type:`v4l2_subdev_routing`.
> > > +
> > > +
> > > +Description
> > > +===========
> > > +
> > > +These ioctls are used to get and set the routing informations associated to
> > > +media pads in a media entity. Routing informations are used to enable or disable
> >
> > The routing is a property of an entity. How about
> >
> > s/the routing informations associated to media pads in/routing/
> >
> > s/R[^\.]+(?=\.)/The routing configuration determines the flows of data
> > inside an entity/
> 
> Thanks, you made me discover regexr.com here

:-) Or just man perlre . :-)

> >
> > > +data connections between stream endpoints of multiplexed media pads.
> > > +
> > > +Drivers report their routing tables using VIDIOC_SUBDEV_G_ROUTING and
> > > +application use the information there reported to enable or disable data
> > > +connections between streams in a pad, by setting or clearing the
> >
> > How about:
> >
> > s/applications\K.*a pad/may enable or disable routes with the
> > VIDIOC_SUBDEV_S_ROUTING IOCTL,
> >
> 
> Yes, reads better
> 
> > > +V4L2_SUBDEV_ROUTE_FL_ACTIVE flag of ``flags`` field of a struct
> > > +:c:type:`v4l2_subdev_route`.
> > > +
> > > +When inspecting routes through VIDIOC_SUBDEV_G_ROUTING and the application
> > > +provided ``num_routes`` is not big enough to contain all the available routes
> > > +the subdevice exposes, drivers return the ENOSPC error code and adjust the
> > > +``num_routes`` value. Application should then reserve enough memory for all the
> >
> > s/the\K$/value of the/
> > s/value\K\./field/
> >
> 
> Will update
> 
> > > +route entries and call VIDIOC_SUBDEV_G_ROUTING again.
> > > +
> > > +.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}|
> > > +
> > > +.. c:type:: v4l2_subdev_routing
> > > +
> > > +.. flat-table:: struct v4l2_subdev_routing
> > > +    :header-rows:  0
> > > +    :stub-columns: 0
> > > +    :widths:       1 1 2
> > > +
> > > +    * - struct :c:type:`v4l2_subdev_route`
> > > +      - ``routes[]``
> > > +      - Array of struct :c:type:`v4l2_subdev_route` entries
> > > +    * - __u32
> > > +      - ``num_routes``
> > > +      - Number of entries of the routes array
> > > +    * - __u32
> > > +      - ``reserved``\ [5]
> > > +      - Reserved for future extensions. Applications and drivers must set
> > > +	the array to zero.
> > > +
> > > +.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}|
> > > +
> > > +.. c:type:: v4l2_subdev_route
> > > +
> > > +.. flat-table:: struct v4l2_subdev_route
> > > +    :header-rows:  0
> > > +    :stub-columns: 0
> > > +    :widths:       1 1 2
> > > +
> > > +    * - __u32
> > > +      - ``sink_pad``
> > > +      - Sink pad number
> > > +    * - __u32
> > > +      - ``sink_stream``
> > > +      - Sink pad stream number
> > > +    * - __u32
> > > +      - ``source_stream``
> > > +      - Source pad stream number
> > > +    * - __u32
> > > +      - ``sink_stream``
> > > +      - Sink pad stream number
> > > +    * - __u32
> > > +      - ``flags``
> > > +      - Route enable/disable flags
> > > +	:ref:`v4l2_subdev_routing_flags <v4l2-subdev-routing-flags>`.
> > > +    * - __u32
> > > +      - ``reserved``\ [5]
> > > +      - Reserved for future extensions. Applications and drivers must set
> > > +	the array to zero.
> > > +
> > > +.. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
> > > +
> > > +.. _v4l2-subdev-routing-flags:
> > > +
> > > +.. flat-table:: enum v4l2_subdev_routing_flags
> > > +    :header-rows:  0
> > > +    :stub-columns: 0
> > > +    :widths:       3 1 4
> > > +
> > > +    * - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> > > +      - 0
> > > +      - The route is enabled. Set by applications.
> > > +    * - V4L2_SUBDEV_ROUTE_FL_IMMUTABLE
> > > +      - 1
> > > +      - The route is immutable. Set by the driver.
> > > +
> > > +Return Value
> > > +============
> > > +
> > > +On success 0 is returned, on error -1 and the ``errno`` variable is set
> > > +appropriately. The generic error codes are described at the
> > > +:ref:`Generic Error Codes <gen-errors>` chapter.
> > > +
> > > +ENOSPC
> > > +   The number of provided route entries is less than the available ones.
> > > +
> > > +EINVAL
> > > +   The sink or source pad identifiers reference a non-existing pad, or refernce
> > > +   pads of different types (ie. the sink_pad identifiers refers to a source pad)
> > > +   The sink or source stream identifiers reference a non-existing stream
> >
> > s/The/or the/
> >
> > > +   in the sink or source pad.
> >
> > s/i/o/
> 
> Thanks, I will update all of these in v4.

Thanks!

-- 
Sakari Ailus
sakari.ailus@linux.intel.com

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 19/31] media: Documentation: Add GS_ROUTING documentation
  2019-03-05 18:51 ` [PATCH v3 19/31] media: Documentation: Add GS_ROUTING documentation Jacopo Mondi
  2019-03-07 15:19   ` Sakari Ailus
@ 2019-03-14 14:43   ` Luca Ceresoli
  2020-02-13 13:36   ` Hans Verkuil
  2 siblings, 0 replies; 59+ messages in thread
From: Luca Ceresoli @ 2019-03-14 14:43 UTC (permalink / raw)
  To: Jacopo Mondi, sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: linux-media, linux-renesas-soc

Hi,

On 05/03/19 19:51, Jacopo Mondi wrote:
> Add documentation for VIDIOC_SUBDEV_G/S_ROUTING ioctl and add
> description of multiplexed media pads and internal routing to the
> V4L2-subdev documentation section.
> 
> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> ---
>  Documentation/media/uapi/v4l/dev-subdev.rst   |  90 +++++++++++
>  Documentation/media/uapi/v4l/user-func.rst    |   1 +
>  .../uapi/v4l/vidioc-subdev-g-routing.rst      | 142 ++++++++++++++++++
>  3 files changed, 233 insertions(+)
>  create mode 100644 Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst
> 
> diff --git a/Documentation/media/uapi/v4l/dev-subdev.rst b/Documentation/media/uapi/v4l/dev-subdev.rst
> index 2c2768c7343b..b9fbb5d2caec 100644
> --- a/Documentation/media/uapi/v4l/dev-subdev.rst
> +++ b/Documentation/media/uapi/v4l/dev-subdev.rst
> @@ -36,6 +36,8 @@ will feature a character device node on which ioctls can be called to
>  
>  -  negotiate image formats on individual pads
>  
> +-  inspect and modify internal data routing between pads of the same entity
> +
>  Sub-device character device nodes, conventionally named
>  ``/dev/v4l-subdev*``, use major number 81.
>  
> @@ -461,3 +463,91 @@ source pads.
>      :maxdepth: 1
>  
>      subdev-formats
> +
> +
> +Multiplexed media pads and internal routing
> +-------------------------------------------
> +
> +Subdevice drivers might expose the internal connections between media pads of an
> +entity by providing a routing table that applications can inspect and manipulate
> +to change the internal routing between sink and source pads' data connection
> +endpoints. A routing table is described by a struct
> +:c:type:`v4l2_subdev_routing`, which contains ``num_routes`` route entries, each
> +one represented by a struct :c:type:`v4l2_subdev_route`.
> +
> +Data routes do not just connect one pad to another in an entity, but they refer
> +instead to the ``streams`` a media pad provides. Streams are data connection
> +endpoints in a media pad. Multiplexed media pads expose multiple ``streams``
> +which represent, when the underlying hardware technology allows that, logical
> +data streams transported over a single physical media bus.
> +
> +One of the most notable examples of logical stream multiplexing techniques is
> +represented by the data interleaving mechanism implemented by mean of Virtual
> +Channels identifiers as defined by the MIPI CSI-2 media bus specifications. A
> +subdevice that implements support for Virtual Channel data interleaving might
> +expose up to 4 data ``streams``, one for each available Virtual Channel, on the
> +source media pad that represents a CSI-2 connection endpoint.
> +
> +Routes are defined as potential data connections between a ``(sink_pad,
> +sink_stream)`` pair and a ``(source_pad, source_stream)`` one, where
> +``sink_pad`` and ``source_pad`` are the indexes of two media pads part of the
> +same media entity, and ``sink_stream`` and ``source_stream`` are the identifiers
> +of the data streams to be connected in the media pads. Media pads that do not
> +support data multiplexing expose a single stream, usually with identifier 0.
> +
> +Routes are reported to applications in a routing table which can be
> +inspected and manipulated using the :ref:`routing <VIDIOC_SUBDEV_G_ROUTING>`
> +ioctls.
> +
> +Routes can be activated and deactivated by setting or clearing the
> +V4L2_SUBDEV_ROUTE_FL_ACTIVE flag in the ``flags`` field of struct

`` around V4L2_SUBDEV_ROUTE_FL_ACTIVE would be better.

> +:c:type:`v4l2_subdev_route`.
> +
> +Subdev driver might create routes that cannot be modified by applications. Such
> +routes are identified by the presence of the V4L2_SUBDEV_ROUTE_FL_IMMUTABLE

`` here too.

> +flag in the ``flags`` field of struct :c:type:`v4l2_subdev_route`.
> +
> +As an example, the routing table of a source pad which supports two logical
> +video streams and can be connected to two sink pads is here below described.

After this sentence I expected to find the routing table, but instead
there's a pad description. I'd rephrase as:

--------8<--------
Take for example a subdevice that has two sink pads and can combnie
their streams on a single source pad as two logical streams. The
following table describes the pads for such subdevice.
--------8<--------

> +
> +.. flat-table::

To make the headers bold as in other tables just add here:
:header-rows:  1

> +    :widths:       1 2 1

The content of the central column is not really larger than the others,
you can set '1 1 1' here or just ditch the :width: line.

> +
> +    * - Pad Index
> +      - Function
> +      - Number of streams
> +    * - 0
> +      - SINK
> +      - 1
> +    * - 1
> +      - SINK
> +      - 1
> +    * - 2
> +      - SOURCE
> +      - 2
> +
> +In such an example, the source media pad will report a routing table with 4
> +entries, one entry for each possible ``(sink_pad, sink_stream) - (source_pad,
> +source_stream)`` combination.
> +
> +.. flat-table:: routing table

Here as well:
:header-rows:  1

> +    :widths:       2 1 2
> +
> +    * - Sink Pad/Sink Stream
> +      - ->
> +      - Source Pad/Source Stream
> +    * - 0/0
> +      - ->
> +      - 2/0
> +    * - 0/0
> +      - ->
> +      - 2/1
> +    * - 1/0
> +      - ->
> +      - 2/0
> +    * - 1/0
> +      - ->
> +      - 2/1
> +
> +Subdev drivers are free to decide how many routes an application can enable on
> +a media pad at the same time, and refuse to enable or disable specific routes.

...

> +Description
> +===========
> +
> +These ioctls are used to get and set the routing informations associated to

"information" is uncountable, thus s/informations/information/ here and
in other places.

> +media pads in a media entity. Routing informations are used to enable or disable
> +data connections between stream endpoints of multiplexed media pads.

...

> +.. flat-table:: struct v4l2_subdev_route
> +    :header-rows:  0
> +    :stub-columns: 0
> +    :widths:       1 1 2
> +
> +    * - __u32
> +      - ``sink_pad``
> +      - Sink pad number
> +    * - __u32
> +      - ``sink_stream``
> +      - Sink pad stream number
> +    * - __u32
> +      - ``source_stream``
> +      - Source pad stream number
> +    * - __u32
> +      - ``sink_stream``
> +      - Sink pad stream number

The 3rd and 4th field are wrong, they should be source_pad and
source_stream.

> +    * - __u32
> +      - ``flags``
> +      - Route enable/disable flags
> +	:ref:`v4l2_subdev_routing_flags <v4l2-subdev-routing-flags>`.
> +    * - __u32
> +      - ``reserved``\ [5]
> +      - Reserved for future extensions. Applications and drivers must set
> +	the array to zero.

-- 
Luca

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 19/31] media: Documentation: Add GS_ROUTING documentation
  2019-03-07 15:19   ` Sakari Ailus
  2019-03-08 13:31     ` Jacopo Mondi
@ 2019-03-14 14:44     ` Luca Ceresoli
  1 sibling, 0 replies; 59+ messages in thread
From: Luca Ceresoli @ 2019-03-14 14:44 UTC (permalink / raw)
  To: Sakari Ailus, Jacopo Mondi
  Cc: laurent.pinchart, niklas.soderlund+renesas, linux-media,
	linux-renesas-soc

Hi,

On 07/03/19 16:19, Sakari Ailus wrote:

...

>> +Description
>> +===========
>> +
>> +These ioctls are used to get and set the routing informations associated to
>> +media pads in a media entity. Routing informations are used to enable or disable
> 
> The routing is a property of an entity. How about
> 
> s/the routing informations associated to media pads in/routing/
> 
> s/R[^\.]+(?=\.)/The routing configuration determines the flows of data
> inside an entity/
> 
>> +data connections between stream endpoints of multiplexed media pads.

In the original wording, mentioning "multiplexed" is incorrect. Routing
can happen even between non-multiplexed streams, as e.g. in the AFE
driver in patch 27. Sakari's rewording is OK in this respect.

-- 
Luca

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 09/31] media: entity: Add media_has_route() function
  2019-03-05 18:51 ` [PATCH v3 09/31] media: entity: Add media_has_route() function Jacopo Mondi
@ 2019-03-14 14:45   ` Luca Ceresoli
  2019-03-15  9:22     ` Jacopo Mondi
  0 siblings, 1 reply; 59+ messages in thread
From: Luca Ceresoli @ 2019-03-14 14:45 UTC (permalink / raw)
  To: Jacopo Mondi, sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: linux-media, linux-renesas-soc, Michal Simek

Hi,

in the Subject line:
s/media_has_route/media_entity_has_route/

On 05/03/19 19:51, Jacopo Mondi wrote:
> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
> This is a wrapper around the media entity has_route operation.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Michal Simek <michal.simek@xilinx.com>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> ---
>  drivers/media/media-entity.c | 19 +++++++++++++++++++
>  include/media/media-entity.h | 17 +++++++++++++++++
>  2 files changed, 36 insertions(+)
> 
> diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
> index 6f5196d05894..8e0ca8b1cfa2 100644
> --- a/drivers/media/media-entity.c
> +++ b/drivers/media/media-entity.c
> @@ -238,6 +238,25 @@ EXPORT_SYMBOL_GPL(media_entity_pads_init);
>   * Graph traversal
>   */
>  
> +bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
> +			    unsigned int pad1)
> +{
> +	if (pad0 >= entity->num_pads || pad1 >= entity->num_pads)
> +		return false;
> +
> +	if (pad0 == pad1)
> +		return true;
> +
> +	if (!entity->ops || !entity->ops->has_route)
> +		return true;

Entities that implement has_route in following patches return false if
called with two sink pads or two source pads. This code behaves
differently. Which behavior is correct? IOW, how do you define "two
entity pads are connected internally"?

> +	if (entity->pads[pad1].index < entity->pads[pad0].index)
> +		swap(pad0, pad1);
> +
> +	return entity->ops->has_route(entity, pad0, pad1);
> +}
> +EXPORT_SYMBOL_GPL(media_entity_has_route);
> +
>  static struct media_pad *
>  media_pad_other(struct media_pad *pad, struct media_link *link)
>  {
> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> index 675bc27b8b3c..205561545d7e 100644
> --- a/include/media/media-entity.h
> +++ b/include/media/media-entity.h
> @@ -919,6 +919,23 @@ int media_entity_get_fwnode_pad(struct media_entity *entity,
>  __must_check int media_graph_walk_init(
>  	struct media_graph *graph, struct media_device *mdev);
>  
> +/**
> + * media_entity_has_route - Check if two entity pads are connected internally
> + *
> + * @entity: The entity
> + * @pad0: The first pad index
> + * @pad1: The second pad index
> + *
> + * This function can be used to check whether two pads of an entity are
> + * connected internally in the entity.
> + *
> + * The caller must hold entity->graph_obj.mdev->mutex.
> + *
> + * Return: true if the pads are connected internally and false otherwise.
> + */
> +bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
> +			    unsigned int pad1);
> +
>  /**
>   * media_graph_walk_cleanup - Release resources used by graph walk.
>   *
> 

-- 
Luca

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 28/31] adv748x: afe: Implement has_route()
  2019-03-05 18:51 ` [PATCH v3 28/31] adv748x: afe: Implement has_route() Jacopo Mondi
  2019-03-07 12:49   ` Sakari Ailus
@ 2019-03-14 14:45   ` Luca Ceresoli
  1 sibling, 0 replies; 59+ messages in thread
From: Luca Ceresoli @ 2019-03-14 14:45 UTC (permalink / raw)
  To: Jacopo Mondi, sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: linux-media, linux-renesas-soc

Hi,

On 05/03/19 19:51, Jacopo Mondi wrote:
> Now that the adv748x subdevice supports internal routing, add an
> has_route() operation used during media graph traversal.
> 
> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> ---
>  drivers/media/i2c/adv748x/adv748x-afe.c | 26 +++++++++++++++++++++++++
>  1 file changed, 26 insertions(+)
> 
> diff --git a/drivers/media/i2c/adv748x/adv748x-afe.c b/drivers/media/i2c/adv748x/adv748x-afe.c
> index 3f770f71413f..39ac55f0adbb 100644
> --- a/drivers/media/i2c/adv748x/adv748x-afe.c
> +++ b/drivers/media/i2c/adv748x/adv748x-afe.c
> @@ -463,6 +463,30 @@ static const struct v4l2_subdev_ops adv748x_afe_ops = {
>  	.pad = &adv748x_afe_pad_ops,
>  };
>  
> +/* -----------------------------------------------------------------------------
> + * media_entity_operations
> + */
> +
> +static bool adv748x_afe_has_route(struct media_entity *entity,
> +				  unsigned int pad0, unsigned int pad1)
> +{
> +	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
> +	struct adv748x_afe *afe = adv748x_sd_to_afe(sd);
> +
> +	/* Only consider direct sink->source routes. */
> +	if (pad0 > ADV748X_AFE_SINK_AIN7 ||
> +	    pad1 != ADV748X_AFE_SOURCE)
> +		return false;
> +
> +	if (pad0 != afe->input)
> +		return false;
> +
> +	return true;
> +}
> +
> +static const struct media_entity_operations adv748x_afe_entity_ops = {
> +	.has_route = adv748x_afe_has_route,
> +};

Missing empty line.

>  /* -----------------------------------------------------------------------------
>   * Controls
>   */

-- 
Luca

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 26/31] adv748x: csi2: add internal routing configuration
  2019-03-05 18:51 ` [PATCH v3 26/31] adv748x: csi2: add internal routing configuration Jacopo Mondi
@ 2019-03-14 14:45   ` Luca Ceresoli
  2019-03-15  9:45     ` Jacopo Mondi
  0 siblings, 1 reply; 59+ messages in thread
From: Luca Ceresoli @ 2019-03-14 14:45 UTC (permalink / raw)
  To: Jacopo Mondi, sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: linux-media, linux-renesas-soc

Hi,

begging your pardon for the noob question below...

On 05/03/19 19:51, Jacopo Mondi wrote:
> From: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> 
> Add support to get and set the internal routing between the adv748x
> CSI-2 transmitters sink pad and its multiplexed source pad. This routing
> includes which stream of the multiplexed pad to use, allowing the user
> to select which CSI-2 virtual channel to use when transmitting the
> stream.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> ---
>  drivers/media/i2c/adv748x/adv748x-csi2.c | 65 ++++++++++++++++++++++++
>  1 file changed, 65 insertions(+)
> 
> diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
> index d8f7cbee86e7..13454af72c6e 100644
> --- a/drivers/media/i2c/adv748x/adv748x-csi2.c
> +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
> @@ -14,6 +14,8 @@
>  
>  #include "adv748x.h"
>  
> +#define ADV748X_CSI2_ROUTES_MAX 4
> +
>  struct adv748x_csi2_format {
>  	unsigned int code;
>  	unsigned int datatype;
> @@ -253,10 +255,73 @@ static int adv748x_csi2_get_frame_desc(struct v4l2_subdev *sd, unsigned int pad,
>  	return 0;
>  }
>  
> +static int adv748x_csi2_get_routing(struct v4l2_subdev *sd,
> +				    struct v4l2_subdev_krouting *routing)
> +{
> +	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> +	struct v4l2_subdev_route *r = routing->routes;
> +	unsigned int vc;
> +
> +	if (routing->num_routes < ADV748X_CSI2_ROUTES_MAX) {
> +		routing->num_routes = ADV748X_CSI2_ROUTES_MAX;
> +		return -ENOSPC;
> +	}
> +
> +	routing->num_routes = ADV748X_CSI2_ROUTES_MAX;
> +
> +	for (vc = 0; vc < ADV748X_CSI2_ROUTES_MAX; vc++) {
> +		r->sink_pad = ADV748X_CSI2_SINK;
> +		r->sink_stream = 0;
> +		r->source_pad = ADV748X_CSI2_SOURCE;
> +		r->source_stream = vc;
> +		r->flags = vc == tx->vc ? V4L2_SUBDEV_ROUTE_FL_ACTIVE : 0;
> +		r++;Begging your pardon for the noob question...

> +	}
> +
> +	return 0;
> +}
> +
> +static int adv748x_csi2_set_routing(struct v4l2_subdev *sd,
> +				    struct v4l2_subdev_krouting *routing)
> +{
> +	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> +	struct v4l2_subdev_route *r = routing->routes;
> +	unsigned int i;
> +	int vc = -1;
> +
> +	if (routing->num_routes > ADV748X_CSI2_ROUTES_MAX)
> +		return -ENOSPC;
> +
> +	for (i = 0; i < routing->num_routes; i++) {
> +		if (r->sink_pad != ADV748X_CSI2_SINK ||
> +		    r->sink_stream != 0 ||
> +		    r->source_pad != ADV748X_CSI2_SOURCE ||
> +		    r->source_stream >= ADV748X_CSI2_ROUTES_MAX)
> +			return -EINVAL;
> +
> +		if (r->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE) {
> +			if (vc != -1)
> +				return -EMLINK;
> +
> +			vc = r->source_stream;
> +		}
> +		r++;
> +	}
> +
> +	if (vc != -1)
> +		tx->vc = vc;
> +
> +	adv748x_csi2_set_virtual_channel(tx, tx->vc);
> +
> +	return 0;
> +}

Not specific to this patch but rather to the set_routing idea as a
whole: can the set_routing ioctl be called while the stream is running?

If it cannot, I find it a limiting factor for nowadays use cases. I also
didn't find where the ioctl is rejected.

If it can, then shouldn't this function call s_stream(stop) through the
sink pad whose route becomes disabled, and a s_stream(start) through the
one that gets enabled?

Thanks,
-- 
Luca

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 09/31] media: entity: Add media_has_route() function
  2019-03-14 14:45   ` Luca Ceresoli
@ 2019-03-15  9:22     ` Jacopo Mondi
  0 siblings, 0 replies; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-15  9:22 UTC (permalink / raw)
  To: Luca Ceresoli
  Cc: Jacopo Mondi, sakari.ailus, laurent.pinchart,
	niklas.soderlund+renesas, linux-media, linux-renesas-soc,
	Michal Simek

[-- Attachment #1: Type: text/plain, Size: 3900 bytes --]

Hi Luca,
   thanks for the comments

On Thu, Mar 14, 2019 at 03:45:00PM +0100, Luca Ceresoli wrote:
> Hi,
>
> in the Subject line:
> s/media_has_route/media_entity_has_route/

Ah! Thanks for noticing this.

>
> On 05/03/19 19:51, Jacopo Mondi wrote:
> > From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> >
> > This is a wrapper around the media entity has_route operation.
> >
> > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > Signed-off-by: Michal Simek <michal.simek@xilinx.com>
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> > ---
> >  drivers/media/media-entity.c | 19 +++++++++++++++++++
> >  include/media/media-entity.h | 17 +++++++++++++++++
> >  2 files changed, 36 insertions(+)
> >
> > diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
> > index 6f5196d05894..8e0ca8b1cfa2 100644
> > --- a/drivers/media/media-entity.c
> > +++ b/drivers/media/media-entity.c
> > @@ -238,6 +238,25 @@ EXPORT_SYMBOL_GPL(media_entity_pads_init);
> >   * Graph traversal
> >   */
> >
> > +bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
> > +			    unsigned int pad1)
> > +{
> > +	if (pad0 >= entity->num_pads || pad1 >= entity->num_pads)
> > +		return false;
> > +
> > +	if (pad0 == pad1)
> > +		return true;
> > +
> > +	if (!entity->ops || !entity->ops->has_route)
> > +		return true;
>
> Entities that implement has_route in following patches return false if
> called with two sink pads or two source pads. This code behaves
> differently. Which behavior is correct? IOW, how do you define "two
> entity pads are connected internally"?

The handling of "indirect routes" (aka routes that connects two
sources or two sinks) is totally up to the device driver and we
decided not to make any assumption (nor introduce helpers, as it was
in v2) to support that in the framework.

Have a look at:
[PATCH v2 15/30] media: entity: Look for indirect routes
where Sakari implemented an helper to support the most common use case
of two source pads connected to the same sink pad. In this case the
two sources are reported as connected, but we decided for now to let
the drivers handle this, and more complex indirect routes, internaly.

The devices for which "has_route" has been implemented in this series
do not support indirect routes, for now, but the framework does not
make assumptions on this.

Thanks
   j

>
> > +	if (entity->pads[pad1].index < entity->pads[pad0].index)
> > +		swap(pad0, pad1);
> > +
> > +	return entity->ops->has_route(entity, pad0, pad1);
> > +}
> > +EXPORT_SYMBOL_GPL(media_entity_has_route);
> > +
> >  static struct media_pad *
> >  media_pad_other(struct media_pad *pad, struct media_link *link)
> >  {
> > diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> > index 675bc27b8b3c..205561545d7e 100644
> > --- a/include/media/media-entity.h
> > +++ b/include/media/media-entity.h
> > @@ -919,6 +919,23 @@ int media_entity_get_fwnode_pad(struct media_entity *entity,
> >  __must_check int media_graph_walk_init(
> >  	struct media_graph *graph, struct media_device *mdev);
> >
> > +/**
> > + * media_entity_has_route - Check if two entity pads are connected internally
> > + *
> > + * @entity: The entity
> > + * @pad0: The first pad index
> > + * @pad1: The second pad index
> > + *
> > + * This function can be used to check whether two pads of an entity are
> > + * connected internally in the entity.
> > + *
> > + * The caller must hold entity->graph_obj.mdev->mutex.
> > + *
> > + * Return: true if the pads are connected internally and false otherwise.
> > + */
> > +bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
> > +			    unsigned int pad1);
> > +
> >  /**
> >   * media_graph_walk_cleanup - Release resources used by graph walk.
> >   *
> >
>
> --
> Luca

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 26/31] adv748x: csi2: add internal routing configuration
  2019-03-14 14:45   ` Luca Ceresoli
@ 2019-03-15  9:45     ` Jacopo Mondi
  2019-03-15 10:06       ` Sakari Ailus
  0 siblings, 1 reply; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-15  9:45 UTC (permalink / raw)
  To: Luca Ceresoli
  Cc: Jacopo Mondi, sakari.ailus, laurent.pinchart,
	niklas.soderlund+renesas, linux-media, linux-renesas-soc

[-- Attachment #1: Type: text/plain, Size: 5156 bytes --]

Hi Luca,

On Thu, Mar 14, 2019 at 03:45:27PM +0100, Luca Ceresoli wrote:
> Hi,
>
> begging your pardon for the noob question below...
>

Let a noob help another noob then

> On 05/03/19 19:51, Jacopo Mondi wrote:
> > From: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> >
> > Add support to get and set the internal routing between the adv748x
> > CSI-2 transmitters sink pad and its multiplexed source pad. This routing
> > includes which stream of the multiplexed pad to use, allowing the user
> > to select which CSI-2 virtual channel to use when transmitting the
> > stream.
> >
> > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> > ---
> >  drivers/media/i2c/adv748x/adv748x-csi2.c | 65 ++++++++++++++++++++++++
> >  1 file changed, 65 insertions(+)
> >
> > diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > index d8f7cbee86e7..13454af72c6e 100644
> > --- a/drivers/media/i2c/adv748x/adv748x-csi2.c
> > +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > @@ -14,6 +14,8 @@
> >
> >  #include "adv748x.h"
> >
> > +#define ADV748X_CSI2_ROUTES_MAX 4
> > +
> >  struct adv748x_csi2_format {
> >  	unsigned int code;
> >  	unsigned int datatype;
> > @@ -253,10 +255,73 @@ static int adv748x_csi2_get_frame_desc(struct v4l2_subdev *sd, unsigned int pad,
> >  	return 0;
> >  }
> >
> > +static int adv748x_csi2_get_routing(struct v4l2_subdev *sd,
> > +				    struct v4l2_subdev_krouting *routing)
> > +{
> > +	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> > +	struct v4l2_subdev_route *r = routing->routes;
> > +	unsigned int vc;
> > +
> > +	if (routing->num_routes < ADV748X_CSI2_ROUTES_MAX) {
> > +		routing->num_routes = ADV748X_CSI2_ROUTES_MAX;
> > +		return -ENOSPC;
> > +	}
> > +
> > +	routing->num_routes = ADV748X_CSI2_ROUTES_MAX;
> > +
> > +	for (vc = 0; vc < ADV748X_CSI2_ROUTES_MAX; vc++) {
> > +		r->sink_pad = ADV748X_CSI2_SINK;
> > +		r->sink_stream = 0;
> > +		r->source_pad = ADV748X_CSI2_SOURCE;
> > +		r->source_stream = vc;
> > +		r->flags = vc == tx->vc ? V4L2_SUBDEV_ROUTE_FL_ACTIVE : 0;
> > +		r++;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int adv748x_csi2_set_routing(struct v4l2_subdev *sd,
> > +				    struct v4l2_subdev_krouting *routing)
> > +{
> > +	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> > +	struct v4l2_subdev_route *r = routing->routes;
> > +	unsigned int i;
> > +	int vc = -1;
> > +
> > +	if (routing->num_routes > ADV748X_CSI2_ROUTES_MAX)
> > +		return -ENOSPC;
> > +
> > +	for (i = 0; i < routing->num_routes; i++) {
> > +		if (r->sink_pad != ADV748X_CSI2_SINK ||
> > +		    r->sink_stream != 0 ||
> > +		    r->source_pad != ADV748X_CSI2_SOURCE ||
> > +		    r->source_stream >= ADV748X_CSI2_ROUTES_MAX)
> > +			return -EINVAL;
> > +
> > +		if (r->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE) {
> > +			if (vc != -1)
> > +				return -EMLINK;
> > +
> > +			vc = r->source_stream;
> > +		}
> > +		r++;
> > +	}
> > +
> > +	if (vc != -1)
> > +		tx->vc = vc;
> > +
> > +	adv748x_csi2_set_virtual_channel(tx, tx->vc);
> > +
> > +	return 0;
> > +}
>
> Not specific to this patch but rather to the set_routing idea as a
> whole: can the set_routing ioctl be called while the stream is running?
>
> If it cannot, I find it a limiting factor for nowadays use cases. I also
> didn't find where the ioctl is rejected.
>

The framework does not make assumptions about that at the moment.

> If it can, then shouldn't this function call s_stream(stop) through the
> sink pad whose route becomes disabled, and a s_stream(start) through the
> one that gets enabled?
>

If I got this right, you're here rightfully pointing out that changing
the routing between pads in an entity migh impact the pipeline as a
whole, and this would require, to activate/deactivate devices that
where not part of the pipeline.

This is probably the wrong patch to use an example, as this one is for
a multiplexed interface, where there is no need to go through an
s_stream() for the two CSI-2 endpoints, but as you pointed out in our
brief offline chat, the AFE->TX routing example for this very device
is a good one: if we change the analogue source that is internally
routed to the CSI-2 output of the adv748x, do we need to s_stream(1)
the now routed entity and s_stream(0) on the not not-anymore-routed
one?

My gut feeling is that this is up to userspace, as it should know
what are the requirements of the devices in the system, but this mean
going through an s_stream(0)/s_stream(1) sequence on the video device,
and that would interrupt the streaming for sure.

At the same time, I don't feel too much at ease with the idea of
s_routing calling s_stream on the entity' remote subdevices, as this
would skip the link format validation that media_pipeline_start()
performs.

So yeah, I understand your point, but I don't have a real answer,
maybe someone else does and want to share his mind?

Thanks
   j

> Thanks,
> --
> Luca

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 26/31] adv748x: csi2: add internal routing configuration
  2019-03-15  9:45     ` Jacopo Mondi
@ 2019-03-15 10:06       ` Sakari Ailus
  2019-03-16 10:23         ` Luca Ceresoli
  0 siblings, 1 reply; 59+ messages in thread
From: Sakari Ailus @ 2019-03-15 10:06 UTC (permalink / raw)
  To: Jacopo Mondi
  Cc: Luca Ceresoli, Jacopo Mondi, laurent.pinchart,
	niklas.soderlund+renesas, linux-media, linux-renesas-soc

Hi Luca, Jacopo,

On Fri, Mar 15, 2019 at 10:45:38AM +0100, Jacopo Mondi wrote:
> Hi Luca,
> 
> On Thu, Mar 14, 2019 at 03:45:27PM +0100, Luca Ceresoli wrote:
> > Hi,
> >
> > begging your pardon for the noob question below...
> >
> 
> Let a noob help another noob then
> 
> > On 05/03/19 19:51, Jacopo Mondi wrote:
> > > From: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > >
> > > Add support to get and set the internal routing between the adv748x
> > > CSI-2 transmitters sink pad and its multiplexed source pad. This routing
> > > includes which stream of the multiplexed pad to use, allowing the user
> > > to select which CSI-2 virtual channel to use when transmitting the
> > > stream.
> > >
> > > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > > Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> > > ---
> > >  drivers/media/i2c/adv748x/adv748x-csi2.c | 65 ++++++++++++++++++++++++
> > >  1 file changed, 65 insertions(+)
> > >
> > > diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > index d8f7cbee86e7..13454af72c6e 100644
> > > --- a/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > @@ -14,6 +14,8 @@
> > >
> > >  #include "adv748x.h"
> > >
> > > +#define ADV748X_CSI2_ROUTES_MAX 4
> > > +
> > >  struct adv748x_csi2_format {
> > >  	unsigned int code;
> > >  	unsigned int datatype;
> > > @@ -253,10 +255,73 @@ static int adv748x_csi2_get_frame_desc(struct v4l2_subdev *sd, unsigned int pad,
> > >  	return 0;
> > >  }
> > >
> > > +static int adv748x_csi2_get_routing(struct v4l2_subdev *sd,
> > > +				    struct v4l2_subdev_krouting *routing)
> > > +{
> > > +	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> > > +	struct v4l2_subdev_route *r = routing->routes;
> > > +	unsigned int vc;
> > > +
> > > +	if (routing->num_routes < ADV748X_CSI2_ROUTES_MAX) {
> > > +		routing->num_routes = ADV748X_CSI2_ROUTES_MAX;
> > > +		return -ENOSPC;
> > > +	}
> > > +
> > > +	routing->num_routes = ADV748X_CSI2_ROUTES_MAX;
> > > +
> > > +	for (vc = 0; vc < ADV748X_CSI2_ROUTES_MAX; vc++) {
> > > +		r->sink_pad = ADV748X_CSI2_SINK;
> > > +		r->sink_stream = 0;
> > > +		r->source_pad = ADV748X_CSI2_SOURCE;
> > > +		r->source_stream = vc;
> > > +		r->flags = vc == tx->vc ? V4L2_SUBDEV_ROUTE_FL_ACTIVE : 0;
> > > +		r++;
> > > +	}
> > > +
> > > +	return 0;
> > > +}
> > > +
> > > +static int adv748x_csi2_set_routing(struct v4l2_subdev *sd,
> > > +				    struct v4l2_subdev_krouting *routing)
> > > +{
> > > +	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> > > +	struct v4l2_subdev_route *r = routing->routes;
> > > +	unsigned int i;
> > > +	int vc = -1;
> > > +
> > > +	if (routing->num_routes > ADV748X_CSI2_ROUTES_MAX)
> > > +		return -ENOSPC;
> > > +
> > > +	for (i = 0; i < routing->num_routes; i++) {
> > > +		if (r->sink_pad != ADV748X_CSI2_SINK ||
> > > +		    r->sink_stream != 0 ||
> > > +		    r->source_pad != ADV748X_CSI2_SOURCE ||
> > > +		    r->source_stream >= ADV748X_CSI2_ROUTES_MAX)
> > > +			return -EINVAL;
> > > +
> > > +		if (r->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE) {
> > > +			if (vc != -1)
> > > +				return -EMLINK;
> > > +
> > > +			vc = r->source_stream;
> > > +		}
> > > +		r++;
> > > +	}
> > > +
> > > +	if (vc != -1)
> > > +		tx->vc = vc;
> > > +
> > > +	adv748x_csi2_set_virtual_channel(tx, tx->vc);
> > > +
> > > +	return 0;
> > > +}
> >
> > Not specific to this patch but rather to the set_routing idea as a
> > whole: can the set_routing ioctl be called while the stream is running?
> >
> > If it cannot, I find it a limiting factor for nowadays use cases. I also
> > didn't find where the ioctl is rejected.
> >
> 
> The framework does not make assumptions about that at the moment.
> 
> > If it can, then shouldn't this function call s_stream(stop) through the
> > sink pad whose route becomes disabled, and a s_stream(start) through the
> > one that gets enabled?
> >
> 
> If I got this right, you're here rightfully pointing out that changing
> the routing between pads in an entity migh impact the pipeline as a
> whole, and this would require, to activate/deactivate devices that
> where not part of the pipeline.

I'd say that ultimately this depends on the devices themselves, whether
they support this or not. In practice I don't think we have any such cases
at the moment, but it's possible in principle. Changes on the framework may
well be needed but likely the biggest complications will still be in
drivers supporting that.

The media links have a dynamic flag for this purpose but I don't think it's
ever been used.

> 
> This is probably the wrong patch to use an example, as this one is for
> a multiplexed interface, where there is no need to go through an
> s_stream() for the two CSI-2 endpoints, but as you pointed out in our
> brief offline chat, the AFE->TX routing example for this very device
> is a good one: if we change the analogue source that is internally
> routed to the CSI-2 output of the adv748x, do we need to s_stream(1)
> the now routed entity and s_stream(0) on the not not-anymore-routed
> one?
> 
> My gut feeling is that this is up to userspace, as it should know
> what are the requirements of the devices in the system, but this mean
> going through an s_stream(0)/s_stream(1) sequence on the video device,
> and that would interrupt the streaming for sure.
> 
> At the same time, I don't feel too much at ease with the idea of
> s_routing calling s_stream on the entity' remote subdevices, as this
> would skip the link format validation that media_pipeline_start()
> performs.

The link validation must be done in this case as well, it may not be
simply skipped.

> 
> So yeah, I understand your point, but I don't have a real answer,
> maybe someone else does and want to share his mind?

-- 
Kind regards,

Sakari Ailus
sakari.ailus@linux.intel.com

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 26/31] adv748x: csi2: add internal routing configuration
  2019-03-15 10:06       ` Sakari Ailus
@ 2019-03-16 10:23         ` Luca Ceresoli
  2019-03-20 17:14           ` Jacopo Mondi
  0 siblings, 1 reply; 59+ messages in thread
From: Luca Ceresoli @ 2019-03-16 10:23 UTC (permalink / raw)
  To: Sakari Ailus, Jacopo Mondi
  Cc: Jacopo Mondi, laurent.pinchart, niklas.soderlund+renesas,
	linux-media, linux-renesas-soc

Hi Jacopo, Sakari,

On 15/03/19 11:06, Sakari Ailus wrote:
> Hi Luca, Jacopo,
> 
> On Fri, Mar 15, 2019 at 10:45:38AM +0100, Jacopo Mondi wrote:
>> Hi Luca,
>>
>> On Thu, Mar 14, 2019 at 03:45:27PM +0100, Luca Ceresoli wrote:
>>> Hi,
>>>
>>> begging your pardon for the noob question below...
>>>
>>
>> Let a noob help another noob then
>>
>>> On 05/03/19 19:51, Jacopo Mondi wrote:
>>>> From: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
>>>>
>>>> Add support to get and set the internal routing between the adv748x
>>>> CSI-2 transmitters sink pad and its multiplexed source pad. This routing
>>>> includes which stream of the multiplexed pad to use, allowing the user
>>>> to select which CSI-2 virtual channel to use when transmitting the
>>>> stream.
>>>>
>>>> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
>>>> Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
>>>> ---
>>>>  drivers/media/i2c/adv748x/adv748x-csi2.c | 65 ++++++++++++++++++++++++
>>>>  1 file changed, 65 insertions(+)
>>>>
>>>> diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
>>>> index d8f7cbee86e7..13454af72c6e 100644
>>>> --- a/drivers/media/i2c/adv748x/adv748x-csi2.c
>>>> +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
>>>> @@ -14,6 +14,8 @@
>>>>
>>>>  #include "adv748x.h"
>>>>
>>>> +#define ADV748X_CSI2_ROUTES_MAX 4
>>>> +
>>>>  struct adv748x_csi2_format {
>>>>  	unsigned int code;
>>>>  	unsigned int datatype;
>>>> @@ -253,10 +255,73 @@ static int adv748x_csi2_get_frame_desc(struct v4l2_subdev *sd, unsigned int pad,
>>>>  	return 0;
>>>>  }
>>>>
>>>> +static int adv748x_csi2_get_routing(struct v4l2_subdev *sd,
>>>> +				    struct v4l2_subdev_krouting *routing)
>>>> +{
>>>> +	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
>>>> +	struct v4l2_subdev_route *r = routing->routes;
>>>> +	unsigned int vc;
>>>> +
>>>> +	if (routing->num_routes < ADV748X_CSI2_ROUTES_MAX) {
>>>> +		routing->num_routes = ADV748X_CSI2_ROUTES_MAX;
>>>> +		return -ENOSPC;
>>>> +	}
>>>> +
>>>> +	routing->num_routes = ADV748X_CSI2_ROUTES_MAX;
>>>> +
>>>> +	for (vc = 0; vc < ADV748X_CSI2_ROUTES_MAX; vc++) {
>>>> +		r->sink_pad = ADV748X_CSI2_SINK;
>>>> +		r->sink_stream = 0;
>>>> +		r->source_pad = ADV748X_CSI2_SOURCE;
>>>> +		r->source_stream = vc;
>>>> +		r->flags = vc == tx->vc ? V4L2_SUBDEV_ROUTE_FL_ACTIVE : 0;
>>>> +		r++;
>>>> +	}
>>>> +
>>>> +	return 0;
>>>> +}
>>>> +
>>>> +static int adv748x_csi2_set_routing(struct v4l2_subdev *sd,
>>>> +				    struct v4l2_subdev_krouting *routing)
>>>> +{
>>>> +	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
>>>> +	struct v4l2_subdev_route *r = routing->routes;
>>>> +	unsigned int i;
>>>> +	int vc = -1;
>>>> +
>>>> +	if (routing->num_routes > ADV748X_CSI2_ROUTES_MAX)
>>>> +		return -ENOSPC;
>>>> +
>>>> +	for (i = 0; i < routing->num_routes; i++) {
>>>> +		if (r->sink_pad != ADV748X_CSI2_SINK ||
>>>> +		    r->sink_stream != 0 ||
>>>> +		    r->source_pad != ADV748X_CSI2_SOURCE ||
>>>> +		    r->source_stream >= ADV748X_CSI2_ROUTES_MAX)
>>>> +			return -EINVAL;
>>>> +
>>>> +		if (r->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE) {
>>>> +			if (vc != -1)
>>>> +				return -EMLINK;
>>>> +
>>>> +			vc = r->source_stream;
>>>> +		}
>>>> +		r++;
>>>> +	}
>>>> +
>>>> +	if (vc != -1)
>>>> +		tx->vc = vc;
>>>> +
>>>> +	adv748x_csi2_set_virtual_channel(tx, tx->vc);
>>>> +
>>>> +	return 0;
>>>> +}
>>>
>>> Not specific to this patch but rather to the set_routing idea as a
>>> whole: can the set_routing ioctl be called while the stream is running?
>>>
>>> If it cannot, I find it a limiting factor for nowadays use cases. I also
>>> didn't find where the ioctl is rejected.
>>>
>>
>> The framework does not make assumptions about that at the moment.
>>
>>> If it can, then shouldn't this function call s_stream(stop) through the
>>> sink pad whose route becomes disabled, and a s_stream(start) through the
>>> one that gets enabled?
>>>
>>
>> If I got this right, you're here rightfully pointing out that changing
>> the routing between pads in an entity migh impact the pipeline as a
>> whole, and this would require, to activate/deactivate devices that
>> where not part of the pipeline.
> 
> I'd say that ultimately this depends on the devices themselves, whether
> they support this or not. In practice I don't think we have any such cases
> at the moment, but it's possible in principle. Changes on the framework may
> well be needed but likely the biggest complications will still be in
> drivers supporting that.

I understand V4L2 currently does not support changing a pipeline that is
running. However there are many use cases that would require it.

Most of the use cases that come to my mind involve a multiplexer with
multiple inputs, one of which can be selected to be forwarded. In those
cases s_routing deselects an input and selects another one. How the can
we handle such cases without sending a s_stream on the two upstreams?
Having all possible inputs always running is not a real solution.

> The media links have a dynamic flag for this purpose but I don't think it's
> ever been used.
> 
>>
>> This is probably the wrong patch to use an example, as this one is for
>> a multiplexed interface, where there is no need to go through an
>> s_stream() for the two CSI-2 endpoints, but as you pointed out in our
>> brief offline chat, the AFE->TX routing example for this very device
>> is a good one: if we change the analogue source that is internally
>> routed to the CSI-2 output of the adv748x, do we need to s_stream(1)
>> the now routed entity and s_stream(0) on the not not-anymore-routed
>> one?
>>
>> My gut feeling is that this is up to userspace, as it should know
>> what are the requirements of the devices in the system, but this mean
>> going through an s_stream(0)/s_stream(1) sequence on the video device,
>> and that would interrupt the streaming for sure.
>>
>> At the same time, I don't feel too much at ease with the idea of
>> s_routing calling s_stream on the entity' remote subdevices, as this
>> would skip the link format validation that media_pipeline_start()
>> performs.
> 
> The link validation must be done in this case as well, it may not be
> simply skipped.

Agreed.

The routing VS pipeline validation point is a very important one. The
current proposed workflow is:

 1. the pipeline is validated as a whole, having knowledge of all the
    entities
 2. streaming is started
 3. s_routing is called on an entity (not on the pipeline!)

Now the s_routing function in the entity driver is not in a good
position to validate the candidate future pipeline as a whole.

Naively I'd say there are two possible solutions:

 1. the s_routing reaches the pipeline first, then the new pipeline is
    computed and verified, and if verification succeeds it is applied
 2. a partial pipeline verification mechanism is added, so the entity
    receiving a s_routing request to e.g. change the sink pad can invoke
    a verification on the part of pipeline that is about to be
    activated, and if verification succeeds it is applied

Somehow I suspect neither is trivial...

-- 
Luca



^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 26/31] adv748x: csi2: add internal routing configuration
  2019-03-16 10:23         ` Luca Ceresoli
@ 2019-03-20 17:14           ` Jacopo Mondi
  2019-03-22 16:52             ` Luca Ceresoli
  0 siblings, 1 reply; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-20 17:14 UTC (permalink / raw)
  To: Luca Ceresoli
  Cc: Sakari Ailus, Jacopo Mondi, laurent.pinchart,
	niklas.soderlund+renesas, linux-media, linux-renesas-soc

[-- Attachment #1: Type: text/plain, Size: 9262 bytes --]

Hi Luca,
   thanks for the input

On Sat, Mar 16, 2019 at 11:23:42AM +0100, Luca Ceresoli wrote:
> Hi Jacopo, Sakari,
>
> On 15/03/19 11:06, Sakari Ailus wrote:
> > Hi Luca, Jacopo,
> >
> > On Fri, Mar 15, 2019 at 10:45:38AM +0100, Jacopo Mondi wrote:
> >> Hi Luca,
> >>
> >> On Thu, Mar 14, 2019 at 03:45:27PM +0100, Luca Ceresoli wrote:
> >>> Hi,
> >>>
> >>> begging your pardon for the noob question below...
> >>>
> >>
> >> Let a noob help another noob then
> >>
> >>> On 05/03/19 19:51, Jacopo Mondi wrote:
> >>>> From: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> >>>>
> >>>> Add support to get and set the internal routing between the adv748x
> >>>> CSI-2 transmitters sink pad and its multiplexed source pad. This routing
> >>>> includes which stream of the multiplexed pad to use, allowing the user
> >>>> to select which CSI-2 virtual channel to use when transmitting the
> >>>> stream.
> >>>>
> >>>> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> >>>> Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> >>>> ---
> >>>>  drivers/media/i2c/adv748x/adv748x-csi2.c | 65 ++++++++++++++++++++++++
> >>>>  1 file changed, 65 insertions(+)
> >>>>
> >>>> diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
> >>>> index d8f7cbee86e7..13454af72c6e 100644
> >>>> --- a/drivers/media/i2c/adv748x/adv748x-csi2.c
> >>>> +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
> >>>> @@ -14,6 +14,8 @@
> >>>>
> >>>>  #include "adv748x.h"
> >>>>
> >>>> +#define ADV748X_CSI2_ROUTES_MAX 4
> >>>> +
> >>>>  struct adv748x_csi2_format {
> >>>>  	unsigned int code;
> >>>>  	unsigned int datatype;
> >>>> @@ -253,10 +255,73 @@ static int adv748x_csi2_get_frame_desc(struct v4l2_subdev *sd, unsigned int pad,
> >>>>  	return 0;
> >>>>  }
> >>>>
> >>>> +static int adv748x_csi2_get_routing(struct v4l2_subdev *sd,
> >>>> +				    struct v4l2_subdev_krouting *routing)
> >>>> +{
> >>>> +	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> >>>> +	struct v4l2_subdev_route *r = routing->routes;
> >>>> +	unsigned int vc;
> >>>> +
> >>>> +	if (routing->num_routes < ADV748X_CSI2_ROUTES_MAX) {
> >>>> +		routing->num_routes = ADV748X_CSI2_ROUTES_MAX;
> >>>> +		return -ENOSPC;
> >>>> +	}
> >>>> +
> >>>> +	routing->num_routes = ADV748X_CSI2_ROUTES_MAX;
> >>>> +
> >>>> +	for (vc = 0; vc < ADV748X_CSI2_ROUTES_MAX; vc++) {
> >>>> +		r->sink_pad = ADV748X_CSI2_SINK;
> >>>> +		r->sink_stream = 0;
> >>>> +		r->source_pad = ADV748X_CSI2_SOURCE;
> >>>> +		r->source_stream = vc;
> >>>> +		r->flags = vc == tx->vc ? V4L2_SUBDEV_ROUTE_FL_ACTIVE : 0;
> >>>> +		r++;
> >>>> +	}
> >>>> +
> >>>> +	return 0;
> >>>> +}
> >>>> +
> >>>> +static int adv748x_csi2_set_routing(struct v4l2_subdev *sd,
> >>>> +				    struct v4l2_subdev_krouting *routing)
> >>>> +{
> >>>> +	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> >>>> +	struct v4l2_subdev_route *r = routing->routes;
> >>>> +	unsigned int i;
> >>>> +	int vc = -1;
> >>>> +
> >>>> +	if (routing->num_routes > ADV748X_CSI2_ROUTES_MAX)
> >>>> +		return -ENOSPC;
> >>>> +
> >>>> +	for (i = 0; i < routing->num_routes; i++) {
> >>>> +		if (r->sink_pad != ADV748X_CSI2_SINK ||
> >>>> +		    r->sink_stream != 0 ||
> >>>> +		    r->source_pad != ADV748X_CSI2_SOURCE ||
> >>>> +		    r->source_stream >= ADV748X_CSI2_ROUTES_MAX)
> >>>> +			return -EINVAL;
> >>>> +
> >>>> +		if (r->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE) {
> >>>> +			if (vc != -1)
> >>>> +				return -EMLINK;
> >>>> +
> >>>> +			vc = r->source_stream;
> >>>> +		}
> >>>> +		r++;
> >>>> +	}
> >>>> +
> >>>> +	if (vc != -1)
> >>>> +		tx->vc = vc;
> >>>> +
> >>>> +	adv748x_csi2_set_virtual_channel(tx, tx->vc);
> >>>> +
> >>>> +	return 0;
> >>>> +}
> >>>
> >>> Not specific to this patch but rather to the set_routing idea as a
> >>> whole: can the set_routing ioctl be called while the stream is running?
> >>>
> >>> If it cannot, I find it a limiting factor for nowadays use cases. I also
> >>> didn't find where the ioctl is rejected.
> >>>
> >>
> >> The framework does not make assumptions about that at the moment.
> >>
> >>> If it can, then shouldn't this function call s_stream(stop) through the
> >>> sink pad whose route becomes disabled, and a s_stream(start) through the
> >>> one that gets enabled?
> >>>
> >>
> >> If I got this right, you're here rightfully pointing out that changing
> >> the routing between pads in an entity migh impact the pipeline as a
> >> whole, and this would require, to activate/deactivate devices that
> >> where not part of the pipeline.
> >
> > I'd say that ultimately this depends on the devices themselves, whether
> > they support this or not. In practice I don't think we have any such cases
> > at the moment, but it's possible in principle. Changes on the framework may
> > well be needed but likely the biggest complications will still be in
> > drivers supporting that.
>
> I understand V4L2 currently does not support changing a pipeline that is
> running. However there are many use cases that would require it.
>
> Most of the use cases that come to my mind involve a multiplexer with
> multiple inputs, one of which can be selected to be forwarded. In those
> cases s_routing deselects an input and selects another one. How the can
> we handle such cases without sending a s_stream on the two upstreams?
> Having all possible inputs always running is not a real solution.

This seems a very specific use case, I might surely be wrong, but if
such a muxing device allows you to switch from one source to another
without interruption, its driver should also take care of several
other things, such as controlling power and format validations. I
don't think that's something that should be considered at framework
level, but again without a real use case I can't tell for real.

>
> > The media links have a dynamic flag for this purpose but I don't think it's
> > ever been used.
> >
> >>
> >> This is probably the wrong patch to use an example, as this one is for
> >> a multiplexed interface, where there is no need to go through an
> >> s_stream() for the two CSI-2 endpoints, but as you pointed out in our
> >> brief offline chat, the AFE->TX routing example for this very device
> >> is a good one: if we change the analogue source that is internally
> >> routed to the CSI-2 output of the adv748x, do we need to s_stream(1)
> >> the now routed entity and s_stream(0) on the not not-anymore-routed
> >> one?
> >>
> >> My gut feeling is that this is up to userspace, as it should know
> >> what are the requirements of the devices in the system, but this mean
> >> going through an s_stream(0)/s_stream(1) sequence on the video device,
> >> and that would interrupt the streaming for sure.
> >>
> >> At the same time, I don't feel too much at ease with the idea of
> >> s_routing calling s_stream on the entity' remote subdevices, as this
> >> would skip the link format validation that media_pipeline_start()
> >> performs.
> >
> > The link validation must be done in this case as well, it may not be
> > simply skipped.
>
> Agreed.
>
> The routing VS pipeline validation point is a very important one. The
> current proposed workflow is:
>
>  1. the pipeline is validated as a whole, having knowledge of all the
>     entities

let me specify this to avoid confusions:
     "all the entities -with an active route in the pipeline-"

>  2. streaming is started
>  3. s_routing is called on an entity (not on the pipeline!)
>
> Now the s_routing function in the entity driver is not in a good
> position to validate the candidate future pipeline as a whole.
>
> Naively I'd say there are two possible solutions:
>
>  1. the s_routing reaches the pipeline first, then the new pipeline is
>     computed and verified, and if verification succeeds it is applied
>  2. a partial pipeline verification mechanism is added, so the entity
>     receiving a s_routing request to e.g. change the sink pad can invoke
>     a verification on the part of pipeline that is about to be
>     activated, and if verification succeeds it is applied
>
> Somehow I suspect neither is trivial...

I would say it is not, but if you have such a device that does not
require going through a s_stream(0)/s_stream(1) cycle and all the
associated re-negotiation and validations, it seems to me nothing
prevents you from handling this in the driver implementation. Maybe it
won't look that great, but this seems to be quite a custom design that
requires all input sources to be linked to your sink pads, their
format validated all at the same time, power, stream activation and
internal mux configuration controlled by s_routing. Am I wrong or
nothing in this series would prevent your from doing this?

tl;dr: I would not make this something the framework should be
concerned about, as there's nothing preventing you from
implementing support for such a use case. But again, without a real
example we can only guess, and I might be overlooking the issue or
missing some relevant detail for sure.

Thanks
   j

>
> --
> Luca
>
>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 26/31] adv748x: csi2: add internal routing configuration
  2019-03-20 17:14           ` Jacopo Mondi
@ 2019-03-22 16:52             ` Luca Ceresoli
  2019-03-28 15:08               ` Jacopo Mondi
  0 siblings, 1 reply; 59+ messages in thread
From: Luca Ceresoli @ 2019-03-22 16:52 UTC (permalink / raw)
  To: Jacopo Mondi
  Cc: Sakari Ailus, Jacopo Mondi, laurent.pinchart,
	niklas.soderlund+renesas, linux-media, linux-renesas-soc

Hi,

thanks for the follow-up.

On 20/03/19 18:14, Jacopo Mondi wrote:
>>>> This is probably the wrong patch to use an example, as this one is for
>>>> a multiplexed interface, where there is no need to go through an
>>>> s_stream() for the two CSI-2 endpoints, but as you pointed out in our
>>>> brief offline chat, the AFE->TX routing example for this very device
>>>> is a good one: if we change the analogue source that is internally
>>>> routed to the CSI-2 output of the adv748x, do we need to s_stream(1)
>>>> the now routed entity and s_stream(0) on the not not-anymore-routed
>>>> one?
>>>>
>>>> My gut feeling is that this is up to userspace, as it should know
>>>> what are the requirements of the devices in the system, but this mean
>>>> going through an s_stream(0)/s_stream(1) sequence on the video device,
>>>> and that would interrupt the streaming for sure.
>>>>
>>>> At the same time, I don't feel too much at ease with the idea of
>>>> s_routing calling s_stream on the entity' remote subdevices, as this
>>>> would skip the link format validation that media_pipeline_start()
>>>> performs.
>>>
>>> The link validation must be done in this case as well, it may not be
>>> simply skipped.
>>
>> Agreed.
>>
>> The routing VS pipeline validation point is a very important one. The
>> current proposed workflow is:
>>
>>  1. the pipeline is validated as a whole, having knowledge of all the
>>     entities
> 
> let me specify this to avoid confusions:
>      "all the entities -with an active route in the pipeline-"
> 
>>  2. streaming is started
>>  3. s_routing is called on an entity (not on the pipeline!)
>>
>> Now the s_routing function in the entity driver is not in a good
>> position to validate the candidate future pipeline as a whole.
>>
>> Naively I'd say there are two possible solutions:
>>
>>  1. the s_routing reaches the pipeline first, then the new pipeline is
>>     computed and verified, and if verification succeeds it is applied
>>  2. a partial pipeline verification mechanism is added, so the entity
>>     receiving a s_routing request to e.g. change the sink pad can invoke
>>     a verification on the part of pipeline that is about to be
>>     activated, and if verification succeeds it is applied
>>
>> Somehow I suspect neither is trivial...
> 
> I would say it is not, but if you have such a device that does not
> require going through a s_stream(0)/s_stream(1) cycle and all the
> associated re-negotiation and validations, it seems to me nothing
> prevents you from handling this in the driver implementation. Maybe it
> won't look that great, but this seems to be quite a custom design that
> requires all input sources to be linked to your sink pads, their
> format validated all at the same time, power, stream activation and
> internal mux configuration controlled by s_routing. Am I wrong or
> nothing in this series would prevent your from doing this?

You're right, nothing prevents me from doing a custom hack for my custom
design. It's what I'm doing right now. My concern is whether the
framework will evolve to allow modifying a running pipeline without
custom hacks.

> tl;dr: I would not make this something the framework should be
> concerned about, as there's nothing preventing you from
> implementing support for such a use case. But again, without a real
> example we can only guess, and I might be overlooking the issue or
> missing some relevant detail for sure.

I'm a bit surprised in observing that my use case looks so strange,
perhaps yours is so different that we don't quite understand each other.
So below is an example of what I have in mind. Can you explain your use
case too?


Here's a use case. Consider a product that takes 3 camera inputs,
selects one of them and produces a continuous video stream with the
camera image and an OSD on top of it. The selected camera can be changed
at any time (e.g. upon user selection).

                  OSD FB ---.
                            |
            .--------.      V
Camera 0 -->|        |   .-----.
Camera 1 -->|  MUX   |-->| OSD |--> DMA --> /dev/video0
Camera 2 -->|        |   `-----'
            `--------'

A prerequisite is obviously that each piece of hardware and software
involved is able to cope with a sudden stream change. Perhaps not that
common, but no rocket science.

It looks to me that each of these pieces can be modeled as an entity and
the s_routing API is a perfect fit for the mux block. Am I wrong?

Thanks,
-- 
Luca

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 26/31] adv748x: csi2: add internal routing configuration
  2019-03-22 16:52             ` Luca Ceresoli
@ 2019-03-28 15:08               ` Jacopo Mondi
  2019-03-28 15:17                 ` Luca Ceresoli
  0 siblings, 1 reply; 59+ messages in thread
From: Jacopo Mondi @ 2019-03-28 15:08 UTC (permalink / raw)
  To: Luca Ceresoli
  Cc: Sakari Ailus, Jacopo Mondi, laurent.pinchart,
	niklas.soderlund+renesas, linux-media, linux-renesas-soc

[-- Attachment #1: Type: text/plain, Size: 5609 bytes --]

Hi Luca,

On Fri, Mar 22, 2019 at 05:52:15PM +0100, Luca Ceresoli wrote:
> Hi,
>
> thanks for the follow-up.
>
> On 20/03/19 18:14, Jacopo Mondi wrote:
> >>>> This is probably the wrong patch to use an example, as this one is for
> >>>> a multiplexed interface, where there is no need to go through an
> >>>> s_stream() for the two CSI-2 endpoints, but as you pointed out in our
> >>>> brief offline chat, the AFE->TX routing example for this very device
> >>>> is a good one: if we change the analogue source that is internally
> >>>> routed to the CSI-2 output of the adv748x, do we need to s_stream(1)
> >>>> the now routed entity and s_stream(0) on the not not-anymore-routed
> >>>> one?
> >>>>
> >>>> My gut feeling is that this is up to userspace, as it should know
> >>>> what are the requirements of the devices in the system, but this mean
> >>>> going through an s_stream(0)/s_stream(1) sequence on the video device,
> >>>> and that would interrupt the streaming for sure.
> >>>>
> >>>> At the same time, I don't feel too much at ease with the idea of
> >>>> s_routing calling s_stream on the entity' remote subdevices, as this
> >>>> would skip the link format validation that media_pipeline_start()
> >>>> performs.
> >>>
> >>> The link validation must be done in this case as well, it may not be
> >>> simply skipped.
> >>
> >> Agreed.
> >>
> >> The routing VS pipeline validation point is a very important one. The
> >> current proposed workflow is:
> >>
> >>  1. the pipeline is validated as a whole, having knowledge of all the
> >>     entities
> >
> > let me specify this to avoid confusions:
> >      "all the entities -with an active route in the pipeline-"
> >
> >>  2. streaming is started
> >>  3. s_routing is called on an entity (not on the pipeline!)
> >>
> >> Now the s_routing function in the entity driver is not in a good
> >> position to validate the candidate future pipeline as a whole.
> >>
> >> Naively I'd say there are two possible solutions:
> >>
> >>  1. the s_routing reaches the pipeline first, then the new pipeline is
> >>     computed and verified, and if verification succeeds it is applied
> >>  2. a partial pipeline verification mechanism is added, so the entity
> >>     receiving a s_routing request to e.g. change the sink pad can invoke
> >>     a verification on the part of pipeline that is about to be
> >>     activated, and if verification succeeds it is applied
> >>
> >> Somehow I suspect neither is trivial...
> >
> > I would say it is not, but if you have such a device that does not
> > require going through a s_stream(0)/s_stream(1) cycle and all the
> > associated re-negotiation and validations, it seems to me nothing
> > prevents you from handling this in the driver implementation. Maybe it
> > won't look that great, but this seems to be quite a custom design that
> > requires all input sources to be linked to your sink pads, their
> > format validated all at the same time, power, stream activation and
> > internal mux configuration controlled by s_routing. Am I wrong or
> > nothing in this series would prevent your from doing this?
>
> You're right, nothing prevents me from doing a custom hack for my custom
> design. It's what I'm doing right now. My concern is whether the
> framework will evolve to allow modifying a running pipeline without
> custom hacks.
>
> > tl;dr: I would not make this something the framework should be
> > concerned about, as there's nothing preventing you from
> > implementing support for such a use case. But again, without a real
> > example we can only guess, and I might be overlooking the issue or
> > missing some relevant detail for sure.
>
> I'm a bit surprised in observing that my use case looks so strange,
> perhaps yours is so different that we don't quite understand each other.
> So below is an example of what I have in mind. Can you explain your use
> case too?

I'm mostly interested in this series as it allows me to add data lane
negotiation at run time. In my case, there are no stream continuity
constraints, but I get your point here.
>
>
> Here's a use case. Consider a product that takes 3 camera inputs,
> selects one of them and produces a continuous video stream with the
> camera image and an OSD on top of it. The selected camera can be changed
> at any time (e.g. upon user selection).
>
>                   OSD FB ---.
>                             |
>             .--------.      V
> Camera 0 -->|        |   .-----.
> Camera 1 -->|  MUX   |-->| OSD |--> DMA --> /dev/video0
> Camera 2 -->|        |   `-----'
>             `--------'
>
> A prerequisite is obviously that each piece of hardware and software
> involved is able to cope with a sudden stream change. Perhaps not that
> common, but no rocket science.
>
> It looks to me that each of these pieces can be modeled as an entity and
> the s_routing API is a perfect fit for the mux block. Am I wrong?
>

Personally, I would say that your muxer driver, if able to switch
source without requiring a pipeline restart, should handle it
internally. There are specific details that the mux should be able to
handle, and we're still pretty vague on the details. As a start, which
is the bus type your sources connects to the muxer with? Parallel?
CSI-2? How is the video stream multiplexed? with CSI-2 VC? Do you need
to control your source power during inactive period? I'm sure there
are more questions... I agree a partial pipeline restart might be
something to consider, but at the same time I think this is not
something strictly required to get this series merged, isn't it?

Thanks
  j


> Thanks,
> --
> Luca

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 26/31] adv748x: csi2: add internal routing configuration
  2019-03-28 15:08               ` Jacopo Mondi
@ 2019-03-28 15:17                 ` Luca Ceresoli
  0 siblings, 0 replies; 59+ messages in thread
From: Luca Ceresoli @ 2019-03-28 15:17 UTC (permalink / raw)
  To: Jacopo Mondi
  Cc: Sakari Ailus, Jacopo Mondi, laurent.pinchart,
	niklas.soderlund+renesas, linux-media, linux-renesas-soc

Hi Jacopo,

On 28/03/19 16:08, Jacopo Mondi wrote:
> Hi Luca,
> 
> On Fri, Mar 22, 2019 at 05:52:15PM +0100, Luca Ceresoli wrote:
>> Hi,
>>
>> thanks for the follow-up.
>>
>> On 20/03/19 18:14, Jacopo Mondi wrote:
>>>>>> This is probably the wrong patch to use an example, as this one is for
>>>>>> a multiplexed interface, where there is no need to go through an
>>>>>> s_stream() for the two CSI-2 endpoints, but as you pointed out in our
>>>>>> brief offline chat, the AFE->TX routing example for this very device
>>>>>> is a good one: if we change the analogue source that is internally
>>>>>> routed to the CSI-2 output of the adv748x, do we need to s_stream(1)
>>>>>> the now routed entity and s_stream(0) on the not not-anymore-routed
>>>>>> one?
>>>>>>
>>>>>> My gut feeling is that this is up to userspace, as it should know
>>>>>> what are the requirements of the devices in the system, but this mean
>>>>>> going through an s_stream(0)/s_stream(1) sequence on the video device,
>>>>>> and that would interrupt the streaming for sure.
>>>>>>
>>>>>> At the same time, I don't feel too much at ease with the idea of
>>>>>> s_routing calling s_stream on the entity' remote subdevices, as this
>>>>>> would skip the link format validation that media_pipeline_start()
>>>>>> performs.
>>>>>
>>>>> The link validation must be done in this case as well, it may not be
>>>>> simply skipped.
>>>>
>>>> Agreed.
>>>>
>>>> The routing VS pipeline validation point is a very important one. The
>>>> current proposed workflow is:
>>>>
>>>>  1. the pipeline is validated as a whole, having knowledge of all the
>>>>     entities
>>>
>>> let me specify this to avoid confusions:
>>>      "all the entities -with an active route in the pipeline-"
>>>
>>>>  2. streaming is started
>>>>  3. s_routing is called on an entity (not on the pipeline!)
>>>>
>>>> Now the s_routing function in the entity driver is not in a good
>>>> position to validate the candidate future pipeline as a whole.
>>>>
>>>> Naively I'd say there are two possible solutions:
>>>>
>>>>  1. the s_routing reaches the pipeline first, then the new pipeline is
>>>>     computed and verified, and if verification succeeds it is applied
>>>>  2. a partial pipeline verification mechanism is added, so the entity
>>>>     receiving a s_routing request to e.g. change the sink pad can invoke
>>>>     a verification on the part of pipeline that is about to be
>>>>     activated, and if verification succeeds it is applied
>>>>
>>>> Somehow I suspect neither is trivial...
>>>
>>> I would say it is not, but if you have such a device that does not
>>> require going through a s_stream(0)/s_stream(1) cycle and all the
>>> associated re-negotiation and validations, it seems to me nothing
>>> prevents you from handling this in the driver implementation. Maybe it
>>> won't look that great, but this seems to be quite a custom design that
>>> requires all input sources to be linked to your sink pads, their
>>> format validated all at the same time, power, stream activation and
>>> internal mux configuration controlled by s_routing. Am I wrong or
>>> nothing in this series would prevent your from doing this?
>>
>> You're right, nothing prevents me from doing a custom hack for my custom
>> design. It's what I'm doing right now. My concern is whether the
>> framework will evolve to allow modifying a running pipeline without
>> custom hacks.
>>
>>> tl;dr: I would not make this something the framework should be
>>> concerned about, as there's nothing preventing you from
>>> implementing support for such a use case. But again, without a real
>>> example we can only guess, and I might be overlooking the issue or
>>> missing some relevant detail for sure.
>>
>> I'm a bit surprised in observing that my use case looks so strange,
>> perhaps yours is so different that we don't quite understand each other.
>> So below is an example of what I have in mind. Can you explain your use
>> case too?
> 
> I'm mostly interested in this series as it allows me to add data lane
> negotiation at run time. In my case, there are no stream continuity
> constraints, but I get your point here.
>>
>>
>> Here's a use case. Consider a product that takes 3 camera inputs,
>> selects one of them and produces a continuous video stream with the
>> camera image and an OSD on top of it. The selected camera can be changed
>> at any time (e.g. upon user selection).
>>
>>                   OSD FB ---.
>>                             |
>>             .--------.      V
>> Camera 0 -->|        |   .-----.
>> Camera 1 -->|  MUX   |-->| OSD |--> DMA --> /dev/video0
>> Camera 2 -->|        |   `-----'
>>             `--------'
>>
>> A prerequisite is obviously that each piece of hardware and software
>> involved is able to cope with a sudden stream change. Perhaps not that
>> common, but no rocket science.
>>
>> It looks to me that each of these pieces can be modeled as an entity and
>> the s_routing API is a perfect fit for the mux block. Am I wrong?
>>
> 
> Personally, I would say that your muxer driver, if able to switch
> source without requiring a pipeline restart, should handle it
> internally. There are specific details that the mux should be able to
> handle, and we're still pretty vague on the details. As a start, which
> is the bus type your sources connects to the muxer with? Parallel?
> CSI-2? How is the video stream multiplexed? with CSI-2 VC? Do you need
> to control your source power during inactive period?

Not completely defined, but it could be either parallel or MIPI CSI-2,
and each input could be connected either directly from the sensor or
through a ser/des. Controlling power would be important as well.

> I'm sure there
> are more questions... I agree a partial pipeline restart might be
> something to consider, but at the same time I think this is not
> something strictly required to get this series merged, isn't it?

Right. As far as I can understand this series is already OK if you need
to change routing while the stream is not running.

-- 
Luca

^ permalink raw reply	[flat|nested] 59+ messages in thread

* Re: [PATCH v3 19/31] media: Documentation: Add GS_ROUTING documentation
  2019-03-05 18:51 ` [PATCH v3 19/31] media: Documentation: Add GS_ROUTING documentation Jacopo Mondi
  2019-03-07 15:19   ` Sakari Ailus
  2019-03-14 14:43   ` Luca Ceresoli
@ 2020-02-13 13:36   ` Hans Verkuil
  2 siblings, 0 replies; 59+ messages in thread
From: Hans Verkuil @ 2020-02-13 13:36 UTC (permalink / raw)
  To: Jacopo Mondi, sakari.ailus, laurent.pinchart, niklas.soderlund+renesas
  Cc: linux-media, linux-renesas-soc

Laurent asked me to review the approach taken in this series since he wants
to restart this.

I'm not reviewing the code: it's old by now, and it is really the uAPI that is
of interest, so I am concentrating on this patch that documents this feature.

On 3/5/19 7:51 PM, Jacopo Mondi wrote:
> Add documentation for VIDIOC_SUBDEV_G/S_ROUTING ioctl and add
> description of multiplexed media pads and internal routing to the
> V4L2-subdev documentation section.
> 
> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> ---
>  Documentation/media/uapi/v4l/dev-subdev.rst   |  90 +++++++++++
>  Documentation/media/uapi/v4l/user-func.rst    |   1 +
>  .../uapi/v4l/vidioc-subdev-g-routing.rst      | 142 ++++++++++++++++++
>  3 files changed, 233 insertions(+)
>  create mode 100644 Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst
> 
> diff --git a/Documentation/media/uapi/v4l/dev-subdev.rst b/Documentation/media/uapi/v4l/dev-subdev.rst
> index 2c2768c7343b..b9fbb5d2caec 100644
> --- a/Documentation/media/uapi/v4l/dev-subdev.rst
> +++ b/Documentation/media/uapi/v4l/dev-subdev.rst
> @@ -36,6 +36,8 @@ will feature a character device node on which ioctls can be called to
>  
>  -  negotiate image formats on individual pads
>  
> +-  inspect and modify internal data routing between pads of the same entity
> +
>  Sub-device character device nodes, conventionally named
>  ``/dev/v4l-subdev*``, use major number 81.
>  
> @@ -461,3 +463,91 @@ source pads.
>      :maxdepth: 1
>  
>      subdev-formats
> +
> +
> +Multiplexed media pads and internal routing
> +-------------------------------------------
> +
> +Subdevice drivers might expose the internal connections between media pads of an
> +entity by providing a routing table that applications can inspect and manipulate
> +to change the internal routing between sink and source pads' data connection
> +endpoints. A routing table is described by a struct
> +:c:type:`v4l2_subdev_routing`, which contains ``num_routes`` route entries, each
> +one represented by a struct :c:type:`v4l2_subdev_route`.
> +
> +Data routes do not just connect one pad to another in an entity, but they refer
> +instead to the ``streams`` a media pad provides. Streams are data connection
> +endpoints in a media pad. Multiplexed media pads expose multiple ``streams``
> +which represent, when the underlying hardware technology allows that, logical
> +data streams transported over a single physical media bus.
> +
> +One of the most notable examples of logical stream multiplexing techniques is
> +represented by the data interleaving mechanism implemented by mean of Virtual
> +Channels identifiers as defined by the MIPI CSI-2 media bus specifications. A
> +subdevice that implements support for Virtual Channel data interleaving might
> +expose up to 4 data ``streams``, one for each available Virtual Channel, on the
> +source media pad that represents a CSI-2 connection endpoint.
> +
> +Routes are defined as potential data connections between a ``(sink_pad,
> +sink_stream)`` pair and a ``(source_pad, source_stream)`` one, where
> +``sink_pad`` and ``source_pad`` are the indexes of two media pads part of the
> +same media entity, and ``sink_stream`` and ``source_stream`` are the identifiers
> +of the data streams to be connected in the media pads. Media pads that do not
> +support data multiplexing expose a single stream, usually with identifier 0.

Why usually? Is there any reason why it can be non-0?

Existing entities do not support routes. Will the new G/S_ROUTING ioctls
fail on those? How does userspace detect that an entity support routing?
I advice adding an entity flag or something similar.

If you have a pad that supports multiplexing, what is then the mediabus
format of that pad? Can each stream have a different mediabus format? If so,
how to you discover that? This really needs to be clarified and documented.

I assume both sink and source pads can be multiplexed, so you can have an
entity with two non-multiplexed sink pads that both route to the same
multiplexed source pad, or vice versa.

Can a single (sink pad, sink stream) be routed to two different source pads?

Can all routes be disabled?

> +
> +Routes are reported to applications in a routing table which can be
> +inspected and manipulated using the :ref:`routing <VIDIOC_SUBDEV_G_ROUTING>`
> +ioctls.
> +
> +Routes can be activated and deactivated by setting or clearing the
> +V4L2_SUBDEV_ROUTE_FL_ACTIVE flag in the ``flags`` field of struct
> +:c:type:`v4l2_subdev_route`.
> +
> +Subdev driver might create routes that cannot be modified by applications. Such
> +routes are identified by the presence of the V4L2_SUBDEV_ROUTE_FL_IMMUTABLE
> +flag in the ``flags`` field of struct :c:type:`v4l2_subdev_route`.
> +
> +As an example, the routing table of a source pad which supports two logical
> +video streams and can be connected to two sink pads is here below described.
> +
> +.. flat-table::
> +    :widths:       1 2 1
> +
> +    * - Pad Index
> +      - Function
> +      - Number of streams
> +    * - 0
> +      - SINK
> +      - 1
> +    * - 1
> +      - SINK
> +      - 1
> +    * - 2
> +      - SOURCE
> +      - 2
> +
> +In such an example, the source media pad will report a routing table with 4
> +entries, one entry for each possible ``(sink_pad, sink_stream) - (source_pad,
> +source_stream)`` combination.
> +
> +.. flat-table:: routing table
> +    :widths:       2 1 2
> +
> +    * - Sink Pad/Sink Stream
> +      - ->
> +      - Source Pad/Source Stream
> +    * - 0/0
> +      - ->
> +      - 2/0
> +    * - 0/0
> +      - ->
> +      - 2/1
> +    * - 1/0
> +      - ->
> +      - 2/0
> +    * - 1/0
> +      - ->
> +      - 2/1
> +
> +Subdev drivers are free to decide how many routes an application can enable on
> +a media pad at the same time, and refuse to enable or disable specific routes.
> diff --git a/Documentation/media/uapi/v4l/user-func.rst b/Documentation/media/uapi/v4l/user-func.rst
> index ca0ef21d77fe..0166446f4ab4 100644
> --- a/Documentation/media/uapi/v4l/user-func.rst
> +++ b/Documentation/media/uapi/v4l/user-func.rst
> @@ -77,6 +77,7 @@ Function Reference
>      vidioc-subdev-g-crop
>      vidioc-subdev-g-fmt
>      vidioc-subdev-g-frame-interval
> +    vidioc-subdev-g-routing
>      vidioc-subdev-g-selection
>      vidioc-subscribe-event
>      func-mmap
> diff --git a/Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst b/Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst
> new file mode 100644
> index 000000000000..8b592722c477
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/vidioc-subdev-g-routing.rst
> @@ -0,0 +1,142 @@
> +.. Permission is granted to copy, distribute and/or modify this
> +.. document under the terms of the GNU Free Documentation License,
> +.. Version 1.1 or any later version published by the Free Software
> +.. Foundation, with no Invariant Sections, no Front-Cover Texts
> +.. and no Back-Cover Texts. A copy of the license is included at
> +.. Documentation/media/uapi/fdl-appendix.rst.
> +..
> +.. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections
> +
> +.. _VIDIOC_SUBDEV_G_ROUTING:
> +
> +******************************************************
> +ioctl VIDIOC_SUBDEV_G_ROUTING, VIDIOC_SUBDEV_S_ROUTING
> +******************************************************
> +
> +Name
> +====
> +
> +VIDIOC_SUBDEV_G_ROUTING - VIDIOC_SUBDEV_S_ROUTING - Get or set routing between streams of media pads in a media entity.
> +
> +
> +Synopsis
> +========
> +
> +.. c:function:: int ioctl( int fd, VIDIOC_SUBDEV_G_ROUTING, struct v4l2_subdev_routing *argp )
> +    :name: VIDIOC_SUBDEV_G_ROUTING
> +
> +.. c:function:: int ioctl( int fd, VIDIOC_SUBDEV_S_ROUTING, struct v4l2_subdev_routing *argp )
> +    :name: VIDIOC_SUBDEV_S_ROUTING
> +
> +
> +Arguments
> +=========
> +
> +``fd``
> +    File descriptor returned by :ref:`open() <func-open>`.
> +
> +``argp``
> +    Pointer to struct :c:type:`v4l2_subdev_routing`.
> +
> +
> +Description
> +===========
> +
> +These ioctls are used to get and set the routing informations associated to
> +media pads in a media entity. Routing informations are used to enable or disable
> +data connections between stream endpoints of multiplexed media pads.
> +
> +Drivers report their routing tables using VIDIOC_SUBDEV_G_ROUTING and
> +application use the information there reported to enable or disable data
> +connections between streams in a pad, by setting or clearing the
> +V4L2_SUBDEV_ROUTE_FL_ACTIVE flag of ``flags`` field of a struct
> +:c:type:`v4l2_subdev_route`.
> +
> +When inspecting routes through VIDIOC_SUBDEV_G_ROUTING and the application
> +provided ``num_routes`` is not big enough to contain all the available routes
> +the subdevice exposes, drivers return the ENOSPC error code and adjust the
> +``num_routes`` value. Application should then reserve enough memory for all the
> +route entries and call VIDIOC_SUBDEV_G_ROUTING again.
> +
> +.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}|
> +
> +.. c:type:: v4l2_subdev_routing
> +
> +.. flat-table:: struct v4l2_subdev_routing
> +    :header-rows:  0
> +    :stub-columns: 0
> +    :widths:       1 1 2
> +
> +    * - struct :c:type:`v4l2_subdev_route`
> +      - ``routes[]``
> +      - Array of struct :c:type:`v4l2_subdev_route` entries
> +    * - __u32
> +      - ``num_routes``
> +      - Number of entries of the routes array
> +    * - __u32
> +      - ``reserved``\ [5]
> +      - Reserved for future extensions. Applications and drivers must set
> +	the array to zero.

I'm wondering why this is done through a subdevice ioctl. Should this be done
through media controller ioctls? You are changing the topology, and if you want
to do that atomically through e.g. a future S_TOPOLOGY ioctl, then the routing
information inside an entity should be part of the topology. In fact, if multiple
sensors have their video streams combined into a multiplexed stream over CSI, and
then the streams are demultiplexed for post-processing, don't you have to be able
to follow the complete route the video takes anyway via the media topology?

It feels to me that you need MEDIA_IOC_ENUM_ROUTES and MEDIA_IOC_SETUP_ROUTE(S)
functions and integrate it into G_TOPOLOGY as well. Also, this isn't V4L2 specific,
it's a general routing issue that is much better handled by the media controller.
Especially since the routing support is added to the MC in the first half of this
series.

Regards,

	Hans

> +
> +.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}|
> +
> +.. c:type:: v4l2_subdev_route
> +
> +.. flat-table:: struct v4l2_subdev_route
> +    :header-rows:  0
> +    :stub-columns: 0
> +    :widths:       1 1 2
> +
> +    * - __u32
> +      - ``sink_pad``
> +      - Sink pad number
> +    * - __u32
> +      - ``sink_stream``
> +      - Sink pad stream number
> +    * - __u32
> +      - ``source_stream``
> +      - Source pad stream number
> +    * - __u32
> +      - ``sink_stream``
> +      - Sink pad stream number
> +    * - __u32
> +      - ``flags``
> +      - Route enable/disable flags
> +	:ref:`v4l2_subdev_routing_flags <v4l2-subdev-routing-flags>`.
> +    * - __u32
> +      - ``reserved``\ [5]
> +      - Reserved for future extensions. Applications and drivers must set
> +	the array to zero.
> +
> +.. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
> +
> +.. _v4l2-subdev-routing-flags:
> +
> +.. flat-table:: enum v4l2_subdev_routing_flags
> +    :header-rows:  0
> +    :stub-columns: 0
> +    :widths:       3 1 4
> +
> +    * - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> +      - 0
> +      - The route is enabled. Set by applications.
> +    * - V4L2_SUBDEV_ROUTE_FL_IMMUTABLE
> +      - 1
> +      - The route is immutable. Set by the driver.
> +
> +Return Value
> +============
> +
> +On success 0 is returned, on error -1 and the ``errno`` variable is set
> +appropriately. The generic error codes are described at the
> +:ref:`Generic Error Codes <gen-errors>` chapter.
> +
> +ENOSPC
> +   The number of provided route entries is less than the available ones.
> +
> +EINVAL
> +   The sink or source pad identifiers reference a non-existing pad, or refernce
> +   pads of different types (ie. the sink_pad identifiers refers to a source pad)
> +   The sink or source stream identifiers reference a non-existing stream
> +   in the sink or source pad.
> +
> 


^ permalink raw reply	[flat|nested] 59+ messages in thread

end of thread, back to index

Thread overview: 59+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-03-05 18:51 [PATCH v3 00/31] v4l: add support for multiplexed streams Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 01/31] media: entity: Use pad as a starting point for graph walk Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 02/31] media: entity: Use pads instead of entities in the media graph walk stack Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 03/31] media: entity: Walk the graph based on pads Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 04/31] v4l: mc: Start walk from a specific pad in use count calculation Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 05/31] media: entity: Add iterator helper for entity pads Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 06/31] media: entity: Move the pipeline from entity to pads Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 07/31] media: entity: Use pad as the starting point for a pipeline Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 08/31] media: entity: Add has_route entity operation Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 09/31] media: entity: Add media_has_route() function Jacopo Mondi
2019-03-14 14:45   ` Luca Ceresoli
2019-03-15  9:22     ` Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 10/31] media: entity: Use routing information during graph traversal Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 11/31] media: entity: Skip link validation for pads to which there is no route to Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 12/31] media: entity: Add an iterator helper for connected pads Jacopo Mondi
2019-03-07 10:09   ` Sakari Ailus
2019-03-07 10:27     ` Ian Arkver
2019-03-07 12:38       ` Sakari Ailus
2019-03-07 20:18       ` Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 13/31] media: entity: Add only connected pads to the pipeline Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 14/31] media: entity: Add debug information in graph walk route check Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 15/31] v4l: Add bus type to frame descriptors Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 16/31] v4l: Add CSI-2 bus configuration " Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 17/31] v4l: Add stream to frame descriptor Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 18/31] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 19/31] media: Documentation: Add GS_ROUTING documentation Jacopo Mondi
2019-03-07 15:19   ` Sakari Ailus
2019-03-08 13:31     ` Jacopo Mondi
2019-03-08 14:14       ` Sakari Ailus
2019-03-14 14:44     ` Luca Ceresoli
2019-03-14 14:43   ` Luca Ceresoli
2020-02-13 13:36   ` Hans Verkuil
2019-03-05 18:51 ` [PATCH v3 20/31] v4l: subdev: Take routing information into account in link validation Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 21/31] v4l: subdev: Improve link format validation debug messages Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 22/31] v4l: mc: Add an S_ROUTING helper function for power state changes Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 23/31] adv748x: csi2: add translation from pixelcode to CSI-2 datatype Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 24/31] adv748x: csi2: only allow formats on sink pads Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 25/31] adv748x: csi2: describe the multiplexed stream Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 26/31] adv748x: csi2: add internal routing configuration Jacopo Mondi
2019-03-14 14:45   ` Luca Ceresoli
2019-03-15  9:45     ` Jacopo Mondi
2019-03-15 10:06       ` Sakari Ailus
2019-03-16 10:23         ` Luca Ceresoli
2019-03-20 17:14           ` Jacopo Mondi
2019-03-22 16:52             ` Luca Ceresoli
2019-03-28 15:08               ` Jacopo Mondi
2019-03-28 15:17                 ` Luca Ceresoli
2019-03-05 18:51 ` [PATCH v3 27/31] adv748x: afe: add routing support Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 28/31] adv748x: afe: Implement has_route() Jacopo Mondi
2019-03-07 12:49   ` Sakari Ailus
2019-03-14 14:45   ` Luca Ceresoli
2019-03-05 18:51 ` [PATCH v3 29/31] rcar-csi2: use frame description information to configure CSI-2 bus Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 30/31] rcar-csi2: expose the subdevice internal routing Jacopo Mondi
2019-03-05 18:51 ` [PATCH v3 31/31] media: rcar-csi2: Implement has_route() Jacopo Mondi
2019-03-07 12:56   ` Sakari Ailus
2019-03-07 22:28     ` Jacopo Mondi
2019-03-07  9:47 ` [PATCH v3 00/31] v4l: add support for multiplexed streams Sakari Ailus
2019-03-08 13:19   ` Jacopo Mondi
2019-03-08 14:12     ` Sakari Ailus

Linux-Media Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-media/0 linux-media/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-media linux-media/ https://lore.kernel.org/linux-media \
		linux-media@vger.kernel.org
	public-inbox-index linux-media

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-media


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git