Linux-Media Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH v5 00/24] v4l: subdev internal routing
@ 2021-04-15 13:04 Tomi Valkeinen
  2021-04-15 13:04 ` [PATCH v5 01/24] media: entity: Use pad as a starting point for graph walk Tomi Valkeinen
                   ` (25 more replies)
  0 siblings, 26 replies; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen

Hi,

This is an RFC for subdev internal routing which is needed for
multiplexed streams support. I believe this is essentially a v5 of the
series, the v4 posted here:

https://lore.kernel.org/linux-media/20190328200608.9463-1-jacopo+renesas@jmondi.org/

Most of the patches are not changed (aside from fixing rebase issues
etc). The major changes in this version are:

1) Added 'which' field to the routing structs. It is currently not used,
as implementing it is not trivial. However, I think it's good to add it
to the uAPI now, and require the field to be set to
V4L2_SUBDEV_FORMAT_ACTIVE for now. See this RFC for an idea how this
could be implemented:

https://lore.kernel.org/linux-media/20210409133659.389544-1-tomi.valkeinen@ideasonboard.com/

2) No hardcoded maximum number of routes. Defining a maximum is not
possible, as there can be an arbitrary amount of routes per pad, and
there can be an arbitrary amount of pads per subdev. This series
allocates space for the routing table dynamically, which unfortunately
leads to not-just-a-few allocs and frees.

3) When searching for a format for a stream, the v4 looked for a
non-multiplexed pad only as far as the "other" side of the subdev. It
wouldn't work for a subdev which has multiplexed sink and source pads.
This series implements a "deep" get-format (v4l2_subdev_get_format_dir)
which follows a stream either towards the original source or the final
sink, while looking for a non-multiplexed pad with a format.

Some thoughts:

1) Link validation and v4l2_subdev_get_format_dir need to look at the
routing, and this leads to multiple allocs to get a copy of the routing
table. There might be a possibility here to keep a table allocated and
re-use it in consecutive get_routing calls.

Or even better, perhaps the kAPI could be changed so that allocs are not
needed. I thought about a kAPI where the subdev just returns a pointer
to its routing table, but then we hit the life-cycle problem: how to
ensure the table won't be freed or changed until the caller is done.

2) The routing uAPI is a bit vague. There is no way for the userspace to
figure out what kind of routing is allowed. Also, the existence of a
route in the routing table already indicates that the route is active,
but we also have V4L2_SUBDEV_ROUTE_FL_ACTIVE. I decided to keep
V4L2_SUBDEV_ROUTE_FL_ACTIVE for now, even if it doesn't really provide
any feature.

3) V4L2_FRAME_DESC_ENTRY_MAX is defined as 8 (I change it from 4 to 8 in
this series). This limits the number of streams per pad to 8. Preferably
the number of frame descs would be unlimited, but I didn't start
tackling this. I believe 8 is quite safe number (4 pixel streams and 4
embedded data stream).

4) Link validation ends up following the same routes multiple times, as
each stream in each subdev is validated separately.

 Tomi

Jacopo Mondi (2):
  media: entity: Add iterator helper for entity pads
  media: Documentation: Add GS_ROUTING documentation

Laurent Pinchart (4):
  media: entity: Add has_route entity operation
  media: entity: Add media_entity_has_route() function
  media: entity: Use routing information during graph traversal
  v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations

Sakari Ailus (14):
  media: entity: Use pad as a starting point for graph walk
  media: entity: Use pads instead of entities in the media graph walk
    stack
  media: entity: Walk the graph based on pads
  v4l: mc: Start walk from a specific pad in use count calculation
  media: entity: Move the pipeline from entity to pads
  media: entity: Use pad as the starting point for a pipeline
  media: entity: Skip link validation for pads to which there is no
    route to
  media: entity: Add an iterator helper for connected pads
  media: entity: Add only connected pads to the pipeline
  media: entity: Add debug information in graph walk route check
  v4l: Add bus type to frame descriptors
  v4l: Add CSI-2 bus configuration to frame descriptors
  v4l: Add stream to frame descriptor
  v4l: mc: Add an S_ROUTING helper function for power state changes

Tomi Valkeinen (4):
  v4l: subdev: routing kernel helper functions
  v4l: subdev: add v4l2_subdev_get_format_dir()
  v4l: subdev: Take routing information into account in link validation
  v4l: subdev: increase V4L2_FRAME_DESC_ENTRY_MAX to 8

 Documentation/driver-api/media/mc-core.rst    |  15 +-
 .../userspace-api/media/v4l/dev-subdev.rst    |  92 +++++
 .../userspace-api/media/v4l/user-func.rst     |   1 +
 .../media/v4l/vidioc-subdev-g-routing.rst     | 143 +++++++
 drivers/media/mc/mc-device.c                  |  13 +-
 drivers/media/mc/mc-entity.c                  | 239 +++++++-----
 drivers/media/pci/intel/ipu3/ipu3-cio2-main.c |   6 +-
 .../media/platform/exynos4-is/fimc-capture.c  |   8 +-
 .../platform/exynos4-is/fimc-isp-video.c      |   8 +-
 drivers/media/platform/exynos4-is/fimc-isp.c  |   2 +-
 drivers/media/platform/exynos4-is/fimc-lite.c |  10 +-
 drivers/media/platform/exynos4-is/media-dev.c |  20 +-
 drivers/media/platform/omap3isp/isp.c         |   2 +-
 drivers/media/platform/omap3isp/ispvideo.c    |  25 +-
 drivers/media/platform/omap3isp/ispvideo.h    |   2 +-
 .../media/platform/qcom/camss/camss-video.c   |   6 +-
 drivers/media/platform/rcar-vin/rcar-core.c   |  16 +-
 drivers/media/platform/rcar-vin/rcar-dma.c    |   8 +-
 .../platform/rockchip/rkisp1/rkisp1-capture.c |   6 +-
 .../media/platform/s3c-camif/camif-capture.c  |   6 +-
 drivers/media/platform/stm32/stm32-dcmi.c     |   6 +-
 .../platform/sunxi/sun4i-csi/sun4i_dma.c      |   6 +-
 .../platform/sunxi/sun6i-csi/sun6i_video.c    |   6 +-
 drivers/media/platform/ti-vpe/cal-video.c     |   6 +-
 drivers/media/platform/vsp1/vsp1_video.c      |  18 +-
 drivers/media/platform/xilinx/xilinx-dma.c    |  20 +-
 drivers/media/platform/xilinx/xilinx-dma.h    |   2 +-
 .../media/test-drivers/vimc/vimc-capture.c    |   6 +-
 drivers/media/usb/au0828/au0828-core.c        |   8 +-
 drivers/media/v4l2-core/v4l2-ioctl.c          |  25 +-
 drivers/media/v4l2-core/v4l2-mc.c             |  77 ++--
 drivers/media/v4l2-core/v4l2-subdev.c         | 368 ++++++++++++++++++
 drivers/staging/media/imx/imx-media-utils.c   |   8 +-
 drivers/staging/media/ipu3/ipu3-v4l2.c        |   6 +-
 drivers/staging/media/omap4iss/iss.c          |   2 +-
 drivers/staging/media/omap4iss/iss_video.c    |  38 +-
 drivers/staging/media/omap4iss/iss_video.h    |   2 +-
 drivers/staging/media/tegra-video/tegra210.c  |   6 +-
 include/media/media-entity.h                  | 143 +++++--
 include/media/v4l2-mc.h                       |  22 ++
 include/media/v4l2-subdev.h                   |  93 ++++-
 include/uapi/linux/v4l2-subdev.h              |  44 +++
 42 files changed, 1241 insertions(+), 299 deletions(-)
 create mode 100644 Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst

-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 01/24] media: entity: Use pad as a starting point for graph walk
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-15 13:04 ` [PATCH v5 02/24] media: entity: Use pads instead of entities in the media graph walk stack Tomi Valkeinen
                   ` (24 subsequent siblings)
  25 siblings, 0 replies; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil

From: Sakari Ailus <sakari.ailus@linux.intel.com>

With the upcoming use of the recently added has_route() media entity op, all
the pads in an entity will no longer be considered interconnected. This has
an effect where the media graph is traversed: the starting pad does make a
difference.

Prepare for this change by using pad instead of the entity as an argument
for the graph walk operations. The actual graph traversal algorithm change
is in further patches.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 Documentation/driver-api/media/mc-core.rst    |  2 +-
 drivers/media/mc/mc-entity.c                  | 17 ++++++++---------
 drivers/media/platform/exynos4-is/media-dev.c |  4 ++--
 drivers/media/platform/omap3isp/ispvideo.c    |  2 +-
 drivers/media/platform/vsp1/vsp1_video.c      |  2 +-
 drivers/media/platform/xilinx/xilinx-dma.c    |  2 +-
 drivers/media/v4l2-core/v4l2-mc.c             |  6 +++---
 drivers/staging/media/omap4iss/iss_video.c    |  4 ++--
 include/media/media-entity.h                  | 10 ++++------
 9 files changed, 23 insertions(+), 26 deletions(-)

diff --git a/Documentation/driver-api/media/mc-core.rst b/Documentation/driver-api/media/mc-core.rst
index 57b5bbba944e..ba0aee982124 100644
--- a/Documentation/driver-api/media/mc-core.rst
+++ b/Documentation/driver-api/media/mc-core.rst
@@ -167,7 +167,7 @@ Drivers initiate a graph traversal by calling
 :c:func:`media_graph_walk_start()`
 
 The graph structure, provided by the caller, is initialized to start graph
-traversal at the given entity.
+traversal at the given pad in an entity.
 
 Drivers can then retrieve the next entity by calling
 :c:func:`media_graph_walk_next()`
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 12b45e669bcc..459a5bb4ab41 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -291,17 +291,16 @@ void media_graph_walk_cleanup(struct media_graph *graph)
 }
 EXPORT_SYMBOL_GPL(media_graph_walk_cleanup);
 
-void media_graph_walk_start(struct media_graph *graph,
-			    struct media_entity *entity)
+void media_graph_walk_start(struct media_graph *graph, struct media_pad *pad)
 {
 	media_entity_enum_zero(&graph->ent_enum);
-	media_entity_enum_set(&graph->ent_enum, entity);
+	media_entity_enum_set(&graph->ent_enum, pad->entity);
 
 	graph->top = 0;
 	graph->stack[graph->top].entity = NULL;
-	stack_push(graph, entity);
-	dev_dbg(entity->graph_obj.mdev->dev,
-		"begin graph walk at '%s'\n", entity->name);
+	stack_push(graph, pad->entity);
+	dev_dbg(pad->graph_obj.mdev->dev,
+		"begin graph walk at '%s':%u\n", pad->entity->name, pad->index);
 }
 EXPORT_SYMBOL_GPL(media_graph_walk_start);
 
@@ -419,7 +418,7 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 			goto error_graph_walk_start;
 	}
 
-	media_graph_walk_start(&pipe->graph, entity);
+	media_graph_walk_start(&pipe->graph, entity->pads);
 
 	while ((entity = media_graph_walk_next(graph))) {
 		DECLARE_BITMAP(active, MEDIA_ENTITY_MAX_PADS);
@@ -503,7 +502,7 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 	 * Link validation on graph failed. We revert what we did and
 	 * return the error.
 	 */
-	media_graph_walk_start(graph, entity_err);
+	media_graph_walk_start(graph, entity_err->pads);
 
 	while ((entity_err = media_graph_walk_next(graph))) {
 		/* Sanity check for negative stream_count */
@@ -554,7 +553,7 @@ void __media_pipeline_stop(struct media_entity *entity)
 	if (WARN_ON(!pipe))
 		return;
 
-	media_graph_walk_start(graph, entity);
+	media_graph_walk_start(graph, entity->pads);
 
 	while ((entity = media_graph_walk_next(graph))) {
 		/* Sanity check for negative stream_count */
diff --git a/drivers/media/platform/exynos4-is/media-dev.c b/drivers/media/platform/exynos4-is/media-dev.c
index 13d192ba4aa6..d90663b65932 100644
--- a/drivers/media/platform/exynos4-is/media-dev.c
+++ b/drivers/media/platform/exynos4-is/media-dev.c
@@ -1175,7 +1175,7 @@ static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
 	 * through active links. This is needed as we cannot power on/off the
 	 * subdevs in random order.
 	 */
-	media_graph_walk_start(graph, entity);
+	media_graph_walk_start(graph, entity->pads);
 
 	while ((entity = media_graph_walk_next(graph))) {
 		if (!is_media_entity_v4l2_video_device(entity))
@@ -1190,7 +1190,7 @@ static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
 	return 0;
 
 err:
-	media_graph_walk_start(graph, entity_err);
+	media_graph_walk_start(graph, entity_err->pads);
 
 	while ((entity_err = media_graph_walk_next(graph))) {
 		if (!is_media_entity_v4l2_video_device(entity_err))
diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
index 8811d6dd4ee7..3c1485d59404 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -234,7 +234,7 @@ static int isp_video_get_graph_data(struct isp_video *video,
 		return ret;
 	}
 
-	media_graph_walk_start(&graph, entity);
+	media_graph_walk_start(&graph, entity->pads);
 
 	while ((entity = media_graph_walk_next(&graph))) {
 		struct isp_video *__video;
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index 044eb5778820..61e4fbaba7b7 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -569,7 +569,7 @@ static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
 	if (ret)
 		return ret;
 
-	media_graph_walk_start(&graph, entity);
+	media_graph_walk_start(&graph, entity->pads);
 
 	while ((entity = media_graph_walk_next(&graph))) {
 		struct v4l2_subdev *subdev;
diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index 2a56201cb853..d64c3bee8b95 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -190,7 +190,7 @@ static int xvip_pipeline_validate(struct xvip_pipeline *pipe,
 		return ret;
 	}
 
-	media_graph_walk_start(&graph, entity);
+	media_graph_walk_start(&graph, entity->pads);
 
 	while ((entity = media_graph_walk_next(&graph))) {
 		struct xvip_dma *dma;
diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
index b01474717dca..d215fe31b9a2 100644
--- a/drivers/media/v4l2-core/v4l2-mc.c
+++ b/drivers/media/v4l2-core/v4l2-mc.c
@@ -436,7 +436,7 @@ static int pipeline_pm_use_count(struct media_entity *entity,
 {
 	int use = 0;
 
-	media_graph_walk_start(graph, entity);
+	media_graph_walk_start(graph, entity->pads);
 
 	while ((entity = media_graph_walk_next(graph))) {
 		if (is_media_entity_v4l2_video_device(entity))
@@ -499,7 +499,7 @@ static int pipeline_pm_power(struct media_entity *entity, int change,
 	if (!change)
 		return 0;
 
-	media_graph_walk_start(graph, entity);
+	media_graph_walk_start(graph, entity->pads);
 
 	while (!ret && (entity = media_graph_walk_next(graph)))
 		if (is_media_entity_v4l2_subdev(entity))
@@ -508,7 +508,7 @@ static int pipeline_pm_power(struct media_entity *entity, int change,
 	if (!ret)
 		return ret;
 
-	media_graph_walk_start(graph, first);
+	media_graph_walk_start(graph, first->pads);
 
 	while ((first = media_graph_walk_next(graph))
 	       && first != entity)
diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
index 66975a37dc85..77bf1b8a56f7 100644
--- a/drivers/staging/media/omap4iss/iss_video.c
+++ b/drivers/staging/media/omap4iss/iss_video.c
@@ -217,7 +217,7 @@ iss_video_far_end(struct iss_video *video)
 		return NULL;
 	}
 
-	media_graph_walk_start(&graph, entity);
+	media_graph_walk_start(&graph, entity->pads);
 
 	while ((entity = media_graph_walk_next(&graph))) {
 		if (entity == &video->video.entity)
@@ -890,7 +890,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 	if (ret < 0)
 		goto err_media_pipeline_start;
 
-	media_graph_walk_start(&graph, entity);
+	media_graph_walk_start(&graph, entity->pads);
 	while ((entity = media_graph_walk_next(&graph)))
 		media_entity_enum_set(&pipe->ent_enum, entity);
 
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index cbdfcb79d0d0..83c4fd93f349 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -897,22 +897,20 @@ __must_check int media_graph_walk_init(
 void media_graph_walk_cleanup(struct media_graph *graph);
 
 /**
- * media_graph_walk_start - Start walking the media graph at a
- *	given entity
+ * media_graph_walk_start - Start walking the media graph at a given pad
  *
  * @graph: Media graph structure that will be used to walk the graph
- * @entity: Starting entity
+ * @pad: Starting pad
  *
  * Before using this function, media_graph_walk_init() must be
  * used to allocate resources used for walking the graph. This
  * function initializes the graph traversal structure to walk the
- * entities graph starting at the given entity. The traversal
+ * entities graph starting at the given pad. The traversal
  * structure must not be modified by the caller during graph
  * traversal. After the graph walk, the resources must be released
  * using media_graph_walk_cleanup().
  */
-void media_graph_walk_start(struct media_graph *graph,
-			    struct media_entity *entity);
+void media_graph_walk_start(struct media_graph *graph, struct media_pad *pad);
 
 /**
  * media_graph_walk_next - Get the next entity in the graph
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 02/24] media: entity: Use pads instead of entities in the media graph walk stack
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
  2021-04-15 13:04 ` [PATCH v5 01/24] media: entity: Use pad as a starting point for graph walk Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-15 13:04 ` [PATCH v5 03/24] media: entity: Walk the graph based on pads Tomi Valkeinen
                   ` (23 subsequent siblings)
  25 siblings, 0 replies; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil

From: Sakari Ailus <sakari.ailus@linux.intel.com>

Change the media graph walk stack structure to use media pads instead of
using media entities. In addition to the entity, the pad contains the
information which pad in the entity are being dealt with.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/mc/mc-entity.c | 53 ++++++++++++++++++------------------
 include/media/media-entity.h |  8 +++---
 2 files changed, 30 insertions(+), 31 deletions(-)

diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 459a5bb4ab41..44a05806b589 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -228,40 +228,39 @@ EXPORT_SYMBOL_GPL(media_entity_pads_init);
  * Graph traversal
  */
 
-static struct media_entity *
-media_entity_other(struct media_entity *entity, struct media_link *link)
+static struct media_pad *
+media_pad_other(struct media_pad *pad, struct media_link *link)
 {
-	if (link->source->entity == entity)
-		return link->sink->entity;
+	if (link->source == pad)
+		return link->sink;
 	else
-		return link->source->entity;
+		return link->source;
 }
 
 /* push an entity to traversal stack */
-static void stack_push(struct media_graph *graph,
-		       struct media_entity *entity)
+static void stack_push(struct media_graph *graph, struct media_pad *pad)
 {
 	if (graph->top == MEDIA_ENTITY_ENUM_MAX_DEPTH - 1) {
 		WARN_ON(1);
 		return;
 	}
 	graph->top++;
-	graph->stack[graph->top].link = entity->links.next;
-	graph->stack[graph->top].entity = entity;
+	graph->stack[graph->top].link = pad->entity->links.next;
+	graph->stack[graph->top].pad = pad;
 }
 
-static struct media_entity *stack_pop(struct media_graph *graph)
+static struct media_pad *stack_pop(struct media_graph *graph)
 {
-	struct media_entity *entity;
+	struct media_pad *pad;
 
-	entity = graph->stack[graph->top].entity;
+	pad = graph->stack[graph->top].pad;
 	graph->top--;
 
-	return entity;
+	return pad;
 }
 
 #define link_top(en)	((en)->stack[(en)->top].link)
-#define stack_top(en)	((en)->stack[(en)->top].entity)
+#define stack_top(en)	((en)->stack[(en)->top].pad)
 
 /**
  * media_graph_walk_init - Allocate resources for graph walk
@@ -297,8 +296,8 @@ void media_graph_walk_start(struct media_graph *graph, struct media_pad *pad)
 	media_entity_enum_set(&graph->ent_enum, pad->entity);
 
 	graph->top = 0;
-	graph->stack[graph->top].entity = NULL;
-	stack_push(graph, pad->entity);
+	graph->stack[graph->top].pad = NULL;
+	stack_push(graph, pad);
 	dev_dbg(pad->graph_obj.mdev->dev,
 		"begin graph walk at '%s':%u\n", pad->entity->name, pad->index);
 }
@@ -306,16 +305,16 @@ EXPORT_SYMBOL_GPL(media_graph_walk_start);
 
 static void media_graph_walk_iter(struct media_graph *graph)
 {
-	struct media_entity *entity = stack_top(graph);
+	struct media_pad *pad = stack_top(graph);
 	struct media_link *link;
-	struct media_entity *next;
+	struct media_pad *next;
 
 	link = list_entry(link_top(graph), typeof(*link), list);
 
 	/* The link is not enabled so we do not follow. */
 	if (!(link->flags & MEDIA_LNK_FL_ENABLED)) {
 		link_top(graph) = link_top(graph)->next;
-		dev_dbg(entity->graph_obj.mdev->dev,
+		dev_dbg(pad->graph_obj.mdev->dev,
 			"walk: skipping disabled link '%s':%u -> '%s':%u\n",
 			link->source->entity->name, link->source->index,
 			link->sink->entity->name, link->sink->index);
@@ -323,22 +322,22 @@ static void media_graph_walk_iter(struct media_graph *graph)
 	}
 
 	/* Get the entity in the other end of the link . */
-	next = media_entity_other(entity, link);
+	next = media_pad_other(pad, link);
 
 	/* Has the entity already been visited? */
-	if (media_entity_enum_test_and_set(&graph->ent_enum, next)) {
+	if (media_entity_enum_test_and_set(&graph->ent_enum, next->entity)) {
 		link_top(graph) = link_top(graph)->next;
-		dev_dbg(entity->graph_obj.mdev->dev,
+		dev_dbg(pad->graph_obj.mdev->dev,
 			"walk: skipping entity '%s' (already seen)\n",
-			next->name);
+			next->entity->name);
 		return;
 	}
 
 	/* Push the new entity to stack and start over. */
 	link_top(graph) = link_top(graph)->next;
 	stack_push(graph, next);
-	dev_dbg(entity->graph_obj.mdev->dev, "walk: pushing '%s' on stack\n",
-		next->name);
+	dev_dbg(next->graph_obj.mdev->dev, "walk: pushing '%s':%u on stack\n",
+		next->entity->name, next->index);
 }
 
 struct media_entity *media_graph_walk_next(struct media_graph *graph)
@@ -353,10 +352,10 @@ struct media_entity *media_graph_walk_next(struct media_graph *graph)
 	 * top of the stack until no more entities on the level can be
 	 * found.
 	 */
-	while (link_top(graph) != &stack_top(graph)->links)
+	while (link_top(graph) != &stack_top(graph)->entity->links)
 		media_graph_walk_iter(graph);
 
-	entity = stack_pop(graph);
+	entity = stack_pop(graph)->entity;
 	dev_dbg(entity->graph_obj.mdev->dev,
 		"walk: returning entity '%s'\n", entity->name);
 
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 83c4fd93f349..97b170cf38eb 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -78,16 +78,16 @@ struct media_entity_enum {
  * struct media_graph - Media graph traversal state
  *
  * @stack:		Graph traversal stack; the stack contains information
- *			on the path the media entities to be walked and the
- *			links through which they were reached.
- * @stack.entity:	pointer to &struct media_entity at the graph.
+ *			on the media pads to be walked and the links through
+ *			which they were reached.
+ * @stack.pad:		pointer to &struct media_pad at the graph.
  * @stack.link:		pointer to &struct list_head.
  * @ent_enum:		Visited entities
  * @top:		The top of the stack
  */
 struct media_graph {
 	struct {
-		struct media_entity *entity;
+		struct media_pad *pad;
 		struct list_head *link;
 	} stack[MEDIA_ENTITY_ENUM_MAX_DEPTH];
 
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 03/24] media: entity: Walk the graph based on pads
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
  2021-04-15 13:04 ` [PATCH v5 01/24] media: entity: Use pad as a starting point for graph walk Tomi Valkeinen
  2021-04-15 13:04 ` [PATCH v5 02/24] media: entity: Use pads instead of entities in the media graph walk stack Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-18 17:47   ` Laurent Pinchart
  2021-04-15 13:04 ` [PATCH v5 04/24] v4l: mc: Start walk from a specific pad in use count calculation Tomi Valkeinen
                   ` (22 subsequent siblings)
  25 siblings, 1 reply; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil

From: Sakari Ailus <sakari.ailus@linux.intel.com>

Instead of iterating over graph entities during the walk, iterate the pads
through which the entity was first reached. This is required in order to
make the entity pipeline pad-based rather than entity based.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 Documentation/driver-api/media/mc-core.rst    |  7 ++-
 drivers/media/mc/mc-entity.c                  | 49 +++++++++++--------
 drivers/media/platform/exynos4-is/media-dev.c | 20 ++++----
 drivers/media/platform/omap3isp/ispvideo.c    | 17 ++++---
 drivers/media/platform/vsp1/vsp1_video.c      | 12 ++---
 drivers/media/platform/xilinx/xilinx-dma.c    | 12 ++---
 drivers/media/v4l2-core/v4l2-mc.c             | 24 ++++-----
 drivers/staging/media/omap4iss/iss_video.c    | 34 +++++++------
 include/media/media-entity.h                  |  7 +--
 9 files changed, 99 insertions(+), 83 deletions(-)

diff --git a/Documentation/driver-api/media/mc-core.rst b/Documentation/driver-api/media/mc-core.rst
index ba0aee982124..8a13640bed56 100644
--- a/Documentation/driver-api/media/mc-core.rst
+++ b/Documentation/driver-api/media/mc-core.rst
@@ -169,8 +169,11 @@ Drivers initiate a graph traversal by calling
 The graph structure, provided by the caller, is initialized to start graph
 traversal at the given pad in an entity.
 
-Drivers can then retrieve the next entity by calling
-:c:func:`media_graph_walk_next()`
+Drivers can then retrieve the next pad by calling
+:c:func:`media_graph_walk_next()`. Only the pad through which the entity
+is first reached is returned. If the caller is interested in knowing which
+further pads would be connected, the :c:func:`media_entity_has_route()`
+function can be used for that.
 
 When the graph traversal is complete the function will return ``NULL``.
 
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 44a05806b589..401fddf320e7 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -340,9 +340,9 @@ static void media_graph_walk_iter(struct media_graph *graph)
 		next->entity->name, next->index);
 }
 
-struct media_entity *media_graph_walk_next(struct media_graph *graph)
+struct media_pad *media_graph_walk_next(struct media_graph *graph)
 {
-	struct media_entity *entity;
+	struct media_pad *pad;
 
 	if (stack_top(graph) == NULL)
 		return NULL;
@@ -355,11 +355,11 @@ struct media_entity *media_graph_walk_next(struct media_graph *graph)
 	while (link_top(graph) != &stack_top(graph)->entity->links)
 		media_graph_walk_iter(graph);
 
-	entity = stack_pop(graph)->entity;
-	dev_dbg(entity->graph_obj.mdev->dev,
-		"walk: returning entity '%s'\n", entity->name);
+	pad = stack_pop(graph);
+	dev_dbg(pad->graph_obj.mdev->dev,
+		"walk: returning pad '%s':%u\n", pad->entity->name, pad->index);
 
-	return entity;
+	return pad;
 }
 EXPORT_SYMBOL_GPL(media_graph_walk_next);
 
@@ -407,7 +407,8 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 {
 	struct media_device *mdev = entity->graph_obj.mdev;
 	struct media_graph *graph = &pipe->graph;
-	struct media_entity *entity_err = entity;
+	struct media_pad *pad = entity->pads;
+	struct media_pad *pad_err = pad;
 	struct media_link *link;
 	int ret;
 
@@ -417,9 +418,11 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 			goto error_graph_walk_start;
 	}
 
-	media_graph_walk_start(&pipe->graph, entity->pads);
+	media_graph_walk_start(&pipe->graph, pad);
+
+	while ((pad = media_graph_walk_next(graph))) {
+		struct media_entity *entity = pad->entity;
 
-	while ((entity = media_graph_walk_next(graph))) {
 		DECLARE_BITMAP(active, MEDIA_ENTITY_MAX_PADS);
 		DECLARE_BITMAP(has_no_links, MEDIA_ENTITY_MAX_PADS);
 
@@ -428,7 +431,7 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 		if (entity->pipe && entity->pipe != pipe) {
 			pr_err("Pipe active for %s. Can't start for %s\n",
 				entity->name,
-				entity_err->name);
+				pad_err->entity->name);
 			ret = -EBUSY;
 			goto error;
 		}
@@ -446,11 +449,12 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 		bitmap_fill(has_no_links, entity->num_pads);
 
 		list_for_each_entry(link, &entity->links, list) {
-			struct media_pad *pad = link->sink->entity == entity
-						? link->sink : link->source;
+			struct media_pad *other_pad =
+				link->sink->entity == entity ?
+				link->sink : link->source;
 
 			/* Mark that a pad is connected by a link. */
-			bitmap_clear(has_no_links, pad->index, 1);
+			bitmap_clear(has_no_links, other_pad->index, 1);
 
 			/*
 			 * Pads that either do not need to connect or
@@ -459,13 +463,13 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 			 */
 			if (!(pad->flags & MEDIA_PAD_FL_MUST_CONNECT) ||
 			    link->flags & MEDIA_LNK_FL_ENABLED)
-				bitmap_set(active, pad->index, 1);
+				bitmap_set(active, other_pad->index, 1);
 
 			/*
 			 * Link validation will only take place for
 			 * sink ends of the link that are enabled.
 			 */
-			if (link->sink != pad ||
+			if (link->sink != other_pad ||
 			    !(link->flags & MEDIA_LNK_FL_ENABLED))
 				continue;
 
@@ -501,9 +505,11 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 	 * Link validation on graph failed. We revert what we did and
 	 * return the error.
 	 */
-	media_graph_walk_start(graph, entity_err->pads);
+	media_graph_walk_start(graph, pad_err);
+
+	while ((pad_err = media_graph_walk_next(graph))) {
+		struct media_entity *entity_err = pad_err->entity;
 
-	while ((entity_err = media_graph_walk_next(graph))) {
 		/* Sanity check for negative stream_count */
 		if (!WARN_ON_ONCE(entity_err->stream_count <= 0)) {
 			entity_err->stream_count--;
@@ -515,7 +521,7 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 		 * We haven't increased stream_count further than this
 		 * so we quit here.
 		 */
-		if (entity_err == entity)
+		if (pad_err == pad)
 			break;
 	}
 
@@ -542,8 +548,9 @@ EXPORT_SYMBOL_GPL(media_pipeline_start);
 
 void __media_pipeline_stop(struct media_entity *entity)
 {
-	struct media_graph *graph = &entity->pipe->graph;
 	struct media_pipeline *pipe = entity->pipe;
+	struct media_graph *graph = &pipe->graph;
+	struct media_pad *pad;
 
 	/*
 	 * If the following check fails, the driver has performed an
@@ -554,7 +561,9 @@ void __media_pipeline_stop(struct media_entity *entity)
 
 	media_graph_walk_start(graph, entity->pads);
 
-	while ((entity = media_graph_walk_next(graph))) {
+	while ((pad = media_graph_walk_next(graph))) {
+		struct media_entity *entity = pad->entity;
+
 		/* Sanity check for negative stream_count */
 		if (!WARN_ON_ONCE(entity->stream_count <= 0)) {
 			entity->stream_count--;
diff --git a/drivers/media/platform/exynos4-is/media-dev.c b/drivers/media/platform/exynos4-is/media-dev.c
index d90663b65932..b910a23b7e23 100644
--- a/drivers/media/platform/exynos4-is/media-dev.c
+++ b/drivers/media/platform/exynos4-is/media-dev.c
@@ -1166,7 +1166,7 @@ static int __fimc_md_modify_pipeline(struct media_entity *entity, bool enable)
 static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
 				      struct media_graph *graph)
 {
-	struct media_entity *entity_err = entity;
+	struct media_pad *pad, *pad_err = entity->pads;
 	int ret;
 
 	/*
@@ -1175,13 +1175,13 @@ static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
 	 * through active links. This is needed as we cannot power on/off the
 	 * subdevs in random order.
 	 */
-	media_graph_walk_start(graph, entity->pads);
+	media_graph_walk_start(graph, pad_err);
 
-	while ((entity = media_graph_walk_next(graph))) {
-		if (!is_media_entity_v4l2_video_device(entity))
+	while ((pad = media_graph_walk_next(graph))) {
+		if (!is_media_entity_v4l2_video_device(pad->entity))
 			continue;
 
-		ret  = __fimc_md_modify_pipeline(entity, enable);
+		ret  = __fimc_md_modify_pipeline(pad->entity, enable);
 
 		if (ret < 0)
 			goto err;
@@ -1190,15 +1190,15 @@ static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
 	return 0;
 
 err:
-	media_graph_walk_start(graph, entity_err->pads);
+	media_graph_walk_start(graph, pad_err);
 
-	while ((entity_err = media_graph_walk_next(graph))) {
-		if (!is_media_entity_v4l2_video_device(entity_err))
+	while ((pad_err = media_graph_walk_next(graph))) {
+		if (!is_media_entity_v4l2_video_device(pad_err->entity))
 			continue;
 
-		__fimc_md_modify_pipeline(entity_err, !enable);
+		__fimc_md_modify_pipeline(pad_err->entity, !enable);
 
-		if (entity_err == entity)
+		if (pad_err == pad)
 			break;
 	}
 
diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
index 3c1485d59404..49cde04bfb21 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -222,8 +222,8 @@ static int isp_video_get_graph_data(struct isp_video *video,
 				    struct isp_pipeline *pipe)
 {
 	struct media_graph graph;
-	struct media_entity *entity = &video->video.entity;
-	struct media_device *mdev = entity->graph_obj.mdev;
+	struct media_pad *pad = video->video.entity.pads;
+	struct media_device *mdev = pad->entity->graph_obj.mdev;
 	struct isp_video *far_end = NULL;
 	int ret;
 
@@ -234,23 +234,24 @@ static int isp_video_get_graph_data(struct isp_video *video,
 		return ret;
 	}
 
-	media_graph_walk_start(&graph, entity->pads);
+	media_graph_walk_start(&graph, pad);
 
-	while ((entity = media_graph_walk_next(&graph))) {
+	while ((pad = media_graph_walk_next(&graph))) {
 		struct isp_video *__video;
 
-		media_entity_enum_set(&pipe->ent_enum, entity);
+		media_entity_enum_set(&pipe->ent_enum, pad->entity);
 
 		if (far_end != NULL)
 			continue;
 
-		if (entity == &video->video.entity)
+		if (pad == video->video.entity.pads)
 			continue;
 
-		if (!is_media_entity_v4l2_video_device(entity))
+		if (!is_media_entity_v4l2_video_device(pad->entity))
 			continue;
 
-		__video = to_isp_video(media_entity_to_video_device(entity));
+		__video = to_isp_video(media_entity_to_video_device(
+					       pad->entity));
 		if (__video->type != video->type)
 			far_end = __video;
 	}
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index 61e4fbaba7b7..39dccf347ce1 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -559,8 +559,8 @@ static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
 				     struct vsp1_video *video)
 {
 	struct media_graph graph;
-	struct media_entity *entity = &video->video.entity;
-	struct media_device *mdev = entity->graph_obj.mdev;
+	struct media_pad *pad = video->video.entity.pads;
+	struct media_device *mdev = pad->entity->graph_obj.mdev;
 	unsigned int i;
 	int ret;
 
@@ -569,17 +569,17 @@ static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
 	if (ret)
 		return ret;
 
-	media_graph_walk_start(&graph, entity->pads);
+	media_graph_walk_start(&graph, pad);
 
-	while ((entity = media_graph_walk_next(&graph))) {
+	while ((pad = media_graph_walk_next(&graph))) {
 		struct v4l2_subdev *subdev;
 		struct vsp1_rwpf *rwpf;
 		struct vsp1_entity *e;
 
-		if (!is_media_entity_v4l2_subdev(entity))
+		if (!is_media_entity_v4l2_subdev(pad->entity))
 			continue;
 
-		subdev = media_entity_to_v4l2_subdev(entity);
+		subdev = media_entity_to_v4l2_subdev(pad->entity);
 		e = to_vsp1_entity(subdev);
 		list_add_tail(&e->list_pipe, &pipe->entities);
 		e->pipe = pipe;
diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index d64c3bee8b95..8df3c43aecbe 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -175,8 +175,8 @@ static int xvip_pipeline_validate(struct xvip_pipeline *pipe,
 				  struct xvip_dma *start)
 {
 	struct media_graph graph;
-	struct media_entity *entity = &start->video.entity;
-	struct media_device *mdev = entity->graph_obj.mdev;
+	struct media_pad *pad = start->video.entity.pads;
+	struct media_device *mdev = pad->entity->graph_obj.mdev;
 	unsigned int num_inputs = 0;
 	unsigned int num_outputs = 0;
 	int ret;
@@ -190,15 +190,15 @@ static int xvip_pipeline_validate(struct xvip_pipeline *pipe,
 		return ret;
 	}
 
-	media_graph_walk_start(&graph, entity->pads);
+	media_graph_walk_start(&graph, pad);
 
-	while ((entity = media_graph_walk_next(&graph))) {
+	while ((pad = media_graph_walk_next(&graph))) {
 		struct xvip_dma *dma;
 
-		if (entity->function != MEDIA_ENT_F_IO_V4L)
+		if (pad->entity->function != MEDIA_ENT_F_IO_V4L)
 			continue;
 
-		dma = to_xvip_dma(media_entity_to_video_device(entity));
+		dma = to_xvip_dma(media_entity_to_video_device(pad->entity));
 
 		if (dma->pad.flags & MEDIA_PAD_FL_SINK) {
 			pipe->output = dma;
diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
index d215fe31b9a2..cbeb580c6754 100644
--- a/drivers/media/v4l2-core/v4l2-mc.c
+++ b/drivers/media/v4l2-core/v4l2-mc.c
@@ -434,13 +434,14 @@ EXPORT_SYMBOL_GPL(v4l2_create_fwnode_links);
 static int pipeline_pm_use_count(struct media_entity *entity,
 	struct media_graph *graph)
 {
+	struct media_pad *pad;
 	int use = 0;
 
 	media_graph_walk_start(graph, entity->pads);
 
-	while ((entity = media_graph_walk_next(graph))) {
-		if (is_media_entity_v4l2_video_device(entity))
-			use += entity->use_count;
+	while ((pad = media_graph_walk_next(graph))) {
+		if (is_media_entity_v4l2_video_device(pad->entity))
+			use += pad->entity->use_count;
 	}
 
 	return use;
@@ -493,7 +494,7 @@ static int pipeline_pm_power_one(struct media_entity *entity, int change)
 static int pipeline_pm_power(struct media_entity *entity, int change,
 	struct media_graph *graph)
 {
-	struct media_entity *first = entity;
+	struct media_pad *tmp_pad, *pad;
 	int ret = 0;
 
 	if (!change)
@@ -501,19 +502,18 @@ static int pipeline_pm_power(struct media_entity *entity, int change,
 
 	media_graph_walk_start(graph, entity->pads);
 
-	while (!ret && (entity = media_graph_walk_next(graph)))
-		if (is_media_entity_v4l2_subdev(entity))
-			ret = pipeline_pm_power_one(entity, change);
+	while (!ret && (pad = media_graph_walk_next(graph)))
+		if (is_media_entity_v4l2_subdev(pad->entity))
+			ret = pipeline_pm_power_one(pad->entity, change);
 
 	if (!ret)
 		return ret;
 
-	media_graph_walk_start(graph, first->pads);
+	media_graph_walk_start(graph, entity->pads);
 
-	while ((first = media_graph_walk_next(graph))
-	       && first != entity)
-		if (is_media_entity_v4l2_subdev(first))
-			pipeline_pm_power_one(first, -change);
+	while ((tmp_pad = media_graph_walk_next(graph)) && tmp_pad != pad)
+		if (is_media_entity_v4l2_subdev(tmp_pad->entity))
+			pipeline_pm_power_one(tmp_pad->entity, -change);
 
 	return ret;
 }
diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
index 77bf1b8a56f7..9f3ff5a37d90 100644
--- a/drivers/staging/media/omap4iss/iss_video.c
+++ b/drivers/staging/media/omap4iss/iss_video.c
@@ -206,8 +206,8 @@ static struct iss_video *
 iss_video_far_end(struct iss_video *video)
 {
 	struct media_graph graph;
-	struct media_entity *entity = &video->video.entity;
-	struct media_device *mdev = entity->graph_obj.mdev;
+	struct media_pad *pad = video->video.entity.pads;
+	struct media_device *mdev = pad->entity->graph_obj.mdev;
 	struct iss_video *far_end = NULL;
 
 	mutex_lock(&mdev->graph_mutex);
@@ -217,16 +217,17 @@ iss_video_far_end(struct iss_video *video)
 		return NULL;
 	}
 
-	media_graph_walk_start(&graph, entity->pads);
+	media_graph_walk_start(&graph, pad);
 
-	while ((entity = media_graph_walk_next(&graph))) {
-		if (entity == &video->video.entity)
+	while ((pad = media_graph_walk_next(&graph))) {
+		if (pad->entity == &video->video.entity)
 			continue;
 
-		if (!is_media_entity_v4l2_video_device(entity))
+		if (!is_media_entity_v4l2_video_device(pad->entity))
 			continue;
 
-		far_end = to_iss_video(media_entity_to_video_device(entity));
+		far_end = to_iss_video(media_entity_to_video_device(
+						pad->entity));
 		if (far_end->type != video->type)
 			break;
 
@@ -853,7 +854,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 	struct iss_video_fh *vfh = to_iss_video_fh(fh);
 	struct iss_video *video = video_drvdata(file);
 	struct media_graph graph;
-	struct media_entity *entity = &video->video.entity;
+	struct media_pad *pad = video->video.entity.pads;
 	enum iss_pipeline_state state;
 	struct iss_pipeline *pipe;
 	struct iss_video *far_end;
@@ -869,30 +870,31 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 	 * Start streaming on the pipeline. No link touching an entity in the
 	 * pipeline can be activated or deactivated once streaming is started.
 	 */
-	pipe = entity->pipe
-	     ? to_iss_pipeline(entity) : &video->pipe;
+	pipe = pad->entity->pipe
+	     ? to_iss_pipeline(pad->entity) : &video->pipe;
 	pipe->external = NULL;
 	pipe->external_rate = 0;
 	pipe->external_bpp = 0;
 
-	ret = media_entity_enum_init(&pipe->ent_enum, entity->graph_obj.mdev);
+	ret = media_entity_enum_init(&pipe->ent_enum,
+				     pad->entity->graph_obj.mdev);
 	if (ret)
 		goto err_graph_walk_init;
 
-	ret = media_graph_walk_init(&graph, entity->graph_obj.mdev);
+	ret = media_graph_walk_init(&graph, pad->entity->graph_obj.mdev);
 	if (ret)
 		goto err_graph_walk_init;
 
 	if (video->iss->pdata->set_constraints)
 		video->iss->pdata->set_constraints(video->iss, true);
 
-	ret = media_pipeline_start(entity, &pipe->pipe);
+	ret = media_pipeline_start(pad->entity, &pipe->pipe);
 	if (ret < 0)
 		goto err_media_pipeline_start;
 
-	media_graph_walk_start(&graph, entity->pads);
-	while ((entity = media_graph_walk_next(&graph)))
-		media_entity_enum_set(&pipe->ent_enum, entity);
+	media_graph_walk_start(&graph, pad);
+	while ((pad = media_graph_walk_next(&graph)))
+		media_entity_enum_set(&pipe->ent_enum, pad->entity);
 
 	/*
 	 * Verify that the currently configured format matches the output of
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 97b170cf38eb..2d45344ca527 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -921,10 +921,11 @@ void media_graph_walk_start(struct media_graph *graph, struct media_pad *pad);
  * The graph structure must have been previously initialized with a call to
  * media_graph_walk_start().
  *
- * Return: returns the next entity in the graph or %NULL if the whole graph
- * have been traversed.
+ * Return: returns the next pad in the graph or %NULL if the whole
+ * graph have been traversed. The pad which is returned is the pad
+ * through which a new entity is reached when parsing the graph.
  */
-struct media_entity *media_graph_walk_next(struct media_graph *graph);
+struct media_pad *media_graph_walk_next(struct media_graph *graph);
 
 /**
  * media_pipeline_start - Mark a pipeline as streaming
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 04/24] v4l: mc: Start walk from a specific pad in use count calculation
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (2 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 03/24] media: entity: Walk the graph based on pads Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-15 13:04 ` [PATCH v5 05/24] media: entity: Add iterator helper for entity pads Tomi Valkeinen
                   ` (21 subsequent siblings)
  25 siblings, 0 replies; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil

From: Sakari Ailus <sakari.ailus@linux.intel.com>

With the addition of the recent has_route() media entity op, the pads of a
media entity are no longer all interconnected. This has to be taken into
account in power management.

Prepare for the addition of a helper function supporting S_ROUTING.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/v4l2-core/v4l2-mc.c | 27 +++++++++++++--------------
 1 file changed, 13 insertions(+), 14 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
index cbeb580c6754..35d18ed89fa5 100644
--- a/drivers/media/v4l2-core/v4l2-mc.c
+++ b/drivers/media/v4l2-core/v4l2-mc.c
@@ -427,17 +427,16 @@ EXPORT_SYMBOL_GPL(v4l2_create_fwnode_links);
 
 /*
  * pipeline_pm_use_count - Count the number of users of a pipeline
- * @entity: The entity
+ * @pad: Any pad along the pipeline
  *
  * Return the total number of users of all video device nodes in the pipeline.
  */
-static int pipeline_pm_use_count(struct media_entity *entity,
-	struct media_graph *graph)
+static int pipeline_pm_use_count(struct media_pad *pad,
+				 struct media_graph *graph)
 {
-	struct media_pad *pad;
 	int use = 0;
 
-	media_graph_walk_start(graph, entity->pads);
+	media_graph_walk_start(graph, pad);
 
 	while ((pad = media_graph_walk_next(graph))) {
 		if (is_media_entity_v4l2_video_device(pad->entity))
@@ -483,7 +482,7 @@ static int pipeline_pm_power_one(struct media_entity *entity, int change)
 
 /*
  * pipeline_pm_power - Apply power change to all entities in a pipeline
- * @entity: The entity
+ * @pad: Any pad along the pipeline
  * @change: Use count change
  *
  * Walk the pipeline to update the use count and the power state of all non-node
@@ -491,16 +490,16 @@ static int pipeline_pm_power_one(struct media_entity *entity, int change)
  *
  * Return 0 on success or a negative error code on failure.
  */
-static int pipeline_pm_power(struct media_entity *entity, int change,
-	struct media_graph *graph)
+static int pipeline_pm_power(struct media_pad *pad, int change,
+			     struct media_graph *graph)
 {
-	struct media_pad *tmp_pad, *pad;
+	struct media_pad *tmp_pad, *first = pad;
 	int ret = 0;
 
 	if (!change)
 		return 0;
 
-	media_graph_walk_start(graph, entity->pads);
+	media_graph_walk_start(graph, pad);
 
 	while (!ret && (pad = media_graph_walk_next(graph)))
 		if (is_media_entity_v4l2_subdev(pad->entity))
@@ -509,7 +508,7 @@ static int pipeline_pm_power(struct media_entity *entity, int change,
 	if (!ret)
 		return ret;
 
-	media_graph_walk_start(graph, entity->pads);
+	media_graph_walk_start(graph, first);
 
 	while ((tmp_pad = media_graph_walk_next(graph)) && tmp_pad != pad)
 		if (is_media_entity_v4l2_subdev(tmp_pad->entity))
@@ -531,7 +530,7 @@ static int v4l2_pipeline_pm_use(struct media_entity *entity, unsigned int use)
 	WARN_ON(entity->use_count < 0);
 
 	/* Apply power change to connected non-nodes. */
-	ret = pipeline_pm_power(entity, change, &mdev->pm_count_walk);
+	ret = pipeline_pm_power(entity->pads, change, &mdev->pm_count_walk);
 	if (ret < 0)
 		entity->use_count -= change;
 
@@ -557,8 +556,8 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 			      unsigned int notification)
 {
 	struct media_graph *graph = &link->graph_obj.mdev->pm_count_walk;
-	struct media_entity *source = link->source->entity;
-	struct media_entity *sink = link->sink->entity;
+	struct media_pad *source = link->source;
+	struct media_pad *sink = link->sink;
 	int source_use;
 	int sink_use;
 	int ret = 0;
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 05/24] media: entity: Add iterator helper for entity pads
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (3 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 04/24] v4l: mc: Start walk from a specific pad in use count calculation Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-18 17:52   ` Laurent Pinchart
  2021-04-15 13:04 ` [PATCH v5 06/24] media: entity: Move the pipeline from entity to pads Tomi Valkeinen
                   ` (20 subsequent siblings)
  25 siblings, 1 reply; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil

From: Jacopo Mondi <jacopo+renesas@jmondi.org>

Add an iterator helper to easily cycle through all pads in an entity and
use it in media-entity and media-device code where appropriate.

Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/mc/mc-device.c | 13 ++++++-------
 drivers/media/mc/mc-entity.c | 11 ++++++-----
 include/media/media-entity.h | 12 ++++++++++++
 3 files changed, 24 insertions(+), 12 deletions(-)

diff --git a/drivers/media/mc/mc-device.c b/drivers/media/mc/mc-device.c
index 9e56d2ad6b94..704ef1360eba 100644
--- a/drivers/media/mc/mc-device.c
+++ b/drivers/media/mc/mc-device.c
@@ -581,7 +581,7 @@ static void __media_device_unregister_entity(struct media_entity *entity)
 	struct media_device *mdev = entity->graph_obj.mdev;
 	struct media_link *link, *tmp;
 	struct media_interface *intf;
-	unsigned int i;
+	struct media_pad *iter;
 
 	ida_free(&mdev->entity_internal_idx, entity->internal_idx);
 
@@ -597,8 +597,8 @@ static void __media_device_unregister_entity(struct media_entity *entity)
 	__media_entity_remove_links(entity);
 
 	/* Remove all pads that belong to this entity */
-	for (i = 0; i < entity->num_pads; i++)
-		media_gobj_destroy(&entity->pads[i].graph_obj);
+	media_entity_for_each_pad(entity, iter)
+		media_gobj_destroy(&iter->graph_obj);
 
 	/* Remove the entity */
 	media_gobj_destroy(&entity->graph_obj);
@@ -617,7 +617,7 @@ int __must_check media_device_register_entity(struct media_device *mdev,
 					      struct media_entity *entity)
 {
 	struct media_entity_notify *notify, *next;
-	unsigned int i;
+	struct media_pad *iter;
 	int ret;
 
 	if (entity->function == MEDIA_ENT_F_V4L2_SUBDEV_UNKNOWN ||
@@ -646,9 +646,8 @@ int __must_check media_device_register_entity(struct media_device *mdev,
 	media_gobj_create(mdev, MEDIA_GRAPH_ENTITY, &entity->graph_obj);
 
 	/* Initialize objects at the pads */
-	for (i = 0; i < entity->num_pads; i++)
-		media_gobj_create(mdev, MEDIA_GRAPH_PAD,
-			       &entity->pads[i].graph_obj);
+	media_entity_for_each_pad(entity, iter)
+		media_gobj_create(mdev, MEDIA_GRAPH_PAD, &iter->graph_obj);
 
 	/* invoke entity_notify callbacks */
 	list_for_each_entry_safe(notify, next, &mdev->entity_notify, list)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 401fddf320e7..830841e0cd28 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -198,7 +198,8 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads,
 			   struct media_pad *pads)
 {
 	struct media_device *mdev = entity->graph_obj.mdev;
-	unsigned int i;
+	struct media_pad *iter;
+	unsigned int i = 0;
 
 	if (num_pads >= MEDIA_ENTITY_MAX_PADS)
 		return -E2BIG;
@@ -209,12 +210,12 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads,
 	if (mdev)
 		mutex_lock(&mdev->graph_mutex);
 
-	for (i = 0; i < num_pads; i++) {
-		pads[i].entity = entity;
-		pads[i].index = i;
+	media_entity_for_each_pad(entity, iter) {
+		iter->entity = entity;
+		iter->index = i++;
 		if (mdev)
 			media_gobj_create(mdev, MEDIA_GRAPH_PAD,
-					&entity->pads[i].graph_obj);
+					&iter->graph_obj);
 	}
 
 	if (mdev)
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 2d45344ca527..52b1a1cab57a 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -1102,3 +1102,15 @@ void media_remove_intf_links(struct media_interface *intf);
 	 (entity)->ops->operation((entity) , ##args) : -ENOIOCTLCMD)
 
 #endif
+
+/**
+ * media_entity_for_each_pad - Iterate on all pads in an entity
+ * @entity: The entity the pads belong to
+ * @iter: The iterator pad
+ *
+ * Iterate on all pads in a media entity.
+ */
+#define media_entity_for_each_pad(entity, iter)			\
+	for (iter = (entity)->pads;				\
+	     iter < &(entity)->pads[(entity)->num_pads];	\
+	     ++iter)
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 06/24] media: entity: Move the pipeline from entity to pads
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (4 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 05/24] media: entity: Add iterator helper for entity pads Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-18 18:00   ` Laurent Pinchart
  2021-04-15 13:04 ` [PATCH v5 07/24] media: entity: Use pad as the starting point for a pipeline Tomi Valkeinen
                   ` (19 subsequent siblings)
  25 siblings, 1 reply; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen

From: Sakari Ailus <sakari.ailus@linux.intel.com>

This moves the pipe and stream_count fields from struct media_entity to
struct media_pad. Effectively streams become pad-specific rather than
being entity specific, allowing several independent streams to traverse a
single entity and an entity to be part of several streams.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

- Update documentation to use 'pads'
- Use the media pad iterator in media_entity.c
- Update rcar-dma.c to use the new per-pad stream count
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>

- Fix cleanup in the error path of __media_pipeline_start()
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/mc/mc-entity.c                  | 68 +++++++++++--------
 drivers/media/platform/exynos4-is/fimc-isp.c  |  2 +-
 drivers/media/platform/exynos4-is/fimc-lite.c |  2 +-
 drivers/media/platform/omap3isp/isp.c         |  2 +-
 drivers/media/platform/omap3isp/ispvideo.c    |  2 +-
 drivers/media/platform/omap3isp/ispvideo.h    |  2 +-
 drivers/media/platform/rcar-vin/rcar-core.c   | 16 +++--
 drivers/media/platform/rcar-vin/rcar-dma.c    |  2 +-
 drivers/media/platform/xilinx/xilinx-dma.c    |  2 +-
 drivers/media/platform/xilinx/xilinx-dma.h    |  2 +-
 drivers/staging/media/imx/imx-media-utils.c   |  2 +-
 drivers/staging/media/omap4iss/iss.c          |  2 +-
 drivers/staging/media/omap4iss/iss_video.c    |  2 +-
 drivers/staging/media/omap4iss/iss_video.h    |  2 +-
 include/media/media-entity.h                  | 21 +++---
 15 files changed, 73 insertions(+), 56 deletions(-)

diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 830841e0cd28..b6e5aa639c26 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -423,24 +423,28 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 
 	while ((pad = media_graph_walk_next(graph))) {
 		struct media_entity *entity = pad->entity;
+		bool skip_validation = pad->pipe;
+		struct media_pad *iter;
 
 		DECLARE_BITMAP(active, MEDIA_ENTITY_MAX_PADS);
 		DECLARE_BITMAP(has_no_links, MEDIA_ENTITY_MAX_PADS);
 
-		entity->stream_count++;
-
-		if (entity->pipe && entity->pipe != pipe) {
-			pr_err("Pipe active for %s. Can't start for %s\n",
-				entity->name,
-				pad_err->entity->name);
-			ret = -EBUSY;
-			goto error;
+		media_entity_for_each_pad(entity, iter) {
+			if (iter->pipe && iter->pipe != pipe) {
+				pr_err("Pipe active for %s. Can't start for %s\n",
+				       entity->name, iter->entity->name);
+				ret = -EBUSY;
+			} else {
+				iter->pipe = pipe;
+			}
+			iter->stream_count++;
 		}
 
-		entity->pipe = pipe;
+		if (ret)
+			goto error;
 
-		/* Already streaming --- no need to check. */
-		if (entity->stream_count > 1)
+		/* Already part of the pipeline, skip validation. */
+		if (skip_validation)
 			continue;
 
 		if (!entity->ops || !entity->ops->link_validate)
@@ -509,20 +513,23 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 	media_graph_walk_start(graph, pad_err);
 
 	while ((pad_err = media_graph_walk_next(graph))) {
-		struct media_entity *entity_err = pad_err->entity;
-
-		/* Sanity check for negative stream_count */
-		if (!WARN_ON_ONCE(entity_err->stream_count <= 0)) {
-			entity_err->stream_count--;
-			if (entity_err->stream_count == 0)
-				entity_err->pipe = NULL;
+		struct media_entity *entity = pad_err->entity;
+		struct media_pad *iter;
+
+		media_entity_for_each_pad(entity, iter) {
+			/* Sanity check for negative stream_count */
+			if (!WARN_ON_ONCE(iter->stream_count <= 0)) {
+				--iter->stream_count;
+				if (iter->stream_count == 0)
+					iter->pipe = NULL;
+			}
 		}
 
 		/*
 		 * We haven't increased stream_count further than this
 		 * so we quit here.
 		 */
-		if (pad_err == pad)
+		if (pad_err->entity == pad->entity)
 			break;
 	}
 
@@ -549,7 +556,7 @@ EXPORT_SYMBOL_GPL(media_pipeline_start);
 
 void __media_pipeline_stop(struct media_entity *entity)
 {
-	struct media_pipeline *pipe = entity->pipe;
+	struct media_pipeline *pipe = entity->pads->pipe;
 	struct media_graph *graph = &pipe->graph;
 	struct media_pad *pad;
 
@@ -564,12 +571,15 @@ void __media_pipeline_stop(struct media_entity *entity)
 
 	while ((pad = media_graph_walk_next(graph))) {
 		struct media_entity *entity = pad->entity;
-
-		/* Sanity check for negative stream_count */
-		if (!WARN_ON_ONCE(entity->stream_count <= 0)) {
-			entity->stream_count--;
-			if (entity->stream_count == 0)
-				entity->pipe = NULL;
+		struct media_pad *iter;
+
+		media_entity_for_each_pad(entity, iter) {
+			/* Sanity check for negative stream_count */
+			if (!WARN_ON_ONCE(iter->stream_count <= 0)) {
+				iter->stream_count--;
+				if (iter->stream_count == 0)
+					iter->pipe = NULL;
+			}
 		}
 	}
 
@@ -839,7 +849,7 @@ int __media_entity_setup_link(struct media_link *link, u32 flags)
 {
 	const u32 mask = MEDIA_LNK_FL_ENABLED;
 	struct media_device *mdev;
-	struct media_entity *source, *sink;
+	struct media_pad *source, *sink;
 	int ret = -EBUSY;
 
 	if (link == NULL)
@@ -855,8 +865,8 @@ int __media_entity_setup_link(struct media_link *link, u32 flags)
 	if (link->flags == flags)
 		return 0;
 
-	source = link->source->entity;
-	sink = link->sink->entity;
+	source = link->source;
+	sink = link->sink;
 
 	if (!(link->flags & MEDIA_LNK_FL_DYNAMIC) &&
 	    (source->stream_count || sink->stream_count))
diff --git a/drivers/media/platform/exynos4-is/fimc-isp.c b/drivers/media/platform/exynos4-is/fimc-isp.c
index a77c49b18511..563d07f630bc 100644
--- a/drivers/media/platform/exynos4-is/fimc-isp.c
+++ b/drivers/media/platform/exynos4-is/fimc-isp.c
@@ -223,7 +223,7 @@ static int fimc_isp_subdev_set_fmt(struct v4l2_subdev *sd,
 			}
 		}
 	} else {
-		if (sd->entity.stream_count == 0) {
+		if (sd->entity.pads->stream_count == 0) {
 			if (fmt->pad == FIMC_ISP_SD_PAD_SINK) {
 				struct v4l2_subdev_format format = *fmt;
 
diff --git a/drivers/media/platform/exynos4-is/fimc-lite.c b/drivers/media/platform/exynos4-is/fimc-lite.c
index fe20af3a7178..56773a0be8d9 100644
--- a/drivers/media/platform/exynos4-is/fimc-lite.c
+++ b/drivers/media/platform/exynos4-is/fimc-lite.c
@@ -1070,7 +1070,7 @@ static int fimc_lite_subdev_set_fmt(struct v4l2_subdev *sd,
 	mutex_lock(&fimc->lock);
 
 	if ((atomic_read(&fimc->out_path) == FIMC_IO_ISP &&
-	    sd->entity.stream_count > 0) ||
+	    sd->entity.pads->stream_count > 0) ||
 	    (atomic_read(&fimc->out_path) == FIMC_IO_DMA &&
 	    vb2_is_busy(&fimc->vb_queue))) {
 		mutex_unlock(&fimc->lock);
diff --git a/drivers/media/platform/omap3isp/isp.c b/drivers/media/platform/omap3isp/isp.c
index a6bb7d9bf75f..4ef623f10a44 100644
--- a/drivers/media/platform/omap3isp/isp.c
+++ b/drivers/media/platform/omap3isp/isp.c
@@ -930,7 +930,7 @@ static int isp_pipeline_is_last(struct media_entity *me)
 	struct isp_pipeline *pipe;
 	struct media_pad *pad;
 
-	if (!me->pipe)
+	if (!me->pads->pipe)
 		return 0;
 	pipe = to_isp_pipeline(me);
 	if (pipe->stream_state == ISP_PIPELINE_STREAM_STOPPED)
diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
index 49cde04bfb21..f3bee79cdd85 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -1094,7 +1094,7 @@ isp_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 	/* Start streaming on the pipeline. No link touching an entity in the
 	 * pipeline can be activated or deactivated once streaming is started.
 	 */
-	pipe = video->video.entity.pipe
+	pipe = video->video.entity.pads->pipe
 	     ? to_isp_pipeline(&video->video.entity) : &video->pipe;
 
 	ret = media_entity_enum_init(&pipe->ent_enum, &video->isp->media_dev);
diff --git a/drivers/media/platform/omap3isp/ispvideo.h b/drivers/media/platform/omap3isp/ispvideo.h
index a0908670c0cf..4c9c5b719ec5 100644
--- a/drivers/media/platform/omap3isp/ispvideo.h
+++ b/drivers/media/platform/omap3isp/ispvideo.h
@@ -100,7 +100,7 @@ struct isp_pipeline {
 };
 
 #define to_isp_pipeline(__e) \
-	container_of((__e)->pipe, struct isp_pipeline, pipe)
+	container_of((__e)->pads->pipe, struct isp_pipeline, pipe)
 
 static inline int isp_pipeline_ready(struct isp_pipeline *pipe)
 {
diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index cb3025992817..07ec008aacc4 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -132,13 +132,17 @@ static int rvin_group_link_notify(struct media_link *link, u32 flags,
 		return 0;
 
 	/*
-	 * Don't allow link changes if any entity in the graph is
-	 * streaming, modifying the CHSEL register fields can disrupt
-	 * running streams.
+	 * Don't allow link changes if any stream in the graph is active as
+	 * modifying the CHSEL register fields can disrupt running streams.
 	 */
-	media_device_for_each_entity(entity, &group->mdev)
-		if (entity->stream_count)
-			return -EBUSY;
+	media_device_for_each_entity(entity, &group->mdev) {
+		struct media_pad *iter;
+
+		media_entity_for_each_pad(entity, iter) {
+			if (iter->stream_count)
+				return -EBUSY;
+		}
+	}
 
 	mutex_lock(&group->lock);
 
diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index f30dafbdf61c..7994262c9b63 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -1231,7 +1231,7 @@ static int rvin_set_stream(struct rvin_dev *vin, int on)
 	 */
 	mdev = vin->vdev.entity.graph_obj.mdev;
 	mutex_lock(&mdev->graph_mutex);
-	pipe = sd->entity.pipe ? sd->entity.pipe : &vin->vdev.pipe;
+	pipe = sd->entity.pads->pipe ? sd->entity.pads->pipe : &vin->vdev.pipe;
 	ret = __media_pipeline_start(&vin->vdev.entity, pipe);
 	mutex_unlock(&mdev->graph_mutex);
 	if (ret)
diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index 8df3c43aecbe..7fa0467dddde 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -403,7 +403,7 @@ static int xvip_dma_start_streaming(struct vb2_queue *vq, unsigned int count)
 	 * Use the pipeline object embedded in the first DMA object that starts
 	 * streaming.
 	 */
-	pipe = dma->video.entity.pipe
+	pipe = dma->video.entity.pads->pipe
 	     ? to_xvip_pipeline(&dma->video.entity) : &dma->pipe;
 
 	ret = media_pipeline_start(&dma->video.entity, &pipe->pipe);
diff --git a/drivers/media/platform/xilinx/xilinx-dma.h b/drivers/media/platform/xilinx/xilinx-dma.h
index 2378bdae57ae..69ced71a5696 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.h
+++ b/drivers/media/platform/xilinx/xilinx-dma.h
@@ -47,7 +47,7 @@ struct xvip_pipeline {
 
 static inline struct xvip_pipeline *to_xvip_pipeline(struct media_entity *e)
 {
-	return container_of(e->pipe, struct xvip_pipeline, pipe);
+	return container_of(e->pads->pipe, struct xvip_pipeline, pipe);
 }
 
 /**
diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
index 5128915a5d6f..04b7c6bdcd85 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -913,7 +913,7 @@ int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
 			__media_pipeline_stop(entity);
 	} else {
 		v4l2_subdev_call(sd, video, s_stream, 0);
-		if (entity->pipe)
+		if (entity->pads->pipe)
 			__media_pipeline_stop(entity);
 	}
 
diff --git a/drivers/staging/media/omap4iss/iss.c b/drivers/staging/media/omap4iss/iss.c
index 085397045b36..4f4573219337 100644
--- a/drivers/staging/media/omap4iss/iss.c
+++ b/drivers/staging/media/omap4iss/iss.c
@@ -543,7 +543,7 @@ static int iss_pipeline_is_last(struct media_entity *me)
 	struct iss_pipeline *pipe;
 	struct media_pad *pad;
 
-	if (!me->pipe)
+	if (!me->pads->pipe)
 		return 0;
 	pipe = to_iss_pipeline(me);
 	if (pipe->stream_state == ISS_PIPELINE_STREAM_STOPPED)
diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
index 9f3ff5a37d90..9d7bf8c85558 100644
--- a/drivers/staging/media/omap4iss/iss_video.c
+++ b/drivers/staging/media/omap4iss/iss_video.c
@@ -870,7 +870,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 	 * Start streaming on the pipeline. No link touching an entity in the
 	 * pipeline can be activated or deactivated once streaming is started.
 	 */
-	pipe = pad->entity->pipe
+	pipe = pad->pipe
 	     ? to_iss_pipeline(pad->entity) : &video->pipe;
 	pipe->external = NULL;
 	pipe->external_rate = 0;
diff --git a/drivers/staging/media/omap4iss/iss_video.h b/drivers/staging/media/omap4iss/iss_video.h
index 526281bf0051..9b8ec27bf87d 100644
--- a/drivers/staging/media/omap4iss/iss_video.h
+++ b/drivers/staging/media/omap4iss/iss_video.h
@@ -91,7 +91,7 @@ struct iss_pipeline {
 };
 
 #define to_iss_pipeline(__e) \
-	container_of((__e)->pipe, struct iss_pipeline, pipe)
+	container_of((__e)->pads->pipe, struct iss_pipeline, pipe)
 
 static inline int iss_pipeline_ready(struct iss_pipeline *pipe)
 {
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 52b1a1cab57a..b8f94662526c 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -180,15 +180,24 @@ enum media_pad_signal_type {
  *
  * @graph_obj:	Embedded structure containing the media object common data
  * @entity:	Entity this pad belongs to
+ * @pipe:	Pipeline this pad belongs to.
+ * @stream_count: Stream count for the pad.
  * @index:	Pad index in the entity pads array, numbered from 0 to n
  * @sig_type:	Type of the signal inside a media pad
  * @flags:	Pad flags, as defined in
  *		:ref:`include/uapi/linux/media.h <media_header>`
  *		(seek for ``MEDIA_PAD_FL_*``)
+ * .. note::
+ *
+ *    @stream_count reference count must never be negative, but is a signed
+ *    integer on purpose: a simple ``WARN_ON(<0)`` check can be used to
+ *    detect reference count bugs that would make them negative.
  */
 struct media_pad {
 	struct media_gobj graph_obj;	/* must be first field in struct */
 	struct media_entity *entity;
+	struct media_pipeline *pipe;
+	int stream_count;
 	u16 index;
 	enum media_pad_signal_type sig_type;
 	unsigned long flags;
@@ -267,9 +276,7 @@ enum media_entity_type {
  * @pads:	Pads array with the size defined by @num_pads.
  * @links:	List of data links.
  * @ops:	Entity operations.
- * @stream_count: Stream count for the entity.
  * @use_count:	Use count for the entity.
- * @pipe:	Pipeline this entity belongs to.
  * @info:	Union with devnode information.  Kept just for backward
  *		compatibility.
  * @info.dev:	Contains device major and minor info.
@@ -282,10 +289,9 @@ enum media_entity_type {
  *
  * .. note::
  *
- *    @stream_count and @use_count reference counts must never be
- *    negative, but are signed integers on purpose: a simple ``WARN_ON(<0)``
- *    check can be used to detect reference count bugs that would make them
- *    negative.
+ *    @use_count reference count must never be negative, but is a signed
+ *    integer on purpose: a simple ``WARN_ON(<0)`` check can be used to
+ *    detect reference count bugs that would make them negative.
  */
 struct media_entity {
 	struct media_gobj graph_obj;	/* must be first field in struct */
@@ -304,11 +310,8 @@ struct media_entity {
 
 	const struct media_entity_operations *ops;
 
-	int stream_count;
 	int use_count;
 
-	struct media_pipeline *pipe;
-
 	union {
 		struct {
 			u32 major;
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 07/24] media: entity: Use pad as the starting point for a pipeline
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (5 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 06/24] media: entity: Move the pipeline from entity to pads Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-15 13:04 ` [PATCH v5 08/24] media: entity: Add has_route entity operation Tomi Valkeinen
                   ` (18 subsequent siblings)
  25 siblings, 0 replies; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil

From: Sakari Ailus <sakari.ailus@linux.intel.com>

The pipeline has been moved from the entity to the pads; reflect this in
the media pipeline function API.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 Documentation/driver-api/media/mc-core.rst    |  6 ++--
 drivers/media/mc/mc-entity.c                  | 24 ++++++-------
 drivers/media/pci/intel/ipu3/ipu3-cio2-main.c |  6 ++--
 .../media/platform/exynos4-is/fimc-capture.c  |  8 ++---
 .../platform/exynos4-is/fimc-isp-video.c      |  8 ++---
 drivers/media/platform/exynos4-is/fimc-lite.c |  8 ++---
 drivers/media/platform/omap3isp/ispvideo.c    |  6 ++--
 .../media/platform/qcom/camss/camss-video.c   |  6 ++--
 drivers/media/platform/rcar-vin/rcar-dma.c    |  6 ++--
 .../platform/rockchip/rkisp1/rkisp1-capture.c |  6 ++--
 .../media/platform/s3c-camif/camif-capture.c  |  6 ++--
 drivers/media/platform/stm32/stm32-dcmi.c     |  6 ++--
 .../platform/sunxi/sun4i-csi/sun4i_dma.c      |  6 ++--
 .../platform/sunxi/sun6i-csi/sun6i_video.c    |  6 ++--
 drivers/media/platform/ti-vpe/cal-video.c     |  6 ++--
 drivers/media/platform/vsp1/vsp1_video.c      |  6 ++--
 drivers/media/platform/xilinx/xilinx-dma.c    |  6 ++--
 .../media/test-drivers/vimc/vimc-capture.c    |  6 ++--
 drivers/media/usb/au0828/au0828-core.c        |  8 ++---
 drivers/staging/media/imx/imx-media-utils.c   |  6 ++--
 drivers/staging/media/ipu3/ipu3-v4l2.c        |  6 ++--
 drivers/staging/media/omap4iss/iss_video.c    |  6 ++--
 drivers/staging/media/tegra-video/tegra210.c  |  6 ++--
 include/media/media-entity.h                  | 34 +++++++++----------
 24 files changed, 98 insertions(+), 100 deletions(-)

diff --git a/Documentation/driver-api/media/mc-core.rst b/Documentation/driver-api/media/mc-core.rst
index 8a13640bed56..69a64279a61f 100644
--- a/Documentation/driver-api/media/mc-core.rst
+++ b/Documentation/driver-api/media/mc-core.rst
@@ -213,11 +213,11 @@ When starting streaming, drivers must notify all entities in the pipeline to
 prevent link states from being modified during streaming by calling
 :c:func:`media_pipeline_start()`.
 
-The function will mark all entities connected to the given entity through
-enabled links, either directly or indirectly, as streaming.
+The function will mark all entities connected to the given pad through
+enabled routes and links, either directly or indirectly, as streaming.
 
 The struct media_pipeline instance pointed to by
-the pipe argument will be stored in every entity in the pipeline.
+the pipe argument will be stored in every pad in the pipeline.
 Drivers should embed the struct media_pipeline
 in higher-level pipeline structures and can then access the
 pipeline through the struct media_entity
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index b6e5aa639c26..45c3d45bc30c 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -403,12 +403,11 @@ EXPORT_SYMBOL_GPL(media_entity_get_fwnode_pad);
  * Pipeline management
  */
 
-__must_check int __media_pipeline_start(struct media_entity *entity,
+__must_check int __media_pipeline_start(struct media_pad *pad,
 					struct media_pipeline *pipe)
 {
-	struct media_device *mdev = entity->graph_obj.mdev;
+	struct media_device *mdev = pad->graph_obj.mdev;
 	struct media_graph *graph = &pipe->graph;
-	struct media_pad *pad = entity->pads;
 	struct media_pad *pad_err = pad;
 	struct media_link *link;
 	int ret;
@@ -541,24 +540,23 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
 }
 EXPORT_SYMBOL_GPL(__media_pipeline_start);
 
-__must_check int media_pipeline_start(struct media_entity *entity,
+__must_check int media_pipeline_start(struct media_pad *pad,
 				      struct media_pipeline *pipe)
 {
-	struct media_device *mdev = entity->graph_obj.mdev;
+	struct media_device *mdev = pad->graph_obj.mdev;
 	int ret;
 
 	mutex_lock(&mdev->graph_mutex);
-	ret = __media_pipeline_start(entity, pipe);
+	ret = __media_pipeline_start(pad, pipe);
 	mutex_unlock(&mdev->graph_mutex);
 	return ret;
 }
 EXPORT_SYMBOL_GPL(media_pipeline_start);
 
-void __media_pipeline_stop(struct media_entity *entity)
+void __media_pipeline_stop(struct media_pad *pad)
 {
-	struct media_pipeline *pipe = entity->pads->pipe;
+	struct media_pipeline *pipe = pad->pipe;
 	struct media_graph *graph = &pipe->graph;
-	struct media_pad *pad;
 
 	/*
 	 * If the following check fails, the driver has performed an
@@ -567,7 +565,7 @@ void __media_pipeline_stop(struct media_entity *entity)
 	if (WARN_ON(!pipe))
 		return;
 
-	media_graph_walk_start(graph, entity->pads);
+	media_graph_walk_start(graph, pad);
 
 	while ((pad = media_graph_walk_next(graph))) {
 		struct media_entity *entity = pad->entity;
@@ -589,12 +587,12 @@ void __media_pipeline_stop(struct media_entity *entity)
 }
 EXPORT_SYMBOL_GPL(__media_pipeline_stop);
 
-void media_pipeline_stop(struct media_entity *entity)
+void media_pipeline_stop(struct media_pad *pad)
 {
-	struct media_device *mdev = entity->graph_obj.mdev;
+	struct media_device *mdev = pad->graph_obj.mdev;
 
 	mutex_lock(&mdev->graph_mutex);
-	__media_pipeline_stop(entity);
+	__media_pipeline_stop(pad);
 	mutex_unlock(&mdev->graph_mutex);
 }
 EXPORT_SYMBOL_GPL(media_pipeline_stop);
diff --git a/drivers/media/pci/intel/ipu3/ipu3-cio2-main.c b/drivers/media/pci/intel/ipu3/ipu3-cio2-main.c
index 6e8c0c230e11..53c87e972048 100644
--- a/drivers/media/pci/intel/ipu3/ipu3-cio2-main.c
+++ b/drivers/media/pci/intel/ipu3/ipu3-cio2-main.c
@@ -982,7 +982,7 @@ static int cio2_vb2_start_streaming(struct vb2_queue *vq, unsigned int count)
 		return r;
 	}
 
-	r = media_pipeline_start(&q->vdev.entity, &q->pipe);
+	r = media_pipeline_start(q->vdev.entity.pads, &q->pipe);
 	if (r)
 		goto fail_pipeline;
 
@@ -1002,7 +1002,7 @@ static int cio2_vb2_start_streaming(struct vb2_queue *vq, unsigned int count)
 fail_csi2_subdev:
 	cio2_hw_exit(cio2, q);
 fail_hw:
-	media_pipeline_stop(&q->vdev.entity);
+	media_pipeline_stop(q->vdev.entity.pads);
 fail_pipeline:
 	dev_dbg(&cio2->pci_dev->dev, "failed to start streaming (%d)\n", r);
 	cio2_vb2_return_all_buffers(q, VB2_BUF_STATE_QUEUED);
@@ -1023,7 +1023,7 @@ static void cio2_vb2_stop_streaming(struct vb2_queue *vq)
 	cio2_hw_exit(cio2, q);
 	synchronize_irq(cio2->pci_dev->irq);
 	cio2_vb2_return_all_buffers(q, VB2_BUF_STATE_ERROR);
-	media_pipeline_stop(&q->vdev.entity);
+	media_pipeline_stop(q->vdev.entity.pads);
 	pm_runtime_put(&cio2->pci_dev->dev);
 	cio2->streaming = false;
 }
diff --git a/drivers/media/platform/exynos4-is/fimc-capture.c b/drivers/media/platform/exynos4-is/fimc-capture.c
index 13c838d3f947..2f67415575db 100644
--- a/drivers/media/platform/exynos4-is/fimc-capture.c
+++ b/drivers/media/platform/exynos4-is/fimc-capture.c
@@ -526,7 +526,7 @@ static int fimc_capture_release(struct file *file)
 	mutex_lock(&fimc->lock);
 
 	if (close && vc->streaming) {
-		media_pipeline_stop(&vc->ve.vdev.entity);
+		media_pipeline_stop(vc->ve.vdev.entity.pads);
 		vc->streaming = false;
 	}
 
@@ -1186,7 +1186,7 @@ static int fimc_cap_streamon(struct file *file, void *priv,
 	if (fimc_capture_active(fimc))
 		return -EBUSY;
 
-	ret = media_pipeline_start(entity, &vc->ve.pipe->mp);
+	ret = media_pipeline_start(entity->pads, &vc->ve.pipe->mp);
 	if (ret < 0)
 		return ret;
 
@@ -1220,7 +1220,7 @@ static int fimc_cap_streamon(struct file *file, void *priv,
 	}
 
 err_p_stop:
-	media_pipeline_stop(entity);
+	media_pipeline_stop(entity->pads);
 	return ret;
 }
 
@@ -1236,7 +1236,7 @@ static int fimc_cap_streamoff(struct file *file, void *priv,
 		return ret;
 
 	if (vc->streaming) {
-		media_pipeline_stop(&vc->ve.vdev.entity);
+		media_pipeline_stop(vc->ve.vdev.entity.pads);
 		vc->streaming = false;
 	}
 
diff --git a/drivers/media/platform/exynos4-is/fimc-isp-video.c b/drivers/media/platform/exynos4-is/fimc-isp-video.c
index 612b9872afc8..d9289ff6949a 100644
--- a/drivers/media/platform/exynos4-is/fimc-isp-video.c
+++ b/drivers/media/platform/exynos4-is/fimc-isp-video.c
@@ -310,7 +310,7 @@ static int isp_video_release(struct file *file)
 	mutex_lock(&isp->video_lock);
 
 	if (v4l2_fh_is_singular_file(file) && ivc->streaming) {
-		media_pipeline_stop(entity);
+		media_pipeline_stop(entity->pads);
 		ivc->streaming = 0;
 	}
 
@@ -491,7 +491,7 @@ static int isp_video_streamon(struct file *file, void *priv,
 	struct media_entity *me = &ve->vdev.entity;
 	int ret;
 
-	ret = media_pipeline_start(me, &ve->pipe->mp);
+	ret = media_pipeline_start(me->pads, &ve->pipe->mp);
 	if (ret < 0)
 		return ret;
 
@@ -506,7 +506,7 @@ static int isp_video_streamon(struct file *file, void *priv,
 	isp->video_capture.streaming = 1;
 	return 0;
 p_stop:
-	media_pipeline_stop(me);
+	media_pipeline_stop(me->pads);
 	return ret;
 }
 
@@ -521,7 +521,7 @@ static int isp_video_streamoff(struct file *file, void *priv,
 	if (ret < 0)
 		return ret;
 
-	media_pipeline_stop(&video->ve.vdev.entity);
+	media_pipeline_stop(video->ve.vdev.entity.pads);
 	video->streaming = 0;
 	return 0;
 }
diff --git a/drivers/media/platform/exynos4-is/fimc-lite.c b/drivers/media/platform/exynos4-is/fimc-lite.c
index 56773a0be8d9..a1570f6ac738 100644
--- a/drivers/media/platform/exynos4-is/fimc-lite.c
+++ b/drivers/media/platform/exynos4-is/fimc-lite.c
@@ -515,7 +515,7 @@ static int fimc_lite_release(struct file *file)
 	if (v4l2_fh_is_singular_file(file) &&
 	    atomic_read(&fimc->out_path) == FIMC_IO_DMA) {
 		if (fimc->streaming) {
-			media_pipeline_stop(entity);
+			media_pipeline_stop(entity->pads);
 			fimc->streaming = false;
 		}
 		fimc_lite_stop_capture(fimc, false);
@@ -819,7 +819,7 @@ static int fimc_lite_streamon(struct file *file, void *priv,
 	if (fimc_lite_active(fimc))
 		return -EBUSY;
 
-	ret = media_pipeline_start(entity, &fimc->ve.pipe->mp);
+	ret = media_pipeline_start(entity->pads, &fimc->ve.pipe->mp);
 	if (ret < 0)
 		return ret;
 
@@ -836,7 +836,7 @@ static int fimc_lite_streamon(struct file *file, void *priv,
 	}
 
 err_p_stop:
-	media_pipeline_stop(entity);
+	media_pipeline_stop(entity->pads);
 	return 0;
 }
 
@@ -850,7 +850,7 @@ static int fimc_lite_streamoff(struct file *file, void *priv,
 	if (ret < 0)
 		return ret;
 
-	media_pipeline_stop(&fimc->ve.vdev.entity);
+	media_pipeline_stop(fimc->ve.vdev.entity.pads);
 	fimc->streaming = false;
 	return 0;
 }
diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
index f3bee79cdd85..ea4b6a90138b 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -1105,7 +1105,7 @@ isp_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 	pipe->l3_ick = clk_get_rate(video->isp->clock[ISP_CLK_L3_ICK]);
 	pipe->max_rate = pipe->l3_ick;
 
-	ret = media_pipeline_start(&video->video.entity, &pipe->pipe);
+	ret = media_pipeline_start(video->video.entity.pads, &pipe->pipe);
 	if (ret < 0)
 		goto err_pipeline_start;
 
@@ -1162,7 +1162,7 @@ isp_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 	return 0;
 
 err_check_format:
-	media_pipeline_stop(&video->video.entity);
+	media_pipeline_stop(video->video.entity.pads);
 err_pipeline_start:
 	/* TODO: Implement PM QoS */
 	/* The DMA queue must be emptied here, otherwise CCDC interrupts that
@@ -1229,7 +1229,7 @@ isp_video_streamoff(struct file *file, void *fh, enum v4l2_buf_type type)
 	video->error = false;
 
 	/* TODO: Implement PM QoS */
-	media_pipeline_stop(&video->video.entity);
+	media_pipeline_stop(video->video.entity.pads);
 
 	media_entity_enum_cleanup(&pipe->ent_enum);
 
diff --git a/drivers/media/platform/qcom/camss/camss-video.c b/drivers/media/platform/qcom/camss/camss-video.c
index f282275af626..5cd494f17589 100644
--- a/drivers/media/platform/qcom/camss/camss-video.c
+++ b/drivers/media/platform/qcom/camss/camss-video.c
@@ -491,7 +491,7 @@ static int video_start_streaming(struct vb2_queue *q, unsigned int count)
 	struct v4l2_subdev *subdev;
 	int ret;
 
-	ret = media_pipeline_start(&vdev->entity, &video->pipe);
+	ret = media_pipeline_start(vdev->entity.pads, &video->pipe);
 	if (ret < 0)
 		return ret;
 
@@ -520,7 +520,7 @@ static int video_start_streaming(struct vb2_queue *q, unsigned int count)
 	return 0;
 
 error:
-	media_pipeline_stop(&vdev->entity);
+	media_pipeline_stop(vdev->entity.pads);
 
 	video->ops->flush_buffers(video, VB2_BUF_STATE_QUEUED);
 
@@ -551,7 +551,7 @@ static void video_stop_streaming(struct vb2_queue *q)
 		v4l2_subdev_call(subdev, video, s_stream, 0);
 	}
 
-	media_pipeline_stop(&vdev->entity);
+	media_pipeline_stop(vdev->entity.pads);
 
 	video->ops->flush_buffers(video, VB2_BUF_STATE_ERROR);
 }
diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index 7994262c9b63..2d930e53c1e2 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -1215,7 +1215,7 @@ static int rvin_set_stream(struct rvin_dev *vin, int on)
 	sd = media_entity_to_v4l2_subdev(pad->entity);
 
 	if (!on) {
-		media_pipeline_stop(&vin->vdev.entity);
+		media_pipeline_stop(vin->vdev.entity.pads);
 		return v4l2_subdev_call(sd, video, s_stream, 0);
 	}
 
@@ -1232,7 +1232,7 @@ static int rvin_set_stream(struct rvin_dev *vin, int on)
 	mdev = vin->vdev.entity.graph_obj.mdev;
 	mutex_lock(&mdev->graph_mutex);
 	pipe = sd->entity.pads->pipe ? sd->entity.pads->pipe : &vin->vdev.pipe;
-	ret = __media_pipeline_start(&vin->vdev.entity, pipe);
+	ret = __media_pipeline_start(vin->vdev.entity.pads, pipe);
 	mutex_unlock(&mdev->graph_mutex);
 	if (ret)
 		return ret;
@@ -1241,7 +1241,7 @@ static int rvin_set_stream(struct rvin_dev *vin, int on)
 	if (ret == -ENOIOCTLCMD)
 		ret = 0;
 	if (ret)
-		media_pipeline_stop(&vin->vdev.entity);
+		media_pipeline_stop(vin->vdev.entity.pads);
 
 	return ret;
 }
diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-capture.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-capture.c
index 5f6c9d1623e4..c2a11fcc9709 100644
--- a/drivers/media/platform/rockchip/rkisp1/rkisp1-capture.c
+++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-capture.c
@@ -979,7 +979,7 @@ static void rkisp1_vb2_stop_streaming(struct vb2_queue *queue)
 
 	rkisp1_dummy_buf_destroy(cap);
 
-	media_pipeline_stop(&node->vdev.entity);
+	media_pipeline_stop(node->vdev.entity.pads);
 
 	mutex_unlock(&cap->rkisp1->stream_lock);
 }
@@ -993,7 +993,7 @@ rkisp1_vb2_start_streaming(struct vb2_queue *queue, unsigned int count)
 
 	mutex_lock(&cap->rkisp1->stream_lock);
 
-	ret = media_pipeline_start(entity, &cap->rkisp1->pipe);
+	ret = media_pipeline_start(entity->pads, &cap->rkisp1->pipe);
 	if (ret) {
 		dev_err(cap->rkisp1->dev, "start pipeline failed %d\n", ret);
 		goto err_ret_buffers;
@@ -1030,7 +1030,7 @@ rkisp1_vb2_start_streaming(struct vb2_queue *queue, unsigned int count)
 err_destroy_dummy:
 	rkisp1_dummy_buf_destroy(cap);
 err_pipeline_stop:
-	media_pipeline_stop(entity);
+	media_pipeline_stop(entity->pads);
 err_ret_buffers:
 	rkisp1_return_all_buffers(cap, VB2_BUF_STATE_QUEUED);
 	mutex_unlock(&cap->rkisp1->stream_lock);
diff --git a/drivers/media/platform/s3c-camif/camif-capture.c b/drivers/media/platform/s3c-camif/camif-capture.c
index 9ca49af29542..22451bc2ef2d 100644
--- a/drivers/media/platform/s3c-camif/camif-capture.c
+++ b/drivers/media/platform/s3c-camif/camif-capture.c
@@ -848,13 +848,13 @@ static int s3c_camif_streamon(struct file *file, void *priv,
 	if (s3c_vp_active(vp))
 		return 0;
 
-	ret = media_pipeline_start(sensor, camif->m_pipeline);
+	ret = media_pipeline_start(sensor->pads, camif->m_pipeline);
 	if (ret < 0)
 		return ret;
 
 	ret = camif_pipeline_validate(camif);
 	if (ret < 0) {
-		media_pipeline_stop(sensor);
+		media_pipeline_stop(sensor->pads);
 		return ret;
 	}
 
@@ -878,7 +878,7 @@ static int s3c_camif_streamoff(struct file *file, void *priv,
 
 	ret = vb2_streamoff(&vp->vb_queue, type);
 	if (ret == 0)
-		media_pipeline_stop(&camif->sensor.sd->entity);
+		media_pipeline_stop(camif->sensor.sd->entity.pads);
 	return ret;
 }
 
diff --git a/drivers/media/platform/stm32/stm32-dcmi.c b/drivers/media/platform/stm32/stm32-dcmi.c
index bbcc2254fa2e..c68fdb6c766b 100644
--- a/drivers/media/platform/stm32/stm32-dcmi.c
+++ b/drivers/media/platform/stm32/stm32-dcmi.c
@@ -730,7 +730,7 @@ static int dcmi_start_streaming(struct vb2_queue *vq, unsigned int count)
 		goto err_pm_put;
 	}
 
-	ret = media_pipeline_start(&dcmi->vdev->entity, &dcmi->pipeline);
+	ret = media_pipeline_start(dcmi->vdev->entity.pads, &dcmi->pipeline);
 	if (ret < 0) {
 		dev_err(dcmi->dev, "%s: Failed to start streaming, media pipeline start error (%d)\n",
 			__func__, ret);
@@ -844,7 +844,7 @@ static int dcmi_start_streaming(struct vb2_queue *vq, unsigned int count)
 	dcmi_pipeline_stop(dcmi);
 
 err_media_pipeline_stop:
-	media_pipeline_stop(&dcmi->vdev->entity);
+	media_pipeline_stop(dcmi->vdev->entity.pads);
 
 err_pm_put:
 	pm_runtime_put(dcmi->dev);
@@ -870,7 +870,7 @@ static void dcmi_stop_streaming(struct vb2_queue *vq)
 
 	dcmi_pipeline_stop(dcmi);
 
-	media_pipeline_stop(&dcmi->vdev->entity);
+	media_pipeline_stop(dcmi->vdev->entity.pads);
 
 	spin_lock_irq(&dcmi->irqlock);
 
diff --git a/drivers/media/platform/sunxi/sun4i-csi/sun4i_dma.c b/drivers/media/platform/sunxi/sun4i-csi/sun4i_dma.c
index 2c39cd7f2862..be0defdf74f1 100644
--- a/drivers/media/platform/sunxi/sun4i-csi/sun4i_dma.c
+++ b/drivers/media/platform/sunxi/sun4i-csi/sun4i_dma.c
@@ -266,7 +266,7 @@ static int sun4i_csi_start_streaming(struct vb2_queue *vq, unsigned int count)
 		goto err_clear_dma_queue;
 	}
 
-	ret = media_pipeline_start(&csi->vdev.entity, &csi->vdev.pipe);
+	ret = media_pipeline_start(csi->vdev.entity.pads, &csi->vdev.pipe);
 	if (ret < 0)
 		goto err_free_scratch_buffer;
 
@@ -330,7 +330,7 @@ static int sun4i_csi_start_streaming(struct vb2_queue *vq, unsigned int count)
 	sun4i_csi_capture_stop(csi);
 
 err_disable_pipeline:
-	media_pipeline_stop(&csi->vdev.entity);
+	media_pipeline_stop(csi->vdev.entity.pads);
 
 err_free_scratch_buffer:
 	dma_free_coherent(csi->dev, csi->scratch.size, csi->scratch.vaddr,
@@ -359,7 +359,7 @@ static void sun4i_csi_stop_streaming(struct vb2_queue *vq)
 	return_all_buffers(csi, VB2_BUF_STATE_ERROR);
 	spin_unlock_irqrestore(&csi->qlock, flags);
 
-	media_pipeline_stop(&csi->vdev.entity);
+	media_pipeline_stop(csi->vdev.entity.pads);
 
 	dma_free_coherent(csi->dev, csi->scratch.size, csi->scratch.vaddr,
 			  csi->scratch.paddr);
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
index 3181d0781b61..537057a75eaa 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
@@ -141,7 +141,7 @@ static int sun6i_video_start_streaming(struct vb2_queue *vq, unsigned int count)
 
 	video->sequence = 0;
 
-	ret = media_pipeline_start(&video->vdev.entity, &video->vdev.pipe);
+	ret = media_pipeline_start(video->vdev.entity.pads, &video->vdev.pipe);
 	if (ret < 0)
 		goto clear_dma_queue;
 
@@ -207,7 +207,7 @@ static int sun6i_video_start_streaming(struct vb2_queue *vq, unsigned int count)
 stop_csi_stream:
 	sun6i_csi_set_stream(video->csi, false);
 stop_media_pipeline:
-	media_pipeline_stop(&video->vdev.entity);
+	media_pipeline_stop(video->vdev.entity.pads);
 clear_dma_queue:
 	spin_lock_irqsave(&video->dma_queue_lock, flags);
 	list_for_each_entry(buf, &video->dma_queue, list)
@@ -231,7 +231,7 @@ static void sun6i_video_stop_streaming(struct vb2_queue *vq)
 
 	sun6i_csi_set_stream(video->csi, false);
 
-	media_pipeline_stop(&video->vdev.entity);
+	media_pipeline_stop(video->vdev.entity.pads);
 
 	/* Release all active buffers */
 	spin_lock_irqsave(&video->dma_queue_lock, flags);
diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 7b7436a355ee..ca75b54311a8 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -675,7 +675,7 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 	dma_addr_t addr;
 	int ret;
 
-	ret = media_pipeline_start(&ctx->vdev.entity, &ctx->phy->pipe);
+	ret = media_pipeline_start(ctx->vdev.entity.pads, &ctx->phy->pipe);
 	if (ret < 0) {
 		ctx_err(ctx, "Failed to start media pipeline: %d\n", ret);
 		goto error_release_buffers;
@@ -719,7 +719,7 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 	pm_runtime_put_sync(ctx->cal->dev);
 
 error_pipeline:
-	media_pipeline_stop(&ctx->vdev.entity);
+	media_pipeline_stop(ctx->vdev.entity.pads);
 error_release_buffers:
 	cal_release_buffers(ctx, VB2_BUF_STATE_QUEUED);
 
@@ -738,7 +738,7 @@ static void cal_stop_streaming(struct vb2_queue *vq)
 
 	cal_release_buffers(ctx, VB2_BUF_STATE_ERROR);
 
-	media_pipeline_stop(&ctx->vdev.entity);
+	media_pipeline_stop(ctx->vdev.entity.pads);
 }
 
 static const struct vb2_ops cal_video_qops = {
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index 39dccf347ce1..426b1c339180 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -927,7 +927,7 @@ static void vsp1_video_stop_streaming(struct vb2_queue *vq)
 	}
 	mutex_unlock(&pipe->lock);
 
-	media_pipeline_stop(&video->video.entity);
+	media_pipeline_stop(video->video.entity.pads);
 	vsp1_video_release_buffers(video);
 	vsp1_video_pipeline_put(pipe);
 }
@@ -1048,7 +1048,7 @@ vsp1_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 		return PTR_ERR(pipe);
 	}
 
-	ret = __media_pipeline_start(&video->video.entity, &pipe->pipe);
+	ret = __media_pipeline_start(video->video.entity.pads, &pipe->pipe);
 	if (ret < 0) {
 		mutex_unlock(&mdev->graph_mutex);
 		goto err_pipe;
@@ -1072,7 +1072,7 @@ vsp1_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 	return 0;
 
 err_stop:
-	media_pipeline_stop(&video->video.entity);
+	media_pipeline_stop(video->video.entity.pads);
 err_pipe:
 	vsp1_video_pipeline_put(pipe);
 	return ret;
diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index 7fa0467dddde..07074eda5f70 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -406,7 +406,7 @@ static int xvip_dma_start_streaming(struct vb2_queue *vq, unsigned int count)
 	pipe = dma->video.entity.pads->pipe
 	     ? to_xvip_pipeline(&dma->video.entity) : &dma->pipe;
 
-	ret = media_pipeline_start(&dma->video.entity, &pipe->pipe);
+	ret = media_pipeline_start(dma->video.entity.pads, &pipe->pipe);
 	if (ret < 0)
 		goto error;
 
@@ -432,7 +432,7 @@ static int xvip_dma_start_streaming(struct vb2_queue *vq, unsigned int count)
 	return 0;
 
 error_stop:
-	media_pipeline_stop(&dma->video.entity);
+	media_pipeline_stop(dma->video.entity.pads);
 
 error:
 	/* Give back all queued buffers to videobuf2. */
@@ -460,7 +460,7 @@ static void xvip_dma_stop_streaming(struct vb2_queue *vq)
 
 	/* Cleanup the pipeline and mark it as being stopped. */
 	xvip_pipeline_cleanup(pipe);
-	media_pipeline_stop(&dma->video.entity);
+	media_pipeline_stop(dma->video.entity.pads);
 
 	/* Give back all queued buffers to videobuf2. */
 	spin_lock_irq(&dma->queued_lock);
diff --git a/drivers/media/test-drivers/vimc/vimc-capture.c b/drivers/media/test-drivers/vimc/vimc-capture.c
index 5e9fd902cd37..10724b0a868c 100644
--- a/drivers/media/test-drivers/vimc/vimc-capture.c
+++ b/drivers/media/test-drivers/vimc/vimc-capture.c
@@ -246,7 +246,7 @@ static int vimc_cap_start_streaming(struct vb2_queue *vq, unsigned int count)
 	vcap->sequence = 0;
 
 	/* Start the media pipeline */
-	ret = media_pipeline_start(entity, &vcap->stream.pipe);
+	ret = media_pipeline_start(entity->pads, &vcap->stream.pipe);
 	if (ret) {
 		vimc_cap_return_all_buffers(vcap, VB2_BUF_STATE_QUEUED);
 		return ret;
@@ -254,7 +254,7 @@ static int vimc_cap_start_streaming(struct vb2_queue *vq, unsigned int count)
 
 	ret = vimc_streamer_s_stream(&vcap->stream, &vcap->ved, 1);
 	if (ret) {
-		media_pipeline_stop(entity);
+		media_pipeline_stop(entity->pads);
 		vimc_cap_return_all_buffers(vcap, VB2_BUF_STATE_QUEUED);
 		return ret;
 	}
@@ -273,7 +273,7 @@ static void vimc_cap_stop_streaming(struct vb2_queue *vq)
 	vimc_streamer_s_stream(&vcap->stream, &vcap->ved, 0);
 
 	/* Stop the media pipeline */
-	media_pipeline_stop(&vcap->vdev.entity);
+	media_pipeline_stop(vcap->vdev.entity.pads);
 
 	/* Release all active buffers */
 	vimc_cap_return_all_buffers(vcap, VB2_BUF_STATE_ERROR);
diff --git a/drivers/media/usb/au0828/au0828-core.c b/drivers/media/usb/au0828/au0828-core.c
index a8a72d5fbd12..93dd7bb0ece0 100644
--- a/drivers/media/usb/au0828/au0828-core.c
+++ b/drivers/media/usb/au0828/au0828-core.c
@@ -410,7 +410,7 @@ static int au0828_enable_source(struct media_entity *entity,
 		goto end;
 	}
 
-	ret = __media_pipeline_start(entity, pipe);
+	ret = __media_pipeline_start(entity->pads, pipe);
 	if (ret) {
 		pr_err("Start Pipeline: %s->%s Error %d\n",
 			source->name, entity->name, ret);
@@ -501,12 +501,12 @@ static void au0828_disable_source(struct media_entity *entity)
 				return;
 
 			/* stop pipeline */
-			__media_pipeline_stop(dev->active_link_owner);
+			__media_pipeline_stop(dev->active_link_owner->pads);
 			pr_debug("Pipeline stop for %s\n",
 				dev->active_link_owner->name);
 
 			ret = __media_pipeline_start(
-					dev->active_link_user,
+					dev->active_link_user->pads,
 					dev->active_link_user_pipe);
 			if (ret) {
 				pr_err("Start Pipeline: %s->%s %d\n",
@@ -532,7 +532,7 @@ static void au0828_disable_source(struct media_entity *entity)
 			return;
 
 		/* stop pipeline */
-		__media_pipeline_stop(dev->active_link_owner);
+		__media_pipeline_stop(dev->active_link_owner->pads);
 		pr_debug("Pipeline stop for %s\n",
 			dev->active_link_owner->name);
 
diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
index 04b7c6bdcd85..b8cffb103f12 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -905,16 +905,16 @@ int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
 	mutex_lock(&imxmd->md.graph_mutex);
 
 	if (on) {
-		ret = __media_pipeline_start(entity, &imxmd->pipe);
+		ret = __media_pipeline_start(entity->pads, &imxmd->pipe);
 		if (ret)
 			goto out;
 		ret = v4l2_subdev_call(sd, video, s_stream, 1);
 		if (ret)
-			__media_pipeline_stop(entity);
+			__media_pipeline_stop(entity->pads);
 	} else {
 		v4l2_subdev_call(sd, video, s_stream, 0);
 		if (entity->pads->pipe)
-			__media_pipeline_stop(entity);
+			__media_pipeline_stop(entity->pads);
 	}
 
 out:
diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c b/drivers/staging/media/ipu3/ipu3-v4l2.c
index 60aa02eb7d2a..7a3e2737401d 100644
--- a/drivers/staging/media/ipu3/ipu3-v4l2.c
+++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
@@ -485,7 +485,7 @@ static int imgu_vb2_start_streaming(struct vb2_queue *vq, unsigned int count)
 
 	pipe = node->pipe;
 	imgu_pipe = &imgu->imgu_pipe[pipe];
-	r = media_pipeline_start(&node->vdev.entity, &imgu_pipe->pipeline);
+	r = media_pipeline_start(node->vdev.entity.pads, &imgu_pipe->pipeline);
 	if (r < 0)
 		goto fail_return_bufs;
 
@@ -510,7 +510,7 @@ static int imgu_vb2_start_streaming(struct vb2_queue *vq, unsigned int count)
 	return 0;
 
 fail_stop_pipeline:
-	media_pipeline_stop(&node->vdev.entity);
+	media_pipeline_stop(node->vdev.entity.pads);
 fail_return_bufs:
 	imgu_return_all_buffers(imgu, node, VB2_BUF_STATE_QUEUED);
 
@@ -550,7 +550,7 @@ static void imgu_vb2_stop_streaming(struct vb2_queue *vq)
 	imgu_return_all_buffers(imgu, node, VB2_BUF_STATE_ERROR);
 	mutex_unlock(&imgu->streaming_lock);
 
-	media_pipeline_stop(&node->vdev.entity);
+	media_pipeline_stop(node->vdev.entity.pads);
 }
 
 /******************** v4l2_ioctl_ops ********************/
diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
index 9d7bf8c85558..46aa91fe12fe 100644
--- a/drivers/staging/media/omap4iss/iss_video.c
+++ b/drivers/staging/media/omap4iss/iss_video.c
@@ -888,7 +888,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 	if (video->iss->pdata->set_constraints)
 		video->iss->pdata->set_constraints(video->iss, true);
 
-	ret = media_pipeline_start(pad->entity, &pipe->pipe);
+	ret = media_pipeline_start(pad, &pipe->pipe);
 	if (ret < 0)
 		goto err_media_pipeline_start;
 
@@ -977,7 +977,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 err_omap4iss_set_stream:
 	vb2_streamoff(&vfh->queue, type);
 err_iss_video_check_format:
-	media_pipeline_stop(&video->video.entity);
+	media_pipeline_stop(video->video.entity.pads);
 err_media_pipeline_start:
 	if (video->iss->pdata->set_constraints)
 		video->iss->pdata->set_constraints(video->iss, false);
@@ -1031,7 +1031,7 @@ iss_video_streamoff(struct file *file, void *fh, enum v4l2_buf_type type)
 
 	if (video->iss->pdata->set_constraints)
 		video->iss->pdata->set_constraints(video->iss, false);
-	media_pipeline_stop(&video->video.entity);
+	media_pipeline_stop(video->video.entity.pads);
 
 done:
 	mutex_unlock(&video->stream_lock);
diff --git a/drivers/staging/media/tegra-video/tegra210.c b/drivers/staging/media/tegra-video/tegra210.c
index f10a041e3e6c..d2d7dd0e8624 100644
--- a/drivers/staging/media/tegra-video/tegra210.c
+++ b/drivers/staging/media/tegra-video/tegra210.c
@@ -547,7 +547,7 @@ static int tegra210_vi_start_streaming(struct vb2_queue *vq, u32 count)
 		       VI_INCR_SYNCPT_NO_STALL);
 
 	/* start the pipeline */
-	ret = media_pipeline_start(&chan->video.entity, pipe);
+	ret = media_pipeline_start(chan->video.entity.pads, pipe);
 	if (ret < 0)
 		goto error_pipeline_start;
 
@@ -595,7 +595,7 @@ static int tegra210_vi_start_streaming(struct vb2_queue *vq, u32 count)
 error_kthread_start:
 	tegra_channel_set_stream(chan, false);
 error_set_stream:
-	media_pipeline_stop(&chan->video.entity);
+	media_pipeline_stop(chan->video.entity.pads);
 error_pipeline_start:
 	tegra_channel_release_buffers(chan, VB2_BUF_STATE_QUEUED);
 	return ret;
@@ -617,7 +617,7 @@ static void tegra210_vi_stop_streaming(struct vb2_queue *vq)
 
 	tegra_channel_release_buffers(chan, VB2_BUF_STATE_ERROR);
 	tegra_channel_set_stream(chan, false);
-	media_pipeline_stop(&chan->video.entity);
+	media_pipeline_stop(chan->video.entity.pads);
 }
 
 /*
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index b8f94662526c..d60ff8bf3f9e 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -932,53 +932,53 @@ struct media_pad *media_graph_walk_next(struct media_graph *graph);
 
 /**
  * media_pipeline_start - Mark a pipeline as streaming
- * @entity: Starting entity
- * @pipe: Media pipeline to be assigned to all entities in the pipeline.
+ * @pad: Starting pad
+ * @pipe: Media pipeline to be assigned to all pads in the pipeline.
  *
- * Mark all entities connected to a given entity through enabled links, either
- * directly or indirectly, as streaming. The given pipeline object is assigned
- * to every entity in the pipeline and stored in the media_entity pipe field.
+ * Mark all pads connected to a given pad through enabled routes or links,
+ * either directly or indirectly, as streaming. The given pipeline object is
+ * assigned to every pad in the pipeline and stored in the media_pad pipe
+ * field.
  *
  * Calls to this function can be nested, in which case the same number of
  * media_pipeline_stop() calls will be required to stop streaming. The
  * pipeline pointer must be identical for all nested calls to
  * media_pipeline_start().
  */
-__must_check int media_pipeline_start(struct media_entity *entity,
+__must_check int media_pipeline_start(struct media_pad *pad,
 				      struct media_pipeline *pipe);
 /**
  * __media_pipeline_start - Mark a pipeline as streaming
  *
- * @entity: Starting entity
- * @pipe: Media pipeline to be assigned to all entities in the pipeline.
+ * @pad: Starting pad
+ * @pipe: Media pipeline to be assigned to all pads in the pipeline.
  *
  * ..note:: This is the non-locking version of media_pipeline_start()
  */
-__must_check int __media_pipeline_start(struct media_entity *entity,
+__must_check int __media_pipeline_start(struct media_pad *pad,
 					struct media_pipeline *pipe);
 
 /**
  * media_pipeline_stop - Mark a pipeline as not streaming
- * @entity: Starting entity
+ * @pad: Starting pad
  *
- * Mark all entities connected to a given entity through enabled links, either
- * directly or indirectly, as not streaming. The media_entity pipe field is
- * reset to %NULL.
+ * Mark all pads connected to a given pad through enabled routes or links,
+ * either directly or indirectly, as not streaming.
  *
  * If multiple calls to media_pipeline_start() have been made, the same
  * number of calls to this function are required to mark the pipeline as not
- * streaming.
+ * streaming and reset the media_pad pipe field to %NULL.
  */
-void media_pipeline_stop(struct media_entity *entity);
+void media_pipeline_stop(struct media_pad *pad);
 
 /**
  * __media_pipeline_stop - Mark a pipeline as not streaming
  *
- * @entity: Starting entity
+ * @pad: Starting pad
  *
  * .. note:: This is the non-locking version of media_pipeline_stop()
  */
-void __media_pipeline_stop(struct media_entity *entity);
+void __media_pipeline_stop(struct media_pad *pad);
 
 /**
  * media_devnode_create() - creates and initializes a device node interface
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 08/24] media: entity: Add has_route entity operation
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (6 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 07/24] media: entity: Use pad as the starting point for a pipeline Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-15 13:04 ` [PATCH v5 09/24] media: entity: Add media_entity_has_route() function Tomi Valkeinen
                   ` (17 subsequent siblings)
  25 siblings, 0 replies; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil, Michal Simek

From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

The optional operation can be used by entities to report whether two
pads are internally connected.

While at there, fix a Sphinx compiler warning in a comment block a few
lines above.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 include/media/media-entity.h | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index d60ff8bf3f9e..466e4357a082 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -187,6 +187,7 @@ enum media_pad_signal_type {
  * @flags:	Pad flags, as defined in
  *		:ref:`include/uapi/linux/media.h <media_header>`
  *		(seek for ``MEDIA_PAD_FL_*``)
+ *
  * .. note::
  *
  *    @stream_count reference count must never be negative, but is a signed
@@ -214,6 +215,10 @@ struct media_pad {
  * @link_validate:	Return whether a link is valid from the entity point of
  *			view. The media_pipeline_start() function
  *			validates all links by calling this operation. Optional.
+ * @has_route:		Return whether a route exists inside the entity between
+ *			two given pads. Pads are passed to the operation ordered
+ *			by index. Optional: If the operation isn't implemented
+ *			all pads will be considered as connected.
  *
  * .. note::
  *
@@ -227,6 +232,8 @@ struct media_entity_operations {
 			  const struct media_pad *local,
 			  const struct media_pad *remote, u32 flags);
 	int (*link_validate)(struct media_link *link);
+	bool (*has_route)(struct media_entity *entity, unsigned int pad0,
+			  unsigned int pad1);
 };
 
 /**
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 09/24] media: entity: Add media_entity_has_route() function
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (7 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 08/24] media: entity: Add has_route entity operation Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-15 13:04 ` [PATCH v5 10/24] media: entity: Use routing information during graph traversal Tomi Valkeinen
                   ` (16 subsequent siblings)
  25 siblings, 0 replies; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil, Michal Simek

From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

This is a wrapper around the media entity has_route operation.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/mc/mc-entity.c | 19 +++++++++++++++++++
 include/media/media-entity.h | 17 +++++++++++++++++
 2 files changed, 36 insertions(+)

diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 45c3d45bc30c..9a3587f25894 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -229,6 +229,25 @@ EXPORT_SYMBOL_GPL(media_entity_pads_init);
  * Graph traversal
  */
 
+bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
+			    unsigned int pad1)
+{
+	if (pad0 >= entity->num_pads || pad1 >= entity->num_pads)
+		return false;
+
+	if (pad0 == pad1)
+		return true;
+
+	if (!entity->ops || !entity->ops->has_route)
+		return true;
+
+	if (entity->pads[pad1].index < entity->pads[pad0].index)
+		swap(pad0, pad1);
+
+	return entity->ops->has_route(entity, pad0, pad1);
+}
+EXPORT_SYMBOL_GPL(media_entity_has_route);
+
 static struct media_pad *
 media_pad_other(struct media_pad *pad, struct media_link *link)
 {
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 466e4357a082..73de1c335e4e 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -899,6 +899,23 @@ int media_entity_get_fwnode_pad(struct media_entity *entity,
 __must_check int media_graph_walk_init(
 	struct media_graph *graph, struct media_device *mdev);
 
+/**
+ * media_entity_has_route - Check if two entity pads are connected internally
+ *
+ * @entity: The entity
+ * @pad0: The first pad index
+ * @pad1: The second pad index
+ *
+ * This function can be used to check whether two pads of an entity are
+ * connected internally in the entity.
+ *
+ * The caller must hold entity->graph_obj.mdev->mutex.
+ *
+ * Return: true if the pads are connected internally and false otherwise.
+ */
+bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
+			    unsigned int pad1);
+
 /**
  * media_graph_walk_cleanup - Release resources used by graph walk.
  *
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 10/24] media: entity: Use routing information during graph traversal
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (8 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 09/24] media: entity: Add media_entity_has_route() function Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-15 13:04 ` [PATCH v5 11/24] media: entity: Skip link validation for pads to which there is no route to Tomi Valkeinen
                   ` (15 subsequent siblings)
  25 siblings, 0 replies; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil, Michal Simek

From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Take internal routing information as reported by the entity has_route
operation into account during graph traversal to avoid following
unrelated links.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/mc/mc-entity.c | 44 ++++++++++++++++++++++--------------
 1 file changed, 27 insertions(+), 17 deletions(-)

diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 9a3587f25894..28d7fd254c77 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -248,15 +248,6 @@ bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
 }
 EXPORT_SYMBOL_GPL(media_entity_has_route);
 
-static struct media_pad *
-media_pad_other(struct media_pad *pad, struct media_link *link)
-{
-	if (link->source == pad)
-		return link->sink;
-	else
-		return link->source;
-}
-
 /* push an entity to traversal stack */
 static void stack_push(struct media_graph *graph, struct media_pad *pad)
 {
@@ -327,7 +318,8 @@ static void media_graph_walk_iter(struct media_graph *graph)
 {
 	struct media_pad *pad = stack_top(graph);
 	struct media_link *link;
-	struct media_pad *next;
+	struct media_pad *remote;
+	struct media_pad *local;
 
 	link = list_entry(link_top(graph), typeof(*link), list);
 
@@ -341,23 +333,41 @@ static void media_graph_walk_iter(struct media_graph *graph)
 		return;
 	}
 
-	/* Get the entity in the other end of the link . */
-	next = media_pad_other(pad, link);
+	/*
+	 * Get the local pad, the remote pad and the entity at the other
+	 * end of the link.
+	 */
+	if (link->source->entity == pad->entity) {
+		remote = link->sink;
+		local = link->source;
+	} else {
+		remote = link->source;
+		local = link->sink;
+	}
+
+	/*
+	 * Are the local pad and the pad we came from connected
+	 * internally in the entity ?
+	 */
+	if (!media_entity_has_route(pad->entity, pad->index, local->index)) {
+		link_top(graph) = link_top(graph)->next;
+		return;
+	}
 
 	/* Has the entity already been visited? */
-	if (media_entity_enum_test_and_set(&graph->ent_enum, next->entity)) {
+	if (media_entity_enum_test_and_set(&graph->ent_enum, remote->entity)) {
 		link_top(graph) = link_top(graph)->next;
 		dev_dbg(pad->graph_obj.mdev->dev,
 			"walk: skipping entity '%s' (already seen)\n",
-			next->entity->name);
+			remote->entity->name);
 		return;
 	}
 
 	/* Push the new entity to stack and start over. */
 	link_top(graph) = link_top(graph)->next;
-	stack_push(graph, next);
-	dev_dbg(next->graph_obj.mdev->dev, "walk: pushing '%s':%u on stack\n",
-		next->entity->name, next->index);
+	stack_push(graph, remote);
+	dev_dbg(remote->graph_obj.mdev->dev, "walk: pushing '%s':%u on stack\n",
+		remote->entity->name, remote->index);
 }
 
 struct media_pad *media_graph_walk_next(struct media_graph *graph)
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 11/24] media: entity: Skip link validation for pads to which there is no route to
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (9 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 10/24] media: entity: Use routing information during graph traversal Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-18 18:06   ` Laurent Pinchart
  2021-04-15 13:04 ` [PATCH v5 12/24] media: entity: Add an iterator helper for connected pads Tomi Valkeinen
                   ` (14 subsequent siblings)
  25 siblings, 1 reply; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil

From: Sakari Ailus <sakari.ailus@linux.intel.com>

Links are validated along the pipeline which is about to start streaming.
Not all the pads in entities that are traversed along that pipeline are
part of the pipeline, however. Skip the link validation for such pads,
and while at there rename "other_pad" to "local_pad" to convey the fact
the route to be checked is internal to the entity.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/mc/mc-entity.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 28d7fd254c77..fe6cb743c85c 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -482,12 +482,17 @@ __must_check int __media_pipeline_start(struct media_pad *pad,
 		bitmap_fill(has_no_links, entity->num_pads);
 
 		list_for_each_entry(link, &entity->links, list) {
-			struct media_pad *other_pad =
+			struct media_pad *local_pad =
 				link->sink->entity == entity ?
 				link->sink : link->source;
 
+			/* Ignore pads to which there is no route. */
+			if (!media_entity_has_route(entity, pad->index,
+						    local_pad->index))
+				continue;
+
 			/* Mark that a pad is connected by a link. */
-			bitmap_clear(has_no_links, other_pad->index, 1);
+			bitmap_clear(has_no_links, local_pad->index, 1);
 
 			/*
 			 * Pads that either do not need to connect or
@@ -496,13 +501,13 @@ __must_check int __media_pipeline_start(struct media_pad *pad,
 			 */
 			if (!(pad->flags & MEDIA_PAD_FL_MUST_CONNECT) ||
 			    link->flags & MEDIA_LNK_FL_ENABLED)
-				bitmap_set(active, other_pad->index, 1);
+				bitmap_set(active, local_pad->index, 1);
 
 			/*
 			 * Link validation will only take place for
 			 * sink ends of the link that are enabled.
 			 */
-			if (link->sink != other_pad ||
+			if (link->sink != local_pad ||
 			    !(link->flags & MEDIA_LNK_FL_ENABLED))
 				continue;
 
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 12/24] media: entity: Add an iterator helper for connected pads
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (10 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 11/24] media: entity: Skip link validation for pads to which there is no route to Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-18 18:20   ` Laurent Pinchart
  2021-04-15 13:04 ` [PATCH v5 13/24] media: entity: Add only connected pads to the pipeline Tomi Valkeinen
                   ` (13 subsequent siblings)
  25 siblings, 1 reply; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil

From: Sakari Ailus <sakari.ailus@linux.intel.com>

Add a helper macro for iterating over pads that are connected through
enabled routes. This can be used to find all the connected pads within an
entity, for instance starting from the pad which has been obtained during
the graph walk.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

- Make __media_entity_next_routed_pad() return NULL and adjust the
  iterator to handle that
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 include/media/media-entity.h | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 73de1c335e4e..edd6f60ed6b4 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -916,6 +916,33 @@ __must_check int media_graph_walk_init(
 bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
 			    unsigned int pad1);
 
+static inline struct media_pad *__media_entity_next_routed_pad(
+	struct media_pad *start, struct media_pad *iter)
+{
+	struct media_entity *entity = start->entity;
+
+	for (; iter < &entity->pads[entity->num_pads]; iter++)
+		if (media_entity_has_route(entity, start->index, iter->index))
+			return iter;
+
+	return NULL;
+}
+
+/**
+ * media_entity_for_each_routed_pad - Iterate over entity pads connected by routes
+ *
+ * @start: The stating pad
+ * @iter: The iterator pad
+ *
+ * Iterate over all pads connected through routes from a given pad
+ * within an entity. The iteration will include the starting pad itself.
+ */
+#define media_entity_for_each_routed_pad(start, iter)			\
+	for (iter = __media_entity_next_routed_pad(			\
+		     start, (start)->entity->pads);			\
+	     iter != NULL;						\
+	     iter = __media_entity_next_routed_pad(start, iter + 1))
+
 /**
  * media_graph_walk_cleanup - Release resources used by graph walk.
  *
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 13/24] media: entity: Add only connected pads to the pipeline
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (11 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 12/24] media: entity: Add an iterator helper for connected pads Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-15 13:04 ` [PATCH v5 14/24] media: entity: Add debug information in graph walk route check Tomi Valkeinen
                   ` (12 subsequent siblings)
  25 siblings, 0 replies; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil

From: Sakari Ailus <sakari.ailus@linux.intel.com>

A single entity may contain multiple pipelines. Only add pads that were
connected to the pad through which the entity was reached to the pipeline.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/mc/mc-entity.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index fe6cb743c85c..40e5544552c0 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -457,7 +457,7 @@ __must_check int __media_pipeline_start(struct media_pad *pad,
 		DECLARE_BITMAP(active, MEDIA_ENTITY_MAX_PADS);
 		DECLARE_BITMAP(has_no_links, MEDIA_ENTITY_MAX_PADS);
 
-		media_entity_for_each_pad(entity, iter) {
+		media_entity_for_each_routed_pad(pad, iter) {
 			if (iter->pipe && iter->pipe != pipe) {
 				pr_err("Pipe active for %s. Can't start for %s\n",
 				       entity->name, iter->entity->name);
@@ -546,10 +546,9 @@ __must_check int __media_pipeline_start(struct media_pad *pad,
 	media_graph_walk_start(graph, pad_err);
 
 	while ((pad_err = media_graph_walk_next(graph))) {
-		struct media_entity *entity = pad_err->entity;
 		struct media_pad *iter;
 
-		media_entity_for_each_pad(entity, iter) {
+		media_entity_for_each_routed_pad(pad_err, iter) {
 			/* Sanity check for negative stream_count */
 			if (!WARN_ON_ONCE(iter->stream_count <= 0)) {
 				--iter->stream_count;
@@ -602,10 +601,9 @@ void __media_pipeline_stop(struct media_pad *pad)
 	media_graph_walk_start(graph, pad);
 
 	while ((pad = media_graph_walk_next(graph))) {
-		struct media_entity *entity = pad->entity;
 		struct media_pad *iter;
 
-		media_entity_for_each_pad(entity, iter) {
+		media_entity_for_each_routed_pad(pad, iter) {
 			/* Sanity check for negative stream_count */
 			if (!WARN_ON_ONCE(iter->stream_count <= 0)) {
 				iter->stream_count--;
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 14/24] media: entity: Add debug information in graph walk route check
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (12 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 13/24] media: entity: Add only connected pads to the pipeline Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-15 13:04 ` [PATCH v5 15/24] v4l: Add bus type to frame descriptors Tomi Valkeinen
                   ` (11 subsequent siblings)
  25 siblings, 0 replies; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil

From: Sakari Ailus <sakari.ailus@linux.intel.com>

Add debug printout in graph walk route check.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 drivers/media/mc/mc-entity.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 40e5544552c0..484a18333231 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -351,6 +351,9 @@ static void media_graph_walk_iter(struct media_graph *graph)
 	 */
 	if (!media_entity_has_route(pad->entity, pad->index, local->index)) {
 		link_top(graph) = link_top(graph)->next;
+		dev_dbg(pad->graph_obj.mdev->dev,
+			"walk: skipping \"%s\":%u -> %u (no route)\n",
+			pad->entity->name, pad->index, local->index);
 		return;
 	}
 
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 15/24] v4l: Add bus type to frame descriptors
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (13 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 14/24] media: entity: Add debug information in graph walk route check Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-18 19:23   ` Laurent Pinchart
  2021-04-15 13:04 ` [PATCH v5 16/24] v4l: Add CSI-2 bus configuration " Tomi Valkeinen
                   ` (10 subsequent siblings)
  25 siblings, 1 reply; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil

From: Sakari Ailus <sakari.ailus@linux.intel.com>

Add the media bus type to the frame descriptor. CSI-2 specific
information will be added in next patch to the frame descriptor.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

- Make the bus type a named enum
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 include/media/v4l2-subdev.h | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index d0e9a5bdb08b..85977abbea46 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -340,12 +340,21 @@ struct v4l2_mbus_frame_desc_entry {
 
 #define V4L2_FRAME_DESC_ENTRY_MAX	4
 
+enum v4l2_mbus_frame_desc_type {
+	V4L2_MBUS_FRAME_DESC_TYPE_PLATFORM,
+	V4L2_MBUS_FRAME_DESC_TYPE_PARALLEL,
+	V4L2_MBUS_FRAME_DESC_TYPE_CCP2,
+	V4L2_MBUS_FRAME_DESC_TYPE_CSI2,
+};
+
 /**
  * struct v4l2_mbus_frame_desc - media bus data frame description
+ * @type: type of the bus (enum v4l2_mbus_frame_desc_type)
  * @entry: frame descriptors array
  * @num_entries: number of entries in @entry array
  */
 struct v4l2_mbus_frame_desc {
+	enum v4l2_mbus_frame_desc_type type;
 	struct v4l2_mbus_frame_desc_entry entry[V4L2_FRAME_DESC_ENTRY_MAX];
 	unsigned short num_entries;
 };
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 16/24] v4l: Add CSI-2 bus configuration to frame descriptors
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (14 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 15/24] v4l: Add bus type to frame descriptors Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-18 19:24   ` Laurent Pinchart
  2021-04-15 13:04 ` [PATCH v5 17/24] v4l: Add stream to frame descriptor Tomi Valkeinen
                   ` (9 subsequent siblings)
  25 siblings, 1 reply; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil

From: Sakari Ailus <sakari.ailus@linux.intel.com>

Add CSI-2 bus specific configuration to the frame descriptors. This allows
obtaining the virtual channel and data type information for each stream
the transmitter is sending.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 include/media/v4l2-subdev.h | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 85977abbea46..30ec011d31e3 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -308,6 +308,17 @@ struct v4l2_subdev_audio_ops {
 	int (*s_stream)(struct v4l2_subdev *sd, int enable);
 };
 
+/**
+ * struct v4l2_mbus_frame_desc_entry_csi2
+ *
+ * @channel: CSI-2 virtual channel
+ * @data_type: CSI-2 data type ID
+ */
+struct v4l2_mbus_frame_desc_entry_csi2 {
+	u8 channel;
+	u8 data_type;
+};
+
 /**
  * enum v4l2_mbus_frame_desc_flags - media bus frame description flags
  *
@@ -331,11 +342,16 @@ enum v4l2_mbus_frame_desc_flags {
  *		%FRAME_DESC_FL_BLOB is not set.
  * @length:	number of octets per frame, valid if @flags
  *		%V4L2_MBUS_FRAME_DESC_FL_LEN_MAX is set.
+ * @bus:	Bus specific frame descriptor parameters
+ * @bus.csi2:	CSI-2 specific bus configuration
  */
 struct v4l2_mbus_frame_desc_entry {
 	enum v4l2_mbus_frame_desc_flags flags;
 	u32 pixelcode;
 	u32 length;
+	union {
+		struct v4l2_mbus_frame_desc_entry_csi2 csi2;
+	} bus;
 };
 
 #define V4L2_FRAME_DESC_ENTRY_MAX	4
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 17/24] v4l: Add stream to frame descriptor
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (15 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 16/24] v4l: Add CSI-2 bus configuration " Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-18 19:27   ` Laurent Pinchart
  2021-04-15 13:04 ` [PATCH v5 18/24] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations Tomi Valkeinen
                   ` (8 subsequent siblings)
  25 siblings, 1 reply; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil

From: Sakari Ailus <sakari.ailus@linux.intel.com>

The stream field identifies the stream this frame descriptor applies to in
routing configuration across a multiplexed link.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 include/media/v4l2-subdev.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 30ec011d31e3..436d0445aafd 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -338,6 +338,7 @@ enum v4l2_mbus_frame_desc_flags {
  * struct v4l2_mbus_frame_desc_entry - media bus frame description structure
  *
  * @flags:	bitmask flags, as defined by &enum v4l2_mbus_frame_desc_flags.
+ * @stream:	stream in routing configuration
  * @pixelcode:	media bus pixel code, valid if @flags
  *		%FRAME_DESC_FL_BLOB is not set.
  * @length:	number of octets per frame, valid if @flags
@@ -347,6 +348,7 @@ enum v4l2_mbus_frame_desc_flags {
  */
 struct v4l2_mbus_frame_desc_entry {
 	enum v4l2_mbus_frame_desc_flags flags;
+	u32 stream;
 	u32 pixelcode;
 	u32 length;
 	union {
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 18/24] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (16 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 17/24] v4l: Add stream to frame descriptor Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-18 18:32   ` Laurent Pinchart
  2021-04-20 16:35   ` Sakari Ailus
  2021-04-15 13:04 ` [PATCH v5 19/24] media: Documentation: Add GS_ROUTING documentation Tomi Valkeinen
                   ` (7 subsequent siblings)
  25 siblings, 2 replies; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil, Michal Simek, Tomi Valkeinen

From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Add support for subdev internal routing. A route is defined as a single
stream from a sink pad to a source pad.

The userspace can configure the routing via two new ioctls,
VIDIOC_SUBDEV_G_ROUTING and VIDIOC_SUBDEV_S_ROUTING, and subdevs can
implement the functionality with v4l2_subdev_pad_ops.get_routing() and
v4l2_subdev_pad_ops.set_routing().

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>

- Add sink and source streams for multiplexed links
- Copy the argument back in case of an error. This is needed to let the
  caller know the number of routes.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

- Expand and refine documentation.
- Make the 'routes' pointer a __u64 __user pointer so that a compat32
  version of the ioctl is not required.
- Add struct v4l2_subdev_krouting to be used for subdevice operations.

Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>

- Fix typecasing warnings
- Check sink & source pad types
- Add 'which' field

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/v4l2-core/v4l2-ioctl.c  | 25 ++++++++++++++-
 drivers/media/v4l2-core/v4l2-subdev.c | 45 +++++++++++++++++++++++++++
 include/media/v4l2-subdev.h           | 24 ++++++++++++++
 include/uapi/linux/v4l2-subdev.h      | 44 ++++++++++++++++++++++++++
 4 files changed, 137 insertions(+), 1 deletion(-)

diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 6a5d1c6d11d6..f5732962753f 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -16,6 +16,7 @@
 #include <linux/kernel.h>
 #include <linux/version.h>
 
+#include <linux/v4l2-subdev.h>
 #include <linux/videodev2.h>
 
 #include <media/v4l2-common.h>
@@ -3108,6 +3109,21 @@ static int check_array_args(unsigned int cmd, void *parg, size_t *array_size,
 		ret = 1;
 		break;
 	}
+
+	case VIDIOC_SUBDEV_G_ROUTING:
+	case VIDIOC_SUBDEV_S_ROUTING: {
+		struct v4l2_subdev_routing *route = parg;
+
+		if (route->num_routes > 256)
+			return -EINVAL;
+
+		*user_ptr = u64_to_user_ptr(route->routes);
+		*kernel_ptr = (void **)&route->routes;
+		*array_size = sizeof(struct v4l2_subdev_route)
+			    * route->num_routes;
+		ret = 1;
+		break;
+	}
 	}
 
 	return ret;
@@ -3369,8 +3385,15 @@ video_usercopy(struct file *file, unsigned int orig_cmd, unsigned long arg,
 	/*
 	 * Some ioctls can return an error, but still have valid
 	 * results that must be returned.
+	 *
+	 * FIXME: subdev IOCTLS are partially handled here and partially in
+	 * v4l2-subdev.c and the 'always_copy' flag can only be set for IOCTLS
+	 * defined here as part of the 'v4l2_ioctls' array. As
+	 * VIDIOC_SUBDEV_G_ROUTING needs to return results to applications even
+	 * in case of failure, but it is not defined here as part of the
+	 * 'v4l2_ioctls' array, insert an ad-hoc check to address that.
 	 */
-	if (err < 0 && !always_copy)
+	if (err < 0 && !always_copy && cmd != VIDIOC_SUBDEV_G_ROUTING)
 		goto out;
 
 out_array_args:
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 956dafab43d4..95a4c3091fa6 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -681,6 +681,51 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg)
 	case VIDIOC_SUBDEV_QUERYSTD:
 		return v4l2_subdev_call(sd, video, querystd, arg);
 
+	case VIDIOC_SUBDEV_G_ROUTING: {
+		struct v4l2_subdev_routing *routing = arg;
+		struct v4l2_subdev_krouting krouting = {
+			.which = routing->which,
+			.num_routes = routing->num_routes,
+			.routes = (struct v4l2_subdev_route *)(uintptr_t)routing->routes,
+		};
+		int ret;
+
+		ret = v4l2_subdev_call(sd, pad, get_routing, &krouting);
+		if (ret)
+			return ret;
+
+		routing->num_routes = krouting.num_routes;
+
+		return 0;
+	}
+
+	case VIDIOC_SUBDEV_S_ROUTING: {
+		struct v4l2_subdev_routing *routing = arg;
+		struct v4l2_subdev_route *route = (struct v4l2_subdev_route *)(uintptr_t)
+						  routing->routes;
+		struct v4l2_subdev_krouting krouting = {};
+		unsigned int i;
+
+		if (routing->which != V4L2_SUBDEV_FORMAT_TRY && ro_subdev)
+			return -EPERM;
+
+		for (i = 0; i < routing->num_routes; ++i) {
+			if (route[i].sink_pad >= sd->entity.num_pads ||
+			    route[i].source_pad >= sd->entity.num_pads)
+				return -EINVAL;
+
+			if (!(sd->entity.pads[route[i].sink_pad].flags & MEDIA_PAD_FL_SINK) ||
+			    !(sd->entity.pads[route[i].source_pad].flags & MEDIA_PAD_FL_SOURCE))
+				return -EINVAL;
+		}
+
+		krouting.which = routing->which;
+		krouting.num_routes = routing->num_routes;
+		krouting.routes = route;
+
+		return v4l2_subdev_call(sd, pad, set_routing, &krouting);
+	}
+
 	default:
 		return v4l2_subdev_call(sd, core, ioctl, cmd, arg);
 	}
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 436d0445aafd..3826ab918731 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -650,6 +650,22 @@ struct v4l2_subdev_pad_config {
 	struct v4l2_rect try_compose;
 };
 
+/**
+ * struct v4l2_subdev_krouting - subdev routing table
+ *
+ * @which: format type (from enum v4l2_subdev_format_whence)
+ * @routes: &struct v4l2_subdev_route
+ * @num_routes: number of routes
+ *
+ * This structure is used to translate argument received from
+ * VIDIOC_SUBDEV_G/S_ROUTING() ioctl to sudev device drivers operations.
+ */
+struct v4l2_subdev_krouting {
+	u32 which;
+	struct v4l2_subdev_route *routes;
+	unsigned int num_routes;
+};
+
 /**
  * struct v4l2_subdev_pad_ops - v4l2-subdev pad level operations
  *
@@ -711,6 +727,10 @@ struct v4l2_subdev_pad_config {
  *		     applied to the hardware. The operation shall fail if the
  *		     pad index it has been called on is not valid or in case of
  *		     unrecoverable failures.
+ *
+ * @get_routing: get the subdevice routing table.
+ * @set_routing: enable or disable data connection routes described in the
+ *		 subdevice routing table.
  */
 struct v4l2_subdev_pad_ops {
 	int (*init_cfg)(struct v4l2_subdev *sd,
@@ -755,6 +775,10 @@ struct v4l2_subdev_pad_ops {
 			       struct v4l2_mbus_config *config);
 	int (*set_mbus_config)(struct v4l2_subdev *sd, unsigned int pad,
 			       struct v4l2_mbus_config *config);
+	int (*get_routing)(struct v4l2_subdev *sd,
+			   struct v4l2_subdev_krouting *route);
+	int (*set_routing)(struct v4l2_subdev *sd,
+			   struct v4l2_subdev_krouting *route);
 };
 
 /**
diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h
index 658106f5b5dc..f2a17cbd1e9a 100644
--- a/include/uapi/linux/v4l2-subdev.h
+++ b/include/uapi/linux/v4l2-subdev.h
@@ -188,6 +188,48 @@ struct v4l2_subdev_capability {
 /* The v4l2 sub-device video device node is registered in read-only mode. */
 #define V4L2_SUBDEV_CAP_RO_SUBDEV		0x00000001
 
+#define V4L2_SUBDEV_ROUTE_FL_ACTIVE		BIT(0)
+#define V4L2_SUBDEV_ROUTE_FL_IMMUTABLE		BIT(1)
+
+/**
+ * struct v4l2_subdev_route - A route inside a subdev
+ *
+ * @sink_pad: the sink pad index
+ * @sink_stream: the sink stream identifier
+ * @source_pad: the source pad index
+ * @source_stream: the source stream identifier
+ * @flags: route flags:
+ *
+ *	V4L2_SUBDEV_ROUTE_FL_ACTIVE: Is the route active? An active
+ *	route will start when streaming is enabled on a video node.
+ *	Set by the user.
+ *
+ *	V4L2_SUBDEV_ROUTE_FL_IMMUTABLE: Is the route immutable, i.e.
+ *	can it be activated and inactivated? Set by the driver.
+ */
+struct v4l2_subdev_route {
+	__u32 sink_pad;
+	__u32 sink_stream;
+	__u32 source_pad;
+	__u32 source_stream;
+	__u32 flags;
+	__u32 reserved[5];
+};
+
+/**
+ * struct v4l2_subdev_routing - Subdev routing information
+ *
+ * @which: format type (from enum v4l2_subdev_format_whence)
+ * @routes: pointer to the routes array
+ * @num_routes: the total number of routes in the routes array
+ */
+struct v4l2_subdev_routing {
+	__u32 which;
+	__u64 routes;
+	__u32 num_routes;
+	__u32 reserved[5];
+};
+
 /* Backwards compatibility define --- to be removed */
 #define v4l2_subdev_edid v4l2_edid
 
@@ -215,5 +257,7 @@ struct v4l2_subdev_capability {
 #define VIDIOC_SUBDEV_ENUM_DV_TIMINGS		_IOWR('V', 98, struct v4l2_enum_dv_timings)
 #define VIDIOC_SUBDEV_QUERY_DV_TIMINGS		_IOR('V', 99, struct v4l2_dv_timings)
 #define VIDIOC_SUBDEV_DV_TIMINGS_CAP		_IOWR('V', 100, struct v4l2_dv_timings_cap)
+#define VIDIOC_SUBDEV_G_ROUTING			_IOWR('V', 38, struct v4l2_subdev_routing)
+#define VIDIOC_SUBDEV_S_ROUTING			_IOWR('V', 39, struct v4l2_subdev_routing)
 
 #endif
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 19/24] media: Documentation: Add GS_ROUTING documentation
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (17 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 18/24] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-15 13:04 ` [PATCH v5 20/24] v4l: mc: Add an S_ROUTING helper function for power state changes Tomi Valkeinen
                   ` (6 subsequent siblings)
  25 siblings, 0 replies; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil

From: Jacopo Mondi <jacopo+renesas@jmondi.org>

Add documentation for VIDIOC_SUBDEV_G/S_ROUTING ioctl and add
description of multiplexed media pads and internal routing to the
V4L2-subdev documentation section.

Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
 .../userspace-api/media/v4l/dev-subdev.rst    |  92 +++++++++++
 .../userspace-api/media/v4l/user-func.rst     |   1 +
 .../media/v4l/vidioc-subdev-g-routing.rst     | 143 ++++++++++++++++++
 3 files changed, 236 insertions(+)
 create mode 100644 Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst

diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
index fd1de0a73a9f..7fd5d5d1350b 100644
--- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
+++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
@@ -29,6 +29,8 @@ will feature a character device node on which ioctls can be called to
 
 -  negotiate image formats on individual pads
 
+-  inspect and modify internal data routing between pads of the same entity
+
 Sub-device character device nodes, conventionally named
 ``/dev/v4l-subdev*``, use major number 81.
 
@@ -501,3 +503,93 @@ source pads.
     :maxdepth: 1
 
     subdev-formats
+
+
+Multiplexed media pads and internal routing
+-------------------------------------------
+
+Subdevice drivers may expose the internal connections between media pads of an
+entity by exposing a routing table that applications can inspect and manipulate
+to change the internal routing between sink and source pads' data connection
+endpoints. A routing table is described by a struct
+:c:type:`v4l2_subdev_routing`, which contains ``num_routes`` route entries, each
+one represented by a struct :c:type:`v4l2_subdev_route`.
+
+Data routes do not just connect one pad to another in an entity, but they refer
+instead to the ``streams`` a media pad provides. Streams are data connection
+endpoints in a media pad. Multiplexed media pads expose multiple ``streams``
+which represent, when the underlying hardware technology allows that, logical
+data flows transported over a single physical media bus.
+
+A noteworthy example of logical stream multiplexing techniques is represented
+by the data interleaving mechanism implemented by mean of Virtual Channels as
+defined by the MIPI CSI-2 media bus specifications. A subdevice that implements
+support for Virtual Channel data interleaving might expose up to 4 data
+``streams``, one for each available Virtual Channel, on the source media pad
+that represents a CSI-2 connection endpoint.
+
+Routes are defined as potential data connections between a ``(sink_pad,
+sink_stream)`` pair and a ``(source_pad, source_stream)`` one, where
+``sink_pad`` and ``source_pad`` are the indexes of two media pads part of the
+same media entity, and ``sink_stream`` and ``source_stream`` are the identifiers
+of the data streams to be connected in the media pads. Media pads that do not
+support data multiplexing expose a single stream, usually with identifier 0.
+
+Routes are reported to applications in a routing table which can be
+inspected and manipulated using the :ref:`routing <VIDIOC_SUBDEV_G_ROUTING>`
+ioctls.
+
+Routes can be activated and deactivated by setting or clearing the
+``V4L2_SUBDEV_ROUTE_FL_ACTIVE`` flag in the ``flags`` field of struct
+:c:type:`v4l2_subdev_route`.
+
+A subdev driver may create routes that cannot be modified by applications.
+Such routes are identified by the presence of the
+``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE`` flag in the ``flags`` field of struct
+:c:type:`v4l2_subdev_route`.
+
+As an example, the routing table of a subdevice that has two sink pads and can
+combine their streams on a single source pad as two logical streams is here
+below described.
+
+.. flat-table::
+    :header-rows:  1
+
+    * - Pad Index
+      - Function
+      - Number of streams
+    * - 0
+      - SINK
+      - 1
+    * - 1
+      - SINK
+      - 1
+    * - 2
+      - SOURCE
+      - 2
+
+In such an example, the source media pad will report a routing table with 4
+entries, one entry for each possible ``(sink_pad, sink_stream) - (source_pad,
+source_stream)`` combination.
+
+.. flat-table:: routing table
+    :header-rows:  1
+
+    * - Sink Pad/Sink Stream
+      - ->
+      - Source Pad/Source Stream
+    * - 0/0
+      - ->
+      - 2/0
+    * - 0/0
+      - ->
+      - 2/1
+    * - 1/0
+      - ->
+      - 2/0
+    * - 1/0
+      - ->
+      - 2/1
+
+Subdev drivers are free to decide how many routes an application can enable on
+a media pad at the same time, and refuse to enable or disable specific routes.
diff --git a/Documentation/userspace-api/media/v4l/user-func.rst b/Documentation/userspace-api/media/v4l/user-func.rst
index 53e604bd7d60..228c1521f190 100644
--- a/Documentation/userspace-api/media/v4l/user-func.rst
+++ b/Documentation/userspace-api/media/v4l/user-func.rst
@@ -70,6 +70,7 @@ Function Reference
     vidioc-subdev-g-crop
     vidioc-subdev-g-fmt
     vidioc-subdev-g-frame-interval
+    vidioc-subdev-g-routing
     vidioc-subdev-g-selection
     vidioc-subdev-querycap
     vidioc-subscribe-event
diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
new file mode 100644
index 000000000000..993847be2446
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
@@ -0,0 +1,143 @@
+.. Permission is granted to copy, distribute and/or modify this
+.. document under the terms of the GNU Free Documentation License,
+.. Version 1.1 or any later version published by the Free Software
+.. Foundation, with no Invariant Sections, no Front-Cover Texts
+.. and no Back-Cover Texts. A copy of the license is included at
+.. Documentation/media/uapi/fdl-appendix.rst.
+..
+.. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections
+
+.. _VIDIOC_SUBDEV_G_ROUTING:
+
+******************************************************
+ioctl VIDIOC_SUBDEV_G_ROUTING, VIDIOC_SUBDEV_S_ROUTING
+******************************************************
+
+Name
+====
+
+VIDIOC_SUBDEV_G_ROUTING - VIDIOC_SUBDEV_S_ROUTING - Get or set routing between streams of media pads in a media entity.
+
+
+Synopsis
+========
+
+.. c:function:: int ioctl( int fd, VIDIOC_SUBDEV_G_ROUTING, struct v4l2_subdev_routing *argp )
+    :name: VIDIOC_SUBDEV_G_ROUTING
+
+.. c:function:: int ioctl( int fd, VIDIOC_SUBDEV_S_ROUTING, struct v4l2_subdev_routing *argp )
+    :name: VIDIOC_SUBDEV_S_ROUTING
+
+
+Arguments
+=========
+
+``fd``
+    File descriptor returned by :ref:`open() <func-open>`.
+
+``argp``
+    Pointer to struct :c:type:`v4l2_subdev_routing`.
+
+
+Description
+===========
+
+These ioctls are used to get and set the routing in a media entity.
+The routing configuration determines the flows of data inside an entity.
+
+Drivers report their routing tables using the ``VIDIOC_SUBDEV_G_ROUTING`` ioctl
+and application may enable or disable routes with the VIDIOC_SUBDEV_S_ROUTING
+ioctl, by setting or clearing the ``V4L2_SUBDEV_ROUTE_FL_ACTIVE`` flag of
+the  ``flags`` field of a struct :c:type:`v4l2_subdev_route`.
+
+When inspecting routes through VIDIOC_SUBDEV_G_ROUTING and the application
+provided ``num_routes`` is not big enough to contain all the available routes
+the subdevice exposes, drivers return the ENOSPC error code and adjust the
+value of the ``num_routes`` field. Application should then reserve enough memory
+for all the route entries and call VIDIOC_SUBDEV_G_ROUTING again.
+
+.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}|
+
+.. c:type:: v4l2_subdev_routing
+
+.. flat-table:: struct v4l2_subdev_routing
+    :header-rows:  0
+    :stub-columns: 0
+    :widths:       1 1 2
+
+    * - __u32
+      - ``which``
+      - Format to modified, from enum
+        :ref:`v4l2_subdev_format_whence <v4l2-subdev-format-whence>`.
+    * - struct :c:type:`v4l2_subdev_route`
+      - ``routes[]``
+      - Array of struct :c:type:`v4l2_subdev_route` entries
+    * - __u32
+      - ``num_routes``
+      - Number of entries of the routes array
+    * - __u32
+      - ``reserved``\ [5]
+      - Reserved for future extensions. Applications and drivers must set
+	the array to zero.
+
+.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}|
+
+.. c:type:: v4l2_subdev_route
+
+.. flat-table:: struct v4l2_subdev_route
+    :header-rows:  0
+    :stub-columns: 0
+    :widths:       1 1 2
+
+    * - __u32
+      - ``sink_pad``
+      - Sink pad number.
+    * - __u32
+      - ``sink_stream``
+      - Sink pad stream number.
+    * - __u32
+      - ``source_pad``
+      - Source pad number.
+    * - __u32
+      - ``source_stream``
+      - Source pad stream number.
+    * - __u32
+      - ``flags``
+      - Route enable/disable flags
+	:ref:`v4l2_subdev_routing_flags <v4l2-subdev-routing-flags>`.
+    * - __u32
+      - ``reserved``\ [5]
+      - Reserved for future extensions. Applications and drivers must set
+	the array to zero.
+
+.. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
+
+.. _v4l2-subdev-routing-flags:
+
+.. flat-table:: enum v4l2_subdev_routing_flags
+    :header-rows:  0
+    :stub-columns: 0
+    :widths:       3 1 4
+
+    * - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+      - 0
+      - The route is enabled. Set by applications.
+    * - V4L2_SUBDEV_ROUTE_FL_IMMUTABLE
+      - 1
+      - The route is immutable. Set by the driver.
+
+Return Value
+============
+
+On success 0 is returned, on error -1 and the ``errno`` variable is set
+appropriately. The generic error codes are described at the
+:ref:`Generic Error Codes <gen-errors>` chapter.
+
+ENOSPC
+   The number of provided route entries is less than the available ones.
+
+EINVAL
+   The sink or source pad identifiers reference a non-existing pad, or reference
+   pads of different types (ie. the sink_pad identifiers refers to a source pad)
+   or the sink or source stream identifiers reference a non-existing stream on
+   the sink or source pad.
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 20/24] v4l: mc: Add an S_ROUTING helper function for power state changes
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (18 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 19/24] media: Documentation: Add GS_ROUTING documentation Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-18 18:55   ` Laurent Pinchart
  2021-04-15 13:04 ` [PATCH v5 21/24] v4l: subdev: routing kernel helper functions Tomi Valkeinen
                   ` (5 subsequent siblings)
  25 siblings, 1 reply; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil

From: Sakari Ailus <sakari.ailus@linux.intel.com>

With the addition of the has_route() media entity operation, all pads of an
entity are no longer interconnected. The S_ROUTING IOCTL for sub-devices can
be used to enable and disable routes for an entity. The consequence is that
the routing information has to be taken into account in use count
calculation: disabling a route has a very similar effect on use counts as
has disabling a link.

Add a helper function for drivers implementing VIDIOC_SUBDEV_S_ROUTING
IOCTL to take the change into account.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/v4l2-core/v4l2-mc.c | 34 +++++++++++++++++++++++++++++++
 include/media/v4l2-mc.h           | 22 ++++++++++++++++++++
 2 files changed, 56 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
index 35d18ed89fa5..71acb389aa7b 100644
--- a/drivers/media/v4l2-core/v4l2-mc.c
+++ b/drivers/media/v4l2-core/v4l2-mc.c
@@ -588,3 +588,37 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 	return ret;
 }
 EXPORT_SYMBOL_GPL(v4l2_pipeline_link_notify);
+
+int v4l2_subdev_routing_pm_use(struct media_entity *entity,
+			       struct v4l2_subdev_route *route)
+{
+	struct media_graph *graph =
+		&entity->graph_obj.mdev->pm_count_walk;
+	struct media_pad *source = &entity->pads[route->source_pad];
+	struct media_pad *sink = &entity->pads[route->sink_pad];
+	int source_use;
+	int sink_use;
+	int ret;
+
+	source_use = pipeline_pm_use_count(source, graph);
+	sink_use = pipeline_pm_use_count(sink, graph);
+
+	if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE)) {
+		/* Route disabled. */
+		pipeline_pm_power(source, -sink_use, graph);
+		pipeline_pm_power(sink, -source_use, graph);
+		return 0;
+	}
+
+	/* Route enabled. */
+	ret = pipeline_pm_power(source, sink_use, graph);
+	if (ret < 0)
+		return ret;
+
+	ret = pipeline_pm_power(sink, source_use, graph);
+	if (ret < 0)
+		pipeline_pm_power(source, -sink_use, graph);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_routing_pm_use);
diff --git a/include/media/v4l2-mc.h b/include/media/v4l2-mc.h
index c181685923d5..ab8f4dc143aa 100644
--- a/include/media/v4l2-mc.h
+++ b/include/media/v4l2-mc.h
@@ -18,6 +18,7 @@
 /* We don't need to include pci.h or usb.h here */
 struct pci_dev;
 struct usb_device;
+struct v4l2_subdev_route;
 
 #ifdef CONFIG_MEDIA_CONTROLLER
 /**
@@ -184,6 +185,22 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);
 int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 			      unsigned int notification);
 
+/**
+ * v4l2_subdev_routing_pm_use - Handle power state changes due to S_ROUTING
+ * @entity: The entity
+ * @route: The new state of the route
+ *
+ * Propagate the use count across a route in a pipeline whenever the
+ * route is enabled or disabled. The function is called before
+ * changing the route state when enabling a route, and after changing
+ * the route state when disabling a route.
+ *
+ * Return 0 on success or a negative error code on failure. Powering entities
+ * off is assumed to never fail. This function will not fail for disconnection
+ * events.
+ */
+int v4l2_subdev_routing_pm_use(struct media_entity *entity,
+			       struct v4l2_subdev_route *route);
 #else /* CONFIG_MEDIA_CONTROLLER */
 
 static inline int v4l2_mc_create_media_graph(struct media_device *mdev)
@@ -219,5 +236,10 @@ static inline int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 	return 0;
 }
 
+static inline int v4l2_subdev_routing_pm_use(struct media_entity *entity,
+					     struct v4l2_subdev_route *route)
+{
+	return 0;
+}
 #endif /* CONFIG_MEDIA_CONTROLLER */
 #endif /* _V4L2_MC_H */
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 21/24] v4l: subdev: routing kernel helper functions
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (19 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 20/24] v4l: mc: Add an S_ROUTING helper function for power state changes Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-18 19:18   ` Laurent Pinchart
  2021-04-15 13:04 ` [PATCH v5 22/24] v4l: subdev: add v4l2_subdev_get_format_dir() Tomi Valkeinen
                   ` (4 subsequent siblings)
  25 siblings, 1 reply; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen

Add helper functions for routing.

TODO: add docs.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 89 +++++++++++++++++++++++++++
 include/media/v4l2-subdev.h           | 14 +++++
 2 files changed, 103 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 95a4c3091fa6..7a4f71d8c6c3 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -909,6 +909,95 @@ v4l2_subdev_link_validate_get_format(struct media_pad *pad,
 	return -EINVAL;
 }
 
+int v4l2_subdev_get_krouting(struct v4l2_subdev *sd,
+			     struct v4l2_subdev_krouting *routing)
+{
+	int ret;
+
+	routing->which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	routing->routes = NULL;
+	routing->num_routes = 0;
+
+	ret = v4l2_subdev_call(sd, pad, get_routing, routing);
+	if (ret == 0)
+		return 0;
+	if (ret != -ENOSPC)
+		return ret;
+
+	routing->routes = kvmalloc_array(routing->num_routes,
+					 sizeof(*routing->routes), GFP_KERNEL);
+	if (!routing->routes)
+		return -ENOMEM;
+
+	ret = v4l2_subdev_call(sd, pad, get_routing, routing);
+	if (ret) {
+		kvfree(routing->routes);
+		return ret;
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_get_krouting);
+
+void v4l2_subdev_free_routing(struct v4l2_subdev_krouting *routing)
+{
+	kvfree(routing->routes);
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_free_routing);
+
+void v4l2_subdev_cpy_routing(struct v4l2_subdev_krouting *dst,
+			     const struct v4l2_subdev_krouting *src)
+{
+	memcpy(dst->routes, src->routes,
+	       src->num_routes * sizeof(*src->routes));
+	dst->num_routes = src->num_routes;
+	dst->which = src->which;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_cpy_routing);
+
+int v4l2_subdev_dup_routing(struct v4l2_subdev_krouting *dst,
+			    const struct v4l2_subdev_krouting *src)
+{
+	if (dst->routes)
+		kvfree(dst->routes);
+
+	if (src->num_routes == 0) {
+		dst->which = src->which;
+		dst->routes = NULL;
+		dst->num_routes = 0;
+		return 0;
+	}
+
+	dst->routes = kvmalloc_array(src->num_routes, sizeof(*src->routes),
+				     GFP_KERNEL);
+	if (!dst->routes)
+		return -ENOMEM;
+
+	v4l2_subdev_cpy_routing(dst, src);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_dup_routing);
+
+bool v4l2_subdev_has_route(struct v4l2_subdev_krouting *routing,
+			   unsigned int pad0, unsigned int pad1)
+{
+	unsigned int i;
+
+	for (i = 0; i < routing->num_routes; ++i) {
+		struct v4l2_subdev_route *route = &routing->routes[i];
+
+		if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+			continue;
+
+		if (route->sink_pad == pad0 && route->source_pad == pad1)
+			return true;
+	}
+
+	return false;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_has_route);
+
 int v4l2_subdev_link_validate(struct media_link *link)
 {
 	struct v4l2_subdev *sink;
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 3826ab918731..1843b77dd843 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -1225,4 +1225,18 @@ extern const struct v4l2_subdev_ops v4l2_subdev_call_wrappers;
 void v4l2_subdev_notify_event(struct v4l2_subdev *sd,
 			      const struct v4l2_event *ev);
 
+int v4l2_subdev_get_krouting(struct v4l2_subdev *sd,
+			     struct v4l2_subdev_krouting *routing);
+
+void v4l2_subdev_free_routing(struct v4l2_subdev_krouting *routing);
+
+int v4l2_subdev_dup_routing(struct v4l2_subdev_krouting *dst,
+			    const struct v4l2_subdev_krouting *src);
+
+void v4l2_subdev_cpy_routing(struct v4l2_subdev_krouting *dst,
+			     const struct v4l2_subdev_krouting *src);
+
+bool v4l2_subdev_has_route(struct v4l2_subdev_krouting *routing,
+			   unsigned int pad0, unsigned int pad1);
+
 #endif
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 22/24] v4l: subdev: add v4l2_subdev_get_format_dir()
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (20 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 21/24] v4l: subdev: routing kernel helper functions Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-18 19:04   ` Laurent Pinchart
  2021-04-15 13:04 ` [PATCH v5 23/24] v4l: subdev: Take routing information into account in link validation Tomi Valkeinen
                   ` (3 subsequent siblings)
  25 siblings, 1 reply; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen

Add v4l2_subdev_get_format_dir() which can be used to find subdev format
for a specific stream on a multiplexed pad. The function will follow the
routes and links until it finds a non-multiplexed pad.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 96 +++++++++++++++++++++++++++
 include/media/v4l2-subdev.h           | 26 ++++++++
 2 files changed, 122 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 7a4f71d8c6c3..430dbdaab080 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -998,6 +998,102 @@ bool v4l2_subdev_has_route(struct v4l2_subdev_krouting *routing,
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_has_route);
 
+int v4l2_subdev_get_format_dir(struct media_pad *pad, u16 stream,
+			       enum v4l2_direction dir,
+			       struct v4l2_subdev_format *fmt)
+{
+	struct device *dev = pad->entity->graph_obj.mdev->dev;
+	int ret;
+	int i;
+
+	dev_dbg(dev, "%s '%s':%u:%u %s\n", __func__,
+		pad->entity->name, pad->index, stream,
+		dir == V4L2_DIR_SOURCEWARD ? "sourceward" : "sinkward");
+
+	while (true) {
+		struct v4l2_subdev_krouting routing;
+		struct v4l2_subdev_route *route;
+
+		if (pad->entity->obj_type != MEDIA_ENTITY_TYPE_V4L2_SUBDEV)
+			return -EINVAL;
+
+		ret = v4l2_subdev_link_validate_get_format(pad, fmt);
+		if (ret == 0)
+			return 0;
+		else if (ret != -ENOIOCTLCMD)
+			return ret;
+
+		if (pad->flags &
+		    (dir == V4L2_DIR_SINKWARD ? MEDIA_PAD_FL_SOURCE :
+						MEDIA_PAD_FL_SINK)) {
+			pad = media_entity_remote_pad(pad);
+
+			if (!pad)
+				return -EINVAL;
+
+			if (pad->entity->obj_type != MEDIA_ENTITY_TYPE_V4L2_SUBDEV)
+				return -EINVAL;
+
+			ret = v4l2_subdev_link_validate_get_format(pad, fmt);
+			if (ret == 0)
+				return 0;
+			else if (ret != -ENOIOCTLCMD)
+				return ret;
+		}
+
+		ret = v4l2_subdev_get_krouting(media_entity_to_v4l2_subdev(pad->entity), &routing);
+		if (ret)
+			return ret;
+
+		route = NULL;
+		for (i = 0; i < routing.num_routes; ++i) {
+			u16 near_pad = dir == V4L2_DIR_SINKWARD ?
+					       routing.routes[i].sink_pad :
+					       routing.routes[i].source_pad;
+			u16 near_stream = dir == V4L2_DIR_SINKWARD ?
+						  routing.routes[i].sink_stream :
+						  routing.routes[i].source_stream;
+
+			if (!(routing.routes[i].flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+				continue;
+
+			if (near_pad != pad->index)
+				continue;
+
+			if (near_stream != stream)
+				continue;
+
+			if (route) {
+				dev_err(dev,
+					"%s: '%s' has multiple active routes for stream %u\n",
+					__func__, pad->entity->name, stream);
+				v4l2_subdev_free_routing(&routing);
+				return -EINVAL;
+			}
+
+			route = &routing.routes[i];
+		}
+
+		if (!route) {
+			dev_err(dev, "%s: no route found in '%s' for stream %u\n",
+				__func__, pad->entity->name, stream);
+			v4l2_subdev_free_routing(&routing);
+			return -EINVAL;
+		}
+
+		if (dir == V4L2_DIR_SINKWARD) {
+			pad = &pad->entity->pads[route->source_pad];
+			stream = route->source_stream;
+		} else {
+			pad = &pad->entity->pads[route->sink_pad];
+			stream = route->sink_stream;
+		}
+
+		v4l2_subdev_free_routing(&routing);
+	}
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_get_format_dir);
+
 int v4l2_subdev_link_validate(struct media_link *link)
 {
 	struct v4l2_subdev *sink;
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 1843b77dd843..730631f9a091 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -1239,4 +1239,30 @@ void v4l2_subdev_cpy_routing(struct v4l2_subdev_krouting *dst,
 bool v4l2_subdev_has_route(struct v4l2_subdev_krouting *routing,
 			   unsigned int pad0, unsigned int pad1);
 
+/**
+ * enum v4l2_direction - Direction either towards the source or the sink
+ *
+ * @V4L2_DIR_SOURCEWARD: Direction towards the source.
+ * @V4L2_DIR_SINKWARD: Direction towards the sink.
+ */
+enum v4l2_direction {
+	V4L2_DIR_SOURCEWARD,
+	V4L2_DIR_SINKWARD,
+};
+
+/**
+ * v4l2_subdev_get_format_dir() - Find format by following streams
+ * @pad: The pad from which to start the search
+ * @stream: The stream for which we want to find the format
+ * @dir: The direction of the search
+ * @fmt: Pointer to &struct v4l2_subdev_format where the found format is stored
+ *
+ * This function attempts to find v4l2_subdev_format for a specific stream on a
+ * multiplexed pad by following the stream using routes and links to the specified
+ * direction, until a non-multiplexed pad is found.
+ */
+int v4l2_subdev_get_format_dir(struct media_pad *pad, u16 stream,
+			       enum v4l2_direction dir,
+			       struct v4l2_subdev_format *fmt);
+
 #endif
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 23/24] v4l: subdev: Take routing information into account in link validation
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (21 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 22/24] v4l: subdev: add v4l2_subdev_get_format_dir() Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-15 13:04 ` [PATCH v5 24/24] v4l: subdev: increase V4L2_FRAME_DESC_ENTRY_MAX to 8 Tomi Valkeinen
                   ` (2 subsequent siblings)
  25 siblings, 0 replies; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen

The routing information is essential in link validation for multiplexed
links: the pads at the ends of a multiplexed link have no single format
defined for them. Instead, the format is accessible in the sink (or
source) pads of the sub-devices at both ends of that link.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 138 ++++++++++++++++++++++++++
 1 file changed, 138 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 430dbdaab080..e0cb5b7d84a7 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -1094,6 +1094,140 @@ int v4l2_subdev_get_format_dir(struct media_pad *pad, u16 stream,
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_get_format_dir);
 
+static int v4l2_subdev_link_validate_routing_stream(
+	struct media_link *link, struct media_pad *sink_pad, u16 sink_stream,
+	struct media_pad *source_pad, u16 source_stream)
+{
+	struct v4l2_subdev_format source_fmt;
+	struct v4l2_subdev_format sink_fmt;
+	struct v4l2_subdev *sink_sd;
+	int ret;
+
+	ret = v4l2_subdev_get_format_dir(sink_pad, sink_stream,
+					 V4L2_DIR_SINKWARD, &sink_fmt);
+	if (ret)
+		return ret;
+
+	ret = v4l2_subdev_get_format_dir(source_pad, source_stream,
+					 V4L2_DIR_SOURCEWARD, &source_fmt);
+	if (ret)
+		return ret;
+
+	sink_sd = media_entity_to_v4l2_subdev(sink_pad->entity);
+
+	ret = v4l2_subdev_call(sink_sd, pad, link_validate, link, &source_fmt,
+			       &sink_fmt);
+	if (ret != -ENOIOCTLCMD)
+		return ret;
+
+	ret = v4l2_subdev_link_validate_default(sink_sd, link, &source_fmt,
+						&sink_fmt);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int v4l2_subdev_link_validate_routing(struct media_link *link)
+{
+	int ret;
+	unsigned int i, j;
+
+	struct route_info {
+		struct v4l2_subdev_krouting routing;
+		struct media_pad *pad;
+		struct v4l2_subdev *subdev;
+	};
+
+	struct route_info source_route_info = {
+		.pad = link->source,
+		.subdev = media_entity_to_v4l2_subdev(link->source->entity),
+	};
+
+	struct route_info sink_route_info = {
+		.pad = link->sink,
+		.subdev = media_entity_to_v4l2_subdev(link->sink->entity),
+	};
+
+	struct device *dev = sink_route_info.subdev->entity.graph_obj.mdev->dev;
+
+	dev_dbg(dev, "validating link \"%s\":%u -> \"%s\":%u\n",
+		link->source->entity->name, link->source->index,
+		link->sink->entity->name, link->sink->index);
+
+	ret = v4l2_subdev_get_krouting(source_route_info.subdev,
+				       &source_route_info.routing);
+	if (ret)
+		return ret;
+
+	ret = v4l2_subdev_get_krouting(sink_route_info.subdev,
+				       &sink_route_info.routing);
+	if (ret) {
+		v4l2_subdev_free_routing(&source_route_info.routing);
+		return ret;
+	}
+
+	/*
+	 * Every active sink route needs an active source route, but it's ok
+	 * to have active source routes without matching sink route.
+	 */
+	for (i = 0; i < sink_route_info.routing.num_routes; ++i) {
+		struct v4l2_subdev_route *sink_route =
+			&sink_route_info.routing.routes[i];
+
+		if (!(sink_route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+			continue;
+
+		if (sink_route->sink_pad != sink_route_info.pad->index)
+			continue;
+
+		for (j = 0; j < source_route_info.routing.num_routes; ++j) {
+			struct v4l2_subdev_route *source_route =
+				&source_route_info.routing.routes[j];
+
+			if (!(source_route->flags &
+			      V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+				continue;
+
+			if (source_route->source_pad !=
+			    source_route_info.pad->index)
+				continue;
+
+			if (source_route->source_stream !=
+			    sink_route->sink_stream)
+				continue;
+
+			ret = v4l2_subdev_link_validate_routing_stream(
+				link,
+				&sink_route_info.pad->entity
+					 ->pads[sink_route->sink_pad],
+				sink_route->sink_stream,
+				&source_route_info.pad->entity
+					 ->pads[source_route->source_pad],
+				source_route->source_stream);
+			if (ret)
+				goto out;
+
+			break;
+		}
+
+		if (j == source_route_info.routing.num_routes) {
+			dev_err(dev,
+				"%s: no active source route found for sink route '%s':%u:%u\n",
+				__func__, sink_route_info.pad->entity->name,
+				sink_route->sink_pad, sink_route->sink_stream);
+			ret = -EINVAL;
+			goto out;
+		}
+	}
+
+out:
+	v4l2_subdev_free_routing(&source_route_info.routing);
+	v4l2_subdev_free_routing(&sink_route_info.routing);
+
+	return ret;
+}
+
 int v4l2_subdev_link_validate(struct media_link *link)
 {
 	struct v4l2_subdev *sink;
@@ -1102,11 +1236,15 @@ int v4l2_subdev_link_validate(struct media_link *link)
 
 	rval = v4l2_subdev_link_validate_get_format(
 		link->source, &source_fmt);
+	if (rval == -ENOIOCTLCMD)
+		return v4l2_subdev_link_validate_routing(link);
 	if (rval < 0)
 		return 0;
 
 	rval = v4l2_subdev_link_validate_get_format(
 		link->sink, &sink_fmt);
+	if (rval == -ENOIOCTLCMD)
+		return v4l2_subdev_link_validate_routing(link);
 	if (rval < 0)
 		return 0;
 
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* [PATCH v5 24/24] v4l: subdev: increase V4L2_FRAME_DESC_ENTRY_MAX to 8
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (22 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 23/24] v4l: subdev: Take routing information into account in link validation Tomi Valkeinen
@ 2021-04-15 13:04 ` Tomi Valkeinen
  2021-04-18 19:06   ` Laurent Pinchart
  2021-04-16  8:38 ` [PATCH v5 00/24] v4l: subdev internal routing Niklas Söderlund
  2021-04-18 17:32 ` Laurent Pinchart
  25 siblings, 1 reply; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-15 13:04 UTC (permalink / raw)
  To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas
  Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen

V4L2_FRAME_DESC_ENTRY_MAX is currently set to 4. In theory it's possible
to have an arbitrary amount of streams in a single pad, so preferably
there should be no hardcoded maximum number.

However, I believe a reasonable max is 8, which would cover a CSI-2 pad
with 4 streams of pixel data and 4 streams of metadata.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 include/media/v4l2-subdev.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 730631f9a091..49969d3699cb 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -356,7 +356,7 @@ struct v4l2_mbus_frame_desc_entry {
 	} bus;
 };
 
-#define V4L2_FRAME_DESC_ENTRY_MAX	4
+#define V4L2_FRAME_DESC_ENTRY_MAX	8
 
 enum v4l2_mbus_frame_desc_type {
 	V4L2_MBUS_FRAME_DESC_TYPE_PLATFORM,
-- 
2.25.1


^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 00/24] v4l: subdev internal routing
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (23 preceding siblings ...)
  2021-04-15 13:04 ` [PATCH v5 24/24] v4l: subdev: increase V4L2_FRAME_DESC_ENTRY_MAX to 8 Tomi Valkeinen
@ 2021-04-16  8:38 ` Niklas Söderlund
  2021-04-16  8:47   ` Tomi Valkeinen
  2021-04-18 17:32 ` Laurent Pinchart
  25 siblings, 1 reply; 72+ messages in thread
From: Niklas Söderlund @ 2021-04-16  8:38 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	Mauro Carvalho Chehab, Hans Verkuil

Hi Tomi,

I'm very happy to see this being worked on again!

Is there code somewhere that demonstrates the v5 API in use? I still 
have old branches of this series and it would be nice to see how the API 
have evolved for drivers.

Likewise are there some user-space code around that can be used to test 
the API? For v2 and v3 I had some hack patches [1], do they still work?  
More likely they have gone stale by now :-)

1. git://git.ragnatech.se/v4l-utils routing

On 2021-04-15 16:04:26 +0300, Tomi Valkeinen wrote:
> Hi,
> 
> This is an RFC for subdev internal routing which is needed for
> multiplexed streams support. I believe this is essentially a v5 of the
> series, the v4 posted here:
> 
> https://lore.kernel.org/linux-media/20190328200608.9463-1-jacopo+renesas@jmondi.org/
> 
> Most of the patches are not changed (aside from fixing rebase issues
> etc). The major changes in this version are:
> 
> 1) Added 'which' field to the routing structs. It is currently not used,
> as implementing it is not trivial. However, I think it's good to add it
> to the uAPI now, and require the field to be set to
> V4L2_SUBDEV_FORMAT_ACTIVE for now. See this RFC for an idea how this
> could be implemented:
> 
> https://lore.kernel.org/linux-media/20210409133659.389544-1-tomi.valkeinen@ideasonboard.com/
> 
> 2) No hardcoded maximum number of routes. Defining a maximum is not
> possible, as there can be an arbitrary amount of routes per pad, and
> there can be an arbitrary amount of pads per subdev. This series
> allocates space for the routing table dynamically, which unfortunately
> leads to not-just-a-few allocs and frees.
> 
> 3) When searching for a format for a stream, the v4 looked for a
> non-multiplexed pad only as far as the "other" side of the subdev. It
> wouldn't work for a subdev which has multiplexed sink and source pads.
> This series implements a "deep" get-format (v4l2_subdev_get_format_dir)
> which follows a stream either towards the original source or the final
> sink, while looking for a non-multiplexed pad with a format.
> 
> Some thoughts:
> 
> 1) Link validation and v4l2_subdev_get_format_dir need to look at the
> routing, and this leads to multiple allocs to get a copy of the routing
> table. There might be a possibility here to keep a table allocated and
> re-use it in consecutive get_routing calls.
> 
> Or even better, perhaps the kAPI could be changed so that allocs are not
> needed. I thought about a kAPI where the subdev just returns a pointer
> to its routing table, but then we hit the life-cycle problem: how to
> ensure the table won't be freed or changed until the caller is done.
> 
> 2) The routing uAPI is a bit vague. There is no way for the userspace to
> figure out what kind of routing is allowed. Also, the existence of a
> route in the routing table already indicates that the route is active,
> but we also have V4L2_SUBDEV_ROUTE_FL_ACTIVE. I decided to keep
> V4L2_SUBDEV_ROUTE_FL_ACTIVE for now, even if it doesn't really provide
> any feature.
> 
> 3) V4L2_FRAME_DESC_ENTRY_MAX is defined as 8 (I change it from 4 to 8 in
> this series). This limits the number of streams per pad to 8. Preferably
> the number of frame descs would be unlimited, but I didn't start
> tackling this. I believe 8 is quite safe number (4 pixel streams and 4
> embedded data stream).
> 
> 4) Link validation ends up following the same routes multiple times, as
> each stream in each subdev is validated separately.
> 
>  Tomi
> 
> Jacopo Mondi (2):
>   media: entity: Add iterator helper for entity pads
>   media: Documentation: Add GS_ROUTING documentation
> 
> Laurent Pinchart (4):
>   media: entity: Add has_route entity operation
>   media: entity: Add media_entity_has_route() function
>   media: entity: Use routing information during graph traversal
>   v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations
> 
> Sakari Ailus (14):
>   media: entity: Use pad as a starting point for graph walk
>   media: entity: Use pads instead of entities in the media graph walk
>     stack
>   media: entity: Walk the graph based on pads
>   v4l: mc: Start walk from a specific pad in use count calculation
>   media: entity: Move the pipeline from entity to pads
>   media: entity: Use pad as the starting point for a pipeline
>   media: entity: Skip link validation for pads to which there is no
>     route to
>   media: entity: Add an iterator helper for connected pads
>   media: entity: Add only connected pads to the pipeline
>   media: entity: Add debug information in graph walk route check
>   v4l: Add bus type to frame descriptors
>   v4l: Add CSI-2 bus configuration to frame descriptors
>   v4l: Add stream to frame descriptor
>   v4l: mc: Add an S_ROUTING helper function for power state changes
> 
> Tomi Valkeinen (4):
>   v4l: subdev: routing kernel helper functions
>   v4l: subdev: add v4l2_subdev_get_format_dir()
>   v4l: subdev: Take routing information into account in link validation
>   v4l: subdev: increase V4L2_FRAME_DESC_ENTRY_MAX to 8
> 
>  Documentation/driver-api/media/mc-core.rst    |  15 +-
>  .../userspace-api/media/v4l/dev-subdev.rst    |  92 +++++
>  .../userspace-api/media/v4l/user-func.rst     |   1 +
>  .../media/v4l/vidioc-subdev-g-routing.rst     | 143 +++++++
>  drivers/media/mc/mc-device.c                  |  13 +-
>  drivers/media/mc/mc-entity.c                  | 239 +++++++-----
>  drivers/media/pci/intel/ipu3/ipu3-cio2-main.c |   6 +-
>  .../media/platform/exynos4-is/fimc-capture.c  |   8 +-
>  .../platform/exynos4-is/fimc-isp-video.c      |   8 +-
>  drivers/media/platform/exynos4-is/fimc-isp.c  |   2 +-
>  drivers/media/platform/exynos4-is/fimc-lite.c |  10 +-
>  drivers/media/platform/exynos4-is/media-dev.c |  20 +-
>  drivers/media/platform/omap3isp/isp.c         |   2 +-
>  drivers/media/platform/omap3isp/ispvideo.c    |  25 +-
>  drivers/media/platform/omap3isp/ispvideo.h    |   2 +-
>  .../media/platform/qcom/camss/camss-video.c   |   6 +-
>  drivers/media/platform/rcar-vin/rcar-core.c   |  16 +-
>  drivers/media/platform/rcar-vin/rcar-dma.c    |   8 +-
>  .../platform/rockchip/rkisp1/rkisp1-capture.c |   6 +-
>  .../media/platform/s3c-camif/camif-capture.c  |   6 +-
>  drivers/media/platform/stm32/stm32-dcmi.c     |   6 +-
>  .../platform/sunxi/sun4i-csi/sun4i_dma.c      |   6 +-
>  .../platform/sunxi/sun6i-csi/sun6i_video.c    |   6 +-
>  drivers/media/platform/ti-vpe/cal-video.c     |   6 +-
>  drivers/media/platform/vsp1/vsp1_video.c      |  18 +-
>  drivers/media/platform/xilinx/xilinx-dma.c    |  20 +-
>  drivers/media/platform/xilinx/xilinx-dma.h    |   2 +-
>  .../media/test-drivers/vimc/vimc-capture.c    |   6 +-
>  drivers/media/usb/au0828/au0828-core.c        |   8 +-
>  drivers/media/v4l2-core/v4l2-ioctl.c          |  25 +-
>  drivers/media/v4l2-core/v4l2-mc.c             |  77 ++--
>  drivers/media/v4l2-core/v4l2-subdev.c         | 368 ++++++++++++++++++
>  drivers/staging/media/imx/imx-media-utils.c   |   8 +-
>  drivers/staging/media/ipu3/ipu3-v4l2.c        |   6 +-
>  drivers/staging/media/omap4iss/iss.c          |   2 +-
>  drivers/staging/media/omap4iss/iss_video.c    |  38 +-
>  drivers/staging/media/omap4iss/iss_video.h    |   2 +-
>  drivers/staging/media/tegra-video/tegra210.c  |   6 +-
>  include/media/media-entity.h                  | 143 +++++--
>  include/media/v4l2-mc.h                       |  22 ++
>  include/media/v4l2-subdev.h                   |  93 ++++-
>  include/uapi/linux/v4l2-subdev.h              |  44 +++
>  42 files changed, 1241 insertions(+), 299 deletions(-)
>  create mode 100644 Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
> 
> -- 
> 2.25.1
> 

-- 
Regards,
Niklas Söderlund

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 00/24] v4l: subdev internal routing
  2021-04-16  8:38 ` [PATCH v5 00/24] v4l: subdev internal routing Niklas Söderlund
@ 2021-04-16  8:47   ` Tomi Valkeinen
  2021-04-16  8:56     ` Niklas Söderlund
  0 siblings, 1 reply; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-16  8:47 UTC (permalink / raw)
  To: Niklas Söderlund
  Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	Mauro Carvalho Chehab, Hans Verkuil

Hi Niklas,

On 16/04/2021 11:38, Niklas Söderlund wrote:
> Hi Tomi,
> 
> I'm very happy to see this being worked on again!
> 
> Is there code somewhere that demonstrates the v5 API in use? I still
> have old branches of this series and it would be nice to see how the API
> have evolved for drivers.
> 
> Likewise are there some user-space code around that can be used to test
> the API? For v2 and v3 I had some hack patches [1], do they still work?
> More likely they have gone stale by now :-)
> 
> 1. git://git.ragnatech.se/v4l-utils routing

Yes for both. I didn't share those as they're not in a presentable state =).

But, with the disclaimer that your eyes may bleed when reading the code:

git://git.kernel.org/pub/scm/linux/kernel/git/tomba/linux.git 
multistream/work

git://github.com/tomba/kmsxx.git multistream

On the kernel side, the CAL driver is in relatively good shape. UB960 
driver is somewhat messy but not totally horrible. OV10635 is horrible, 
as it's used to fake metadata stream even if the sensor doesn't really 
have such a thing.

For testing I have used kms++ with python bindings. My test script is 
py/tests/cam.py

The uAPI has changed as there's now the 'which' field. But I think 
that's the only clear change, although the behavior could be slightly 
different wrt. setting formats.

  Tomi

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 00/24] v4l: subdev internal routing
  2021-04-16  8:47   ` Tomi Valkeinen
@ 2021-04-16  8:56     ` Niklas Söderlund
  0 siblings, 0 replies; 72+ messages in thread
From: Niklas Söderlund @ 2021-04-16  8:56 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
	Mauro Carvalho Chehab, Hans Verkuil

Hi Tomi,

On 2021-04-16 11:47:46 +0300, Tomi Valkeinen wrote:
> Hi Niklas,
> 
> On 16/04/2021 11:38, Niklas Söderlund wrote:
> > Hi Tomi,
> > 
> > I'm very happy to see this being worked on again!
> > 
> > Is there code somewhere that demonstrates the v5 API in use? I still
> > have old branches of this series and it would be nice to see how the API
> > have evolved for drivers.
> > 
> > Likewise are there some user-space code around that can be used to test
> > the API? For v2 and v3 I had some hack patches [1], do they still work?
> > More likely they have gone stale by now :-)
> > 
> > 1. git://git.ragnatech.se/v4l-utils routing
> 
> Yes for both. I didn't share those as they're not in a presentable state =).
> 
> But, with the disclaimer that your eyes may bleed when reading the code:
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/tomba/linux.git
> multistream/work
> 
> git://github.com/tomba/kmsxx.git multistream
> 
> On the kernel side, the CAL driver is in relatively good shape. UB960 driver
> is somewhat messy but not totally horrible. OV10635 is horrible, as it's
> used to fake metadata stream even if the sensor doesn't really have such a
> thing.
> 
> For testing I have used kms++ with python bindings. My test script is
> py/tests/cam.py
> 
> The uAPI has changed as there's now the 'which' field. But I think that's
> the only clear change, although the behavior could be slightly different
> wrt. setting formats.

Thanks!

> 
>  Tomi

-- 
Regards,
Niklas Söderlund

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 00/24] v4l: subdev internal routing
  2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
                   ` (24 preceding siblings ...)
  2021-04-16  8:38 ` [PATCH v5 00/24] v4l: subdev internal routing Niklas Söderlund
@ 2021-04-18 17:32 ` Laurent Pinchart
  2021-04-21 12:57   ` Tomi Valkeinen
  25 siblings, 1 reply; 72+ messages in thread
From: Laurent Pinchart @ 2021-04-18 17:32 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Tomi,

Nice to see a v5 of this plagued patch series :-)

On Thu, Apr 15, 2021 at 04:04:26PM +0300, Tomi Valkeinen wrote:
> Hi,
> 
> This is an RFC for subdev internal routing which is needed for
> multiplexed streams support. I believe this is essentially a v5 of the
> series, the v4 posted here:
> 
> https://lore.kernel.org/linux-media/20190328200608.9463-1-jacopo+renesas@jmondi.org/
> 
> Most of the patches are not changed (aside from fixing rebase issues
> etc). The major changes in this version are:
> 
> 1) Added 'which' field to the routing structs. It is currently not used,
> as implementing it is not trivial. However, I think it's good to add it
> to the uAPI now, and require the field to be set to
> V4L2_SUBDEV_FORMAT_ACTIVE for now. See this RFC for an idea how this
> could be implemented:
> 
> https://lore.kernel.org/linux-media/20210409133659.389544-1-tomi.valkeinen@ideasonboard.com/

I've reviewed that, and I like it, but it's not straightforward to
understand from that patch how you envision TRY to be implemented.

> 2) No hardcoded maximum number of routes. Defining a maximum is not
> possible, as there can be an arbitrary amount of routes per pad, and
> there can be an arbitrary amount of pads per subdev. This series
> allocates space for the routing table dynamically, which unfortunately
> leads to not-just-a-few allocs and frees.
> 
> 3) When searching for a format for a stream, the v4 looked for a
> non-multiplexed pad only as far as the "other" side of the subdev. It
> wouldn't work for a subdev which has multiplexed sink and source pads.
> This series implements a "deep" get-format (v4l2_subdev_get_format_dir)
> which follows a stream either towards the original source or the final
> sink, while looking for a non-multiplexed pad with a format.
> 
> Some thoughts:
> 
> 1) Link validation and v4l2_subdev_get_format_dir need to look at the
> routing, and this leads to multiple allocs to get a copy of the routing
> table. There might be a possibility here to keep a table allocated and
> re-use it in consecutive get_routing calls.
> 
> Or even better, perhaps the kAPI could be changed so that allocs are not
> needed. I thought about a kAPI where the subdev just returns a pointer
> to its routing table, but then we hit the life-cycle problem: how to
> ensure the table won't be freed or changed until the caller is done.

Storing the routing table in the v4l2_subdev_config (or
v4l2_subdev_state) would be one way to do so, and I'd like to explore
that direction. State lifetime is indeed an issue, and one simple option
would be to just take the graph lock to modify the routing.

> 2) The routing uAPI is a bit vague. There is no way for the userspace to
> figure out what kind of routing is allowed. Also, the existence of a
> route in the routing table already indicates that the route is active,
> but we also have V4L2_SUBDEV_ROUTE_FL_ACTIVE. I decided to keep
> V4L2_SUBDEV_ROUTE_FL_ACTIVE for now, even if it doesn't really provide
> any feature.

We can't report all possible routes if we take streams into account, but
maybe we could report all possible routes between pads ? This could go
through a separate ioctl.

> 3) V4L2_FRAME_DESC_ENTRY_MAX is defined as 8 (I change it from 4 to 8 in
> this series). This limits the number of streams per pad to 8. Preferably
> the number of frame descs would be unlimited, but I didn't start
> tackling this. I believe 8 is quite safe number (4 pixel streams and 4
> embedded data stream).

A more dynamic solution would be nice, but if this is internal to the
kernel, I suppose we can stored with a fixed limit.

> 4) Link validation ends up following the same routes multiple times, as
> each stream in each subdev is validated separately.

Caching routes would likely be too much trouble for too little gain. If
we can avoid the allocation of routing tables every time, then I think
this is an acceptable limitation.

> Jacopo Mondi (2):
>   media: entity: Add iterator helper for entity pads
>   media: Documentation: Add GS_ROUTING documentation
> 
> Laurent Pinchart (4):
>   media: entity: Add has_route entity operation
>   media: entity: Add media_entity_has_route() function
>   media: entity: Use routing information during graph traversal
>   v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations
> 
> Sakari Ailus (14):
>   media: entity: Use pad as a starting point for graph walk
>   media: entity: Use pads instead of entities in the media graph walk
>     stack
>   media: entity: Walk the graph based on pads
>   v4l: mc: Start walk from a specific pad in use count calculation
>   media: entity: Move the pipeline from entity to pads
>   media: entity: Use pad as the starting point for a pipeline
>   media: entity: Skip link validation for pads to which there is no
>     route to
>   media: entity: Add an iterator helper for connected pads
>   media: entity: Add only connected pads to the pipeline
>   media: entity: Add debug information in graph walk route check
>   v4l: Add bus type to frame descriptors
>   v4l: Add CSI-2 bus configuration to frame descriptors
>   v4l: Add stream to frame descriptor
>   v4l: mc: Add an S_ROUTING helper function for power state changes
> 
> Tomi Valkeinen (4):
>   v4l: subdev: routing kernel helper functions
>   v4l: subdev: add v4l2_subdev_get_format_dir()
>   v4l: subdev: Take routing information into account in link validation
>   v4l: subdev: increase V4L2_FRAME_DESC_ENTRY_MAX to 8
> 
>  Documentation/driver-api/media/mc-core.rst    |  15 +-
>  .../userspace-api/media/v4l/dev-subdev.rst    |  92 +++++
>  .../userspace-api/media/v4l/user-func.rst     |   1 +
>  .../media/v4l/vidioc-subdev-g-routing.rst     | 143 +++++++
>  drivers/media/mc/mc-device.c                  |  13 +-
>  drivers/media/mc/mc-entity.c                  | 239 +++++++-----
>  drivers/media/pci/intel/ipu3/ipu3-cio2-main.c |   6 +-
>  .../media/platform/exynos4-is/fimc-capture.c  |   8 +-
>  .../platform/exynos4-is/fimc-isp-video.c      |   8 +-
>  drivers/media/platform/exynos4-is/fimc-isp.c  |   2 +-
>  drivers/media/platform/exynos4-is/fimc-lite.c |  10 +-
>  drivers/media/platform/exynos4-is/media-dev.c |  20 +-
>  drivers/media/platform/omap3isp/isp.c         |   2 +-
>  drivers/media/platform/omap3isp/ispvideo.c    |  25 +-
>  drivers/media/platform/omap3isp/ispvideo.h    |   2 +-
>  .../media/platform/qcom/camss/camss-video.c   |   6 +-
>  drivers/media/platform/rcar-vin/rcar-core.c   |  16 +-
>  drivers/media/platform/rcar-vin/rcar-dma.c    |   8 +-
>  .../platform/rockchip/rkisp1/rkisp1-capture.c |   6 +-
>  .../media/platform/s3c-camif/camif-capture.c  |   6 +-
>  drivers/media/platform/stm32/stm32-dcmi.c     |   6 +-
>  .../platform/sunxi/sun4i-csi/sun4i_dma.c      |   6 +-
>  .../platform/sunxi/sun6i-csi/sun6i_video.c    |   6 +-
>  drivers/media/platform/ti-vpe/cal-video.c     |   6 +-
>  drivers/media/platform/vsp1/vsp1_video.c      |  18 +-
>  drivers/media/platform/xilinx/xilinx-dma.c    |  20 +-
>  drivers/media/platform/xilinx/xilinx-dma.h    |   2 +-
>  .../media/test-drivers/vimc/vimc-capture.c    |   6 +-
>  drivers/media/usb/au0828/au0828-core.c        |   8 +-
>  drivers/media/v4l2-core/v4l2-ioctl.c          |  25 +-
>  drivers/media/v4l2-core/v4l2-mc.c             |  77 ++--
>  drivers/media/v4l2-core/v4l2-subdev.c         | 368 ++++++++++++++++++
>  drivers/staging/media/imx/imx-media-utils.c   |   8 +-
>  drivers/staging/media/ipu3/ipu3-v4l2.c        |   6 +-
>  drivers/staging/media/omap4iss/iss.c          |   2 +-
>  drivers/staging/media/omap4iss/iss_video.c    |  38 +-
>  drivers/staging/media/omap4iss/iss_video.h    |   2 +-
>  drivers/staging/media/tegra-video/tegra210.c  |   6 +-
>  include/media/media-entity.h                  | 143 +++++--
>  include/media/v4l2-mc.h                       |  22 ++
>  include/media/v4l2-subdev.h                   |  93 ++++-
>  include/uapi/linux/v4l2-subdev.h              |  44 +++
>  42 files changed, 1241 insertions(+), 299 deletions(-)
>  create mode 100644 Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 03/24] media: entity: Walk the graph based on pads
  2021-04-15 13:04 ` [PATCH v5 03/24] media: entity: Walk the graph based on pads Tomi Valkeinen
@ 2021-04-18 17:47   ` Laurent Pinchart
  2021-04-20 11:30     ` Sakari Ailus
  0 siblings, 1 reply; 72+ messages in thread
From: Laurent Pinchart @ 2021-04-18 17:47 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Tomi,

Thank you for the patch.

On Thu, Apr 15, 2021 at 04:04:29PM +0300, Tomi Valkeinen wrote:
> From: Sakari Ailus <sakari.ailus@linux.intel.com>

And thank you Sakari too.

> Instead of iterating over graph entities during the walk, iterate the pads
> through which the entity was first reached. This is required in order to
> make the entity pipeline pad-based rather than entity based.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> ---
>  Documentation/driver-api/media/mc-core.rst    |  7 ++-
>  drivers/media/mc/mc-entity.c                  | 49 +++++++++++--------
>  drivers/media/platform/exynos4-is/media-dev.c | 20 ++++----
>  drivers/media/platform/omap3isp/ispvideo.c    | 17 ++++---
>  drivers/media/platform/vsp1/vsp1_video.c      | 12 ++---
>  drivers/media/platform/xilinx/xilinx-dma.c    | 12 ++---
>  drivers/media/v4l2-core/v4l2-mc.c             | 24 ++++-----
>  drivers/staging/media/omap4iss/iss_video.c    | 34 +++++++------
>  include/media/media-entity.h                  |  7 +--
>  9 files changed, 99 insertions(+), 83 deletions(-)
> 
> diff --git a/Documentation/driver-api/media/mc-core.rst b/Documentation/driver-api/media/mc-core.rst
> index ba0aee982124..8a13640bed56 100644
> --- a/Documentation/driver-api/media/mc-core.rst
> +++ b/Documentation/driver-api/media/mc-core.rst
> @@ -169,8 +169,11 @@ Drivers initiate a graph traversal by calling
>  The graph structure, provided by the caller, is initialized to start graph
>  traversal at the given pad in an entity.
>  
> -Drivers can then retrieve the next entity by calling
> -:c:func:`media_graph_walk_next()`
> +Drivers can then retrieve the next pad by calling
> +:c:func:`media_graph_walk_next()`. Only the pad through which the entity
> +is first reached is returned. If the caller is interested in knowing which
> +further pads would be connected, the :c:func:`media_entity_has_route()`
> +function can be used for that.
>  
>  When the graph traversal is complete the function will return ``NULL``.
>  
> diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
> index 44a05806b589..401fddf320e7 100644
> --- a/drivers/media/mc/mc-entity.c
> +++ b/drivers/media/mc/mc-entity.c
> @@ -340,9 +340,9 @@ static void media_graph_walk_iter(struct media_graph *graph)
>  		next->entity->name, next->index);
>  }
>  
> -struct media_entity *media_graph_walk_next(struct media_graph *graph)
> +struct media_pad *media_graph_walk_next(struct media_graph *graph)
>  {
> -	struct media_entity *entity;
> +	struct media_pad *pad;
>  
>  	if (stack_top(graph) == NULL)
>  		return NULL;
> @@ -355,11 +355,11 @@ struct media_entity *media_graph_walk_next(struct media_graph *graph)
>  	while (link_top(graph) != &stack_top(graph)->entity->links)
>  		media_graph_walk_iter(graph);
>  
> -	entity = stack_pop(graph)->entity;
> -	dev_dbg(entity->graph_obj.mdev->dev,
> -		"walk: returning entity '%s'\n", entity->name);
> +	pad = stack_pop(graph);
> +	dev_dbg(pad->graph_obj.mdev->dev,
> +		"walk: returning pad '%s':%u\n", pad->entity->name, pad->index);
>  
> -	return entity;
> +	return pad;
>  }
>  EXPORT_SYMBOL_GPL(media_graph_walk_next);
>  
> @@ -407,7 +407,8 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
>  {
>  	struct media_device *mdev = entity->graph_obj.mdev;
>  	struct media_graph *graph = &pipe->graph;
> -	struct media_entity *entity_err = entity;
> +	struct media_pad *pad = entity->pads;
> +	struct media_pad *pad_err = pad;
>  	struct media_link *link;
>  	int ret;
>  
> @@ -417,9 +418,11 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
>  			goto error_graph_walk_start;
>  	}
>  
> -	media_graph_walk_start(&pipe->graph, entity->pads);
> +	media_graph_walk_start(&pipe->graph, pad);
> +
> +	while ((pad = media_graph_walk_next(graph))) {
> +		struct media_entity *entity = pad->entity;
>  
> -	while ((entity = media_graph_walk_next(graph))) {
>  		DECLARE_BITMAP(active, MEDIA_ENTITY_MAX_PADS);
>  		DECLARE_BITMAP(has_no_links, MEDIA_ENTITY_MAX_PADS);
>  
> @@ -428,7 +431,7 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
>  		if (entity->pipe && entity->pipe != pipe) {
>  			pr_err("Pipe active for %s. Can't start for %s\n",
>  				entity->name,
> -				entity_err->name);
> +				pad_err->entity->name);
>  			ret = -EBUSY;
>  			goto error;
>  		}
> @@ -446,11 +449,12 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
>  		bitmap_fill(has_no_links, entity->num_pads);
>  
>  		list_for_each_entry(link, &entity->links, list) {
> -			struct media_pad *pad = link->sink->entity == entity
> -						? link->sink : link->source;
> +			struct media_pad *other_pad =
> +				link->sink->entity == entity ?
> +				link->sink : link->source;
>  
>  			/* Mark that a pad is connected by a link. */
> -			bitmap_clear(has_no_links, pad->index, 1);
> +			bitmap_clear(has_no_links, other_pad->index, 1);
>  
>  			/*
>  			 * Pads that either do not need to connect or
> @@ -459,13 +463,13 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
>  			 */
>  			if (!(pad->flags & MEDIA_PAD_FL_MUST_CONNECT) ||

Shouldn't this be other_pad->flags ?

>  			    link->flags & MEDIA_LNK_FL_ENABLED)
> -				bitmap_set(active, pad->index, 1);
> +				bitmap_set(active, other_pad->index, 1);
>  
>  			/*
>  			 * Link validation will only take place for
>  			 * sink ends of the link that are enabled.
>  			 */
> -			if (link->sink != pad ||
> +			if (link->sink != other_pad ||
>  			    !(link->flags & MEDIA_LNK_FL_ENABLED))
>  				continue;
>  
> @@ -501,9 +505,11 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
>  	 * Link validation on graph failed. We revert what we did and
>  	 * return the error.
>  	 */
> -	media_graph_walk_start(graph, entity_err->pads);
> +	media_graph_walk_start(graph, pad_err);
> +
> +	while ((pad_err = media_graph_walk_next(graph))) {
> +		struct media_entity *entity_err = pad_err->entity;
>  
> -	while ((entity_err = media_graph_walk_next(graph))) {
>  		/* Sanity check for negative stream_count */
>  		if (!WARN_ON_ONCE(entity_err->stream_count <= 0)) {
>  			entity_err->stream_count--;
> @@ -515,7 +521,7 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
>  		 * We haven't increased stream_count further than this
>  		 * so we quit here.
>  		 */
> -		if (entity_err == entity)
> +		if (pad_err == pad)
>  			break;
>  	}
>  
> @@ -542,8 +548,9 @@ EXPORT_SYMBOL_GPL(media_pipeline_start);
>  
>  void __media_pipeline_stop(struct media_entity *entity)
>  {
> -	struct media_graph *graph = &entity->pipe->graph;
>  	struct media_pipeline *pipe = entity->pipe;
> +	struct media_graph *graph = &pipe->graph;
> +	struct media_pad *pad;
>  
>  	/*
>  	 * If the following check fails, the driver has performed an
> @@ -554,7 +561,9 @@ void __media_pipeline_stop(struct media_entity *entity)
>  
>  	media_graph_walk_start(graph, entity->pads);
>  
> -	while ((entity = media_graph_walk_next(graph))) {
> +	while ((pad = media_graph_walk_next(graph))) {
> +		struct media_entity *entity = pad->entity;
> +
>  		/* Sanity check for negative stream_count */
>  		if (!WARN_ON_ONCE(entity->stream_count <= 0)) {
>  			entity->stream_count--;
> diff --git a/drivers/media/platform/exynos4-is/media-dev.c b/drivers/media/platform/exynos4-is/media-dev.c
> index d90663b65932..b910a23b7e23 100644
> --- a/drivers/media/platform/exynos4-is/media-dev.c
> +++ b/drivers/media/platform/exynos4-is/media-dev.c
> @@ -1166,7 +1166,7 @@ static int __fimc_md_modify_pipeline(struct media_entity *entity, bool enable)
>  static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
>  				      struct media_graph *graph)
>  {
> -	struct media_entity *entity_err = entity;
> +	struct media_pad *pad, *pad_err = entity->pads;
>  	int ret;
>  
>  	/*
> @@ -1175,13 +1175,13 @@ static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
>  	 * through active links. This is needed as we cannot power on/off the
>  	 * subdevs in random order.
>  	 */
> -	media_graph_walk_start(graph, entity->pads);
> +	media_graph_walk_start(graph, pad_err);
>  
> -	while ((entity = media_graph_walk_next(graph))) {
> -		if (!is_media_entity_v4l2_video_device(entity))
> +	while ((pad = media_graph_walk_next(graph))) {
> +		if (!is_media_entity_v4l2_video_device(pad->entity))
>  			continue;
>  
> -		ret  = __fimc_md_modify_pipeline(entity, enable);
> +		ret  = __fimc_md_modify_pipeline(pad->entity, enable);
>  
>  		if (ret < 0)
>  			goto err;
> @@ -1190,15 +1190,15 @@ static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
>  	return 0;
>  
>  err:
> -	media_graph_walk_start(graph, entity_err->pads);
> +	media_graph_walk_start(graph, pad_err);
>  
> -	while ((entity_err = media_graph_walk_next(graph))) {
> -		if (!is_media_entity_v4l2_video_device(entity_err))
> +	while ((pad_err = media_graph_walk_next(graph))) {
> +		if (!is_media_entity_v4l2_video_device(pad_err->entity))
>  			continue;
>  
> -		__fimc_md_modify_pipeline(entity_err, !enable);
> +		__fimc_md_modify_pipeline(pad_err->entity, !enable);
>  
> -		if (entity_err == entity)
> +		if (pad_err == pad)
>  			break;
>  	}
>  
> diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
> index 3c1485d59404..49cde04bfb21 100644
> --- a/drivers/media/platform/omap3isp/ispvideo.c
> +++ b/drivers/media/platform/omap3isp/ispvideo.c
> @@ -222,8 +222,8 @@ static int isp_video_get_graph_data(struct isp_video *video,
>  				    struct isp_pipeline *pipe)
>  {
>  	struct media_graph graph;
> -	struct media_entity *entity = &video->video.entity;
> -	struct media_device *mdev = entity->graph_obj.mdev;
> +	struct media_pad *pad = video->video.entity.pads;
> +	struct media_device *mdev = pad->entity->graph_obj.mdev;

Maybe video->video.entity.graph_obj.mdev ? Same below.

>  	struct isp_video *far_end = NULL;
>  	int ret;
>  
> @@ -234,23 +234,24 @@ static int isp_video_get_graph_data(struct isp_video *video,
>  		return ret;
>  	}
>  
> -	media_graph_walk_start(&graph, entity->pads);
> +	media_graph_walk_start(&graph, pad);
>  
> -	while ((entity = media_graph_walk_next(&graph))) {
> +	while ((pad = media_graph_walk_next(&graph))) {
>  		struct isp_video *__video;
>  
> -		media_entity_enum_set(&pipe->ent_enum, entity);
> +		media_entity_enum_set(&pipe->ent_enum, pad->entity);
>  
>  		if (far_end != NULL)
>  			continue;
>  
> -		if (entity == &video->video.entity)
> +		if (pad == video->video.entity.pads)
>  			continue;
>  
> -		if (!is_media_entity_v4l2_video_device(entity))
> +		if (!is_media_entity_v4l2_video_device(pad->entity))
>  			continue;
>  
> -		__video = to_isp_video(media_entity_to_video_device(entity));
> +		__video = to_isp_video(media_entity_to_video_device(
> +					       pad->entity));
>  		if (__video->type != video->type)
>  			far_end = __video;
>  	}
> diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
> index 61e4fbaba7b7..39dccf347ce1 100644
> --- a/drivers/media/platform/vsp1/vsp1_video.c
> +++ b/drivers/media/platform/vsp1/vsp1_video.c
> @@ -559,8 +559,8 @@ static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
>  				     struct vsp1_video *video)
>  {
>  	struct media_graph graph;
> -	struct media_entity *entity = &video->video.entity;
> -	struct media_device *mdev = entity->graph_obj.mdev;
> +	struct media_pad *pad = video->video.entity.pads;
> +	struct media_device *mdev = pad->entity->graph_obj.mdev;
>  	unsigned int i;
>  	int ret;
>  
> @@ -569,17 +569,17 @@ static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
>  	if (ret)
>  		return ret;
>  
> -	media_graph_walk_start(&graph, entity->pads);
> +	media_graph_walk_start(&graph, pad);
>  
> -	while ((entity = media_graph_walk_next(&graph))) {
> +	while ((pad = media_graph_walk_next(&graph))) {
>  		struct v4l2_subdev *subdev;
>  		struct vsp1_rwpf *rwpf;
>  		struct vsp1_entity *e;
>  
> -		if (!is_media_entity_v4l2_subdev(entity))
> +		if (!is_media_entity_v4l2_subdev(pad->entity))
>  			continue;
>  
> -		subdev = media_entity_to_v4l2_subdev(entity);
> +		subdev = media_entity_to_v4l2_subdev(pad->entity);
>  		e = to_vsp1_entity(subdev);
>  		list_add_tail(&e->list_pipe, &pipe->entities);
>  		e->pipe = pipe;
> diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
> index d64c3bee8b95..8df3c43aecbe 100644
> --- a/drivers/media/platform/xilinx/xilinx-dma.c
> +++ b/drivers/media/platform/xilinx/xilinx-dma.c
> @@ -175,8 +175,8 @@ static int xvip_pipeline_validate(struct xvip_pipeline *pipe,
>  				  struct xvip_dma *start)
>  {
>  	struct media_graph graph;
> -	struct media_entity *entity = &start->video.entity;
> -	struct media_device *mdev = entity->graph_obj.mdev;
> +	struct media_pad *pad = start->video.entity.pads;
> +	struct media_device *mdev = pad->entity->graph_obj.mdev;
>  	unsigned int num_inputs = 0;
>  	unsigned int num_outputs = 0;
>  	int ret;
> @@ -190,15 +190,15 @@ static int xvip_pipeline_validate(struct xvip_pipeline *pipe,
>  		return ret;
>  	}
>  
> -	media_graph_walk_start(&graph, entity->pads);
> +	media_graph_walk_start(&graph, pad);
>  
> -	while ((entity = media_graph_walk_next(&graph))) {
> +	while ((pad = media_graph_walk_next(&graph))) {
>  		struct xvip_dma *dma;
>  
> -		if (entity->function != MEDIA_ENT_F_IO_V4L)
> +		if (pad->entity->function != MEDIA_ENT_F_IO_V4L)
>  			continue;
>  
> -		dma = to_xvip_dma(media_entity_to_video_device(entity));
> +		dma = to_xvip_dma(media_entity_to_video_device(pad->entity));
>  
>  		if (dma->pad.flags & MEDIA_PAD_FL_SINK) {
>  			pipe->output = dma;
> diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
> index d215fe31b9a2..cbeb580c6754 100644
> --- a/drivers/media/v4l2-core/v4l2-mc.c
> +++ b/drivers/media/v4l2-core/v4l2-mc.c
> @@ -434,13 +434,14 @@ EXPORT_SYMBOL_GPL(v4l2_create_fwnode_links);
>  static int pipeline_pm_use_count(struct media_entity *entity,
>  	struct media_graph *graph)
>  {
> +	struct media_pad *pad;
>  	int use = 0;
>  
>  	media_graph_walk_start(graph, entity->pads);
>  
> -	while ((entity = media_graph_walk_next(graph))) {
> -		if (is_media_entity_v4l2_video_device(entity))
> -			use += entity->use_count;
> +	while ((pad = media_graph_walk_next(graph))) {
> +		if (is_media_entity_v4l2_video_device(pad->entity))
> +			use += pad->entity->use_count;
>  	}
>  
>  	return use;
> @@ -493,7 +494,7 @@ static int pipeline_pm_power_one(struct media_entity *entity, int change)
>  static int pipeline_pm_power(struct media_entity *entity, int change,
>  	struct media_graph *graph)
>  {
> -	struct media_entity *first = entity;
> +	struct media_pad *tmp_pad, *pad;
>  	int ret = 0;
>  
>  	if (!change)
> @@ -501,19 +502,18 @@ static int pipeline_pm_power(struct media_entity *entity, int change,
>  
>  	media_graph_walk_start(graph, entity->pads);
>  
> -	while (!ret && (entity = media_graph_walk_next(graph)))
> -		if (is_media_entity_v4l2_subdev(entity))
> -			ret = pipeline_pm_power_one(entity, change);
> +	while (!ret && (pad = media_graph_walk_next(graph)))
> +		if (is_media_entity_v4l2_subdev(pad->entity))
> +			ret = pipeline_pm_power_one(pad->entity, change);
>  
>  	if (!ret)
>  		return ret;
>  
> -	media_graph_walk_start(graph, first->pads);
> +	media_graph_walk_start(graph, entity->pads);
>  
> -	while ((first = media_graph_walk_next(graph))
> -	       && first != entity)
> -		if (is_media_entity_v4l2_subdev(first))
> -			pipeline_pm_power_one(first, -change);
> +	while ((tmp_pad = media_graph_walk_next(graph)) && tmp_pad != pad)
> +		if (is_media_entity_v4l2_subdev(tmp_pad->entity))
> +			pipeline_pm_power_one(tmp_pad->entity, -change);
>  
>  	return ret;
>  }
> diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
> index 77bf1b8a56f7..9f3ff5a37d90 100644
> --- a/drivers/staging/media/omap4iss/iss_video.c
> +++ b/drivers/staging/media/omap4iss/iss_video.c
> @@ -206,8 +206,8 @@ static struct iss_video *
>  iss_video_far_end(struct iss_video *video)
>  {
>  	struct media_graph graph;
> -	struct media_entity *entity = &video->video.entity;
> -	struct media_device *mdev = entity->graph_obj.mdev;
> +	struct media_pad *pad = video->video.entity.pads;
> +	struct media_device *mdev = pad->entity->graph_obj.mdev;
>  	struct iss_video *far_end = NULL;
>  
>  	mutex_lock(&mdev->graph_mutex);
> @@ -217,16 +217,17 @@ iss_video_far_end(struct iss_video *video)
>  		return NULL;
>  	}
>  
> -	media_graph_walk_start(&graph, entity->pads);
> +	media_graph_walk_start(&graph, pad);
>  
> -	while ((entity = media_graph_walk_next(&graph))) {
> -		if (entity == &video->video.entity)
> +	while ((pad = media_graph_walk_next(&graph))) {
> +		if (pad->entity == &video->video.entity)
>  			continue;
>  
> -		if (!is_media_entity_v4l2_video_device(entity))
> +		if (!is_media_entity_v4l2_video_device(pad->entity))
>  			continue;
>  
> -		far_end = to_iss_video(media_entity_to_video_device(entity));
> +		far_end = to_iss_video(media_entity_to_video_device(
> +						pad->entity));
>  		if (far_end->type != video->type)
>  			break;
>  
> @@ -853,7 +854,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
>  	struct iss_video_fh *vfh = to_iss_video_fh(fh);
>  	struct iss_video *video = video_drvdata(file);
>  	struct media_graph graph;
> -	struct media_entity *entity = &video->video.entity;
> +	struct media_pad *pad = video->video.entity.pads;
>  	enum iss_pipeline_state state;
>  	struct iss_pipeline *pipe;
>  	struct iss_video *far_end;
> @@ -869,30 +870,31 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
>  	 * Start streaming on the pipeline. No link touching an entity in the
>  	 * pipeline can be activated or deactivated once streaming is started.
>  	 */
> -	pipe = entity->pipe
> -	     ? to_iss_pipeline(entity) : &video->pipe;
> +	pipe = pad->entity->pipe
> +	     ? to_iss_pipeline(pad->entity) : &video->pipe;
>  	pipe->external = NULL;
>  	pipe->external_rate = 0;
>  	pipe->external_bpp = 0;
>  
> -	ret = media_entity_enum_init(&pipe->ent_enum, entity->graph_obj.mdev);
> +	ret = media_entity_enum_init(&pipe->ent_enum,
> +				     pad->entity->graph_obj.mdev);
>  	if (ret)
>  		goto err_graph_walk_init;
>  
> -	ret = media_graph_walk_init(&graph, entity->graph_obj.mdev);
> +	ret = media_graph_walk_init(&graph, pad->entity->graph_obj.mdev);
>  	if (ret)
>  		goto err_graph_walk_init;
>  
>  	if (video->iss->pdata->set_constraints)
>  		video->iss->pdata->set_constraints(video->iss, true);
>  
> -	ret = media_pipeline_start(entity, &pipe->pipe);
> +	ret = media_pipeline_start(pad->entity, &pipe->pipe);
>  	if (ret < 0)
>  		goto err_media_pipeline_start;
>  
> -	media_graph_walk_start(&graph, entity->pads);
> -	while ((entity = media_graph_walk_next(&graph)))
> -		media_entity_enum_set(&pipe->ent_enum, entity);
> +	media_graph_walk_start(&graph, pad);
> +	while ((pad = media_graph_walk_next(&graph)))
> +		media_entity_enum_set(&pipe->ent_enum, pad->entity);
>  
>  	/*
>  	 * Verify that the currently configured format matches the output of
> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> index 97b170cf38eb..2d45344ca527 100644
> --- a/include/media/media-entity.h
> +++ b/include/media/media-entity.h
> @@ -921,10 +921,11 @@ void media_graph_walk_start(struct media_graph *graph, struct media_pad *pad);
>   * The graph structure must have been previously initialized with a call to
>   * media_graph_walk_start().
>   *
> - * Return: returns the next entity in the graph or %NULL if the whole graph
> - * have been traversed.
> + * Return: returns the next pad in the graph or %NULL if the whole
> + * graph have been traversed. The pad which is returned is the pad
> + * through which a new entity is reached when parsing the graph.

"next pad in the graph" is a bit confusing. The next sentence clarifies
it, but maybe we can do better:

 * Return: returns the next entity in the graph, identified by the pad through
 * which it has been reached. If the whole graph has been traversed, return
 * %NULL.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

>   */
> -struct media_entity *media_graph_walk_next(struct media_graph *graph);
> +struct media_pad *media_graph_walk_next(struct media_graph *graph);
>  
>  /**
>   * media_pipeline_start - Mark a pipeline as streaming

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 05/24] media: entity: Add iterator helper for entity pads
  2021-04-15 13:04 ` [PATCH v5 05/24] media: entity: Add iterator helper for entity pads Tomi Valkeinen
@ 2021-04-18 17:52   ` Laurent Pinchart
  2021-04-22 12:04     ` Tomi Valkeinen
  0 siblings, 1 reply; 72+ messages in thread
From: Laurent Pinchart @ 2021-04-18 17:52 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Tomi and Jacopo,

Thank you for the patch.

On Thu, Apr 15, 2021 at 04:04:31PM +0300, Tomi Valkeinen wrote:
> From: Jacopo Mondi <jacopo+renesas@jmondi.org>
> 
> Add an iterator helper to easily cycle through all pads in an entity and
> use it in media-entity and media-device code where appropriate.
> 
> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> ---
>  drivers/media/mc/mc-device.c | 13 ++++++-------
>  drivers/media/mc/mc-entity.c | 11 ++++++-----
>  include/media/media-entity.h | 12 ++++++++++++
>  3 files changed, 24 insertions(+), 12 deletions(-)
> 
> diff --git a/drivers/media/mc/mc-device.c b/drivers/media/mc/mc-device.c
> index 9e56d2ad6b94..704ef1360eba 100644
> --- a/drivers/media/mc/mc-device.c
> +++ b/drivers/media/mc/mc-device.c
> @@ -581,7 +581,7 @@ static void __media_device_unregister_entity(struct media_entity *entity)
>  	struct media_device *mdev = entity->graph_obj.mdev;
>  	struct media_link *link, *tmp;
>  	struct media_interface *intf;
> -	unsigned int i;
> +	struct media_pad *iter;
>  
>  	ida_free(&mdev->entity_internal_idx, entity->internal_idx);
>  
> @@ -597,8 +597,8 @@ static void __media_device_unregister_entity(struct media_entity *entity)
>  	__media_entity_remove_links(entity);
>  
>  	/* Remove all pads that belong to this entity */
> -	for (i = 0; i < entity->num_pads; i++)
> -		media_gobj_destroy(&entity->pads[i].graph_obj);
> +	media_entity_for_each_pad(entity, iter)
> +		media_gobj_destroy(&iter->graph_obj);
>  
>  	/* Remove the entity */
>  	media_gobj_destroy(&entity->graph_obj);
> @@ -617,7 +617,7 @@ int __must_check media_device_register_entity(struct media_device *mdev,
>  					      struct media_entity *entity)
>  {
>  	struct media_entity_notify *notify, *next;
> -	unsigned int i;
> +	struct media_pad *iter;
>  	int ret;
>  
>  	if (entity->function == MEDIA_ENT_F_V4L2_SUBDEV_UNKNOWN ||
> @@ -646,9 +646,8 @@ int __must_check media_device_register_entity(struct media_device *mdev,
>  	media_gobj_create(mdev, MEDIA_GRAPH_ENTITY, &entity->graph_obj);
>  
>  	/* Initialize objects at the pads */
> -	for (i = 0; i < entity->num_pads; i++)
> -		media_gobj_create(mdev, MEDIA_GRAPH_PAD,
> -			       &entity->pads[i].graph_obj);
> +	media_entity_for_each_pad(entity, iter)
> +		media_gobj_create(mdev, MEDIA_GRAPH_PAD, &iter->graph_obj);
>  
>  	/* invoke entity_notify callbacks */
>  	list_for_each_entry_safe(notify, next, &mdev->entity_notify, list)
> diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
> index 401fddf320e7..830841e0cd28 100644
> --- a/drivers/media/mc/mc-entity.c
> +++ b/drivers/media/mc/mc-entity.c
> @@ -198,7 +198,8 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads,
>  			   struct media_pad *pads)
>  {
>  	struct media_device *mdev = entity->graph_obj.mdev;
> -	unsigned int i;
> +	struct media_pad *iter;
> +	unsigned int i = 0;
>  
>  	if (num_pads >= MEDIA_ENTITY_MAX_PADS)
>  		return -E2BIG;
> @@ -209,12 +210,12 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads,
>  	if (mdev)
>  		mutex_lock(&mdev->graph_mutex);
>  
> -	for (i = 0; i < num_pads; i++) {
> -		pads[i].entity = entity;
> -		pads[i].index = i;
> +	media_entity_for_each_pad(entity, iter) {
> +		iter->entity = entity;
> +		iter->index = i++;
>  		if (mdev)
>  			media_gobj_create(mdev, MEDIA_GRAPH_PAD,
> -					&entity->pads[i].graph_obj);
> +					&iter->graph_obj);
>  	}
>  
>  	if (mdev)
> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> index 2d45344ca527..52b1a1cab57a 100644
> --- a/include/media/media-entity.h
> +++ b/include/media/media-entity.h
> @@ -1102,3 +1102,15 @@ void media_remove_intf_links(struct media_interface *intf);
>  	 (entity)->ops->operation((entity) , ##args) : -ENOIOCTLCMD)
>  
>  #endif
> +
> +/**
> + * media_entity_for_each_pad - Iterate on all pads in an entity
> + * @entity: The entity the pads belong to
> + * @iter: The iterator pad

I would name the variable pad, here and in the code above.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> + *
> + * Iterate on all pads in a media entity.
> + */
> +#define media_entity_for_each_pad(entity, iter)			\
> +	for (iter = (entity)->pads;				\
> +	     iter < &(entity)->pads[(entity)->num_pads];	\
> +	     ++iter)

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 06/24] media: entity: Move the pipeline from entity to pads
  2021-04-15 13:04 ` [PATCH v5 06/24] media: entity: Move the pipeline from entity to pads Tomi Valkeinen
@ 2021-04-18 18:00   ` Laurent Pinchart
  2021-04-20 11:38     ` Sakari Ailus
  0 siblings, 1 reply; 72+ messages in thread
From: Laurent Pinchart @ 2021-04-18 18:00 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Tomi and Sakari,

Thank you for the patch.

On Thu, Apr 15, 2021 at 04:04:32PM +0300, Tomi Valkeinen wrote:
> From: Sakari Ailus <sakari.ailus@linux.intel.com>
> 
> This moves the pipe and stream_count fields from struct media_entity to
> struct media_pad. Effectively streams become pad-specific rather than
> being entity specific, allowing several independent streams to traverse a
> single entity and an entity to be part of several streams.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
> - Update documentation to use 'pads'
> - Use the media pad iterator in media_entity.c
> - Update rcar-dma.c to use the new per-pad stream count
> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> 
> - Fix cleanup in the error path of __media_pipeline_start()
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  drivers/media/mc/mc-entity.c                  | 68 +++++++++++--------
>  drivers/media/platform/exynos4-is/fimc-isp.c  |  2 +-
>  drivers/media/platform/exynos4-is/fimc-lite.c |  2 +-
>  drivers/media/platform/omap3isp/isp.c         |  2 +-
>  drivers/media/platform/omap3isp/ispvideo.c    |  2 +-
>  drivers/media/platform/omap3isp/ispvideo.h    |  2 +-
>  drivers/media/platform/rcar-vin/rcar-core.c   | 16 +++--
>  drivers/media/platform/rcar-vin/rcar-dma.c    |  2 +-
>  drivers/media/platform/xilinx/xilinx-dma.c    |  2 +-
>  drivers/media/platform/xilinx/xilinx-dma.h    |  2 +-
>  drivers/staging/media/imx/imx-media-utils.c   |  2 +-
>  drivers/staging/media/omap4iss/iss.c          |  2 +-
>  drivers/staging/media/omap4iss/iss_video.c    |  2 +-
>  drivers/staging/media/omap4iss/iss_video.h    |  2 +-
>  include/media/media-entity.h                  | 21 +++---
>  15 files changed, 73 insertions(+), 56 deletions(-)
> 
> diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
> index 830841e0cd28..b6e5aa639c26 100644
> --- a/drivers/media/mc/mc-entity.c
> +++ b/drivers/media/mc/mc-entity.c
> @@ -423,24 +423,28 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
>  
>  	while ((pad = media_graph_walk_next(graph))) {
>  		struct media_entity *entity = pad->entity;
> +		bool skip_validation = pad->pipe;

I'd write

		bool skip_validation = pad->pipe != NULL;

> +		struct media_pad *iter;
>  
>  		DECLARE_BITMAP(active, MEDIA_ENTITY_MAX_PADS);
>  		DECLARE_BITMAP(has_no_links, MEDIA_ENTITY_MAX_PADS);
>  
> -		entity->stream_count++;
> -
> -		if (entity->pipe && entity->pipe != pipe) {
> -			pr_err("Pipe active for %s. Can't start for %s\n",
> -				entity->name,
> -				pad_err->entity->name);
> -			ret = -EBUSY;
> -			goto error;
> +		media_entity_for_each_pad(entity, iter) {
> +			if (iter->pipe && iter->pipe != pipe) {
> +				pr_err("Pipe active for %s. Can't start for %s\n",
> +				       entity->name, iter->entity->name);
> +				ret = -EBUSY;
> +			} else {
> +				iter->pipe = pipe;
> +			}
> +			iter->stream_count++;
>  		}
>  
> -		entity->pipe = pipe;
> +		if (ret)
> +			goto error;
>  
> -		/* Already streaming --- no need to check. */
> -		if (entity->stream_count > 1)
> +		/* Already part of the pipeline, skip validation. */
> +		if (skip_validation)
>  			continue;
>  
>  		if (!entity->ops || !entity->ops->link_validate)
> @@ -509,20 +513,23 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
>  	media_graph_walk_start(graph, pad_err);
>  
>  	while ((pad_err = media_graph_walk_next(graph))) {
> -		struct media_entity *entity_err = pad_err->entity;
> -
> -		/* Sanity check for negative stream_count */
> -		if (!WARN_ON_ONCE(entity_err->stream_count <= 0)) {
> -			entity_err->stream_count--;
> -			if (entity_err->stream_count == 0)
> -				entity_err->pipe = NULL;
> +		struct media_entity *entity = pad_err->entity;
> +		struct media_pad *iter;
> +
> +		media_entity_for_each_pad(entity, iter) {
> +			/* Sanity check for negative stream_count */
> +			if (!WARN_ON_ONCE(iter->stream_count <= 0)) {
> +				--iter->stream_count;
> +				if (iter->stream_count == 0)
> +					iter->pipe = NULL;
> +			}
>  		}
>  
>  		/*
>  		 * We haven't increased stream_count further than this
>  		 * so we quit here.
>  		 */
> -		if (pad_err == pad)
> +		if (pad_err->entity == pad->entity)
>  			break;
>  	}
>  
> @@ -549,7 +556,7 @@ EXPORT_SYMBOL_GPL(media_pipeline_start);
>  
>  void __media_pipeline_stop(struct media_entity *entity)
>  {
> -	struct media_pipeline *pipe = entity->pipe;
> +	struct media_pipeline *pipe = entity->pads->pipe;
>  	struct media_graph *graph = &pipe->graph;
>  	struct media_pad *pad;
>  
> @@ -564,12 +571,15 @@ void __media_pipeline_stop(struct media_entity *entity)
>  
>  	while ((pad = media_graph_walk_next(graph))) {
>  		struct media_entity *entity = pad->entity;
> -
> -		/* Sanity check for negative stream_count */
> -		if (!WARN_ON_ONCE(entity->stream_count <= 0)) {
> -			entity->stream_count--;
> -			if (entity->stream_count == 0)
> -				entity->pipe = NULL;
> +		struct media_pad *iter;
> +
> +		media_entity_for_each_pad(entity, iter) {
> +			/* Sanity check for negative stream_count */
> +			if (!WARN_ON_ONCE(iter->stream_count <= 0)) {
> +				iter->stream_count--;
> +				if (iter->stream_count == 0)
> +					iter->pipe = NULL;
> +			}
>  		}
>  	}
>  
> @@ -839,7 +849,7 @@ int __media_entity_setup_link(struct media_link *link, u32 flags)
>  {
>  	const u32 mask = MEDIA_LNK_FL_ENABLED;
>  	struct media_device *mdev;
> -	struct media_entity *source, *sink;
> +	struct media_pad *source, *sink;
>  	int ret = -EBUSY;
>  
>  	if (link == NULL)
> @@ -855,8 +865,8 @@ int __media_entity_setup_link(struct media_link *link, u32 flags)
>  	if (link->flags == flags)
>  		return 0;
>  
> -	source = link->source->entity;
> -	sink = link->sink->entity;
> +	source = link->source;
> +	sink = link->sink;
>  
>  	if (!(link->flags & MEDIA_LNK_FL_DYNAMIC) &&
>  	    (source->stream_count || sink->stream_count))
> diff --git a/drivers/media/platform/exynos4-is/fimc-isp.c b/drivers/media/platform/exynos4-is/fimc-isp.c
> index a77c49b18511..563d07f630bc 100644
> --- a/drivers/media/platform/exynos4-is/fimc-isp.c
> +++ b/drivers/media/platform/exynos4-is/fimc-isp.c
> @@ -223,7 +223,7 @@ static int fimc_isp_subdev_set_fmt(struct v4l2_subdev *sd,
>  			}
>  		}
>  	} else {
> -		if (sd->entity.stream_count == 0) {
> +		if (sd->entity.pads->stream_count == 0) {
>  			if (fmt->pad == FIMC_ISP_SD_PAD_SINK) {
>  				struct v4l2_subdev_format format = *fmt;
>  
> diff --git a/drivers/media/platform/exynos4-is/fimc-lite.c b/drivers/media/platform/exynos4-is/fimc-lite.c
> index fe20af3a7178..56773a0be8d9 100644
> --- a/drivers/media/platform/exynos4-is/fimc-lite.c
> +++ b/drivers/media/platform/exynos4-is/fimc-lite.c
> @@ -1070,7 +1070,7 @@ static int fimc_lite_subdev_set_fmt(struct v4l2_subdev *sd,
>  	mutex_lock(&fimc->lock);
>  
>  	if ((atomic_read(&fimc->out_path) == FIMC_IO_ISP &&
> -	    sd->entity.stream_count > 0) ||
> +	    sd->entity.pads->stream_count > 0) ||
>  	    (atomic_read(&fimc->out_path) == FIMC_IO_DMA &&
>  	    vb2_is_busy(&fimc->vb_queue))) {
>  		mutex_unlock(&fimc->lock);
> diff --git a/drivers/media/platform/omap3isp/isp.c b/drivers/media/platform/omap3isp/isp.c
> index a6bb7d9bf75f..4ef623f10a44 100644
> --- a/drivers/media/platform/omap3isp/isp.c
> +++ b/drivers/media/platform/omap3isp/isp.c
> @@ -930,7 +930,7 @@ static int isp_pipeline_is_last(struct media_entity *me)
>  	struct isp_pipeline *pipe;
>  	struct media_pad *pad;
>  
> -	if (!me->pipe)
> +	if (!me->pads->pipe)
>  		return 0;
>  	pipe = to_isp_pipeline(me);
>  	if (pipe->stream_state == ISP_PIPELINE_STREAM_STOPPED)
> diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
> index 49cde04bfb21..f3bee79cdd85 100644
> --- a/drivers/media/platform/omap3isp/ispvideo.c
> +++ b/drivers/media/platform/omap3isp/ispvideo.c
> @@ -1094,7 +1094,7 @@ isp_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
>  	/* Start streaming on the pipeline. No link touching an entity in the
>  	 * pipeline can be activated or deactivated once streaming is started.
>  	 */
> -	pipe = video->video.entity.pipe
> +	pipe = video->video.entity.pads->pipe
>  	     ? to_isp_pipeline(&video->video.entity) : &video->pipe;
>  
>  	ret = media_entity_enum_init(&pipe->ent_enum, &video->isp->media_dev);
> diff --git a/drivers/media/platform/omap3isp/ispvideo.h b/drivers/media/platform/omap3isp/ispvideo.h
> index a0908670c0cf..4c9c5b719ec5 100644
> --- a/drivers/media/platform/omap3isp/ispvideo.h
> +++ b/drivers/media/platform/omap3isp/ispvideo.h
> @@ -100,7 +100,7 @@ struct isp_pipeline {
>  };
>  
>  #define to_isp_pipeline(__e) \
> -	container_of((__e)->pipe, struct isp_pipeline, pipe)
> +	container_of((__e)->pads->pipe, struct isp_pipeline, pipe)
>  
>  static inline int isp_pipeline_ready(struct isp_pipeline *pipe)
>  {
> diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
> index cb3025992817..07ec008aacc4 100644
> --- a/drivers/media/platform/rcar-vin/rcar-core.c
> +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> @@ -132,13 +132,17 @@ static int rvin_group_link_notify(struct media_link *link, u32 flags,
>  		return 0;
>  
>  	/*
> -	 * Don't allow link changes if any entity in the graph is
> -	 * streaming, modifying the CHSEL register fields can disrupt
> -	 * running streams.
> +	 * Don't allow link changes if any stream in the graph is active as
> +	 * modifying the CHSEL register fields can disrupt running streams.
>  	 */
> -	media_device_for_each_entity(entity, &group->mdev)
> -		if (entity->stream_count)
> -			return -EBUSY;
> +	media_device_for_each_entity(entity, &group->mdev) {
> +		struct media_pad *iter;
> +
> +		media_entity_for_each_pad(entity, iter) {
> +			if (iter->stream_count)
> +				return -EBUSY;
> +		}

Would it be enough to test entity->pads->stream_count, as the stream
count should be the same for all pads ?

> +	}
>  
>  	mutex_lock(&group->lock);
>  
> diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
> index f30dafbdf61c..7994262c9b63 100644
> --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> @@ -1231,7 +1231,7 @@ static int rvin_set_stream(struct rvin_dev *vin, int on)
>  	 */
>  	mdev = vin->vdev.entity.graph_obj.mdev;
>  	mutex_lock(&mdev->graph_mutex);
> -	pipe = sd->entity.pipe ? sd->entity.pipe : &vin->vdev.pipe;
> +	pipe = sd->entity.pads->pipe ? sd->entity.pads->pipe : &vin->vdev.pipe;
>  	ret = __media_pipeline_start(&vin->vdev.entity, pipe);
>  	mutex_unlock(&mdev->graph_mutex);
>  	if (ret)
> diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
> index 8df3c43aecbe..7fa0467dddde 100644
> --- a/drivers/media/platform/xilinx/xilinx-dma.c
> +++ b/drivers/media/platform/xilinx/xilinx-dma.c
> @@ -403,7 +403,7 @@ static int xvip_dma_start_streaming(struct vb2_queue *vq, unsigned int count)
>  	 * Use the pipeline object embedded in the first DMA object that starts
>  	 * streaming.
>  	 */
> -	pipe = dma->video.entity.pipe
> +	pipe = dma->video.entity.pads->pipe
>  	     ? to_xvip_pipeline(&dma->video.entity) : &dma->pipe;
>  
>  	ret = media_pipeline_start(&dma->video.entity, &pipe->pipe);
> diff --git a/drivers/media/platform/xilinx/xilinx-dma.h b/drivers/media/platform/xilinx/xilinx-dma.h
> index 2378bdae57ae..69ced71a5696 100644
> --- a/drivers/media/platform/xilinx/xilinx-dma.h
> +++ b/drivers/media/platform/xilinx/xilinx-dma.h
> @@ -47,7 +47,7 @@ struct xvip_pipeline {
>  
>  static inline struct xvip_pipeline *to_xvip_pipeline(struct media_entity *e)
>  {
> -	return container_of(e->pipe, struct xvip_pipeline, pipe);
> +	return container_of(e->pads->pipe, struct xvip_pipeline, pipe);
>  }
>  
>  /**
> diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
> index 5128915a5d6f..04b7c6bdcd85 100644
> --- a/drivers/staging/media/imx/imx-media-utils.c
> +++ b/drivers/staging/media/imx/imx-media-utils.c
> @@ -913,7 +913,7 @@ int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
>  			__media_pipeline_stop(entity);
>  	} else {
>  		v4l2_subdev_call(sd, video, s_stream, 0);
> -		if (entity->pipe)
> +		if (entity->pads->pipe)
>  			__media_pipeline_stop(entity);
>  	}
>  
> diff --git a/drivers/staging/media/omap4iss/iss.c b/drivers/staging/media/omap4iss/iss.c
> index 085397045b36..4f4573219337 100644
> --- a/drivers/staging/media/omap4iss/iss.c
> +++ b/drivers/staging/media/omap4iss/iss.c
> @@ -543,7 +543,7 @@ static int iss_pipeline_is_last(struct media_entity *me)
>  	struct iss_pipeline *pipe;
>  	struct media_pad *pad;
>  
> -	if (!me->pipe)
> +	if (!me->pads->pipe)
>  		return 0;
>  	pipe = to_iss_pipeline(me);
>  	if (pipe->stream_state == ISS_PIPELINE_STREAM_STOPPED)
> diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
> index 9f3ff5a37d90..9d7bf8c85558 100644
> --- a/drivers/staging/media/omap4iss/iss_video.c
> +++ b/drivers/staging/media/omap4iss/iss_video.c
> @@ -870,7 +870,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
>  	 * Start streaming on the pipeline. No link touching an entity in the
>  	 * pipeline can be activated or deactivated once streaming is started.
>  	 */
> -	pipe = pad->entity->pipe
> +	pipe = pad->pipe
>  	     ? to_iss_pipeline(pad->entity) : &video->pipe;
>  	pipe->external = NULL;
>  	pipe->external_rate = 0;
> diff --git a/drivers/staging/media/omap4iss/iss_video.h b/drivers/staging/media/omap4iss/iss_video.h
> index 526281bf0051..9b8ec27bf87d 100644
> --- a/drivers/staging/media/omap4iss/iss_video.h
> +++ b/drivers/staging/media/omap4iss/iss_video.h
> @@ -91,7 +91,7 @@ struct iss_pipeline {
>  };
>  
>  #define to_iss_pipeline(__e) \
> -	container_of((__e)->pipe, struct iss_pipeline, pipe)
> +	container_of((__e)->pads->pipe, struct iss_pipeline, pipe)
>  
>  static inline int iss_pipeline_ready(struct iss_pipeline *pipe)
>  {
> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> index 52b1a1cab57a..b8f94662526c 100644
> --- a/include/media/media-entity.h
> +++ b/include/media/media-entity.h
> @@ -180,15 +180,24 @@ enum media_pad_signal_type {
>   *
>   * @graph_obj:	Embedded structure containing the media object common data
>   * @entity:	Entity this pad belongs to
> + * @pipe:	Pipeline this pad belongs to.
> + * @stream_count: Stream count for the pad.

No period at the end of the lines.

>   * @index:	Pad index in the entity pads array, numbered from 0 to n
>   * @sig_type:	Type of the signal inside a media pad
>   * @flags:	Pad flags, as defined in
>   *		:ref:`include/uapi/linux/media.h <media_header>`
>   *		(seek for ``MEDIA_PAD_FL_*``)
> + * .. note::
> + *
> + *    @stream_count reference count must never be negative, but is a signed
> + *    integer on purpose: a simple ``WARN_ON(<0)`` check can be used to
> + *    detect reference count bugs that would make them negative.

s/them/it/

>   */
>  struct media_pad {
>  	struct media_gobj graph_obj;	/* must be first field in struct */
>  	struct media_entity *entity;
> +	struct media_pipeline *pipe;
> +	int stream_count;
>  	u16 index;
>  	enum media_pad_signal_type sig_type;
>  	unsigned long flags;
> @@ -267,9 +276,7 @@ enum media_entity_type {
>   * @pads:	Pads array with the size defined by @num_pads.
>   * @links:	List of data links.
>   * @ops:	Entity operations.
> - * @stream_count: Stream count for the entity.
>   * @use_count:	Use count for the entity.
> - * @pipe:	Pipeline this entity belongs to.
>   * @info:	Union with devnode information.  Kept just for backward
>   *		compatibility.
>   * @info.dev:	Contains device major and minor info.
> @@ -282,10 +289,9 @@ enum media_entity_type {
>   *
>   * .. note::
>   *
> - *    @stream_count and @use_count reference counts must never be
> - *    negative, but are signed integers on purpose: a simple ``WARN_ON(<0)``
> - *    check can be used to detect reference count bugs that would make them
> - *    negative.
> + *    @use_count reference count must never be negative, but is a signed
> + *    integer on purpose: a simple ``WARN_ON(<0)`` check can be used to
> + *    detect reference count bugs that would make them negative.

s/them/it/

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

>   */
>  struct media_entity {
>  	struct media_gobj graph_obj;	/* must be first field in struct */
> @@ -304,11 +310,8 @@ struct media_entity {
>  
>  	const struct media_entity_operations *ops;
>  
> -	int stream_count;
>  	int use_count;
>  
> -	struct media_pipeline *pipe;
> -
>  	union {
>  		struct {
>  			u32 major;

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 11/24] media: entity: Skip link validation for pads to which there is no route to
  2021-04-15 13:04 ` [PATCH v5 11/24] media: entity: Skip link validation for pads to which there is no route to Tomi Valkeinen
@ 2021-04-18 18:06   ` Laurent Pinchart
  2021-04-20 11:41     ` Sakari Ailus
  0 siblings, 1 reply; 72+ messages in thread
From: Laurent Pinchart @ 2021-04-18 18:06 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Tomi and Sakari,

Thank you for the patch.

There's an extra "to" in the subject line.

On Thu, Apr 15, 2021 at 04:04:37PM +0300, Tomi Valkeinen wrote:
> From: Sakari Ailus <sakari.ailus@linux.intel.com>
> 
> Links are validated along the pipeline which is about to start streaming.
> Not all the pads in entities that are traversed along that pipeline are
> part of the pipeline, however. Skip the link validation for such pads,
> and while at there rename "other_pad" to "local_pad" to convey the fact
> the route to be checked is internal to the entity.

Both "pad" and "local_pad" are local. I would have kept the "other_pad"
variable name.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> ---
>  drivers/media/mc/mc-entity.c | 13 +++++++++----
>  1 file changed, 9 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
> index 28d7fd254c77..fe6cb743c85c 100644
> --- a/drivers/media/mc/mc-entity.c
> +++ b/drivers/media/mc/mc-entity.c
> @@ -482,12 +482,17 @@ __must_check int __media_pipeline_start(struct media_pad *pad,
>  		bitmap_fill(has_no_links, entity->num_pads);
>  
>  		list_for_each_entry(link, &entity->links, list) {
> -			struct media_pad *other_pad =
> +			struct media_pad *local_pad =
>  				link->sink->entity == entity ?
>  				link->sink : link->source;
>  
> +			/* Ignore pads to which there is no route. */
> +			if (!media_entity_has_route(entity, pad->index,
> +						    local_pad->index))
> +				continue;
> +
>  			/* Mark that a pad is connected by a link. */
> -			bitmap_clear(has_no_links, other_pad->index, 1);
> +			bitmap_clear(has_no_links, local_pad->index, 1);
>  
>  			/*
>  			 * Pads that either do not need to connect or
> @@ -496,13 +501,13 @@ __must_check int __media_pipeline_start(struct media_pad *pad,
>  			 */
>  			if (!(pad->flags & MEDIA_PAD_FL_MUST_CONNECT) ||
>  			    link->flags & MEDIA_LNK_FL_ENABLED)
> -				bitmap_set(active, other_pad->index, 1);
> +				bitmap_set(active, local_pad->index, 1);
>  
>  			/*
>  			 * Link validation will only take place for
>  			 * sink ends of the link that are enabled.
>  			 */
> -			if (link->sink != other_pad ||
> +			if (link->sink != local_pad ||
>  			    !(link->flags & MEDIA_LNK_FL_ENABLED))
>  				continue;
>  

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 12/24] media: entity: Add an iterator helper for connected pads
  2021-04-15 13:04 ` [PATCH v5 12/24] media: entity: Add an iterator helper for connected pads Tomi Valkeinen
@ 2021-04-18 18:20   ` Laurent Pinchart
  2021-04-20 11:48     ` Sakari Ailus
  0 siblings, 1 reply; 72+ messages in thread
From: Laurent Pinchart @ 2021-04-18 18:20 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Tomi and Sakari,

Thank you for the patch.

On Thu, Apr 15, 2021 at 04:04:38PM +0300, Tomi Valkeinen wrote:
> From: Sakari Ailus <sakari.ailus@linux.intel.com>
> 
> Add a helper macro for iterating over pads that are connected through
> enabled routes. This can be used to find all the connected pads within an
> entity, for instance starting from the pad which has been obtained during
> the graph walk.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> 
> - Make __media_entity_next_routed_pad() return NULL and adjust the
>   iterator to handle that
> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> ---
>  include/media/media-entity.h | 27 +++++++++++++++++++++++++++
>  1 file changed, 27 insertions(+)
> 
> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> index 73de1c335e4e..edd6f60ed6b4 100644
> --- a/include/media/media-entity.h
> +++ b/include/media/media-entity.h
> @@ -916,6 +916,33 @@ __must_check int media_graph_walk_init(
>  bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
>  			    unsigned int pad1);
>  
> +static inline struct media_pad *__media_entity_next_routed_pad(
> +	struct media_pad *start, struct media_pad *iter)
> +{
> +	struct media_entity *entity = start->entity;
> +
> +	for (; iter < &entity->pads[entity->num_pads]; iter++)
> +		if (media_entity_has_route(entity, start->index, iter->index))
> +			return iter;

I'd use curly braces.

> +
> +	return NULL;
> +}

Does this need to be inlined ?

> +
> +/**
> + * media_entity_for_each_routed_pad - Iterate over entity pads connected by routes

"routed" sounds a bit weird. Would media_entity_for_each_connected_pad()
be a better name ?

> + *
> + * @start: The stating pad

s/stating/starting/

> + * @iter: The iterator pad
> + *
> + * Iterate over all pads connected through routes from a given pad

"from the @start pad"

> + * within an entity. The iteration will include the starting pad itself.

s/starting/@start/

I wonder if it wouldn't be more logical to not include the start pad.
That wouldn't match the current usage patterns, which would need to be
adapted accordingly, but I'm worried that including the start pad will
lead to annoying bugs in the future. Maybe I worry too much.

And now that I reread the patch, I also wonder if "start" is a good
name, as it implies we start the enumeration from a given pad, while we
enumerate all pads connected to a given pad. I'm not sure what a better
name would be though, maybe just pad ?

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> + */
> +#define media_entity_for_each_routed_pad(start, iter)			\
> +	for (iter = __media_entity_next_routed_pad(			\
> +		     start, (start)->entity->pads);			\
> +	     iter != NULL;						\
> +	     iter = __media_entity_next_routed_pad(start, iter + 1))
> +
>  /**
>   * media_graph_walk_cleanup - Release resources used by graph walk.
>   *

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 18/24] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations
  2021-04-15 13:04 ` [PATCH v5 18/24] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations Tomi Valkeinen
@ 2021-04-18 18:32   ` Laurent Pinchart
  2021-04-22 11:16     ` Tomi Valkeinen
  2021-04-20 16:35   ` Sakari Ailus
  1 sibling, 1 reply; 72+ messages in thread
From: Laurent Pinchart @ 2021-04-18 18:32 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
	Michal Simek

Hi Tomi,

Thank you for the patch.

On Thu, Apr 15, 2021 at 04:04:44PM +0300, Tomi Valkeinen wrote:
> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
> Add support for subdev internal routing. A route is defined as a single
> stream from a sink pad to a source pad.
> 
> The userspace can configure the routing via two new ioctls,
> VIDIOC_SUBDEV_G_ROUTING and VIDIOC_SUBDEV_S_ROUTING, and subdevs can
> implement the functionality with v4l2_subdev_pad_ops.get_routing() and
> v4l2_subdev_pad_ops.set_routing().
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Michal Simek <michal.simek@xilinx.com>
> 
> - Add sink and source streams for multiplexed links
> - Copy the argument back in case of an error. This is needed to let the
>   caller know the number of routes.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> 
> - Expand and refine documentation.
> - Make the 'routes' pointer a __u64 __user pointer so that a compat32
>   version of the ioctl is not required.
> - Add struct v4l2_subdev_krouting to be used for subdevice operations.
> 
> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> 
> - Fix typecasing warnings
> - Check sink & source pad types
> - Add 'which' field
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>

Seems few people haven't contributed to this patch one way or another
:-)

> ---
>  drivers/media/v4l2-core/v4l2-ioctl.c  | 25 ++++++++++++++-
>  drivers/media/v4l2-core/v4l2-subdev.c | 45 +++++++++++++++++++++++++++
>  include/media/v4l2-subdev.h           | 24 ++++++++++++++
>  include/uapi/linux/v4l2-subdev.h      | 44 ++++++++++++++++++++++++++
>  4 files changed, 137 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 6a5d1c6d11d6..f5732962753f 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -16,6 +16,7 @@
>  #include <linux/kernel.h>
>  #include <linux/version.h>
>  
> +#include <linux/v4l2-subdev.h>
>  #include <linux/videodev2.h>
>  
>  #include <media/v4l2-common.h>
> @@ -3108,6 +3109,21 @@ static int check_array_args(unsigned int cmd, void *parg, size_t *array_size,
>  		ret = 1;
>  		break;
>  	}
> +
> +	case VIDIOC_SUBDEV_G_ROUTING:
> +	case VIDIOC_SUBDEV_S_ROUTING: {
> +		struct v4l2_subdev_routing *route = parg;

s/route/routing/ to match the code below.

> +
> +		if (route->num_routes > 256)
> +			return -EINVAL;
> +
> +		*user_ptr = u64_to_user_ptr(route->routes);
> +		*kernel_ptr = (void **)&route->routes;
> +		*array_size = sizeof(struct v4l2_subdev_route)
> +			    * route->num_routes;
> +		ret = 1;
> +		break;
> +	}
>  	}
>  
>  	return ret;
> @@ -3369,8 +3385,15 @@ video_usercopy(struct file *file, unsigned int orig_cmd, unsigned long arg,
>  	/*
>  	 * Some ioctls can return an error, but still have valid
>  	 * results that must be returned.
> +	 *
> +	 * FIXME: subdev IOCTLS are partially handled here and partially in
> +	 * v4l2-subdev.c and the 'always_copy' flag can only be set for IOCTLS
> +	 * defined here as part of the 'v4l2_ioctls' array. As
> +	 * VIDIOC_SUBDEV_G_ROUTING needs to return results to applications even
> +	 * in case of failure, but it is not defined here as part of the
> +	 * 'v4l2_ioctls' array, insert an ad-hoc check to address that.
>  	 */
> -	if (err < 0 && !always_copy)
> +	if (err < 0 && !always_copy && cmd != VIDIOC_SUBDEV_G_ROUTING)
>  		goto out;
>  
>  out_array_args:
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 956dafab43d4..95a4c3091fa6 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -681,6 +681,51 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg)
>  	case VIDIOC_SUBDEV_QUERYSTD:
>  		return v4l2_subdev_call(sd, video, querystd, arg);
>  
> +	case VIDIOC_SUBDEV_G_ROUTING: {
> +		struct v4l2_subdev_routing *routing = arg;
> +		struct v4l2_subdev_krouting krouting = {
> +			.which = routing->which,
> +			.num_routes = routing->num_routes,
> +			.routes = (struct v4l2_subdev_route *)(uintptr_t)routing->routes,
> +		};
> +		int ret;
> +
> +		ret = v4l2_subdev_call(sd, pad, get_routing, &krouting);
> +		if (ret)
> +			return ret;
> +
> +		routing->num_routes = krouting.num_routes;

Shouldn't num_routes be set even in case of errors ?

> +
> +		return 0;
> +	}
> +
> +	case VIDIOC_SUBDEV_S_ROUTING: {
> +		struct v4l2_subdev_routing *routing = arg;
> +		struct v4l2_subdev_route *route = (struct v4l2_subdev_route *)(uintptr_t)
> +						  routing->routes;

I'd name the variable routes as it points to an array.

> +		struct v4l2_subdev_krouting krouting = {};
> +		unsigned int i;
> +
> +		if (routing->which != V4L2_SUBDEV_FORMAT_TRY && ro_subdev)
> +			return -EPERM;
> +
> +		for (i = 0; i < routing->num_routes; ++i) {
> +			if (route[i].sink_pad >= sd->entity.num_pads ||
> +			    route[i].source_pad >= sd->entity.num_pads)
> +				return -EINVAL;
> +
> +			if (!(sd->entity.pads[route[i].sink_pad].flags & MEDIA_PAD_FL_SINK) ||
> +			    !(sd->entity.pads[route[i].source_pad].flags & MEDIA_PAD_FL_SOURCE))
> +				return -EINVAL;
> +		}
> +
> +		krouting.which = routing->which;
> +		krouting.num_routes = routing->num_routes;
> +		krouting.routes = route;
> +
> +		return v4l2_subdev_call(sd, pad, set_routing, &krouting);
> +	}
> +
>  	default:
>  		return v4l2_subdev_call(sd, core, ioctl, cmd, arg);
>  	}
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index 436d0445aafd..3826ab918731 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -650,6 +650,22 @@ struct v4l2_subdev_pad_config {
>  	struct v4l2_rect try_compose;
>  };
>  
> +/**
> + * struct v4l2_subdev_krouting - subdev routing table
> + *
> + * @which: format type (from enum v4l2_subdev_format_whence)
> + * @routes: &struct v4l2_subdev_route
> + * @num_routes: number of routes
> + *
> + * This structure is used to translate argument received from

s/argument/the arguments/

> + * VIDIOC_SUBDEV_G/S_ROUTING() ioctl to sudev device drivers operations.

s/sudev/subdev/

> + */
> +struct v4l2_subdev_krouting {
> +	u32 which;
> +	struct v4l2_subdev_route *routes;
> +	unsigned int num_routes;
> +};
> +
>  /**
>   * struct v4l2_subdev_pad_ops - v4l2-subdev pad level operations
>   *
> @@ -711,6 +727,10 @@ struct v4l2_subdev_pad_config {
>   *		     applied to the hardware. The operation shall fail if the
>   *		     pad index it has been called on is not valid or in case of
>   *		     unrecoverable failures.
> + *
> + * @get_routing: get the subdevice routing table.
> + * @set_routing: enable or disable data connection routes described in the
> + *		 subdevice routing table.
>   */
>  struct v4l2_subdev_pad_ops {
>  	int (*init_cfg)(struct v4l2_subdev *sd,
> @@ -755,6 +775,10 @@ struct v4l2_subdev_pad_ops {
>  			       struct v4l2_mbus_config *config);
>  	int (*set_mbus_config)(struct v4l2_subdev *sd, unsigned int pad,
>  			       struct v4l2_mbus_config *config);
> +	int (*get_routing)(struct v4l2_subdev *sd,
> +			   struct v4l2_subdev_krouting *route);
> +	int (*set_routing)(struct v4l2_subdev *sd,
> +			   struct v4l2_subdev_krouting *route);
>  };
>  
>  /**
> diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h
> index 658106f5b5dc..f2a17cbd1e9a 100644
> --- a/include/uapi/linux/v4l2-subdev.h
> +++ b/include/uapi/linux/v4l2-subdev.h
> @@ -188,6 +188,48 @@ struct v4l2_subdev_capability {
>  /* The v4l2 sub-device video device node is registered in read-only mode. */
>  #define V4L2_SUBDEV_CAP_RO_SUBDEV		0x00000001
>  
> +#define V4L2_SUBDEV_ROUTE_FL_ACTIVE		BIT(0)
> +#define V4L2_SUBDEV_ROUTE_FL_IMMUTABLE		BIT(1)
> +
> +/**
> + * struct v4l2_subdev_route - A route inside a subdev
> + *
> + * @sink_pad: the sink pad index
> + * @sink_stream: the sink stream identifier
> + * @source_pad: the source pad index
> + * @source_stream: the source stream identifier
> + * @flags: route flags:
> + *
> + *	V4L2_SUBDEV_ROUTE_FL_ACTIVE: Is the route active? An active
> + *	route will start when streaming is enabled on a video node.
> + *	Set by the user.
> + *
> + *	V4L2_SUBDEV_ROUTE_FL_IMMUTABLE: Is the route immutable, i.e.
> + *	can it be activated and inactivated? Set by the driver.

This should be moved to the macros.

> + */
> +struct v4l2_subdev_route {
> +	__u32 sink_pad;
> +	__u32 sink_stream;
> +	__u32 source_pad;
> +	__u32 source_stream;
> +	__u32 flags;
> +	__u32 reserved[5];
> +};
> +
> +/**
> + * struct v4l2_subdev_routing - Subdev routing information
> + *
> + * @which: format type (from enum v4l2_subdev_format_whence)

v4l2_subdev_format_whence shoudn't have had "format" in its name :-S We
can't do much about that, but should "format type" be changed to "routes
type", or maybe "configuration type" ?

> + * @routes: pointer to the routes array
> + * @num_routes: the total number of routes in the routes array

Won't kerneldoc complain about the undocumented reserved field ?

> + */
> +struct v4l2_subdev_routing {
> +	__u32 which;
> +	__u64 routes;
> +	__u32 num_routes;
> +	__u32 reserved[5];
> +};
> +
>  /* Backwards compatibility define --- to be removed */
>  #define v4l2_subdev_edid v4l2_edid
>  
> @@ -215,5 +257,7 @@ struct v4l2_subdev_capability {
>  #define VIDIOC_SUBDEV_ENUM_DV_TIMINGS		_IOWR('V', 98, struct v4l2_enum_dv_timings)
>  #define VIDIOC_SUBDEV_QUERY_DV_TIMINGS		_IOR('V', 99, struct v4l2_dv_timings)
>  #define VIDIOC_SUBDEV_DV_TIMINGS_CAP		_IOWR('V', 100, struct v4l2_dv_timings_cap)
> +#define VIDIOC_SUBDEV_G_ROUTING			_IOWR('V', 38, struct v4l2_subdev_routing)
> +#define VIDIOC_SUBDEV_S_ROUTING			_IOWR('V', 39, struct v4l2_subdev_routing)

I would keep the ioctls sorted by number.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

>  
>  #endif

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 20/24] v4l: mc: Add an S_ROUTING helper function for power state changes
  2021-04-15 13:04 ` [PATCH v5 20/24] v4l: mc: Add an S_ROUTING helper function for power state changes Tomi Valkeinen
@ 2021-04-18 18:55   ` Laurent Pinchart
  2021-04-20 16:41     ` Sakari Ailus
  0 siblings, 1 reply; 72+ messages in thread
From: Laurent Pinchart @ 2021-04-18 18:55 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Tomi and Sakari,

Thank you for the patch.

On Thu, Apr 15, 2021 at 04:04:46PM +0300, Tomi Valkeinen wrote:
> From: Sakari Ailus <sakari.ailus@linux.intel.com>
> 
> With the addition of the has_route() media entity operation, all pads of an
> entity are no longer interconnected. The S_ROUTING IOCTL for sub-devices can
> be used to enable and disable routes for an entity. The consequence is that
> the routing information has to be taken into account in use count
> calculation: disabling a route has a very similar effect on use counts as
> has disabling a link.
> 
> Add a helper function for drivers implementing VIDIOC_SUBDEV_S_ROUTING
> IOCTL to take the change into account.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> ---
>  drivers/media/v4l2-core/v4l2-mc.c | 34 +++++++++++++++++++++++++++++++
>  include/media/v4l2-mc.h           | 22 ++++++++++++++++++++
>  2 files changed, 56 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
> index 35d18ed89fa5..71acb389aa7b 100644
> --- a/drivers/media/v4l2-core/v4l2-mc.c
> +++ b/drivers/media/v4l2-core/v4l2-mc.c
> @@ -588,3 +588,37 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
>  	return ret;
>  }
>  EXPORT_SYMBOL_GPL(v4l2_pipeline_link_notify);
> +
> +int v4l2_subdev_routing_pm_use(struct media_entity *entity,
> +			       struct v4l2_subdev_route *route)
> +{
> +	struct media_graph *graph =
> +		&entity->graph_obj.mdev->pm_count_walk;
> +	struct media_pad *source = &entity->pads[route->source_pad];
> +	struct media_pad *sink = &entity->pads[route->sink_pad];
> +	int source_use;
> +	int sink_use;
> +	int ret;
> +
> +	source_use = pipeline_pm_use_count(source, graph);
> +	sink_use = pipeline_pm_use_count(sink, graph);
> +
> +	if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE)) {
> +		/* Route disabled. */
> +		pipeline_pm_power(source, -sink_use, graph);
> +		pipeline_pm_power(sink, -source_use, graph);
> +		return 0;
> +	}
> +
> +	/* Route enabled. */
> +	ret = pipeline_pm_power(source, sink_use, graph);
> +	if (ret < 0)
> +		return ret;
> +
> +	ret = pipeline_pm_power(sink, source_use, graph);
> +	if (ret < 0)
> +		pipeline_pm_power(source, -sink_use, graph);
> +
> +	return ret;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_routing_pm_use);

.s_power() is getting deprecated for sensors. Sakari, should we move
away from pipeline-based power management and drop this patch, or is it
too early ?

> diff --git a/include/media/v4l2-mc.h b/include/media/v4l2-mc.h
> index c181685923d5..ab8f4dc143aa 100644
> --- a/include/media/v4l2-mc.h
> +++ b/include/media/v4l2-mc.h
> @@ -18,6 +18,7 @@
>  /* We don't need to include pci.h or usb.h here */
>  struct pci_dev;
>  struct usb_device;
> +struct v4l2_subdev_route;
>  
>  #ifdef CONFIG_MEDIA_CONTROLLER
>  /**
> @@ -184,6 +185,22 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);
>  int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
>  			      unsigned int notification);
>  
> +/**
> + * v4l2_subdev_routing_pm_use - Handle power state changes due to S_ROUTING
> + * @entity: The entity
> + * @route: The new state of the route
> + *
> + * Propagate the use count across a route in a pipeline whenever the
> + * route is enabled or disabled. The function is called before
> + * changing the route state when enabling a route, and after changing
> + * the route state when disabling a route.
> + *
> + * Return 0 on success or a negative error code on failure. Powering entities
> + * off is assumed to never fail. This function will not fail for disconnection
> + * events.

"disconnection events" make me think about hotplug. How about "will not
fail when disabling a route" ?

> + */
> +int v4l2_subdev_routing_pm_use(struct media_entity *entity,
> +			       struct v4l2_subdev_route *route);
>  #else /* CONFIG_MEDIA_CONTROLLER */
>  
>  static inline int v4l2_mc_create_media_graph(struct media_device *mdev)
> @@ -219,5 +236,10 @@ static inline int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
>  	return 0;
>  }
>  
> +static inline int v4l2_subdev_routing_pm_use(struct media_entity *entity,
> +					     struct v4l2_subdev_route *route)
> +{
> +	return 0;
> +}

Missing blank line.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

(assuming we want to keep the patch)

>  #endif /* CONFIG_MEDIA_CONTROLLER */
>  #endif /* _V4L2_MC_H */

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 22/24] v4l: subdev: add v4l2_subdev_get_format_dir()
  2021-04-15 13:04 ` [PATCH v5 22/24] v4l: subdev: add v4l2_subdev_get_format_dir() Tomi Valkeinen
@ 2021-04-18 19:04   ` Laurent Pinchart
  2021-04-21 13:04     ` Tomi Valkeinen
  0 siblings, 1 reply; 72+ messages in thread
From: Laurent Pinchart @ 2021-04-18 19:04 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Tomi,

Thank you for the patch.

On Thu, Apr 15, 2021 at 04:04:48PM +0300, Tomi Valkeinen wrote:
> Add v4l2_subdev_get_format_dir() which can be used to find subdev format
> for a specific stream on a multiplexed pad. The function will follow the
> routes and links until it finds a non-multiplexed pad.
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  drivers/media/v4l2-core/v4l2-subdev.c | 96 +++++++++++++++++++++++++++
>  include/media/v4l2-subdev.h           | 26 ++++++++
>  2 files changed, 122 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 7a4f71d8c6c3..430dbdaab080 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -998,6 +998,102 @@ bool v4l2_subdev_has_route(struct v4l2_subdev_krouting *routing,
>  }
>  EXPORT_SYMBOL_GPL(v4l2_subdev_has_route);
>  
> +int v4l2_subdev_get_format_dir(struct media_pad *pad, u16 stream,
> +			       enum v4l2_direction dir,
> +			       struct v4l2_subdev_format *fmt)
> +{
> +	struct device *dev = pad->entity->graph_obj.mdev->dev;
> +	int ret;
> +	int i;
> +
> +	dev_dbg(dev, "%s '%s':%u:%u %s\n", __func__,
> +		pad->entity->name, pad->index, stream,
> +		dir == V4L2_DIR_SOURCEWARD ? "sourceward" : "sinkward");
> +
> +	while (true) {
> +		struct v4l2_subdev_krouting routing;
> +		struct v4l2_subdev_route *route;
> +
> +		if (pad->entity->obj_type != MEDIA_ENTITY_TYPE_V4L2_SUBDEV)
> +			return -EINVAL;
> +
> +		ret = v4l2_subdev_link_validate_get_format(pad, fmt);
> +		if (ret == 0)
> +			return 0;
> +		else if (ret != -ENOIOCTLCMD)
> +			return ret;
> +
> +		if (pad->flags &
> +		    (dir == V4L2_DIR_SINKWARD ? MEDIA_PAD_FL_SOURCE :
> +						MEDIA_PAD_FL_SINK)) {
> +			pad = media_entity_remote_pad(pad);
> +
> +			if (!pad)
> +				return -EINVAL;
> +
> +			if (pad->entity->obj_type != MEDIA_ENTITY_TYPE_V4L2_SUBDEV)
> +				return -EINVAL;
> +
> +			ret = v4l2_subdev_link_validate_get_format(pad, fmt);
> +			if (ret == 0)
> +				return 0;
> +			else if (ret != -ENOIOCTLCMD)
> +				return ret;
> +		}
> +
> +		ret = v4l2_subdev_get_krouting(media_entity_to_v4l2_subdev(pad->entity), &routing);
> +		if (ret)
> +			return ret;
> +
> +		route = NULL;
> +		for (i = 0; i < routing.num_routes; ++i) {
> +			u16 near_pad = dir == V4L2_DIR_SINKWARD ?
> +					       routing.routes[i].sink_pad :
> +					       routing.routes[i].source_pad;
> +			u16 near_stream = dir == V4L2_DIR_SINKWARD ?
> +						  routing.routes[i].sink_stream :
> +						  routing.routes[i].source_stream;
> +
> +			if (!(routing.routes[i].flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
> +				continue;
> +
> +			if (near_pad != pad->index)
> +				continue;
> +
> +			if (near_stream != stream)
> +				continue;
> +
> +			if (route) {
> +				dev_err(dev,
> +					"%s: '%s' has multiple active routes for stream %u\n",
> +					__func__, pad->entity->name, stream);
> +				v4l2_subdev_free_routing(&routing);
> +				return -EINVAL;
> +			}
> +
> +			route = &routing.routes[i];
> +		}
> +
> +		if (!route) {
> +			dev_err(dev, "%s: no route found in '%s' for stream %u\n",
> +				__func__, pad->entity->name, stream);
> +			v4l2_subdev_free_routing(&routing);
> +			return -EINVAL;
> +		}
> +
> +		if (dir == V4L2_DIR_SINKWARD) {
> +			pad = &pad->entity->pads[route->source_pad];
> +			stream = route->source_stream;
> +		} else {
> +			pad = &pad->entity->pads[route->sink_pad];
> +			stream = route->sink_stream;
> +		}
> +
> +		v4l2_subdev_free_routing(&routing);
> +	}
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_get_format_dir);
> +
>  int v4l2_subdev_link_validate(struct media_link *link)
>  {
>  	struct v4l2_subdev *sink;
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index 1843b77dd843..730631f9a091 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -1239,4 +1239,30 @@ void v4l2_subdev_cpy_routing(struct v4l2_subdev_krouting *dst,
>  bool v4l2_subdev_has_route(struct v4l2_subdev_krouting *routing,
>  			   unsigned int pad0, unsigned int pad1);
>  
> +/**
> + * enum v4l2_direction - Direction either towards the source or the sink
> + *
> + * @V4L2_DIR_SOURCEWARD: Direction towards the source.
> + * @V4L2_DIR_SINKWARD: Direction towards the sink.
> + */
> +enum v4l2_direction {
> +	V4L2_DIR_SOURCEWARD,
> +	V4L2_DIR_SINKWARD,
> +};
> +
> +/**
> + * v4l2_subdev_get_format_dir() - Find format by following streams

The name is a bit cryptic, and the usage pattern error-prone. Can we do
better ?  In particular, if we could limit the usage of this function to
be called on a non-multiplexed pad, we could drop the stream argument.
Deducing the direction argument from the type of pad would also make the
API simpler.

> + * @pad: The pad from which to start the search
> + * @stream: The stream for which we want to find the format
> + * @dir: The direction of the search
> + * @fmt: Pointer to &struct v4l2_subdev_format where the found format is stored
> + *
> + * This function attempts to find v4l2_subdev_format for a specific stream on a
> + * multiplexed pad by following the stream using routes and links to the specified
> + * direction, until a non-multiplexed pad is found.
> + */
> +int v4l2_subdev_get_format_dir(struct media_pad *pad, u16 stream,
> +			       enum v4l2_direction dir,
> +			       struct v4l2_subdev_format *fmt);
> +
>  #endif

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 24/24] v4l: subdev: increase V4L2_FRAME_DESC_ENTRY_MAX to 8
  2021-04-15 13:04 ` [PATCH v5 24/24] v4l: subdev: increase V4L2_FRAME_DESC_ENTRY_MAX to 8 Tomi Valkeinen
@ 2021-04-18 19:06   ` Laurent Pinchart
  0 siblings, 0 replies; 72+ messages in thread
From: Laurent Pinchart @ 2021-04-18 19:06 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Tomi,

Thank you for the patch.

On Thu, Apr 15, 2021 at 04:04:50PM +0300, Tomi Valkeinen wrote:
> V4L2_FRAME_DESC_ENTRY_MAX is currently set to 4. In theory it's possible
> to have an arbitrary amount of streams in a single pad, so preferably
> there should be no hardcoded maximum number.
> 
> However, I believe a reasonable max is 8, which would cover a CSI-2 pad
> with 4 streams of pixel data and 4 streams of metadata.
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  include/media/v4l2-subdev.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index 730631f9a091..49969d3699cb 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -356,7 +356,7 @@ struct v4l2_mbus_frame_desc_entry {
>  	} bus;
>  };
>  

With

 /*
  * FIXME: If this number is too small, it should be dropped altogether and the
  * API switched to a dynamic number of frame descriptor entries.
  */

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> -#define V4L2_FRAME_DESC_ENTRY_MAX	4
> +#define V4L2_FRAME_DESC_ENTRY_MAX	8
>  
>  enum v4l2_mbus_frame_desc_type {
>  	V4L2_MBUS_FRAME_DESC_TYPE_PLATFORM,

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 21/24] v4l: subdev: routing kernel helper functions
  2021-04-15 13:04 ` [PATCH v5 21/24] v4l: subdev: routing kernel helper functions Tomi Valkeinen
@ 2021-04-18 19:18   ` Laurent Pinchart
  0 siblings, 0 replies; 72+ messages in thread
From: Laurent Pinchart @ 2021-04-18 19:18 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Tomi,

Thank you for the patch.

On Thu, Apr 15, 2021 at 04:04:47PM +0300, Tomi Valkeinen wrote:
> Add helper functions for routing.
> 
> TODO: add docs.

I was going to mention that :-) Before doing this, however, we should
consider integration of the routing table in the v4l2_subdev_state, as
it will change the API needs. How does v4l2_subdev_duplicate_state()
sound ? ;-)

> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  drivers/media/v4l2-core/v4l2-subdev.c | 89 +++++++++++++++++++++++++++
>  include/media/v4l2-subdev.h           | 14 +++++
>  2 files changed, 103 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 95a4c3091fa6..7a4f71d8c6c3 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -909,6 +909,95 @@ v4l2_subdev_link_validate_get_format(struct media_pad *pad,
>  	return -EINVAL;
>  }
>  
> +int v4l2_subdev_get_krouting(struct v4l2_subdev *sd,
> +			     struct v4l2_subdev_krouting *routing)
> +{
> +	int ret;
> +
> +	routing->which = V4L2_SUBDEV_FORMAT_ACTIVE;
> +	routing->routes = NULL;
> +	routing->num_routes = 0;
> +
> +	ret = v4l2_subdev_call(sd, pad, get_routing, routing);
> +	if (ret == 0)
> +		return 0;
> +	if (ret != -ENOSPC)
> +		return ret;
> +
> +	routing->routes = kvmalloc_array(routing->num_routes,
> +					 sizeof(*routing->routes), GFP_KERNEL);
> +	if (!routing->routes)
> +		return -ENOMEM;
> +
> +	ret = v4l2_subdev_call(sd, pad, get_routing, routing);
> +	if (ret) {
> +		kvfree(routing->routes);
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_get_krouting);
> +
> +void v4l2_subdev_free_routing(struct v4l2_subdev_krouting *routing)
> +{
> +	kvfree(routing->routes);
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_free_routing);
> +
> +void v4l2_subdev_cpy_routing(struct v4l2_subdev_krouting *dst,
> +			     const struct v4l2_subdev_krouting *src)
> +{
> +	memcpy(dst->routes, src->routes,
> +	       src->num_routes * sizeof(*src->routes));
> +	dst->num_routes = src->num_routes;
> +	dst->which = src->which;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_cpy_routing);
> +
> +int v4l2_subdev_dup_routing(struct v4l2_subdev_krouting *dst,
> +			    const struct v4l2_subdev_krouting *src)
> +{
> +	if (dst->routes)
> +		kvfree(dst->routes);
> +
> +	if (src->num_routes == 0) {
> +		dst->which = src->which;
> +		dst->routes = NULL;
> +		dst->num_routes = 0;
> +		return 0;
> +	}
> +
> +	dst->routes = kvmalloc_array(src->num_routes, sizeof(*src->routes),
> +				     GFP_KERNEL);
> +	if (!dst->routes)
> +		return -ENOMEM;
> +
> +	v4l2_subdev_cpy_routing(dst, src);
> +
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_dup_routing);
> +
> +bool v4l2_subdev_has_route(struct v4l2_subdev_krouting *routing,
> +			   unsigned int pad0, unsigned int pad1)
> +{
> +	unsigned int i;
> +
> +	for (i = 0; i < routing->num_routes; ++i) {
> +		struct v4l2_subdev_route *route = &routing->routes[i];
> +
> +		if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
> +			continue;
> +
> +		if (route->sink_pad == pad0 && route->source_pad == pad1)
> +			return true;
> +	}
> +
> +	return false;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_has_route);
> +
>  int v4l2_subdev_link_validate(struct media_link *link)
>  {
>  	struct v4l2_subdev *sink;
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index 3826ab918731..1843b77dd843 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -1225,4 +1225,18 @@ extern const struct v4l2_subdev_ops v4l2_subdev_call_wrappers;
>  void v4l2_subdev_notify_event(struct v4l2_subdev *sd,
>  			      const struct v4l2_event *ev);
>  
> +int v4l2_subdev_get_krouting(struct v4l2_subdev *sd,
> +			     struct v4l2_subdev_krouting *routing);
> +
> +void v4l2_subdev_free_routing(struct v4l2_subdev_krouting *routing);
> +
> +int v4l2_subdev_dup_routing(struct v4l2_subdev_krouting *dst,
> +			    const struct v4l2_subdev_krouting *src);
> +
> +void v4l2_subdev_cpy_routing(struct v4l2_subdev_krouting *dst,
> +			     const struct v4l2_subdev_krouting *src);
> +
> +bool v4l2_subdev_has_route(struct v4l2_subdev_krouting *routing,
> +			   unsigned int pad0, unsigned int pad1);
> +
>  #endif

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 15/24] v4l: Add bus type to frame descriptors
  2021-04-15 13:04 ` [PATCH v5 15/24] v4l: Add bus type to frame descriptors Tomi Valkeinen
@ 2021-04-18 19:23   ` Laurent Pinchart
  2021-04-20 11:50     ` Sakari Ailus
  0 siblings, 1 reply; 72+ messages in thread
From: Laurent Pinchart @ 2021-04-18 19:23 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Tomi and Sakari,

Thank you for the patch.

On Thu, Apr 15, 2021 at 04:04:41PM +0300, Tomi Valkeinen wrote:
> From: Sakari Ailus <sakari.ailus@linux.intel.com>
> 
> Add the media bus type to the frame descriptor. CSI-2 specific
> information will be added in next patch to the frame descriptor.

I'd squash the next patch with this one.

> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> 
> - Make the bus type a named enum
> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> ---
>  include/media/v4l2-subdev.h | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index d0e9a5bdb08b..85977abbea46 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -340,12 +340,21 @@ struct v4l2_mbus_frame_desc_entry {
>  
>  #define V4L2_FRAME_DESC_ENTRY_MAX	4
>  
> +enum v4l2_mbus_frame_desc_type {
> +	V4L2_MBUS_FRAME_DESC_TYPE_PLATFORM,
> +	V4L2_MBUS_FRAME_DESC_TYPE_PARALLEL,
> +	V4L2_MBUS_FRAME_DESC_TYPE_CCP2,
> +	V4L2_MBUS_FRAME_DESC_TYPE_CSI2,
> +};

This should be documented. In particular, I have no idea what
V4L2_MBUS_FRAME_DESC_TYPE_PLATFORM is. I also wonder if we shouldn't
drop CCP2 (at least for now), does anyone use that anymore ?

> +
>  /**
>   * struct v4l2_mbus_frame_desc - media bus data frame description
> + * @type: type of the bus (enum v4l2_mbus_frame_desc_type)
>   * @entry: frame descriptors array
>   * @num_entries: number of entries in @entry array
>   */
>  struct v4l2_mbus_frame_desc {
> +	enum v4l2_mbus_frame_desc_type type;
>  	struct v4l2_mbus_frame_desc_entry entry[V4L2_FRAME_DESC_ENTRY_MAX];
>  	unsigned short num_entries;
>  };

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 16/24] v4l: Add CSI-2 bus configuration to frame descriptors
  2021-04-15 13:04 ` [PATCH v5 16/24] v4l: Add CSI-2 bus configuration " Tomi Valkeinen
@ 2021-04-18 19:24   ` Laurent Pinchart
  2021-04-20 16:32     ` Sakari Ailus
  0 siblings, 1 reply; 72+ messages in thread
From: Laurent Pinchart @ 2021-04-18 19:24 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Tomi and Sakari,

Thank you for the patch.

On Thu, Apr 15, 2021 at 04:04:42PM +0300, Tomi Valkeinen wrote:
> From: Sakari Ailus <sakari.ailus@linux.intel.com>
> 
> Add CSI-2 bus specific configuration to the frame descriptors. This allows
> obtaining the virtual channel and data type information for each stream
> the transmitter is sending.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> ---
>  include/media/v4l2-subdev.h | 16 ++++++++++++++++
>  1 file changed, 16 insertions(+)
> 
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index 85977abbea46..30ec011d31e3 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -308,6 +308,17 @@ struct v4l2_subdev_audio_ops {
>  	int (*s_stream)(struct v4l2_subdev *sd, int enable);
>  };
>  
> +/**
> + * struct v4l2_mbus_frame_desc_entry_csi2
> + *
> + * @channel: CSI-2 virtual channel

Maybe s/channel/virtual_channel/ ? Or vc and dt ?

> + * @data_type: CSI-2 data type ID
> + */
> +struct v4l2_mbus_frame_desc_entry_csi2 {
> +	u8 channel;
> +	u8 data_type;
> +};
> +
>  /**
>   * enum v4l2_mbus_frame_desc_flags - media bus frame description flags
>   *
> @@ -331,11 +342,16 @@ enum v4l2_mbus_frame_desc_flags {
>   *		%FRAME_DESC_FL_BLOB is not set.
>   * @length:	number of octets per frame, valid if @flags
>   *		%V4L2_MBUS_FRAME_DESC_FL_LEN_MAX is set.
> + * @bus:	Bus specific frame descriptor parameters

s/Bus specific/Bus-specific/

> + * @bus.csi2:	CSI-2 specific bus configuration

Ditto.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

>   */
>  struct v4l2_mbus_frame_desc_entry {
>  	enum v4l2_mbus_frame_desc_flags flags;
>  	u32 pixelcode;
>  	u32 length;
> +	union {
> +		struct v4l2_mbus_frame_desc_entry_csi2 csi2;
> +	} bus;
>  };
>  
>  #define V4L2_FRAME_DESC_ENTRY_MAX	4

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 17/24] v4l: Add stream to frame descriptor
  2021-04-15 13:04 ` [PATCH v5 17/24] v4l: Add stream to frame descriptor Tomi Valkeinen
@ 2021-04-18 19:27   ` Laurent Pinchart
  2021-04-22 12:47     ` Tomi Valkeinen
  0 siblings, 1 reply; 72+ messages in thread
From: Laurent Pinchart @ 2021-04-18 19:27 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Tomi and Sakari,

Thank you for the patch.

On Thu, Apr 15, 2021 at 04:04:43PM +0300, Tomi Valkeinen wrote:
> From: Sakari Ailus <sakari.ailus@linux.intel.com>
> 
> The stream field identifies the stream this frame descriptor applies to in
> routing configuration across a multiplexed link.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> ---
>  include/media/v4l2-subdev.h | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index 30ec011d31e3..436d0445aafd 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -338,6 +338,7 @@ enum v4l2_mbus_frame_desc_flags {
>   * struct v4l2_mbus_frame_desc_entry - media bus frame description structure
>   *
>   * @flags:	bitmask flags, as defined by &enum v4l2_mbus_frame_desc_flags.
> + * @stream:	stream in routing configuration
>   * @pixelcode:	media bus pixel code, valid if @flags
>   *		%FRAME_DESC_FL_BLOB is not set.
>   * @length:	number of octets per frame, valid if @flags
> @@ -347,6 +348,7 @@ enum v4l2_mbus_frame_desc_flags {
>   */
>  struct v4l2_mbus_frame_desc_entry {
>  	enum v4l2_mbus_frame_desc_flags flags;
> +	u32 stream;

As this isn't used in this series, I can't really tell how it will be
used, so it's hard to review the patch. Should we postpone it to the
next patch series ?

>  	u32 pixelcode;
>  	u32 length;
>  	union {

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 03/24] media: entity: Walk the graph based on pads
  2021-04-18 17:47   ` Laurent Pinchart
@ 2021-04-20 11:30     ` Sakari Ailus
  0 siblings, 0 replies; 72+ messages in thread
From: Sakari Ailus @ 2021-04-20 11:30 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: Tomi Valkeinen, linux-media, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Laurent,

On Sun, Apr 18, 2021 at 08:47:37PM +0300, Laurent Pinchart wrote:
> Hi Tomi,
> 
> Thank you for the patch.
> 
> On Thu, Apr 15, 2021 at 04:04:29PM +0300, Tomi Valkeinen wrote:
> > From: Sakari Ailus <sakari.ailus@linux.intel.com>
> 
> And thank you Sakari too.

Thank you for the review!

> 
> > Instead of iterating over graph entities during the walk, iterate the pads
> > through which the entity was first reached. This is required in order to
> > make the entity pipeline pad-based rather than entity based.
> > 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> > ---
> >  Documentation/driver-api/media/mc-core.rst    |  7 ++-
> >  drivers/media/mc/mc-entity.c                  | 49 +++++++++++--------
> >  drivers/media/platform/exynos4-is/media-dev.c | 20 ++++----
> >  drivers/media/platform/omap3isp/ispvideo.c    | 17 ++++---
> >  drivers/media/platform/vsp1/vsp1_video.c      | 12 ++---
> >  drivers/media/platform/xilinx/xilinx-dma.c    | 12 ++---
> >  drivers/media/v4l2-core/v4l2-mc.c             | 24 ++++-----
> >  drivers/staging/media/omap4iss/iss_video.c    | 34 +++++++------
> >  include/media/media-entity.h                  |  7 +--
> >  9 files changed, 99 insertions(+), 83 deletions(-)
> > 
> > diff --git a/Documentation/driver-api/media/mc-core.rst b/Documentation/driver-api/media/mc-core.rst
> > index ba0aee982124..8a13640bed56 100644
> > --- a/Documentation/driver-api/media/mc-core.rst
> > +++ b/Documentation/driver-api/media/mc-core.rst
> > @@ -169,8 +169,11 @@ Drivers initiate a graph traversal by calling
> >  The graph structure, provided by the caller, is initialized to start graph
> >  traversal at the given pad in an entity.
> >  
> > -Drivers can then retrieve the next entity by calling
> > -:c:func:`media_graph_walk_next()`
> > +Drivers can then retrieve the next pad by calling
> > +:c:func:`media_graph_walk_next()`. Only the pad through which the entity
> > +is first reached is returned. If the caller is interested in knowing which
> > +further pads would be connected, the :c:func:`media_entity_has_route()`
> > +function can be used for that.
> >  
> >  When the graph traversal is complete the function will return ``NULL``.
> >  
> > diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
> > index 44a05806b589..401fddf320e7 100644
> > --- a/drivers/media/mc/mc-entity.c
> > +++ b/drivers/media/mc/mc-entity.c
> > @@ -340,9 +340,9 @@ static void media_graph_walk_iter(struct media_graph *graph)
> >  		next->entity->name, next->index);
> >  }
> >  
> > -struct media_entity *media_graph_walk_next(struct media_graph *graph)
> > +struct media_pad *media_graph_walk_next(struct media_graph *graph)
> >  {
> > -	struct media_entity *entity;
> > +	struct media_pad *pad;
> >  
> >  	if (stack_top(graph) == NULL)
> >  		return NULL;
> > @@ -355,11 +355,11 @@ struct media_entity *media_graph_walk_next(struct media_graph *graph)
> >  	while (link_top(graph) != &stack_top(graph)->entity->links)
> >  		media_graph_walk_iter(graph);
> >  
> > -	entity = stack_pop(graph)->entity;
> > -	dev_dbg(entity->graph_obj.mdev->dev,
> > -		"walk: returning entity '%s'\n", entity->name);
> > +	pad = stack_pop(graph);
> > +	dev_dbg(pad->graph_obj.mdev->dev,
> > +		"walk: returning pad '%s':%u\n", pad->entity->name, pad->index);
> >  
> > -	return entity;
> > +	return pad;
> >  }
> >  EXPORT_SYMBOL_GPL(media_graph_walk_next);
> >  
> > @@ -407,7 +407,8 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
> >  {
> >  	struct media_device *mdev = entity->graph_obj.mdev;
> >  	struct media_graph *graph = &pipe->graph;
> > -	struct media_entity *entity_err = entity;
> > +	struct media_pad *pad = entity->pads;
> > +	struct media_pad *pad_err = pad;
> >  	struct media_link *link;
> >  	int ret;
> >  
> > @@ -417,9 +418,11 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
> >  			goto error_graph_walk_start;
> >  	}
> >  
> > -	media_graph_walk_start(&pipe->graph, entity->pads);
> > +	media_graph_walk_start(&pipe->graph, pad);
> > +
> > +	while ((pad = media_graph_walk_next(graph))) {
> > +		struct media_entity *entity = pad->entity;
> >  
> > -	while ((entity = media_graph_walk_next(graph))) {
> >  		DECLARE_BITMAP(active, MEDIA_ENTITY_MAX_PADS);
> >  		DECLARE_BITMAP(has_no_links, MEDIA_ENTITY_MAX_PADS);
> >  
> > @@ -428,7 +431,7 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
> >  		if (entity->pipe && entity->pipe != pipe) {
> >  			pr_err("Pipe active for %s. Can't start for %s\n",
> >  				entity->name,
> > -				entity_err->name);
> > +				pad_err->entity->name);
> >  			ret = -EBUSY;
> >  			goto error;
> >  		}
> > @@ -446,11 +449,12 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
> >  		bitmap_fill(has_no_links, entity->num_pads);
> >  
> >  		list_for_each_entry(link, &entity->links, list) {
> > -			struct media_pad *pad = link->sink->entity == entity
> > -						? link->sink : link->source;
> > +			struct media_pad *other_pad =
> > +				link->sink->entity == entity ?
> > +				link->sink : link->source;
> >  
> >  			/* Mark that a pad is connected by a link. */
> > -			bitmap_clear(has_no_links, pad->index, 1);
> > +			bitmap_clear(has_no_links, other_pad->index, 1);
> >  
> >  			/*
> >  			 * Pads that either do not need to connect or
> > @@ -459,13 +463,13 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
> >  			 */
> >  			if (!(pad->flags & MEDIA_PAD_FL_MUST_CONNECT) ||
> 
> Shouldn't this be other_pad->flags ?

Correct. Nice find!

> 
> >  			    link->flags & MEDIA_LNK_FL_ENABLED)
> > -				bitmap_set(active, pad->index, 1);
> > +				bitmap_set(active, other_pad->index, 1);
> >  
> >  			/*
> >  			 * Link validation will only take place for
> >  			 * sink ends of the link that are enabled.
> >  			 */
> > -			if (link->sink != pad ||
> > +			if (link->sink != other_pad ||
> >  			    !(link->flags & MEDIA_LNK_FL_ENABLED))
> >  				continue;
> >  
> > @@ -501,9 +505,11 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
> >  	 * Link validation on graph failed. We revert what we did and
> >  	 * return the error.
> >  	 */
> > -	media_graph_walk_start(graph, entity_err->pads);
> > +	media_graph_walk_start(graph, pad_err);
> > +
> > +	while ((pad_err = media_graph_walk_next(graph))) {
> > +		struct media_entity *entity_err = pad_err->entity;
> >  
> > -	while ((entity_err = media_graph_walk_next(graph))) {
> >  		/* Sanity check for negative stream_count */
> >  		if (!WARN_ON_ONCE(entity_err->stream_count <= 0)) {
> >  			entity_err->stream_count--;
> > @@ -515,7 +521,7 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
> >  		 * We haven't increased stream_count further than this
> >  		 * so we quit here.
> >  		 */
> > -		if (entity_err == entity)
> > +		if (pad_err == pad)
> >  			break;
> >  	}
> >  
> > @@ -542,8 +548,9 @@ EXPORT_SYMBOL_GPL(media_pipeline_start);
> >  
> >  void __media_pipeline_stop(struct media_entity *entity)
> >  {
> > -	struct media_graph *graph = &entity->pipe->graph;
> >  	struct media_pipeline *pipe = entity->pipe;
> > +	struct media_graph *graph = &pipe->graph;
> > +	struct media_pad *pad;
> >  
> >  	/*
> >  	 * If the following check fails, the driver has performed an
> > @@ -554,7 +561,9 @@ void __media_pipeline_stop(struct media_entity *entity)
> >  
> >  	media_graph_walk_start(graph, entity->pads);
> >  
> > -	while ((entity = media_graph_walk_next(graph))) {
> > +	while ((pad = media_graph_walk_next(graph))) {
> > +		struct media_entity *entity = pad->entity;
> > +
> >  		/* Sanity check for negative stream_count */
> >  		if (!WARN_ON_ONCE(entity->stream_count <= 0)) {
> >  			entity->stream_count--;
> > diff --git a/drivers/media/platform/exynos4-is/media-dev.c b/drivers/media/platform/exynos4-is/media-dev.c
> > index d90663b65932..b910a23b7e23 100644
> > --- a/drivers/media/platform/exynos4-is/media-dev.c
> > +++ b/drivers/media/platform/exynos4-is/media-dev.c
> > @@ -1166,7 +1166,7 @@ static int __fimc_md_modify_pipeline(struct media_entity *entity, bool enable)
> >  static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
> >  				      struct media_graph *graph)
> >  {
> > -	struct media_entity *entity_err = entity;
> > +	struct media_pad *pad, *pad_err = entity->pads;
> >  	int ret;
> >  
> >  	/*
> > @@ -1175,13 +1175,13 @@ static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
> >  	 * through active links. This is needed as we cannot power on/off the
> >  	 * subdevs in random order.
> >  	 */
> > -	media_graph_walk_start(graph, entity->pads);
> > +	media_graph_walk_start(graph, pad_err);
> >  
> > -	while ((entity = media_graph_walk_next(graph))) {
> > -		if (!is_media_entity_v4l2_video_device(entity))
> > +	while ((pad = media_graph_walk_next(graph))) {
> > +		if (!is_media_entity_v4l2_video_device(pad->entity))
> >  			continue;
> >  
> > -		ret  = __fimc_md_modify_pipeline(entity, enable);
> > +		ret  = __fimc_md_modify_pipeline(pad->entity, enable);
> >  
> >  		if (ret < 0)
> >  			goto err;
> > @@ -1190,15 +1190,15 @@ static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
> >  	return 0;
> >  
> >  err:
> > -	media_graph_walk_start(graph, entity_err->pads);
> > +	media_graph_walk_start(graph, pad_err);
> >  
> > -	while ((entity_err = media_graph_walk_next(graph))) {
> > -		if (!is_media_entity_v4l2_video_device(entity_err))
> > +	while ((pad_err = media_graph_walk_next(graph))) {
> > +		if (!is_media_entity_v4l2_video_device(pad_err->entity))
> >  			continue;
> >  
> > -		__fimc_md_modify_pipeline(entity_err, !enable);
> > +		__fimc_md_modify_pipeline(pad_err->entity, !enable);
> >  
> > -		if (entity_err == entity)
> > +		if (pad_err == pad)
> >  			break;
> >  	}
> >  
> > diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
> > index 3c1485d59404..49cde04bfb21 100644
> > --- a/drivers/media/platform/omap3isp/ispvideo.c
> > +++ b/drivers/media/platform/omap3isp/ispvideo.c
> > @@ -222,8 +222,8 @@ static int isp_video_get_graph_data(struct isp_video *video,
> >  				    struct isp_pipeline *pipe)
> >  {
> >  	struct media_graph graph;
> > -	struct media_entity *entity = &video->video.entity;
> > -	struct media_device *mdev = entity->graph_obj.mdev;
> > +	struct media_pad *pad = video->video.entity.pads;
> > +	struct media_device *mdev = pad->entity->graph_obj.mdev;
> 
> Maybe video->video.entity.graph_obj.mdev ? Same below.

Could be. That doesn't really matter though.

> 
> >  	struct isp_video *far_end = NULL;
> >  	int ret;
> >  
> > @@ -234,23 +234,24 @@ static int isp_video_get_graph_data(struct isp_video *video,
> >  		return ret;
> >  	}
> >  
> > -	media_graph_walk_start(&graph, entity->pads);
> > +	media_graph_walk_start(&graph, pad);
> >  
> > -	while ((entity = media_graph_walk_next(&graph))) {
> > +	while ((pad = media_graph_walk_next(&graph))) {
> >  		struct isp_video *__video;
> >  
> > -		media_entity_enum_set(&pipe->ent_enum, entity);
> > +		media_entity_enum_set(&pipe->ent_enum, pad->entity);
> >  
> >  		if (far_end != NULL)
> >  			continue;
> >  
> > -		if (entity == &video->video.entity)
> > +		if (pad == video->video.entity.pads)
> >  			continue;
> >  
> > -		if (!is_media_entity_v4l2_video_device(entity))
> > +		if (!is_media_entity_v4l2_video_device(pad->entity))
> >  			continue;
> >  
> > -		__video = to_isp_video(media_entity_to_video_device(entity));
> > +		__video = to_isp_video(media_entity_to_video_device(
> > +					       pad->entity));
> >  		if (__video->type != video->type)
> >  			far_end = __video;
> >  	}
> > diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
> > index 61e4fbaba7b7..39dccf347ce1 100644
> > --- a/drivers/media/platform/vsp1/vsp1_video.c
> > +++ b/drivers/media/platform/vsp1/vsp1_video.c
> > @@ -559,8 +559,8 @@ static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
> >  				     struct vsp1_video *video)
> >  {
> >  	struct media_graph graph;
> > -	struct media_entity *entity = &video->video.entity;
> > -	struct media_device *mdev = entity->graph_obj.mdev;
> > +	struct media_pad *pad = video->video.entity.pads;
> > +	struct media_device *mdev = pad->entity->graph_obj.mdev;
> >  	unsigned int i;
> >  	int ret;
> >  
> > @@ -569,17 +569,17 @@ static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
> >  	if (ret)
> >  		return ret;
> >  
> > -	media_graph_walk_start(&graph, entity->pads);
> > +	media_graph_walk_start(&graph, pad);
> >  
> > -	while ((entity = media_graph_walk_next(&graph))) {
> > +	while ((pad = media_graph_walk_next(&graph))) {
> >  		struct v4l2_subdev *subdev;
> >  		struct vsp1_rwpf *rwpf;
> >  		struct vsp1_entity *e;
> >  
> > -		if (!is_media_entity_v4l2_subdev(entity))
> > +		if (!is_media_entity_v4l2_subdev(pad->entity))
> >  			continue;
> >  
> > -		subdev = media_entity_to_v4l2_subdev(entity);
> > +		subdev = media_entity_to_v4l2_subdev(pad->entity);
> >  		e = to_vsp1_entity(subdev);
> >  		list_add_tail(&e->list_pipe, &pipe->entities);
> >  		e->pipe = pipe;
> > diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
> > index d64c3bee8b95..8df3c43aecbe 100644
> > --- a/drivers/media/platform/xilinx/xilinx-dma.c
> > +++ b/drivers/media/platform/xilinx/xilinx-dma.c
> > @@ -175,8 +175,8 @@ static int xvip_pipeline_validate(struct xvip_pipeline *pipe,
> >  				  struct xvip_dma *start)
> >  {
> >  	struct media_graph graph;
> > -	struct media_entity *entity = &start->video.entity;
> > -	struct media_device *mdev = entity->graph_obj.mdev;
> > +	struct media_pad *pad = start->video.entity.pads;
> > +	struct media_device *mdev = pad->entity->graph_obj.mdev;
> >  	unsigned int num_inputs = 0;
> >  	unsigned int num_outputs = 0;
> >  	int ret;
> > @@ -190,15 +190,15 @@ static int xvip_pipeline_validate(struct xvip_pipeline *pipe,
> >  		return ret;
> >  	}
> >  
> > -	media_graph_walk_start(&graph, entity->pads);
> > +	media_graph_walk_start(&graph, pad);
> >  
> > -	while ((entity = media_graph_walk_next(&graph))) {
> > +	while ((pad = media_graph_walk_next(&graph))) {
> >  		struct xvip_dma *dma;
> >  
> > -		if (entity->function != MEDIA_ENT_F_IO_V4L)
> > +		if (pad->entity->function != MEDIA_ENT_F_IO_V4L)
> >  			continue;
> >  
> > -		dma = to_xvip_dma(media_entity_to_video_device(entity));
> > +		dma = to_xvip_dma(media_entity_to_video_device(pad->entity));
> >  
> >  		if (dma->pad.flags & MEDIA_PAD_FL_SINK) {
> >  			pipe->output = dma;
> > diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
> > index d215fe31b9a2..cbeb580c6754 100644
> > --- a/drivers/media/v4l2-core/v4l2-mc.c
> > +++ b/drivers/media/v4l2-core/v4l2-mc.c
> > @@ -434,13 +434,14 @@ EXPORT_SYMBOL_GPL(v4l2_create_fwnode_links);
> >  static int pipeline_pm_use_count(struct media_entity *entity,
> >  	struct media_graph *graph)
> >  {
> > +	struct media_pad *pad;
> >  	int use = 0;
> >  
> >  	media_graph_walk_start(graph, entity->pads);
> >  
> > -	while ((entity = media_graph_walk_next(graph))) {
> > -		if (is_media_entity_v4l2_video_device(entity))
> > -			use += entity->use_count;
> > +	while ((pad = media_graph_walk_next(graph))) {
> > +		if (is_media_entity_v4l2_video_device(pad->entity))
> > +			use += pad->entity->use_count;
> >  	}
> >  
> >  	return use;
> > @@ -493,7 +494,7 @@ static int pipeline_pm_power_one(struct media_entity *entity, int change)
> >  static int pipeline_pm_power(struct media_entity *entity, int change,
> >  	struct media_graph *graph)
> >  {
> > -	struct media_entity *first = entity;
> > +	struct media_pad *tmp_pad, *pad;
> >  	int ret = 0;
> >  
> >  	if (!change)
> > @@ -501,19 +502,18 @@ static int pipeline_pm_power(struct media_entity *entity, int change,
> >  
> >  	media_graph_walk_start(graph, entity->pads);
> >  
> > -	while (!ret && (entity = media_graph_walk_next(graph)))
> > -		if (is_media_entity_v4l2_subdev(entity))
> > -			ret = pipeline_pm_power_one(entity, change);
> > +	while (!ret && (pad = media_graph_walk_next(graph)))
> > +		if (is_media_entity_v4l2_subdev(pad->entity))
> > +			ret = pipeline_pm_power_one(pad->entity, change);
> >  
> >  	if (!ret)
> >  		return ret;
> >  
> > -	media_graph_walk_start(graph, first->pads);
> > +	media_graph_walk_start(graph, entity->pads);
> >  
> > -	while ((first = media_graph_walk_next(graph))
> > -	       && first != entity)
> > -		if (is_media_entity_v4l2_subdev(first))
> > -			pipeline_pm_power_one(first, -change);
> > +	while ((tmp_pad = media_graph_walk_next(graph)) && tmp_pad != pad)
> > +		if (is_media_entity_v4l2_subdev(tmp_pad->entity))
> > +			pipeline_pm_power_one(tmp_pad->entity, -change);
> >  
> >  	return ret;
> >  }
> > diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
> > index 77bf1b8a56f7..9f3ff5a37d90 100644
> > --- a/drivers/staging/media/omap4iss/iss_video.c
> > +++ b/drivers/staging/media/omap4iss/iss_video.c
> > @@ -206,8 +206,8 @@ static struct iss_video *
> >  iss_video_far_end(struct iss_video *video)
> >  {
> >  	struct media_graph graph;
> > -	struct media_entity *entity = &video->video.entity;
> > -	struct media_device *mdev = entity->graph_obj.mdev;
> > +	struct media_pad *pad = video->video.entity.pads;
> > +	struct media_device *mdev = pad->entity->graph_obj.mdev;
> >  	struct iss_video *far_end = NULL;
> >  
> >  	mutex_lock(&mdev->graph_mutex);
> > @@ -217,16 +217,17 @@ iss_video_far_end(struct iss_video *video)
> >  		return NULL;
> >  	}
> >  
> > -	media_graph_walk_start(&graph, entity->pads);
> > +	media_graph_walk_start(&graph, pad);
> >  
> > -	while ((entity = media_graph_walk_next(&graph))) {
> > -		if (entity == &video->video.entity)
> > +	while ((pad = media_graph_walk_next(&graph))) {
> > +		if (pad->entity == &video->video.entity)
> >  			continue;
> >  
> > -		if (!is_media_entity_v4l2_video_device(entity))
> > +		if (!is_media_entity_v4l2_video_device(pad->entity))
> >  			continue;
> >  
> > -		far_end = to_iss_video(media_entity_to_video_device(entity));
> > +		far_end = to_iss_video(media_entity_to_video_device(
> > +						pad->entity));
> >  		if (far_end->type != video->type)
> >  			break;
> >  
> > @@ -853,7 +854,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
> >  	struct iss_video_fh *vfh = to_iss_video_fh(fh);
> >  	struct iss_video *video = video_drvdata(file);
> >  	struct media_graph graph;
> > -	struct media_entity *entity = &video->video.entity;
> > +	struct media_pad *pad = video->video.entity.pads;
> >  	enum iss_pipeline_state state;
> >  	struct iss_pipeline *pipe;
> >  	struct iss_video *far_end;
> > @@ -869,30 +870,31 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
> >  	 * Start streaming on the pipeline. No link touching an entity in the
> >  	 * pipeline can be activated or deactivated once streaming is started.
> >  	 */
> > -	pipe = entity->pipe
> > -	     ? to_iss_pipeline(entity) : &video->pipe;
> > +	pipe = pad->entity->pipe
> > +	     ? to_iss_pipeline(pad->entity) : &video->pipe;
> >  	pipe->external = NULL;
> >  	pipe->external_rate = 0;
> >  	pipe->external_bpp = 0;
> >  
> > -	ret = media_entity_enum_init(&pipe->ent_enum, entity->graph_obj.mdev);
> > +	ret = media_entity_enum_init(&pipe->ent_enum,
> > +				     pad->entity->graph_obj.mdev);
> >  	if (ret)
> >  		goto err_graph_walk_init;
> >  
> > -	ret = media_graph_walk_init(&graph, entity->graph_obj.mdev);
> > +	ret = media_graph_walk_init(&graph, pad->entity->graph_obj.mdev);
> >  	if (ret)
> >  		goto err_graph_walk_init;
> >  
> >  	if (video->iss->pdata->set_constraints)
> >  		video->iss->pdata->set_constraints(video->iss, true);
> >  
> > -	ret = media_pipeline_start(entity, &pipe->pipe);
> > +	ret = media_pipeline_start(pad->entity, &pipe->pipe);
> >  	if (ret < 0)
> >  		goto err_media_pipeline_start;
> >  
> > -	media_graph_walk_start(&graph, entity->pads);
> > -	while ((entity = media_graph_walk_next(&graph)))
> > -		media_entity_enum_set(&pipe->ent_enum, entity);
> > +	media_graph_walk_start(&graph, pad);
> > +	while ((pad = media_graph_walk_next(&graph)))
> > +		media_entity_enum_set(&pipe->ent_enum, pad->entity);
> >  
> >  	/*
> >  	 * Verify that the currently configured format matches the output of
> > diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> > index 97b170cf38eb..2d45344ca527 100644
> > --- a/include/media/media-entity.h
> > +++ b/include/media/media-entity.h
> > @@ -921,10 +921,11 @@ void media_graph_walk_start(struct media_graph *graph, struct media_pad *pad);
> >   * The graph structure must have been previously initialized with a call to
> >   * media_graph_walk_start().
> >   *
> > - * Return: returns the next entity in the graph or %NULL if the whole graph
> > - * have been traversed.
> > + * Return: returns the next pad in the graph or %NULL if the whole
> > + * graph have been traversed. The pad which is returned is the pad
> > + * through which a new entity is reached when parsing the graph.
> 
> "next pad in the graph" is a bit confusing. The next sentence clarifies
> it, but maybe we can do better:
> 
>  * Return: returns the next entity in the graph, identified by the pad through
>  * which it has been reached. If the whole graph has been traversed, return
>  * %NULL.
> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Thanks!

> 
> >   */
> > -struct media_entity *media_graph_walk_next(struct media_graph *graph);
> > +struct media_pad *media_graph_walk_next(struct media_graph *graph);
> >  
> >  /**
> >   * media_pipeline_start - Mark a pipeline as streaming
> 

-- 
Regards,

Sakari Ailus

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 06/24] media: entity: Move the pipeline from entity to pads
  2021-04-18 18:00   ` Laurent Pinchart
@ 2021-04-20 11:38     ` Sakari Ailus
  0 siblings, 0 replies; 72+ messages in thread
From: Sakari Ailus @ 2021-04-20 11:38 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: Tomi Valkeinen, linux-media, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Laurent,

On Sun, Apr 18, 2021 at 09:00:21PM +0300, Laurent Pinchart wrote:
> Hi Tomi and Sakari,
> 
> Thank you for the patch.
> 
> On Thu, Apr 15, 2021 at 04:04:32PM +0300, Tomi Valkeinen wrote:
> > From: Sakari Ailus <sakari.ailus@linux.intel.com>
> > 
> > This moves the pipe and stream_count fields from struct media_entity to
> > struct media_pad. Effectively streams become pad-specific rather than
> > being entity specific, allowing several independent streams to traverse a
> > single entity and an entity to be part of several streams.
> > 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > 
> > - Update documentation to use 'pads'
> > - Use the media pad iterator in media_entity.c
> > - Update rcar-dma.c to use the new per-pad stream count
> > Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> > 
> > - Fix cleanup in the error path of __media_pipeline_start()
> > Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> > ---
> >  drivers/media/mc/mc-entity.c                  | 68 +++++++++++--------
> >  drivers/media/platform/exynos4-is/fimc-isp.c  |  2 +-
> >  drivers/media/platform/exynos4-is/fimc-lite.c |  2 +-
> >  drivers/media/platform/omap3isp/isp.c         |  2 +-
> >  drivers/media/platform/omap3isp/ispvideo.c    |  2 +-
> >  drivers/media/platform/omap3isp/ispvideo.h    |  2 +-
> >  drivers/media/platform/rcar-vin/rcar-core.c   | 16 +++--
> >  drivers/media/platform/rcar-vin/rcar-dma.c    |  2 +-
> >  drivers/media/platform/xilinx/xilinx-dma.c    |  2 +-
> >  drivers/media/platform/xilinx/xilinx-dma.h    |  2 +-
> >  drivers/staging/media/imx/imx-media-utils.c   |  2 +-
> >  drivers/staging/media/omap4iss/iss.c          |  2 +-
> >  drivers/staging/media/omap4iss/iss_video.c    |  2 +-
> >  drivers/staging/media/omap4iss/iss_video.h    |  2 +-
> >  include/media/media-entity.h                  | 21 +++---
> >  15 files changed, 73 insertions(+), 56 deletions(-)
> > 
> > diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
> > index 830841e0cd28..b6e5aa639c26 100644
> > --- a/drivers/media/mc/mc-entity.c
> > +++ b/drivers/media/mc/mc-entity.c
> > @@ -423,24 +423,28 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
> >  
> >  	while ((pad = media_graph_walk_next(graph))) {
> >  		struct media_entity *entity = pad->entity;
> > +		bool skip_validation = pad->pipe;
> 
> I'd write
> 
> 		bool skip_validation = pad->pipe != NULL;
> 
> > +		struct media_pad *iter;
> >  
> >  		DECLARE_BITMAP(active, MEDIA_ENTITY_MAX_PADS);
> >  		DECLARE_BITMAP(has_no_links, MEDIA_ENTITY_MAX_PADS);
> >  
> > -		entity->stream_count++;
> > -
> > -		if (entity->pipe && entity->pipe != pipe) {
> > -			pr_err("Pipe active for %s. Can't start for %s\n",
> > -				entity->name,
> > -				pad_err->entity->name);
> > -			ret = -EBUSY;
> > -			goto error;
> > +		media_entity_for_each_pad(entity, iter) {
> > +			if (iter->pipe && iter->pipe != pipe) {
> > +				pr_err("Pipe active for %s. Can't start for %s\n",
> > +				       entity->name, iter->entity->name);
> > +				ret = -EBUSY;
> > +			} else {
> > +				iter->pipe = pipe;
> > +			}
> > +			iter->stream_count++;
> >  		}
> >  
> > -		entity->pipe = pipe;
> > +		if (ret)
> > +			goto error;
> >  
> > -		/* Already streaming --- no need to check. */
> > -		if (entity->stream_count > 1)
> > +		/* Already part of the pipeline, skip validation. */
> > +		if (skip_validation)
> >  			continue;
> >  
> >  		if (!entity->ops || !entity->ops->link_validate)
> > @@ -509,20 +513,23 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
> >  	media_graph_walk_start(graph, pad_err);
> >  
> >  	while ((pad_err = media_graph_walk_next(graph))) {
> > -		struct media_entity *entity_err = pad_err->entity;
> > -
> > -		/* Sanity check for negative stream_count */
> > -		if (!WARN_ON_ONCE(entity_err->stream_count <= 0)) {
> > -			entity_err->stream_count--;
> > -			if (entity_err->stream_count == 0)
> > -				entity_err->pipe = NULL;
> > +		struct media_entity *entity = pad_err->entity;
> > +		struct media_pad *iter;
> > +
> > +		media_entity_for_each_pad(entity, iter) {
> > +			/* Sanity check for negative stream_count */
> > +			if (!WARN_ON_ONCE(iter->stream_count <= 0)) {
> > +				--iter->stream_count;
> > +				if (iter->stream_count == 0)
> > +					iter->pipe = NULL;
> > +			}
> >  		}
> >  
> >  		/*
> >  		 * We haven't increased stream_count further than this
> >  		 * so we quit here.
> >  		 */
> > -		if (pad_err == pad)
> > +		if (pad_err->entity == pad->entity)
> >  			break;
> >  	}
> >  
> > @@ -549,7 +556,7 @@ EXPORT_SYMBOL_GPL(media_pipeline_start);
> >  
> >  void __media_pipeline_stop(struct media_entity *entity)
> >  {
> > -	struct media_pipeline *pipe = entity->pipe;
> > +	struct media_pipeline *pipe = entity->pads->pipe;
> >  	struct media_graph *graph = &pipe->graph;
> >  	struct media_pad *pad;
> >  
> > @@ -564,12 +571,15 @@ void __media_pipeline_stop(struct media_entity *entity)
> >  
> >  	while ((pad = media_graph_walk_next(graph))) {
> >  		struct media_entity *entity = pad->entity;
> > -
> > -		/* Sanity check for negative stream_count */
> > -		if (!WARN_ON_ONCE(entity->stream_count <= 0)) {
> > -			entity->stream_count--;
> > -			if (entity->stream_count == 0)
> > -				entity->pipe = NULL;
> > +		struct media_pad *iter;
> > +
> > +		media_entity_for_each_pad(entity, iter) {
> > +			/* Sanity check for negative stream_count */
> > +			if (!WARN_ON_ONCE(iter->stream_count <= 0)) {
> > +				iter->stream_count--;
> > +				if (iter->stream_count == 0)
> > +					iter->pipe = NULL;
> > +			}
> >  		}
> >  	}
> >  
> > @@ -839,7 +849,7 @@ int __media_entity_setup_link(struct media_link *link, u32 flags)
> >  {
> >  	const u32 mask = MEDIA_LNK_FL_ENABLED;
> >  	struct media_device *mdev;
> > -	struct media_entity *source, *sink;
> > +	struct media_pad *source, *sink;
> >  	int ret = -EBUSY;
> >  
> >  	if (link == NULL)
> > @@ -855,8 +865,8 @@ int __media_entity_setup_link(struct media_link *link, u32 flags)
> >  	if (link->flags == flags)
> >  		return 0;
> >  
> > -	source = link->source->entity;
> > -	sink = link->sink->entity;
> > +	source = link->source;
> > +	sink = link->sink;
> >  
> >  	if (!(link->flags & MEDIA_LNK_FL_DYNAMIC) &&
> >  	    (source->stream_count || sink->stream_count))
> > diff --git a/drivers/media/platform/exynos4-is/fimc-isp.c b/drivers/media/platform/exynos4-is/fimc-isp.c
> > index a77c49b18511..563d07f630bc 100644
> > --- a/drivers/media/platform/exynos4-is/fimc-isp.c
> > +++ b/drivers/media/platform/exynos4-is/fimc-isp.c
> > @@ -223,7 +223,7 @@ static int fimc_isp_subdev_set_fmt(struct v4l2_subdev *sd,
> >  			}
> >  		}
> >  	} else {
> > -		if (sd->entity.stream_count == 0) {
> > +		if (sd->entity.pads->stream_count == 0) {
> >  			if (fmt->pad == FIMC_ISP_SD_PAD_SINK) {
> >  				struct v4l2_subdev_format format = *fmt;
> >  
> > diff --git a/drivers/media/platform/exynos4-is/fimc-lite.c b/drivers/media/platform/exynos4-is/fimc-lite.c
> > index fe20af3a7178..56773a0be8d9 100644
> > --- a/drivers/media/platform/exynos4-is/fimc-lite.c
> > +++ b/drivers/media/platform/exynos4-is/fimc-lite.c
> > @@ -1070,7 +1070,7 @@ static int fimc_lite_subdev_set_fmt(struct v4l2_subdev *sd,
> >  	mutex_lock(&fimc->lock);
> >  
> >  	if ((atomic_read(&fimc->out_path) == FIMC_IO_ISP &&
> > -	    sd->entity.stream_count > 0) ||
> > +	    sd->entity.pads->stream_count > 0) ||
> >  	    (atomic_read(&fimc->out_path) == FIMC_IO_DMA &&
> >  	    vb2_is_busy(&fimc->vb_queue))) {
> >  		mutex_unlock(&fimc->lock);
> > diff --git a/drivers/media/platform/omap3isp/isp.c b/drivers/media/platform/omap3isp/isp.c
> > index a6bb7d9bf75f..4ef623f10a44 100644
> > --- a/drivers/media/platform/omap3isp/isp.c
> > +++ b/drivers/media/platform/omap3isp/isp.c
> > @@ -930,7 +930,7 @@ static int isp_pipeline_is_last(struct media_entity *me)
> >  	struct isp_pipeline *pipe;
> >  	struct media_pad *pad;
> >  
> > -	if (!me->pipe)
> > +	if (!me->pads->pipe)
> >  		return 0;
> >  	pipe = to_isp_pipeline(me);
> >  	if (pipe->stream_state == ISP_PIPELINE_STREAM_STOPPED)
> > diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
> > index 49cde04bfb21..f3bee79cdd85 100644
> > --- a/drivers/media/platform/omap3isp/ispvideo.c
> > +++ b/drivers/media/platform/omap3isp/ispvideo.c
> > @@ -1094,7 +1094,7 @@ isp_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
> >  	/* Start streaming on the pipeline. No link touching an entity in the
> >  	 * pipeline can be activated or deactivated once streaming is started.
> >  	 */
> > -	pipe = video->video.entity.pipe
> > +	pipe = video->video.entity.pads->pipe
> >  	     ? to_isp_pipeline(&video->video.entity) : &video->pipe;
> >  
> >  	ret = media_entity_enum_init(&pipe->ent_enum, &video->isp->media_dev);
> > diff --git a/drivers/media/platform/omap3isp/ispvideo.h b/drivers/media/platform/omap3isp/ispvideo.h
> > index a0908670c0cf..4c9c5b719ec5 100644
> > --- a/drivers/media/platform/omap3isp/ispvideo.h
> > +++ b/drivers/media/platform/omap3isp/ispvideo.h
> > @@ -100,7 +100,7 @@ struct isp_pipeline {
> >  };
> >  
> >  #define to_isp_pipeline(__e) \
> > -	container_of((__e)->pipe, struct isp_pipeline, pipe)
> > +	container_of((__e)->pads->pipe, struct isp_pipeline, pipe)
> >  
> >  static inline int isp_pipeline_ready(struct isp_pipeline *pipe)
> >  {
> > diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
> > index cb3025992817..07ec008aacc4 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-core.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-core.c
> > @@ -132,13 +132,17 @@ static int rvin_group_link_notify(struct media_link *link, u32 flags,
> >  		return 0;
> >  
> >  	/*
> > -	 * Don't allow link changes if any entity in the graph is
> > -	 * streaming, modifying the CHSEL register fields can disrupt
> > -	 * running streams.
> > +	 * Don't allow link changes if any stream in the graph is active as
> > +	 * modifying the CHSEL register fields can disrupt running streams.
> >  	 */
> > -	media_device_for_each_entity(entity, &group->mdev)
> > -		if (entity->stream_count)
> > -			return -EBUSY;
> > +	media_device_for_each_entity(entity, &group->mdev) {
> > +		struct media_pad *iter;
> > +
> > +		media_entity_for_each_pad(entity, iter) {
> > +			if (iter->stream_count)
> > +				return -EBUSY;
> > +		}
> 
> Would it be enough to test entity->pads->stream_count, as the stream
> count should be the same for all pads ?

For now, yes. But once routes are added, it wouldn't be enough anymore,
would it?

> 
> > +	}
> >  
> >  	mutex_lock(&group->lock);
> >  
> > diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
> > index f30dafbdf61c..7994262c9b63 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> > @@ -1231,7 +1231,7 @@ static int rvin_set_stream(struct rvin_dev *vin, int on)
> >  	 */
> >  	mdev = vin->vdev.entity.graph_obj.mdev;
> >  	mutex_lock(&mdev->graph_mutex);
> > -	pipe = sd->entity.pipe ? sd->entity.pipe : &vin->vdev.pipe;
> > +	pipe = sd->entity.pads->pipe ? sd->entity.pads->pipe : &vin->vdev.pipe;
> >  	ret = __media_pipeline_start(&vin->vdev.entity, pipe);
> >  	mutex_unlock(&mdev->graph_mutex);
> >  	if (ret)
> > diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
> > index 8df3c43aecbe..7fa0467dddde 100644
> > --- a/drivers/media/platform/xilinx/xilinx-dma.c
> > +++ b/drivers/media/platform/xilinx/xilinx-dma.c
> > @@ -403,7 +403,7 @@ static int xvip_dma_start_streaming(struct vb2_queue *vq, unsigned int count)
> >  	 * Use the pipeline object embedded in the first DMA object that starts
> >  	 * streaming.
> >  	 */
> > -	pipe = dma->video.entity.pipe
> > +	pipe = dma->video.entity.pads->pipe
> >  	     ? to_xvip_pipeline(&dma->video.entity) : &dma->pipe;
> >  
> >  	ret = media_pipeline_start(&dma->video.entity, &pipe->pipe);
> > diff --git a/drivers/media/platform/xilinx/xilinx-dma.h b/drivers/media/platform/xilinx/xilinx-dma.h
> > index 2378bdae57ae..69ced71a5696 100644
> > --- a/drivers/media/platform/xilinx/xilinx-dma.h
> > +++ b/drivers/media/platform/xilinx/xilinx-dma.h
> > @@ -47,7 +47,7 @@ struct xvip_pipeline {
> >  
> >  static inline struct xvip_pipeline *to_xvip_pipeline(struct media_entity *e)
> >  {
> > -	return container_of(e->pipe, struct xvip_pipeline, pipe);
> > +	return container_of(e->pads->pipe, struct xvip_pipeline, pipe);
> >  }
> >  
> >  /**
> > diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
> > index 5128915a5d6f..04b7c6bdcd85 100644
> > --- a/drivers/staging/media/imx/imx-media-utils.c
> > +++ b/drivers/staging/media/imx/imx-media-utils.c
> > @@ -913,7 +913,7 @@ int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
> >  			__media_pipeline_stop(entity);
> >  	} else {
> >  		v4l2_subdev_call(sd, video, s_stream, 0);
> > -		if (entity->pipe)
> > +		if (entity->pads->pipe)
> >  			__media_pipeline_stop(entity);
> >  	}
> >  
> > diff --git a/drivers/staging/media/omap4iss/iss.c b/drivers/staging/media/omap4iss/iss.c
> > index 085397045b36..4f4573219337 100644
> > --- a/drivers/staging/media/omap4iss/iss.c
> > +++ b/drivers/staging/media/omap4iss/iss.c
> > @@ -543,7 +543,7 @@ static int iss_pipeline_is_last(struct media_entity *me)
> >  	struct iss_pipeline *pipe;
> >  	struct media_pad *pad;
> >  
> > -	if (!me->pipe)
> > +	if (!me->pads->pipe)
> >  		return 0;
> >  	pipe = to_iss_pipeline(me);
> >  	if (pipe->stream_state == ISS_PIPELINE_STREAM_STOPPED)
> > diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
> > index 9f3ff5a37d90..9d7bf8c85558 100644
> > --- a/drivers/staging/media/omap4iss/iss_video.c
> > +++ b/drivers/staging/media/omap4iss/iss_video.c
> > @@ -870,7 +870,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
> >  	 * Start streaming on the pipeline. No link touching an entity in the
> >  	 * pipeline can be activated or deactivated once streaming is started.
> >  	 */
> > -	pipe = pad->entity->pipe
> > +	pipe = pad->pipe
> >  	     ? to_iss_pipeline(pad->entity) : &video->pipe;
> >  	pipe->external = NULL;
> >  	pipe->external_rate = 0;
> > diff --git a/drivers/staging/media/omap4iss/iss_video.h b/drivers/staging/media/omap4iss/iss_video.h
> > index 526281bf0051..9b8ec27bf87d 100644
> > --- a/drivers/staging/media/omap4iss/iss_video.h
> > +++ b/drivers/staging/media/omap4iss/iss_video.h
> > @@ -91,7 +91,7 @@ struct iss_pipeline {
> >  };
> >  
> >  #define to_iss_pipeline(__e) \
> > -	container_of((__e)->pipe, struct iss_pipeline, pipe)
> > +	container_of((__e)->pads->pipe, struct iss_pipeline, pipe)
> >  
> >  static inline int iss_pipeline_ready(struct iss_pipeline *pipe)
> >  {
> > diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> > index 52b1a1cab57a..b8f94662526c 100644
> > --- a/include/media/media-entity.h
> > +++ b/include/media/media-entity.h
> > @@ -180,15 +180,24 @@ enum media_pad_signal_type {
> >   *
> >   * @graph_obj:	Embedded structure containing the media object common data
> >   * @entity:	Entity this pad belongs to
> > + * @pipe:	Pipeline this pad belongs to.
> > + * @stream_count: Stream count for the pad.
> 
> No period at the end of the lines.
> 
> >   * @index:	Pad index in the entity pads array, numbered from 0 to n
> >   * @sig_type:	Type of the signal inside a media pad
> >   * @flags:	Pad flags, as defined in
> >   *		:ref:`include/uapi/linux/media.h <media_header>`
> >   *		(seek for ``MEDIA_PAD_FL_*``)
> > + * .. note::
> > + *
> > + *    @stream_count reference count must never be negative, but is a signed
> > + *    integer on purpose: a simple ``WARN_ON(<0)`` check can be used to
> > + *    detect reference count bugs that would make them negative.
> 
> s/them/it/
> 
> >   */
> >  struct media_pad {
> >  	struct media_gobj graph_obj;	/* must be first field in struct */
> >  	struct media_entity *entity;
> > +	struct media_pipeline *pipe;
> > +	int stream_count;
> >  	u16 index;
> >  	enum media_pad_signal_type sig_type;
> >  	unsigned long flags;
> > @@ -267,9 +276,7 @@ enum media_entity_type {
> >   * @pads:	Pads array with the size defined by @num_pads.
> >   * @links:	List of data links.
> >   * @ops:	Entity operations.
> > - * @stream_count: Stream count for the entity.
> >   * @use_count:	Use count for the entity.
> > - * @pipe:	Pipeline this entity belongs to.
> >   * @info:	Union with devnode information.  Kept just for backward
> >   *		compatibility.
> >   * @info.dev:	Contains device major and minor info.
> > @@ -282,10 +289,9 @@ enum media_entity_type {
> >   *
> >   * .. note::
> >   *
> > - *    @stream_count and @use_count reference counts must never be
> > - *    negative, but are signed integers on purpose: a simple ``WARN_ON(<0)``
> > - *    check can be used to detect reference count bugs that would make them
> > - *    negative.
> > + *    @use_count reference count must never be negative, but is a signed
> > + *    integer on purpose: a simple ``WARN_ON(<0)`` check can be used to
> > + *    detect reference count bugs that would make them negative.
> 
> s/them/it/
> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
> >   */
> >  struct media_entity {
> >  	struct media_gobj graph_obj;	/* must be first field in struct */
> > @@ -304,11 +310,8 @@ struct media_entity {
> >  
> >  	const struct media_entity_operations *ops;
> >  
> > -	int stream_count;
> >  	int use_count;
> >  
> > -	struct media_pipeline *pipe;
> > -
> >  	union {
> >  		struct {
> >  			u32 major;
> 

-- 
Sakari Ailus

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 11/24] media: entity: Skip link validation for pads to which there is no route to
  2021-04-18 18:06   ` Laurent Pinchart
@ 2021-04-20 11:41     ` Sakari Ailus
  2021-04-23 12:37       ` Tomi Valkeinen
  0 siblings, 1 reply; 72+ messages in thread
From: Sakari Ailus @ 2021-04-20 11:41 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: Tomi Valkeinen, linux-media, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Laurent,

On Sun, Apr 18, 2021 at 09:06:11PM +0300, Laurent Pinchart wrote:
> Hi Tomi and Sakari,
> 
> Thank you for the patch.
> 
> There's an extra "to" in the subject line.
> 
> On Thu, Apr 15, 2021 at 04:04:37PM +0300, Tomi Valkeinen wrote:
> > From: Sakari Ailus <sakari.ailus@linux.intel.com>
> > 
> > Links are validated along the pipeline which is about to start streaming.
> > Not all the pads in entities that are traversed along that pipeline are
> > part of the pipeline, however. Skip the link validation for such pads,
> > and while at there rename "other_pad" to "local_pad" to convey the fact
> > the route to be checked is internal to the entity.
> 
> Both "pad" and "local_pad" are local. I would have kept the "other_pad"

The pad that in the remote entity is not local. The other one could be
called remote_pad though.

> variable name.
> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Thanks!

> 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> > ---
> >  drivers/media/mc/mc-entity.c | 13 +++++++++----
> >  1 file changed, 9 insertions(+), 4 deletions(-)
> > 
> > diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
> > index 28d7fd254c77..fe6cb743c85c 100644
> > --- a/drivers/media/mc/mc-entity.c
> > +++ b/drivers/media/mc/mc-entity.c
> > @@ -482,12 +482,17 @@ __must_check int __media_pipeline_start(struct media_pad *pad,
> >  		bitmap_fill(has_no_links, entity->num_pads);
> >  
> >  		list_for_each_entry(link, &entity->links, list) {
> > -			struct media_pad *other_pad =
> > +			struct media_pad *local_pad =
> >  				link->sink->entity == entity ?
> >  				link->sink : link->source;
> >  
> > +			/* Ignore pads to which there is no route. */
> > +			if (!media_entity_has_route(entity, pad->index,
> > +						    local_pad->index))
> > +				continue;
> > +
> >  			/* Mark that a pad is connected by a link. */
> > -			bitmap_clear(has_no_links, other_pad->index, 1);
> > +			bitmap_clear(has_no_links, local_pad->index, 1);
> >  
> >  			/*
> >  			 * Pads that either do not need to connect or
> > @@ -496,13 +501,13 @@ __must_check int __media_pipeline_start(struct media_pad *pad,
> >  			 */
> >  			if (!(pad->flags & MEDIA_PAD_FL_MUST_CONNECT) ||
> >  			    link->flags & MEDIA_LNK_FL_ENABLED)
> > -				bitmap_set(active, other_pad->index, 1);
> > +				bitmap_set(active, local_pad->index, 1);
> >  
> >  			/*
> >  			 * Link validation will only take place for
> >  			 * sink ends of the link that are enabled.
> >  			 */
> > -			if (link->sink != other_pad ||
> > +			if (link->sink != local_pad ||
> >  			    !(link->flags & MEDIA_LNK_FL_ENABLED))
> >  				continue;
> >  
> 

-- 
Sakari Ailus

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 12/24] media: entity: Add an iterator helper for connected pads
  2021-04-18 18:20   ` Laurent Pinchart
@ 2021-04-20 11:48     ` Sakari Ailus
  2021-04-29  1:33       ` Laurent Pinchart
  0 siblings, 1 reply; 72+ messages in thread
From: Sakari Ailus @ 2021-04-20 11:48 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: Tomi Valkeinen, linux-media, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Laurent,

Thanks for the review.

On Sun, Apr 18, 2021 at 09:20:03PM +0300, Laurent Pinchart wrote:
> Hi Tomi and Sakari,
> 
> Thank you for the patch.
> 
> On Thu, Apr 15, 2021 at 04:04:38PM +0300, Tomi Valkeinen wrote:
> > From: Sakari Ailus <sakari.ailus@linux.intel.com>
> > 
> > Add a helper macro for iterating over pads that are connected through
> > enabled routes. This can be used to find all the connected pads within an
> > entity, for instance starting from the pad which has been obtained during
> > the graph walk.
> > 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > 
> > - Make __media_entity_next_routed_pad() return NULL and adjust the
> >   iterator to handle that
> > Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> > ---
> >  include/media/media-entity.h | 27 +++++++++++++++++++++++++++
> >  1 file changed, 27 insertions(+)
> > 
> > diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> > index 73de1c335e4e..edd6f60ed6b4 100644
> > --- a/include/media/media-entity.h
> > +++ b/include/media/media-entity.h
> > @@ -916,6 +916,33 @@ __must_check int media_graph_walk_init(
> >  bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
> >  			    unsigned int pad1);
> >  
> > +static inline struct media_pad *__media_entity_next_routed_pad(
> > +	struct media_pad *start, struct media_pad *iter)
> > +{
> > +	struct media_entity *entity = start->entity;
> > +
> > +	for (; iter < &entity->pads[entity->num_pads]; iter++)
> > +		if (media_entity_has_route(entity, start->index, iter->index))
> > +			return iter;
> 
> I'd use curly braces.
> 
> > +
> > +	return NULL;
> > +}
> 
> Does this need to be inlined ?

I guess it doesn't have to. It's used inside loops and it's rather small so
I think it should be fine that way.

> 
> > +
> > +/**
> > + * media_entity_for_each_routed_pad - Iterate over entity pads connected by routes
> 
> "routed" sounds a bit weird. Would media_entity_for_each_connected_pad()
> be a better name ?

"Connected" is often used in context of links. We're dealing with routes
here, so I thought "routed" is appropriate to avoid confusion.

> 
> > + *
> > + * @start: The stating pad
> 
> s/stating/starting/
> 
> > + * @iter: The iterator pad
> > + *
> > + * Iterate over all pads connected through routes from a given pad
> 
> "from the @start pad"
> 
> > + * within an entity. The iteration will include the starting pad itself.
> 
> s/starting/@start/
> 
> I wonder if it wouldn't be more logical to not include the start pad.
> That wouldn't match the current usage patterns, which would need to be
> adapted accordingly, but I'm worried that including the start pad will
> lead to annoying bugs in the future. Maybe I worry too much.

The aim here is to find all pads that are routed to another pad within the
same entity. If you remove the start pad, it becomes a task harder than
difficult.

> 
> And now that I reread the patch, I also wonder if "start" is a good
> name, as it implies we start the enumeration from a given pad, while we
> enumerate all pads connected to a given pad. I'm not sure what a better
> name would be though, maybe just pad ?

There are two pads here. Therefore explicitly calling them something else
makes sense IMO.

> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

I agree on the comments I didn't reply to.

Thank you!

> 
> > + */
> > +#define media_entity_for_each_routed_pad(start, iter)			\
> > +	for (iter = __media_entity_next_routed_pad(			\
> > +		     start, (start)->entity->pads);			\
> > +	     iter != NULL;						\
> > +	     iter = __media_entity_next_routed_pad(start, iter + 1))
> > +
> >  /**
> >   * media_graph_walk_cleanup - Release resources used by graph walk.
> >   *
> 

-- 
Kind regards,

Sakari Ailus

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 15/24] v4l: Add bus type to frame descriptors
  2021-04-18 19:23   ` Laurent Pinchart
@ 2021-04-20 11:50     ` Sakari Ailus
  2021-04-22 12:30       ` Tomi Valkeinen
  0 siblings, 1 reply; 72+ messages in thread
From: Sakari Ailus @ 2021-04-20 11:50 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: Tomi Valkeinen, linux-media, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Laurent,

On Sun, Apr 18, 2021 at 10:23:35PM +0300, Laurent Pinchart wrote:
> Hi Tomi and Sakari,
> 
> Thank you for the patch.
> 
> On Thu, Apr 15, 2021 at 04:04:41PM +0300, Tomi Valkeinen wrote:
> > From: Sakari Ailus <sakari.ailus@linux.intel.com>
> > 
> > Add the media bus type to the frame descriptor. CSI-2 specific
> > information will be added in next patch to the frame descriptor.
> 
> I'd squash the next patch with this one.
> 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > 
> > - Make the bus type a named enum
> > Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> > ---
> >  include/media/v4l2-subdev.h | 9 +++++++++
> >  1 file changed, 9 insertions(+)
> > 
> > diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> > index d0e9a5bdb08b..85977abbea46 100644
> > --- a/include/media/v4l2-subdev.h
> > +++ b/include/media/v4l2-subdev.h
> > @@ -340,12 +340,21 @@ struct v4l2_mbus_frame_desc_entry {
> >  
> >  #define V4L2_FRAME_DESC_ENTRY_MAX	4
> >  
> > +enum v4l2_mbus_frame_desc_type {
> > +	V4L2_MBUS_FRAME_DESC_TYPE_PLATFORM,
> > +	V4L2_MBUS_FRAME_DESC_TYPE_PARALLEL,
> > +	V4L2_MBUS_FRAME_DESC_TYPE_CCP2,
> > +	V4L2_MBUS_FRAME_DESC_TYPE_CSI2,
> > +};
> 
> This should be documented. In particular, I have no idea what
> V4L2_MBUS_FRAME_DESC_TYPE_PLATFORM is. I also wonder if we shouldn't
> drop CCP2 (at least for now), does anyone use that anymore ?

I guess we don't need one here, not now at least.

I agree on the documentation.

> 
> > +
> >  /**
> >   * struct v4l2_mbus_frame_desc - media bus data frame description
> > + * @type: type of the bus (enum v4l2_mbus_frame_desc_type)
> >   * @entry: frame descriptors array
> >   * @num_entries: number of entries in @entry array
> >   */
> >  struct v4l2_mbus_frame_desc {
> > +	enum v4l2_mbus_frame_desc_type type;
> >  	struct v4l2_mbus_frame_desc_entry entry[V4L2_FRAME_DESC_ENTRY_MAX];
> >  	unsigned short num_entries;
> >  };
> 

-- 
Sakari Ailus

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 16/24] v4l: Add CSI-2 bus configuration to frame descriptors
  2021-04-18 19:24   ` Laurent Pinchart
@ 2021-04-20 16:32     ` Sakari Ailus
  0 siblings, 0 replies; 72+ messages in thread
From: Sakari Ailus @ 2021-04-20 16:32 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: Tomi Valkeinen, linux-media, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Laurent,

On Sun, Apr 18, 2021 at 10:24:54PM +0300, Laurent Pinchart wrote:
> Hi Tomi and Sakari,
> 
> Thank you for the patch.
> 
> On Thu, Apr 15, 2021 at 04:04:42PM +0300, Tomi Valkeinen wrote:
> > From: Sakari Ailus <sakari.ailus@linux.intel.com>
> > 
> > Add CSI-2 bus specific configuration to the frame descriptors. This allows
> > obtaining the virtual channel and data type information for each stream
> > the transmitter is sending.
> > 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> > ---
> >  include/media/v4l2-subdev.h | 16 ++++++++++++++++
> >  1 file changed, 16 insertions(+)
> > 
> > diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> > index 85977abbea46..30ec011d31e3 100644
> > --- a/include/media/v4l2-subdev.h
> > +++ b/include/media/v4l2-subdev.h
> > @@ -308,6 +308,17 @@ struct v4l2_subdev_audio_ops {
> >  	int (*s_stream)(struct v4l2_subdev *sd, int enable);
> >  };
> >  
> > +/**
> > + * struct v4l2_mbus_frame_desc_entry_csi2
> > + *
> > + * @channel: CSI-2 virtual channel
> 
> Maybe s/channel/virtual_channel/ ? Or vc and dt ?

Either seems good.

It's inherently about CSI-2 so maybe vc and dt below?

> 
> > + * @data_type: CSI-2 data type ID
> > + */
> > +struct v4l2_mbus_frame_desc_entry_csi2 {
> > +	u8 channel;
> > +	u8 data_type;
> > +};
> > +
> >  /**
> >   * enum v4l2_mbus_frame_desc_flags - media bus frame description flags
> >   *
> > @@ -331,11 +342,16 @@ enum v4l2_mbus_frame_desc_flags {
> >   *		%FRAME_DESC_FL_BLOB is not set.
> >   * @length:	number of octets per frame, valid if @flags
> >   *		%V4L2_MBUS_FRAME_DESC_FL_LEN_MAX is set.
> > + * @bus:	Bus specific frame descriptor parameters
> 
> s/Bus specific/Bus-specific/
> 
> > + * @bus.csi2:	CSI-2 specific bus configuration
> 
> Ditto.
> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Thanks!

-- 
Sakari Ailus

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 18/24] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations
  2021-04-15 13:04 ` [PATCH v5 18/24] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations Tomi Valkeinen
  2021-04-18 18:32   ` Laurent Pinchart
@ 2021-04-20 16:35   ` Sakari Ailus
  1 sibling, 0 replies; 72+ messages in thread
From: Sakari Ailus @ 2021-04-20 16:35 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: linux-media, Jacopo Mondi, Laurent Pinchart,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
	Michal Simek

Moi,

Thanks for the update.

On Thu, Apr 15, 2021 at 04:04:44PM +0300, Tomi Valkeinen wrote:
> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
> Add support for subdev internal routing. A route is defined as a single
> stream from a sink pad to a source pad.
> 
> The userspace can configure the routing via two new ioctls,
> VIDIOC_SUBDEV_G_ROUTING and VIDIOC_SUBDEV_S_ROUTING, and subdevs can
> implement the functionality with v4l2_subdev_pad_ops.get_routing() and
> v4l2_subdev_pad_ops.set_routing().
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Michal Simek <michal.simek@xilinx.com>
> 
> - Add sink and source streams for multiplexed links
> - Copy the argument back in case of an error. This is needed to let the
>   caller know the number of routes.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> 
> - Expand and refine documentation.
> - Make the 'routes' pointer a __u64 __user pointer so that a compat32
>   version of the ioctl is not required.
> - Add struct v4l2_subdev_krouting to be used for subdevice operations.
> 
> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> 
> - Fix typecasing warnings
> - Check sink & source pad types
> - Add 'which' field
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  drivers/media/v4l2-core/v4l2-ioctl.c  | 25 ++++++++++++++-
>  drivers/media/v4l2-core/v4l2-subdev.c | 45 +++++++++++++++++++++++++++
>  include/media/v4l2-subdev.h           | 24 ++++++++++++++
>  include/uapi/linux/v4l2-subdev.h      | 44 ++++++++++++++++++++++++++
>  4 files changed, 137 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 6a5d1c6d11d6..f5732962753f 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -16,6 +16,7 @@
>  #include <linux/kernel.h>
>  #include <linux/version.h>
>  
> +#include <linux/v4l2-subdev.h>
>  #include <linux/videodev2.h>
>  
>  #include <media/v4l2-common.h>
> @@ -3108,6 +3109,21 @@ static int check_array_args(unsigned int cmd, void *parg, size_t *array_size,
>  		ret = 1;
>  		break;
>  	}
> +
> +	case VIDIOC_SUBDEV_G_ROUTING:
> +	case VIDIOC_SUBDEV_S_ROUTING: {
> +		struct v4l2_subdev_routing *route = parg;
> +
> +		if (route->num_routes > 256)
> +			return -EINVAL;
> +
> +		*user_ptr = u64_to_user_ptr(route->routes);
> +		*kernel_ptr = (void **)&route->routes;
> +		*array_size = sizeof(struct v4l2_subdev_route)
> +			    * route->num_routes;
> +		ret = 1;
> +		break;
> +	}
>  	}
>  
>  	return ret;
> @@ -3369,8 +3385,15 @@ video_usercopy(struct file *file, unsigned int orig_cmd, unsigned long arg,
>  	/*
>  	 * Some ioctls can return an error, but still have valid
>  	 * results that must be returned.
> +	 *
> +	 * FIXME: subdev IOCTLS are partially handled here and partially in
> +	 * v4l2-subdev.c and the 'always_copy' flag can only be set for IOCTLS
> +	 * defined here as part of the 'v4l2_ioctls' array. As
> +	 * VIDIOC_SUBDEV_G_ROUTING needs to return results to applications even
> +	 * in case of failure, but it is not defined here as part of the
> +	 * 'v4l2_ioctls' array, insert an ad-hoc check to address that.
>  	 */
> -	if (err < 0 && !always_copy)
> +	if (err < 0 && !always_copy && cmd != VIDIOC_SUBDEV_G_ROUTING)
>  		goto out;
>  
>  out_array_args:
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 956dafab43d4..95a4c3091fa6 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -681,6 +681,51 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg)
>  	case VIDIOC_SUBDEV_QUERYSTD:
>  		return v4l2_subdev_call(sd, video, querystd, arg);
>  
> +	case VIDIOC_SUBDEV_G_ROUTING: {
> +		struct v4l2_subdev_routing *routing = arg;
> +		struct v4l2_subdev_krouting krouting = {
> +			.which = routing->which,
> +			.num_routes = routing->num_routes,
> +			.routes = (struct v4l2_subdev_route *)(uintptr_t)routing->routes,

Please wrap to stay under 80 characters per line. Same below.

> +		};
> +		int ret;
> +
> +		ret = v4l2_subdev_call(sd, pad, get_routing, &krouting);
> +		if (ret)
> +			return ret;
> +
> +		routing->num_routes = krouting.num_routes;
> +
> +		return 0;
> +	}
> +
> +	case VIDIOC_SUBDEV_S_ROUTING: {
> +		struct v4l2_subdev_routing *routing = arg;
> +		struct v4l2_subdev_route *route = (struct v4l2_subdev_route *)(uintptr_t)
> +						  routing->routes;
> +		struct v4l2_subdev_krouting krouting = {};
> +		unsigned int i;
> +
> +		if (routing->which != V4L2_SUBDEV_FORMAT_TRY && ro_subdev)
> +			return -EPERM;
> +
> +		for (i = 0; i < routing->num_routes; ++i) {
> +			if (route[i].sink_pad >= sd->entity.num_pads ||
> +			    route[i].source_pad >= sd->entity.num_pads)
> +				return -EINVAL;
> +
> +			if (!(sd->entity.pads[route[i].sink_pad].flags & MEDIA_PAD_FL_SINK) ||
> +			    !(sd->entity.pads[route[i].source_pad].flags & MEDIA_PAD_FL_SOURCE))
> +				return -EINVAL;
> +		}
> +
> +		krouting.which = routing->which;
> +		krouting.num_routes = routing->num_routes;
> +		krouting.routes = route;
> +
> +		return v4l2_subdev_call(sd, pad, set_routing, &krouting);
> +	}
> +
>  	default:
>  		return v4l2_subdev_call(sd, core, ioctl, cmd, arg);
>  	}
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index 436d0445aafd..3826ab918731 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -650,6 +650,22 @@ struct v4l2_subdev_pad_config {
>  	struct v4l2_rect try_compose;
>  };
>  
> +/**
> + * struct v4l2_subdev_krouting - subdev routing table
> + *
> + * @which: format type (from enum v4l2_subdev_format_whence)
> + * @routes: &struct v4l2_subdev_route
> + * @num_routes: number of routes
> + *
> + * This structure is used to translate argument received from
> + * VIDIOC_SUBDEV_G/S_ROUTING() ioctl to sudev device drivers operations.
> + */
> +struct v4l2_subdev_krouting {
> +	u32 which;
> +	struct v4l2_subdev_route *routes;
> +	unsigned int num_routes;
> +};
> +
>  /**
>   * struct v4l2_subdev_pad_ops - v4l2-subdev pad level operations
>   *
> @@ -711,6 +727,10 @@ struct v4l2_subdev_pad_config {
>   *		     applied to the hardware. The operation shall fail if the
>   *		     pad index it has been called on is not valid or in case of
>   *		     unrecoverable failures.
> + *
> + * @get_routing: get the subdevice routing table.
> + * @set_routing: enable or disable data connection routes described in the
> + *		 subdevice routing table.
>   */
>  struct v4l2_subdev_pad_ops {
>  	int (*init_cfg)(struct v4l2_subdev *sd,
> @@ -755,6 +775,10 @@ struct v4l2_subdev_pad_ops {
>  			       struct v4l2_mbus_config *config);
>  	int (*set_mbus_config)(struct v4l2_subdev *sd, unsigned int pad,
>  			       struct v4l2_mbus_config *config);
> +	int (*get_routing)(struct v4l2_subdev *sd,
> +			   struct v4l2_subdev_krouting *route);
> +	int (*set_routing)(struct v4l2_subdev *sd,
> +			   struct v4l2_subdev_krouting *route);
>  };
>  
>  /**
> diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h
> index 658106f5b5dc..f2a17cbd1e9a 100644
> --- a/include/uapi/linux/v4l2-subdev.h
> +++ b/include/uapi/linux/v4l2-subdev.h
> @@ -188,6 +188,48 @@ struct v4l2_subdev_capability {
>  /* The v4l2 sub-device video device node is registered in read-only mode. */
>  #define V4L2_SUBDEV_CAP_RO_SUBDEV		0x00000001
>  
> +#define V4L2_SUBDEV_ROUTE_FL_ACTIVE		BIT(0)
> +#define V4L2_SUBDEV_ROUTE_FL_IMMUTABLE		BIT(1)
> +
> +/**
> + * struct v4l2_subdev_route - A route inside a subdev
> + *
> + * @sink_pad: the sink pad index
> + * @sink_stream: the sink stream identifier
> + * @source_pad: the source pad index
> + * @source_stream: the source stream identifier
> + * @flags: route flags:
> + *
> + *	V4L2_SUBDEV_ROUTE_FL_ACTIVE: Is the route active? An active
> + *	route will start when streaming is enabled on a video node.
> + *	Set by the user.
> + *
> + *	V4L2_SUBDEV_ROUTE_FL_IMMUTABLE: Is the route immutable, i.e.
> + *	can it be activated and inactivated? Set by the driver.
> + */
> +struct v4l2_subdev_route {
> +	__u32 sink_pad;
> +	__u32 sink_stream;
> +	__u32 source_pad;
> +	__u32 source_stream;
> +	__u32 flags;
> +	__u32 reserved[5];
> +};
> +
> +/**
> + * struct v4l2_subdev_routing - Subdev routing information
> + *
> + * @which: format type (from enum v4l2_subdev_format_whence)
> + * @routes: pointer to the routes array
> + * @num_routes: the total number of routes in the routes array
> + */
> +struct v4l2_subdev_routing {
> +	__u32 which;
> +	__u64 routes;
> +	__u32 num_routes;
> +	__u32 reserved[5];
> +};
> +
>  /* Backwards compatibility define --- to be removed */
>  #define v4l2_subdev_edid v4l2_edid
>  
> @@ -215,5 +257,7 @@ struct v4l2_subdev_capability {
>  #define VIDIOC_SUBDEV_ENUM_DV_TIMINGS		_IOWR('V', 98, struct v4l2_enum_dv_timings)
>  #define VIDIOC_SUBDEV_QUERY_DV_TIMINGS		_IOR('V', 99, struct v4l2_dv_timings)
>  #define VIDIOC_SUBDEV_DV_TIMINGS_CAP		_IOWR('V', 100, struct v4l2_dv_timings_cap)
> +#define VIDIOC_SUBDEV_G_ROUTING			_IOWR('V', 38, struct v4l2_subdev_routing)
> +#define VIDIOC_SUBDEV_S_ROUTING			_IOWR('V', 39, struct v4l2_subdev_routing)
>  
>  #endif
> -- 
> 2.25.1
> 

-- 
Sakari Ailus

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 20/24] v4l: mc: Add an S_ROUTING helper function for power state changes
  2021-04-18 18:55   ` Laurent Pinchart
@ 2021-04-20 16:41     ` Sakari Ailus
  0 siblings, 0 replies; 72+ messages in thread
From: Sakari Ailus @ 2021-04-20 16:41 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: Tomi Valkeinen, linux-media, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Laurent,

On Sun, Apr 18, 2021 at 09:55:40PM +0300, Laurent Pinchart wrote:
> Hi Tomi and Sakari,
> 
> Thank you for the patch.
> 
> On Thu, Apr 15, 2021 at 04:04:46PM +0300, Tomi Valkeinen wrote:
> > From: Sakari Ailus <sakari.ailus@linux.intel.com>
> > 
> > With the addition of the has_route() media entity operation, all pads of an
> > entity are no longer interconnected. The S_ROUTING IOCTL for sub-devices can
> > be used to enable and disable routes for an entity. The consequence is that
> > the routing information has to be taken into account in use count
> > calculation: disabling a route has a very similar effect on use counts as
> > has disabling a link.
> > 
> > Add a helper function for drivers implementing VIDIOC_SUBDEV_S_ROUTING
> > IOCTL to take the change into account.
> > 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > ---
> >  drivers/media/v4l2-core/v4l2-mc.c | 34 +++++++++++++++++++++++++++++++
> >  include/media/v4l2-mc.h           | 22 ++++++++++++++++++++
> >  2 files changed, 56 insertions(+)
> > 
> > diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
> > index 35d18ed89fa5..71acb389aa7b 100644
> > --- a/drivers/media/v4l2-core/v4l2-mc.c
> > +++ b/drivers/media/v4l2-core/v4l2-mc.c
> > @@ -588,3 +588,37 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
> >  	return ret;
> >  }
> >  EXPORT_SYMBOL_GPL(v4l2_pipeline_link_notify);
> > +
> > +int v4l2_subdev_routing_pm_use(struct media_entity *entity,
> > +			       struct v4l2_subdev_route *route)
> > +{
> > +	struct media_graph *graph =
> > +		&entity->graph_obj.mdev->pm_count_walk;
> > +	struct media_pad *source = &entity->pads[route->source_pad];
> > +	struct media_pad *sink = &entity->pads[route->sink_pad];
> > +	int source_use;
> > +	int sink_use;
> > +	int ret;
> > +
> > +	source_use = pipeline_pm_use_count(source, graph);
> > +	sink_use = pipeline_pm_use_count(sink, graph);
> > +
> > +	if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE)) {
> > +		/* Route disabled. */
> > +		pipeline_pm_power(source, -sink_use, graph);
> > +		pipeline_pm_power(sink, -source_use, graph);
> > +		return 0;
> > +	}
> > +
> > +	/* Route enabled. */
> > +	ret = pipeline_pm_power(source, sink_use, graph);
> > +	if (ret < 0)
> > +		return ret;
> > +
> > +	ret = pipeline_pm_power(sink, source_use, graph);
> > +	if (ret < 0)
> > +		pipeline_pm_power(source, -sink_use, graph);
> > +
> > +	return ret;
> > +}
> > +EXPORT_SYMBOL_GPL(v4l2_subdev_routing_pm_use);
> 
> .s_power() is getting deprecated for sensors. Sakari, should we move
> away from pipeline-based power management and drop this patch, or is it
> too early ?

Good question.

I've previously requested that instead of adding s_power support for bridge
and ISP drivers, sensor drivers used with them should be converted to
runtime PM instead. So proceeding along those lines, this code isn't
necessary. I'd actually drop it if there's no sound use case (I don't think
so).

> 
> > diff --git a/include/media/v4l2-mc.h b/include/media/v4l2-mc.h
> > index c181685923d5..ab8f4dc143aa 100644
> > --- a/include/media/v4l2-mc.h
> > +++ b/include/media/v4l2-mc.h
> > @@ -18,6 +18,7 @@
> >  /* We don't need to include pci.h or usb.h here */
> >  struct pci_dev;
> >  struct usb_device;
> > +struct v4l2_subdev_route;
> >  
> >  #ifdef CONFIG_MEDIA_CONTROLLER
> >  /**
> > @@ -184,6 +185,22 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);
> >  int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
> >  			      unsigned int notification);
> >  
> > +/**
> > + * v4l2_subdev_routing_pm_use - Handle power state changes due to S_ROUTING
> > + * @entity: The entity
> > + * @route: The new state of the route
> > + *
> > + * Propagate the use count across a route in a pipeline whenever the
> > + * route is enabled or disabled. The function is called before
> > + * changing the route state when enabling a route, and after changing
> > + * the route state when disabling a route.
> > + *
> > + * Return 0 on success or a negative error code on failure. Powering entities
> > + * off is assumed to never fail. This function will not fail for disconnection
> > + * events.
> 
> "disconnection events" make me think about hotplug. How about "will not
> fail when disabling a route" ?
> 
> > + */
> > +int v4l2_subdev_routing_pm_use(struct media_entity *entity,
> > +			       struct v4l2_subdev_route *route);
> >  #else /* CONFIG_MEDIA_CONTROLLER */
> >  
> >  static inline int v4l2_mc_create_media_graph(struct media_device *mdev)
> > @@ -219,5 +236,10 @@ static inline int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
> >  	return 0;
> >  }
> >  
> > +static inline int v4l2_subdev_routing_pm_use(struct media_entity *entity,
> > +					     struct v4l2_subdev_route *route)
> > +{
> > +	return 0;
> > +}
> 
> Missing blank line.
> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
> (assuming we want to keep the patch)
> 
> >  #endif /* CONFIG_MEDIA_CONTROLLER */
> >  #endif /* _V4L2_MC_H */
> 
> -- 
> Regards,
> 
> Laurent Pinchart

-- 
Sakari Ailus

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 00/24] v4l: subdev internal routing
  2021-04-18 17:32 ` Laurent Pinchart
@ 2021-04-21 12:57   ` Tomi Valkeinen
  2021-04-29  1:27     ` Laurent Pinchart
  0 siblings, 1 reply; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-21 12:57 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

On 18/04/2021 20:32, Laurent Pinchart wrote:
> Hi Tomi,
> 
> Nice to see a v5 of this plagued patch series :-)
> 
> On Thu, Apr 15, 2021 at 04:04:26PM +0300, Tomi Valkeinen wrote:
>> Hi,
>>
>> This is an RFC for subdev internal routing which is needed for
>> multiplexed streams support. I believe this is essentially a v5 of the
>> series, the v4 posted here:
>>
>> https://lore.kernel.org/linux-media/20190328200608.9463-1-jacopo+renesas@jmondi.org/
>>
>> Most of the patches are not changed (aside from fixing rebase issues
>> etc). The major changes in this version are:
>>
>> 1) Added 'which' field to the routing structs. It is currently not used,
>> as implementing it is not trivial. However, I think it's good to add it
>> to the uAPI now, and require the field to be set to
>> V4L2_SUBDEV_FORMAT_ACTIVE for now. See this RFC for an idea how this
>> could be implemented:
>>
>> https://lore.kernel.org/linux-media/20210409133659.389544-1-tomi.valkeinen@ideasonboard.com/
> 
> I've reviewed that, and I like it, but it's not straightforward to
> understand from that patch how you envision TRY to be implemented.

To be honest I don't have much at all experience with TRY. But, afaics, 
if we have means to store the TRY routes, and that is passed to the 
relevant ioctls in the subdev's, isn't that enough? It's up to the 
driver to implement the TRY functionality.

Although currently the S_ROUTING won't return anything to the userspace, 
it's supposed to either accept the routing table or return an error, 
whereas the S_FMT will do it's best to come up with a working setup and 
return it.

>> 2) No hardcoded maximum number of routes. Defining a maximum is not
>> possible, as there can be an arbitrary amount of routes per pad, and
>> there can be an arbitrary amount of pads per subdev. This series
>> allocates space for the routing table dynamically, which unfortunately
>> leads to not-just-a-few allocs and frees.
>>
>> 3) When searching for a format for a stream, the v4 looked for a
>> non-multiplexed pad only as far as the "other" side of the subdev. It
>> wouldn't work for a subdev which has multiplexed sink and source pads.
>> This series implements a "deep" get-format (v4l2_subdev_get_format_dir)
>> which follows a stream either towards the original source or the final
>> sink, while looking for a non-multiplexed pad with a format.
>>
>> Some thoughts:
>>
>> 1) Link validation and v4l2_subdev_get_format_dir need to look at the
>> routing, and this leads to multiple allocs to get a copy of the routing
>> table. There might be a possibility here to keep a table allocated and
>> re-use it in consecutive get_routing calls.
>>
>> Or even better, perhaps the kAPI could be changed so that allocs are not
>> needed. I thought about a kAPI where the subdev just returns a pointer
>> to its routing table, but then we hit the life-cycle problem: how to
>> ensure the table won't be freed or changed until the caller is done.
> 
> Storing the routing table in the v4l2_subdev_config (or
> v4l2_subdev_state) would be one way to do so, and I'd like to explore
> that direction. State lifetime is indeed an issue, and one simple option
> would be to just take the graph lock to modify the routing.
> 
>> 2) The routing uAPI is a bit vague. There is no way for the userspace to
>> figure out what kind of routing is allowed. Also, the existence of a
>> route in the routing table already indicates that the route is active,
>> but we also have V4L2_SUBDEV_ROUTE_FL_ACTIVE. I decided to keep
>> V4L2_SUBDEV_ROUTE_FL_ACTIVE for now, even if it doesn't really provide
>> any feature.
> 
> We can't report all possible routes if we take streams into account, but
> maybe we could report all possible routes between pads ? This could go
> through a separate ioctl.

That should be doable, but I wonder how much it helps. We should also 
somehow indicate if, say, routes from two source pads can go to the same 
sink pads, or can two streams from a single source pad go to separate 
sink pads, and so on. Is it better just to document what the driver 
supports for a specific hardware, than try to come up with a machine 
readable representation of the possible routings.

  Tomi

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 22/24] v4l: subdev: add v4l2_subdev_get_format_dir()
  2021-04-18 19:04   ` Laurent Pinchart
@ 2021-04-21 13:04     ` Tomi Valkeinen
  2021-04-29  1:43       ` Laurent Pinchart
  0 siblings, 1 reply; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-21 13:04 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Laurent,

On 18/04/2021 22:04, Laurent Pinchart wrote:
> Hi Tomi,
> 
> Thank you for the patch.
> 
> On Thu, Apr 15, 2021 at 04:04:48PM +0300, Tomi Valkeinen wrote:
>> Add v4l2_subdev_get_format_dir() which can be used to find subdev format
>> for a specific stream on a multiplexed pad. The function will follow the
>> routes and links until it finds a non-multiplexed pad.
>>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> ---
>>   drivers/media/v4l2-core/v4l2-subdev.c | 96 +++++++++++++++++++++++++++
>>   include/media/v4l2-subdev.h           | 26 ++++++++
>>   2 files changed, 122 insertions(+)
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
>> index 7a4f71d8c6c3..430dbdaab080 100644
>> --- a/drivers/media/v4l2-core/v4l2-subdev.c
>> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
>> @@ -998,6 +998,102 @@ bool v4l2_subdev_has_route(struct v4l2_subdev_krouting *routing,
>>   }
>>   EXPORT_SYMBOL_GPL(v4l2_subdev_has_route);
>>   
>> +int v4l2_subdev_get_format_dir(struct media_pad *pad, u16 stream,
>> +			       enum v4l2_direction dir,
>> +			       struct v4l2_subdev_format *fmt)
>> +{
>> +	struct device *dev = pad->entity->graph_obj.mdev->dev;
>> +	int ret;
>> +	int i;
>> +
>> +	dev_dbg(dev, "%s '%s':%u:%u %s\n", __func__,
>> +		pad->entity->name, pad->index, stream,
>> +		dir == V4L2_DIR_SOURCEWARD ? "sourceward" : "sinkward");
>> +
>> +	while (true) {
>> +		struct v4l2_subdev_krouting routing;
>> +		struct v4l2_subdev_route *route;
>> +
>> +		if (pad->entity->obj_type != MEDIA_ENTITY_TYPE_V4L2_SUBDEV)
>> +			return -EINVAL;
>> +
>> +		ret = v4l2_subdev_link_validate_get_format(pad, fmt);
>> +		if (ret == 0)
>> +			return 0;
>> +		else if (ret != -ENOIOCTLCMD)
>> +			return ret;
>> +
>> +		if (pad->flags &
>> +		    (dir == V4L2_DIR_SINKWARD ? MEDIA_PAD_FL_SOURCE :
>> +						MEDIA_PAD_FL_SINK)) {
>> +			pad = media_entity_remote_pad(pad);
>> +
>> +			if (!pad)
>> +				return -EINVAL;
>> +
>> +			if (pad->entity->obj_type != MEDIA_ENTITY_TYPE_V4L2_SUBDEV)
>> +				return -EINVAL;
>> +
>> +			ret = v4l2_subdev_link_validate_get_format(pad, fmt);
>> +			if (ret == 0)
>> +				return 0;
>> +			else if (ret != -ENOIOCTLCMD)
>> +				return ret;
>> +		}
>> +
>> +		ret = v4l2_subdev_get_krouting(media_entity_to_v4l2_subdev(pad->entity), &routing);
>> +		if (ret)
>> +			return ret;
>> +
>> +		route = NULL;
>> +		for (i = 0; i < routing.num_routes; ++i) {
>> +			u16 near_pad = dir == V4L2_DIR_SINKWARD ?
>> +					       routing.routes[i].sink_pad :
>> +					       routing.routes[i].source_pad;
>> +			u16 near_stream = dir == V4L2_DIR_SINKWARD ?
>> +						  routing.routes[i].sink_stream :
>> +						  routing.routes[i].source_stream;
>> +
>> +			if (!(routing.routes[i].flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
>> +				continue;
>> +
>> +			if (near_pad != pad->index)
>> +				continue;
>> +
>> +			if (near_stream != stream)
>> +				continue;
>> +
>> +			if (route) {
>> +				dev_err(dev,
>> +					"%s: '%s' has multiple active routes for stream %u\n",
>> +					__func__, pad->entity->name, stream);
>> +				v4l2_subdev_free_routing(&routing);
>> +				return -EINVAL;
>> +			}
>> +
>> +			route = &routing.routes[i];
>> +		}
>> +
>> +		if (!route) {
>> +			dev_err(dev, "%s: no route found in '%s' for stream %u\n",
>> +				__func__, pad->entity->name, stream);
>> +			v4l2_subdev_free_routing(&routing);
>> +			return -EINVAL;
>> +		}
>> +
>> +		if (dir == V4L2_DIR_SINKWARD) {
>> +			pad = &pad->entity->pads[route->source_pad];
>> +			stream = route->source_stream;
>> +		} else {
>> +			pad = &pad->entity->pads[route->sink_pad];
>> +			stream = route->sink_stream;
>> +		}
>> +
>> +		v4l2_subdev_free_routing(&routing);
>> +	}
>> +}
>> +EXPORT_SYMBOL_GPL(v4l2_subdev_get_format_dir);
>> +
>>   int v4l2_subdev_link_validate(struct media_link *link)
>>   {
>>   	struct v4l2_subdev *sink;
>> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
>> index 1843b77dd843..730631f9a091 100644
>> --- a/include/media/v4l2-subdev.h
>> +++ b/include/media/v4l2-subdev.h
>> @@ -1239,4 +1239,30 @@ void v4l2_subdev_cpy_routing(struct v4l2_subdev_krouting *dst,
>>   bool v4l2_subdev_has_route(struct v4l2_subdev_krouting *routing,
>>   			   unsigned int pad0, unsigned int pad1);
>>   
>> +/**
>> + * enum v4l2_direction - Direction either towards the source or the sink
>> + *
>> + * @V4L2_DIR_SOURCEWARD: Direction towards the source.
>> + * @V4L2_DIR_SINKWARD: Direction towards the sink.
>> + */
>> +enum v4l2_direction {
>> +	V4L2_DIR_SOURCEWARD,
>> +	V4L2_DIR_SINKWARD,
>> +};
>> +
>> +/**
>> + * v4l2_subdev_get_format_dir() - Find format by following streams
> 
> The name is a bit cryptic, and the usage pattern error-prone. Can we do
> better ?  In particular, if we could limit the usage of this function to
> be called on a non-multiplexed pad, we could drop the stream argument.
> Deducing the direction argument from the type of pad would also make the
> API simpler.

Hmm, but that's not what the function does. It follows a specific 
stream, from a multiplexed pad, so it has to get the stream as a parameter.

We can't deduct the direction from the type of the pad. We can of course 
define that given a source pad this function goes sourceward. But if 
that's not what the caller wants, then the caller needs to first follow 
the stream either direction to get a sink pad, and then call this 
function, which doesn't make sense.

>> + * @pad: The pad from which to start the search
>> + * @stream: The stream for which we want to find the format
>> + * @dir: The direction of the search
>> + * @fmt: Pointer to &struct v4l2_subdev_format where the found format is stored
>> + *
>> + * This function attempts to find v4l2_subdev_format for a specific stream on a
>> + * multiplexed pad by following the stream using routes and links to the specified
>> + * direction, until a non-multiplexed pad is found.
>> + */
>> +int v4l2_subdev_get_format_dir(struct media_pad *pad, u16 stream,
>> +			       enum v4l2_direction dir,
>> +			       struct v4l2_subdev_format *fmt);
>> +
>>   #endif
> 


^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 18/24] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations
  2021-04-18 18:32   ` Laurent Pinchart
@ 2021-04-22 11:16     ` Tomi Valkeinen
  2021-04-22 16:20       ` Laurent Pinchart
  0 siblings, 1 reply; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-22 11:16 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
	Michal Simek

On 18/04/2021 21:32, Laurent Pinchart wrote:
> Hi Tomi,
> 
> Thank you for the patch.
> 
> On Thu, Apr 15, 2021 at 04:04:44PM +0300, Tomi Valkeinen wrote:
>> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>
>> Add support for subdev internal routing. A route is defined as a single
>> stream from a sink pad to a source pad.
>>
>> The userspace can configure the routing via two new ioctls,
>> VIDIOC_SUBDEV_G_ROUTING and VIDIOC_SUBDEV_S_ROUTING, and subdevs can
>> implement the functionality with v4l2_subdev_pad_ops.get_routing() and
>> v4l2_subdev_pad_ops.set_routing().
>>
>> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>> Signed-off-by: Michal Simek <michal.simek@xilinx.com>
>>
>> - Add sink and source streams for multiplexed links
>> - Copy the argument back in case of an error. This is needed to let the
>>    caller know the number of routes.
>>
>> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
>> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
>>
>> - Expand and refine documentation.
>> - Make the 'routes' pointer a __u64 __user pointer so that a compat32
>>    version of the ioctl is not required.
>> - Add struct v4l2_subdev_krouting to be used for subdevice operations.
>>
>> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
>>
>> - Fix typecasing warnings
>> - Check sink & source pad types
>> - Add 'which' field
>>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> 
> Seems few people haven't contributed to this patch one way or another
> :-)

The more the merrier!

>> ---
>>   drivers/media/v4l2-core/v4l2-ioctl.c  | 25 ++++++++++++++-
>>   drivers/media/v4l2-core/v4l2-subdev.c | 45 +++++++++++++++++++++++++++
>>   include/media/v4l2-subdev.h           | 24 ++++++++++++++
>>   include/uapi/linux/v4l2-subdev.h      | 44 ++++++++++++++++++++++++++
>>   4 files changed, 137 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>> index 6a5d1c6d11d6..f5732962753f 100644
>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>> @@ -16,6 +16,7 @@
>>   #include <linux/kernel.h>
>>   #include <linux/version.h>
>>   
>> +#include <linux/v4l2-subdev.h>
>>   #include <linux/videodev2.h>
>>   
>>   #include <media/v4l2-common.h>
>> @@ -3108,6 +3109,21 @@ static int check_array_args(unsigned int cmd, void *parg, size_t *array_size,
>>   		ret = 1;
>>   		break;
>>   	}
>> +
>> +	case VIDIOC_SUBDEV_G_ROUTING:
>> +	case VIDIOC_SUBDEV_S_ROUTING: {
>> +		struct v4l2_subdev_routing *route = parg;
> 
> s/route/routing/ to match the code below.
> 
>> +
>> +		if (route->num_routes > 256)
>> +			return -EINVAL;
>> +
>> +		*user_ptr = u64_to_user_ptr(route->routes);
>> +		*kernel_ptr = (void **)&route->routes;
>> +		*array_size = sizeof(struct v4l2_subdev_route)
>> +			    * route->num_routes;
>> +		ret = 1;
>> +		break;
>> +	}
>>   	}
>>   
>>   	return ret;
>> @@ -3369,8 +3385,15 @@ video_usercopy(struct file *file, unsigned int orig_cmd, unsigned long arg,
>>   	/*
>>   	 * Some ioctls can return an error, but still have valid
>>   	 * results that must be returned.
>> +	 *
>> +	 * FIXME: subdev IOCTLS are partially handled here and partially in
>> +	 * v4l2-subdev.c and the 'always_copy' flag can only be set for IOCTLS
>> +	 * defined here as part of the 'v4l2_ioctls' array. As
>> +	 * VIDIOC_SUBDEV_G_ROUTING needs to return results to applications even
>> +	 * in case of failure, but it is not defined here as part of the
>> +	 * 'v4l2_ioctls' array, insert an ad-hoc check to address that.
>>   	 */
>> -	if (err < 0 && !always_copy)
>> +	if (err < 0 && !always_copy && cmd != VIDIOC_SUBDEV_G_ROUTING)
>>   		goto out;
>>   
>>   out_array_args:
>> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
>> index 956dafab43d4..95a4c3091fa6 100644
>> --- a/drivers/media/v4l2-core/v4l2-subdev.c
>> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
>> @@ -681,6 +681,51 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg)
>>   	case VIDIOC_SUBDEV_QUERYSTD:
>>   		return v4l2_subdev_call(sd, video, querystd, arg);
>>   
>> +	case VIDIOC_SUBDEV_G_ROUTING: {
>> +		struct v4l2_subdev_routing *routing = arg;
>> +		struct v4l2_subdev_krouting krouting = {
>> +			.which = routing->which,
>> +			.num_routes = routing->num_routes,
>> +			.routes = (struct v4l2_subdev_route *)(uintptr_t)routing->routes,
>> +		};
>> +		int ret;
>> +
>> +		ret = v4l2_subdev_call(sd, pad, get_routing, &krouting);
>> +		if (ret)
>> +			return ret;
>> +
>> +		routing->num_routes = krouting.num_routes;
> 
> Shouldn't num_routes be set even in case of errors ?

Yes, you're right.

>> +
>> +		return 0;
>> +	}
>> +
>> +	case VIDIOC_SUBDEV_S_ROUTING: {
>> +		struct v4l2_subdev_routing *routing = arg;
>> +		struct v4l2_subdev_route *route = (struct v4l2_subdev_route *)(uintptr_t)
>> +						  routing->routes;
> 
> I'd name the variable routes as it points to an array.
> 
>> +		struct v4l2_subdev_krouting krouting = {};
>> +		unsigned int i;
>> +
>> +		if (routing->which != V4L2_SUBDEV_FORMAT_TRY && ro_subdev)
>> +			return -EPERM;
>> +
>> +		for (i = 0; i < routing->num_routes; ++i) {
>> +			if (route[i].sink_pad >= sd->entity.num_pads ||
>> +			    route[i].source_pad >= sd->entity.num_pads)
>> +				return -EINVAL;
>> +
>> +			if (!(sd->entity.pads[route[i].sink_pad].flags & MEDIA_PAD_FL_SINK) ||
>> +			    !(sd->entity.pads[route[i].source_pad].flags & MEDIA_PAD_FL_SOURCE))
>> +				return -EINVAL;
>> +		}
>> +
>> +		krouting.which = routing->which;
>> +		krouting.num_routes = routing->num_routes;
>> +		krouting.routes = route;
>> +
>> +		return v4l2_subdev_call(sd, pad, set_routing, &krouting);
>> +	}
>> +
>>   	default:
>>   		return v4l2_subdev_call(sd, core, ioctl, cmd, arg);
>>   	}
>> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
>> index 436d0445aafd..3826ab918731 100644
>> --- a/include/media/v4l2-subdev.h
>> +++ b/include/media/v4l2-subdev.h
>> @@ -650,6 +650,22 @@ struct v4l2_subdev_pad_config {
>>   	struct v4l2_rect try_compose;
>>   };
>>   
>> +/**
>> + * struct v4l2_subdev_krouting - subdev routing table
>> + *
>> + * @which: format type (from enum v4l2_subdev_format_whence)
>> + * @routes: &struct v4l2_subdev_route
>> + * @num_routes: number of routes
>> + *
>> + * This structure is used to translate argument received from
> 
> s/argument/the arguments/
> 
>> + * VIDIOC_SUBDEV_G/S_ROUTING() ioctl to sudev device drivers operations.
> 
> s/sudev/subdev/
> 
>> + */
>> +struct v4l2_subdev_krouting {
>> +	u32 which;
>> +	struct v4l2_subdev_route *routes;
>> +	unsigned int num_routes;
>> +};
>> +
>>   /**
>>    * struct v4l2_subdev_pad_ops - v4l2-subdev pad level operations
>>    *
>> @@ -711,6 +727,10 @@ struct v4l2_subdev_pad_config {
>>    *		     applied to the hardware. The operation shall fail if the
>>    *		     pad index it has been called on is not valid or in case of
>>    *		     unrecoverable failures.
>> + *
>> + * @get_routing: get the subdevice routing table.
>> + * @set_routing: enable or disable data connection routes described in the
>> + *		 subdevice routing table.
>>    */
>>   struct v4l2_subdev_pad_ops {
>>   	int (*init_cfg)(struct v4l2_subdev *sd,
>> @@ -755,6 +775,10 @@ struct v4l2_subdev_pad_ops {
>>   			       struct v4l2_mbus_config *config);
>>   	int (*set_mbus_config)(struct v4l2_subdev *sd, unsigned int pad,
>>   			       struct v4l2_mbus_config *config);
>> +	int (*get_routing)(struct v4l2_subdev *sd,
>> +			   struct v4l2_subdev_krouting *route);
>> +	int (*set_routing)(struct v4l2_subdev *sd,
>> +			   struct v4l2_subdev_krouting *route);
>>   };
>>   
>>   /**
>> diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h
>> index 658106f5b5dc..f2a17cbd1e9a 100644
>> --- a/include/uapi/linux/v4l2-subdev.h
>> +++ b/include/uapi/linux/v4l2-subdev.h
>> @@ -188,6 +188,48 @@ struct v4l2_subdev_capability {
>>   /* The v4l2 sub-device video device node is registered in read-only mode. */
>>   #define V4L2_SUBDEV_CAP_RO_SUBDEV		0x00000001
>>   
>> +#define V4L2_SUBDEV_ROUTE_FL_ACTIVE		BIT(0)
>> +#define V4L2_SUBDEV_ROUTE_FL_IMMUTABLE		BIT(1)
>> +
>> +/**
>> + * struct v4l2_subdev_route - A route inside a subdev
>> + *
>> + * @sink_pad: the sink pad index
>> + * @sink_stream: the sink stream identifier
>> + * @source_pad: the source pad index
>> + * @source_stream: the source stream identifier
>> + * @flags: route flags:
>> + *
>> + *	V4L2_SUBDEV_ROUTE_FL_ACTIVE: Is the route active? An active
>> + *	route will start when streaming is enabled on a video node.
>> + *	Set by the user.
>> + *
>> + *	V4L2_SUBDEV_ROUTE_FL_IMMUTABLE: Is the route immutable, i.e.
>> + *	can it be activated and inactivated? Set by the driver.
> 
> This should be moved to the macros.
> 
>> + */
>> +struct v4l2_subdev_route {
>> +	__u32 sink_pad;
>> +	__u32 sink_stream;
>> +	__u32 source_pad;
>> +	__u32 source_stream;
>> +	__u32 flags;
>> +	__u32 reserved[5];
>> +};
>> +
>> +/**
>> + * struct v4l2_subdev_routing - Subdev routing information
>> + *
>> + * @which: format type (from enum v4l2_subdev_format_whence)
> 
> v4l2_subdev_format_whence shoudn't have had "format" in its name :-S We
> can't do much about that, but should "format type" be changed to "routes
> type", or maybe "configuration type" ?
> 
>> + * @routes: pointer to the routes array
>> + * @num_routes: the total number of routes in the routes array
> 
> Won't kerneldoc complain about the undocumented reserved field ?
> 
>> + */
>> +struct v4l2_subdev_routing {
>> +	__u32 which;
>> +	__u64 routes;
>> +	__u32 num_routes;
>> +	__u32 reserved[5];
>> +};
>> +
>>   /* Backwards compatibility define --- to be removed */
>>   #define v4l2_subdev_edid v4l2_edid
>>   
>> @@ -215,5 +257,7 @@ struct v4l2_subdev_capability {
>>   #define VIDIOC_SUBDEV_ENUM_DV_TIMINGS		_IOWR('V', 98, struct v4l2_enum_dv_timings)
>>   #define VIDIOC_SUBDEV_QUERY_DV_TIMINGS		_IOR('V', 99, struct v4l2_dv_timings)
>>   #define VIDIOC_SUBDEV_DV_TIMINGS_CAP		_IOWR('V', 100, struct v4l2_dv_timings_cap)
>> +#define VIDIOC_SUBDEV_G_ROUTING			_IOWR('V', 38, struct v4l2_subdev_routing)
>> +#define VIDIOC_SUBDEV_S_ROUTING			_IOWR('V', 39, struct v4l2_subdev_routing)
> 
> I would keep the ioctls sorted by number.

The ioctls are not sorted (even if it may look like it above =). I'm not 
sure if there's a better place for them than the bottom.

  Tomi

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 05/24] media: entity: Add iterator helper for entity pads
  2021-04-18 17:52   ` Laurent Pinchart
@ 2021-04-22 12:04     ` Tomi Valkeinen
  0 siblings, 0 replies; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-22 12:04 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

On 18/04/2021 20:52, Laurent Pinchart wrote:
> Hi Tomi and Jacopo,
> 
> Thank you for the patch.
> 
> On Thu, Apr 15, 2021 at 04:04:31PM +0300, Tomi Valkeinen wrote:
>> From: Jacopo Mondi <jacopo+renesas@jmondi.org>
>>
>> Add an iterator helper to easily cycle through all pads in an entity and
>> use it in media-entity and media-device code where appropriate.
>>
>> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
>> ---
>>   drivers/media/mc/mc-device.c | 13 ++++++-------
>>   drivers/media/mc/mc-entity.c | 11 ++++++-----
>>   include/media/media-entity.h | 12 ++++++++++++
>>   3 files changed, 24 insertions(+), 12 deletions(-)
>>
>> diff --git a/drivers/media/mc/mc-device.c b/drivers/media/mc/mc-device.c
>> index 9e56d2ad6b94..704ef1360eba 100644
>> --- a/drivers/media/mc/mc-device.c
>> +++ b/drivers/media/mc/mc-device.c
>> @@ -581,7 +581,7 @@ static void __media_device_unregister_entity(struct media_entity *entity)
>>   	struct media_device *mdev = entity->graph_obj.mdev;
>>   	struct media_link *link, *tmp;
>>   	struct media_interface *intf;
>> -	unsigned int i;
>> +	struct media_pad *iter;
>>   
>>   	ida_free(&mdev->entity_internal_idx, entity->internal_idx);
>>   
>> @@ -597,8 +597,8 @@ static void __media_device_unregister_entity(struct media_entity *entity)
>>   	__media_entity_remove_links(entity);
>>   
>>   	/* Remove all pads that belong to this entity */
>> -	for (i = 0; i < entity->num_pads; i++)
>> -		media_gobj_destroy(&entity->pads[i].graph_obj);
>> +	media_entity_for_each_pad(entity, iter)
>> +		media_gobj_destroy(&iter->graph_obj);
>>   
>>   	/* Remove the entity */
>>   	media_gobj_destroy(&entity->graph_obj);
>> @@ -617,7 +617,7 @@ int __must_check media_device_register_entity(struct media_device *mdev,
>>   					      struct media_entity *entity)
>>   {
>>   	struct media_entity_notify *notify, *next;
>> -	unsigned int i;
>> +	struct media_pad *iter;
>>   	int ret;
>>   
>>   	if (entity->function == MEDIA_ENT_F_V4L2_SUBDEV_UNKNOWN ||
>> @@ -646,9 +646,8 @@ int __must_check media_device_register_entity(struct media_device *mdev,
>>   	media_gobj_create(mdev, MEDIA_GRAPH_ENTITY, &entity->graph_obj);
>>   
>>   	/* Initialize objects at the pads */
>> -	for (i = 0; i < entity->num_pads; i++)
>> -		media_gobj_create(mdev, MEDIA_GRAPH_PAD,
>> -			       &entity->pads[i].graph_obj);
>> +	media_entity_for_each_pad(entity, iter)
>> +		media_gobj_create(mdev, MEDIA_GRAPH_PAD, &iter->graph_obj);
>>   
>>   	/* invoke entity_notify callbacks */
>>   	list_for_each_entry_safe(notify, next, &mdev->entity_notify, list)
>> diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
>> index 401fddf320e7..830841e0cd28 100644
>> --- a/drivers/media/mc/mc-entity.c
>> +++ b/drivers/media/mc/mc-entity.c
>> @@ -198,7 +198,8 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads,
>>   			   struct media_pad *pads)
>>   {
>>   	struct media_device *mdev = entity->graph_obj.mdev;
>> -	unsigned int i;
>> +	struct media_pad *iter;
>> +	unsigned int i = 0;
>>   
>>   	if (num_pads >= MEDIA_ENTITY_MAX_PADS)
>>   		return -E2BIG;
>> @@ -209,12 +210,12 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads,
>>   	if (mdev)
>>   		mutex_lock(&mdev->graph_mutex);
>>   
>> -	for (i = 0; i < num_pads; i++) {
>> -		pads[i].entity = entity;
>> -		pads[i].index = i;
>> +	media_entity_for_each_pad(entity, iter) {
>> +		iter->entity = entity;
>> +		iter->index = i++;
>>   		if (mdev)
>>   			media_gobj_create(mdev, MEDIA_GRAPH_PAD,
>> -					&entity->pads[i].graph_obj);
>> +					&iter->graph_obj);
>>   	}
>>   
>>   	if (mdev)
>> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
>> index 2d45344ca527..52b1a1cab57a 100644
>> --- a/include/media/media-entity.h
>> +++ b/include/media/media-entity.h
>> @@ -1102,3 +1102,15 @@ void media_remove_intf_links(struct media_interface *intf);
>>   	 (entity)->ops->operation((entity) , ##args) : -ENOIOCTLCMD)
>>   
>>   #endif
>> +
>> +/**
>> + * media_entity_for_each_pad - Iterate on all pads in an entity
>> + * @entity: The entity the pads belong to
>> + * @iter: The iterator pad
> 
> I would name the variable pad, here and in the code above.

'iter' name is used in other similar functions/macros here, so I'd 
rather keep this the same.

  Tomi

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 15/24] v4l: Add bus type to frame descriptors
  2021-04-20 11:50     ` Sakari Ailus
@ 2021-04-22 12:30       ` Tomi Valkeinen
  2021-04-29 11:58         ` Sakari Ailus
  0 siblings, 1 reply; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-22 12:30 UTC (permalink / raw)
  To: Sakari Ailus, Laurent Pinchart
  Cc: linux-media, Jacopo Mondi, niklas.soderlund+renesas,
	Mauro Carvalho Chehab, Hans Verkuil

On 20/04/2021 14:50, Sakari Ailus wrote:
> Hi Laurent,
> 
> On Sun, Apr 18, 2021 at 10:23:35PM +0300, Laurent Pinchart wrote:
>> Hi Tomi and Sakari,
>>
>> Thank you for the patch.
>>
>> On Thu, Apr 15, 2021 at 04:04:41PM +0300, Tomi Valkeinen wrote:
>>> From: Sakari Ailus <sakari.ailus@linux.intel.com>
>>>
>>> Add the media bus type to the frame descriptor. CSI-2 specific
>>> information will be added in next patch to the frame descriptor.
>>
>> I'd squash the next patch with this one.
>>
>>> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
>>> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
>>>
>>> - Make the bus type a named enum
>>> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
>>> ---
>>>   include/media/v4l2-subdev.h | 9 +++++++++
>>>   1 file changed, 9 insertions(+)
>>>
>>> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
>>> index d0e9a5bdb08b..85977abbea46 100644
>>> --- a/include/media/v4l2-subdev.h
>>> +++ b/include/media/v4l2-subdev.h
>>> @@ -340,12 +340,21 @@ struct v4l2_mbus_frame_desc_entry {
>>>   
>>>   #define V4L2_FRAME_DESC_ENTRY_MAX	4
>>>   
>>> +enum v4l2_mbus_frame_desc_type {
>>> +	V4L2_MBUS_FRAME_DESC_TYPE_PLATFORM,
>>> +	V4L2_MBUS_FRAME_DESC_TYPE_PARALLEL,
>>> +	V4L2_MBUS_FRAME_DESC_TYPE_CCP2,
>>> +	V4L2_MBUS_FRAME_DESC_TYPE_CSI2,
>>> +};
>>
>> This should be documented. In particular, I have no idea what
>> V4L2_MBUS_FRAME_DESC_TYPE_PLATFORM is. I also wonder if we shouldn't
>> drop CCP2 (at least for now), does anyone use that anymore ?
> 
> I guess we don't need one here, not now at least.
> 
> I agree on the documentation.

As it's the first one in the list, I think it really means "undefined", 
so that current users have a value there (I presume they initialize the 
struct to 0). Sakari?

  Tomi

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 17/24] v4l: Add stream to frame descriptor
  2021-04-18 19:27   ` Laurent Pinchart
@ 2021-04-22 12:47     ` Tomi Valkeinen
  2021-04-22 16:18       ` Laurent Pinchart
  0 siblings, 1 reply; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-22 12:47 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

On 18/04/2021 22:27, Laurent Pinchart wrote:
> Hi Tomi and Sakari,
> 
> Thank you for the patch.
> 
> On Thu, Apr 15, 2021 at 04:04:43PM +0300, Tomi Valkeinen wrote:
>> From: Sakari Ailus <sakari.ailus@linux.intel.com>
>>
>> The stream field identifies the stream this frame descriptor applies to in
>> routing configuration across a multiplexed link.
>>
>> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
>> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
>> Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
>> ---
>>   include/media/v4l2-subdev.h | 2 ++
>>   1 file changed, 2 insertions(+)
>>
>> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
>> index 30ec011d31e3..436d0445aafd 100644
>> --- a/include/media/v4l2-subdev.h
>> +++ b/include/media/v4l2-subdev.h
>> @@ -338,6 +338,7 @@ enum v4l2_mbus_frame_desc_flags {
>>    * struct v4l2_mbus_frame_desc_entry - media bus frame description structure
>>    *
>>    * @flags:	bitmask flags, as defined by &enum v4l2_mbus_frame_desc_flags.
>> + * @stream:	stream in routing configuration
>>    * @pixelcode:	media bus pixel code, valid if @flags
>>    *		%FRAME_DESC_FL_BLOB is not set.
>>    * @length:	number of octets per frame, valid if @flags
>> @@ -347,6 +348,7 @@ enum v4l2_mbus_frame_desc_flags {
>>    */
>>   struct v4l2_mbus_frame_desc_entry {
>>   	enum v4l2_mbus_frame_desc_flags flags;
>> +	u32 stream;
> 
> As this isn't used in this series, I can't really tell how it will be
> used, so it's hard to review the patch. Should we postpone it to the
> next patch series ?

It's not used here, but it is needed allow the drivers use the features 
introduced in this series. It is used to match routes to frame_descs.

Its usage is quite simple: e.g. when a driver has a route 
(v4l2_subdev_route), it can get the frame_descs from the entity behind 
v4l2_subdev_route.source_pad, and then find the matching frame_desc by 
searching for

v4l2_subdev_route.source_stream == v4l2_mbus_frame_desc_entry.stream

  Tomi

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 17/24] v4l: Add stream to frame descriptor
  2021-04-22 12:47     ` Tomi Valkeinen
@ 2021-04-22 16:18       ` Laurent Pinchart
  0 siblings, 0 replies; 72+ messages in thread
From: Laurent Pinchart @ 2021-04-22 16:18 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Tomi,

On Thu, Apr 22, 2021 at 03:47:05PM +0300, Tomi Valkeinen wrote:
> On 18/04/2021 22:27, Laurent Pinchart wrote:
> > On Thu, Apr 15, 2021 at 04:04:43PM +0300, Tomi Valkeinen wrote:
> >> From: Sakari Ailus <sakari.ailus@linux.intel.com>
> >>
> >> The stream field identifies the stream this frame descriptor applies to in
> >> routing configuration across a multiplexed link.
> >>
> >> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> >> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> >> Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> >> ---
> >>   include/media/v4l2-subdev.h | 2 ++
> >>   1 file changed, 2 insertions(+)
> >>
> >> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> >> index 30ec011d31e3..436d0445aafd 100644
> >> --- a/include/media/v4l2-subdev.h
> >> +++ b/include/media/v4l2-subdev.h
> >> @@ -338,6 +338,7 @@ enum v4l2_mbus_frame_desc_flags {
> >>    * struct v4l2_mbus_frame_desc_entry - media bus frame description structure
> >>    *
> >>    * @flags:	bitmask flags, as defined by &enum v4l2_mbus_frame_desc_flags.
> >> + * @stream:	stream in routing configuration
> >>    * @pixelcode:	media bus pixel code, valid if @flags
> >>    *		%FRAME_DESC_FL_BLOB is not set.
> >>    * @length:	number of octets per frame, valid if @flags
> >> @@ -347,6 +348,7 @@ enum v4l2_mbus_frame_desc_flags {
> >>    */
> >>   struct v4l2_mbus_frame_desc_entry {
> >>   	enum v4l2_mbus_frame_desc_flags flags;
> >> +	u32 stream;
> > 
> > As this isn't used in this series, I can't really tell how it will be
> > used, so it's hard to review the patch. Should we postpone it to the
> > next patch series ?
> 
> It's not used here, but it is needed allow the drivers use the features 
> introduced in this series. It is used to match routes to frame_descs.

Sure, I'm not saying it should be dropped, but it would be easier to
review it in the context of the next series, while the rest of this
series bring a self-contained feature that is already useful without
streams support.

> Its usage is quite simple: e.g. when a driver has a route 
> (v4l2_subdev_route), it can get the frame_descs from the entity behind 
> v4l2_subdev_route.source_pad, and then find the matching frame_desc by 
> searching for
> 
> v4l2_subdev_route.source_stream == v4l2_mbus_frame_desc_entry.stream

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 18/24] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations
  2021-04-22 11:16     ` Tomi Valkeinen
@ 2021-04-22 16:20       ` Laurent Pinchart
  2021-04-22 16:58         ` Tomi Valkeinen
  0 siblings, 1 reply; 72+ messages in thread
From: Laurent Pinchart @ 2021-04-22 16:20 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
	Michal Simek

Hi Tomi,

On Thu, Apr 22, 2021 at 02:16:10PM +0300, Tomi Valkeinen wrote:
> On 18/04/2021 21:32, Laurent Pinchart wrote:
> > On Thu, Apr 15, 2021 at 04:04:44PM +0300, Tomi Valkeinen wrote:
> >> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> >>
> >> Add support for subdev internal routing. A route is defined as a single
> >> stream from a sink pad to a source pad.
> >>
> >> The userspace can configure the routing via two new ioctls,
> >> VIDIOC_SUBDEV_G_ROUTING and VIDIOC_SUBDEV_S_ROUTING, and subdevs can
> >> implement the functionality with v4l2_subdev_pad_ops.get_routing() and
> >> v4l2_subdev_pad_ops.set_routing().
> >>
> >> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> >> Signed-off-by: Michal Simek <michal.simek@xilinx.com>
> >>
> >> - Add sink and source streams for multiplexed links
> >> - Copy the argument back in case of an error. This is needed to let the
> >>    caller know the number of routes.
> >>
> >> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> >> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> >>
> >> - Expand and refine documentation.
> >> - Make the 'routes' pointer a __u64 __user pointer so that a compat32
> >>    version of the ioctl is not required.
> >> - Add struct v4l2_subdev_krouting to be used for subdevice operations.
> >>
> >> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> >>
> >> - Fix typecasing warnings
> >> - Check sink & source pad types
> >> - Add 'which' field
> >>
> >> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> > 
> > Seems few people haven't contributed to this patch one way or another
> > :-)
> 
> The more the merrier!
> 
> >> ---
> >>   drivers/media/v4l2-core/v4l2-ioctl.c  | 25 ++++++++++++++-
> >>   drivers/media/v4l2-core/v4l2-subdev.c | 45 +++++++++++++++++++++++++++
> >>   include/media/v4l2-subdev.h           | 24 ++++++++++++++
> >>   include/uapi/linux/v4l2-subdev.h      | 44 ++++++++++++++++++++++++++
> >>   4 files changed, 137 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> >> index 6a5d1c6d11d6..f5732962753f 100644
> >> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> >> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> >> @@ -16,6 +16,7 @@
> >>   #include <linux/kernel.h>
> >>   #include <linux/version.h>
> >>   
> >> +#include <linux/v4l2-subdev.h>
> >>   #include <linux/videodev2.h>
> >>   
> >>   #include <media/v4l2-common.h>
> >> @@ -3108,6 +3109,21 @@ static int check_array_args(unsigned int cmd, void *parg, size_t *array_size,
> >>   		ret = 1;
> >>   		break;
> >>   	}
> >> +
> >> +	case VIDIOC_SUBDEV_G_ROUTING:
> >> +	case VIDIOC_SUBDEV_S_ROUTING: {
> >> +		struct v4l2_subdev_routing *route = parg;
> > 
> > s/route/routing/ to match the code below.
> > 
> >> +
> >> +		if (route->num_routes > 256)
> >> +			return -EINVAL;
> >> +
> >> +		*user_ptr = u64_to_user_ptr(route->routes);
> >> +		*kernel_ptr = (void **)&route->routes;
> >> +		*array_size = sizeof(struct v4l2_subdev_route)
> >> +			    * route->num_routes;
> >> +		ret = 1;
> >> +		break;
> >> +	}
> >>   	}
> >>   
> >>   	return ret;
> >> @@ -3369,8 +3385,15 @@ video_usercopy(struct file *file, unsigned int orig_cmd, unsigned long arg,
> >>   	/*
> >>   	 * Some ioctls can return an error, but still have valid
> >>   	 * results that must be returned.
> >> +	 *
> >> +	 * FIXME: subdev IOCTLS are partially handled here and partially in
> >> +	 * v4l2-subdev.c and the 'always_copy' flag can only be set for IOCTLS
> >> +	 * defined here as part of the 'v4l2_ioctls' array. As
> >> +	 * VIDIOC_SUBDEV_G_ROUTING needs to return results to applications even
> >> +	 * in case of failure, but it is not defined here as part of the
> >> +	 * 'v4l2_ioctls' array, insert an ad-hoc check to address that.
> >>   	 */
> >> -	if (err < 0 && !always_copy)
> >> +	if (err < 0 && !always_copy && cmd != VIDIOC_SUBDEV_G_ROUTING)
> >>   		goto out;
> >>   
> >>   out_array_args:
> >> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> >> index 956dafab43d4..95a4c3091fa6 100644
> >> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> >> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> >> @@ -681,6 +681,51 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg)
> >>   	case VIDIOC_SUBDEV_QUERYSTD:
> >>   		return v4l2_subdev_call(sd, video, querystd, arg);
> >>   
> >> +	case VIDIOC_SUBDEV_G_ROUTING: {
> >> +		struct v4l2_subdev_routing *routing = arg;
> >> +		struct v4l2_subdev_krouting krouting = {
> >> +			.which = routing->which,
> >> +			.num_routes = routing->num_routes,
> >> +			.routes = (struct v4l2_subdev_route *)(uintptr_t)routing->routes,
> >> +		};
> >> +		int ret;
> >> +
> >> +		ret = v4l2_subdev_call(sd, pad, get_routing, &krouting);
> >> +		if (ret)
> >> +			return ret;
> >> +
> >> +		routing->num_routes = krouting.num_routes;
> > 
> > Shouldn't num_routes be set even in case of errors ?
> 
> Yes, you're right.
> 
> >> +
> >> +		return 0;
> >> +	}
> >> +
> >> +	case VIDIOC_SUBDEV_S_ROUTING: {
> >> +		struct v4l2_subdev_routing *routing = arg;
> >> +		struct v4l2_subdev_route *route = (struct v4l2_subdev_route *)(uintptr_t)
> >> +						  routing->routes;
> > 
> > I'd name the variable routes as it points to an array.
> > 
> >> +		struct v4l2_subdev_krouting krouting = {};
> >> +		unsigned int i;
> >> +
> >> +		if (routing->which != V4L2_SUBDEV_FORMAT_TRY && ro_subdev)
> >> +			return -EPERM;
> >> +
> >> +		for (i = 0; i < routing->num_routes; ++i) {
> >> +			if (route[i].sink_pad >= sd->entity.num_pads ||
> >> +			    route[i].source_pad >= sd->entity.num_pads)
> >> +				return -EINVAL;
> >> +
> >> +			if (!(sd->entity.pads[route[i].sink_pad].flags & MEDIA_PAD_FL_SINK) ||
> >> +			    !(sd->entity.pads[route[i].source_pad].flags & MEDIA_PAD_FL_SOURCE))
> >> +				return -EINVAL;
> >> +		}
> >> +
> >> +		krouting.which = routing->which;
> >> +		krouting.num_routes = routing->num_routes;
> >> +		krouting.routes = route;
> >> +
> >> +		return v4l2_subdev_call(sd, pad, set_routing, &krouting);
> >> +	}
> >> +
> >>   	default:
> >>   		return v4l2_subdev_call(sd, core, ioctl, cmd, arg);
> >>   	}
> >> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> >> index 436d0445aafd..3826ab918731 100644
> >> --- a/include/media/v4l2-subdev.h
> >> +++ b/include/media/v4l2-subdev.h
> >> @@ -650,6 +650,22 @@ struct v4l2_subdev_pad_config {
> >>   	struct v4l2_rect try_compose;
> >>   };
> >>   
> >> +/**
> >> + * struct v4l2_subdev_krouting - subdev routing table
> >> + *
> >> + * @which: format type (from enum v4l2_subdev_format_whence)
> >> + * @routes: &struct v4l2_subdev_route
> >> + * @num_routes: number of routes
> >> + *
> >> + * This structure is used to translate argument received from
> > 
> > s/argument/the arguments/
> > 
> >> + * VIDIOC_SUBDEV_G/S_ROUTING() ioctl to sudev device drivers operations.
> > 
> > s/sudev/subdev/
> > 
> >> + */
> >> +struct v4l2_subdev_krouting {
> >> +	u32 which;
> >> +	struct v4l2_subdev_route *routes;
> >> +	unsigned int num_routes;
> >> +};
> >> +
> >>   /**
> >>    * struct v4l2_subdev_pad_ops - v4l2-subdev pad level operations
> >>    *
> >> @@ -711,6 +727,10 @@ struct v4l2_subdev_pad_config {
> >>    *		     applied to the hardware. The operation shall fail if the
> >>    *		     pad index it has been called on is not valid or in case of
> >>    *		     unrecoverable failures.
> >> + *
> >> + * @get_routing: get the subdevice routing table.
> >> + * @set_routing: enable or disable data connection routes described in the
> >> + *		 subdevice routing table.
> >>    */
> >>   struct v4l2_subdev_pad_ops {
> >>   	int (*init_cfg)(struct v4l2_subdev *sd,
> >> @@ -755,6 +775,10 @@ struct v4l2_subdev_pad_ops {
> >>   			       struct v4l2_mbus_config *config);
> >>   	int (*set_mbus_config)(struct v4l2_subdev *sd, unsigned int pad,
> >>   			       struct v4l2_mbus_config *config);
> >> +	int (*get_routing)(struct v4l2_subdev *sd,
> >> +			   struct v4l2_subdev_krouting *route);
> >> +	int (*set_routing)(struct v4l2_subdev *sd,
> >> +			   struct v4l2_subdev_krouting *route);
> >>   };
> >>   
> >>   /**
> >> diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h
> >> index 658106f5b5dc..f2a17cbd1e9a 100644
> >> --- a/include/uapi/linux/v4l2-subdev.h
> >> +++ b/include/uapi/linux/v4l2-subdev.h
> >> @@ -188,6 +188,48 @@ struct v4l2_subdev_capability {
> >>   /* The v4l2 sub-device video device node is registered in read-only mode. */
> >>   #define V4L2_SUBDEV_CAP_RO_SUBDEV		0x00000001
> >>   
> >> +#define V4L2_SUBDEV_ROUTE_FL_ACTIVE		BIT(0)
> >> +#define V4L2_SUBDEV_ROUTE_FL_IMMUTABLE		BIT(1)
> >> +
> >> +/**
> >> + * struct v4l2_subdev_route - A route inside a subdev
> >> + *
> >> + * @sink_pad: the sink pad index
> >> + * @sink_stream: the sink stream identifier
> >> + * @source_pad: the source pad index
> >> + * @source_stream: the source stream identifier
> >> + * @flags: route flags:
> >> + *
> >> + *	V4L2_SUBDEV_ROUTE_FL_ACTIVE: Is the route active? An active
> >> + *	route will start when streaming is enabled on a video node.
> >> + *	Set by the user.
> >> + *
> >> + *	V4L2_SUBDEV_ROUTE_FL_IMMUTABLE: Is the route immutable, i.e.
> >> + *	can it be activated and inactivated? Set by the driver.
> > 
> > This should be moved to the macros.
> > 
> >> + */
> >> +struct v4l2_subdev_route {
> >> +	__u32 sink_pad;
> >> +	__u32 sink_stream;
> >> +	__u32 source_pad;
> >> +	__u32 source_stream;
> >> +	__u32 flags;
> >> +	__u32 reserved[5];
> >> +};
> >> +
> >> +/**
> >> + * struct v4l2_subdev_routing - Subdev routing information
> >> + *
> >> + * @which: format type (from enum v4l2_subdev_format_whence)
> > 
> > v4l2_subdev_format_whence shoudn't have had "format" in its name :-S We
> > can't do much about that, but should "format type" be changed to "routes
> > type", or maybe "configuration type" ?
> > 
> >> + * @routes: pointer to the routes array
> >> + * @num_routes: the total number of routes in the routes array
> > 
> > Won't kerneldoc complain about the undocumented reserved field ?
> > 
> >> + */
> >> +struct v4l2_subdev_routing {
> >> +	__u32 which;
> >> +	__u64 routes;
> >> +	__u32 num_routes;
> >> +	__u32 reserved[5];
> >> +};
> >> +
> >>   /* Backwards compatibility define --- to be removed */
> >>   #define v4l2_subdev_edid v4l2_edid
> >>   
> >> @@ -215,5 +257,7 @@ struct v4l2_subdev_capability {
> >>   #define VIDIOC_SUBDEV_ENUM_DV_TIMINGS		_IOWR('V', 98, struct v4l2_enum_dv_timings)
> >>   #define VIDIOC_SUBDEV_QUERY_DV_TIMINGS		_IOR('V', 99, struct v4l2_dv_timings)
> >>   #define VIDIOC_SUBDEV_DV_TIMINGS_CAP		_IOWR('V', 100, struct v4l2_dv_timings_cap)
> >> +#define VIDIOC_SUBDEV_G_ROUTING			_IOWR('V', 38, struct v4l2_subdev_routing)
> >> +#define VIDIOC_SUBDEV_S_ROUTING			_IOWR('V', 39, struct v4l2_subdev_routing)
> > 
> > I would keep the ioctls sorted by number.
> 
> The ioctls are not sorted (even if it may look like it above =). I'm not 
> sure if there's a better place for them than the bottom.

Isn't the last block of ioctls (the ones reusing the V4L2 video node
numbers) sorted ?

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 18/24] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations
  2021-04-22 16:20       ` Laurent Pinchart
@ 2021-04-22 16:58         ` Tomi Valkeinen
  0 siblings, 0 replies; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-22 16:58 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
	Michal Simek

On 22/04/2021 19:20, Laurent Pinchart wrote:
> Hi Tomi,
> 
> On Thu, Apr 22, 2021 at 02:16:10PM +0300, Tomi Valkeinen wrote:
>> On 18/04/2021 21:32, Laurent Pinchart wrote:
>>> On Thu, Apr 15, 2021 at 04:04:44PM +0300, Tomi Valkeinen wrote:
>>>> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>>>
>>>> Add support for subdev internal routing. A route is defined as a single
>>>> stream from a sink pad to a source pad.
>>>>
>>>> The userspace can configure the routing via two new ioctls,
>>>> VIDIOC_SUBDEV_G_ROUTING and VIDIOC_SUBDEV_S_ROUTING, and subdevs can
>>>> implement the functionality with v4l2_subdev_pad_ops.get_routing() and
>>>> v4l2_subdev_pad_ops.set_routing().
>>>>
>>>> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>>> Signed-off-by: Michal Simek <michal.simek@xilinx.com>
>>>>
>>>> - Add sink and source streams for multiplexed links
>>>> - Copy the argument back in case of an error. This is needed to let the
>>>>     caller know the number of routes.
>>>>
>>>> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
>>>> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
>>>>
>>>> - Expand and refine documentation.
>>>> - Make the 'routes' pointer a __u64 __user pointer so that a compat32
>>>>     version of the ioctl is not required.
>>>> - Add struct v4l2_subdev_krouting to be used for subdevice operations.
>>>>
>>>> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
>>>>
>>>> - Fix typecasing warnings
>>>> - Check sink & source pad types
>>>> - Add 'which' field
>>>>
>>>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>>>
>>> Seems few people haven't contributed to this patch one way or another
>>> :-)
>>
>> The more the merrier!
>>
>>>> ---
>>>>    drivers/media/v4l2-core/v4l2-ioctl.c  | 25 ++++++++++++++-
>>>>    drivers/media/v4l2-core/v4l2-subdev.c | 45 +++++++++++++++++++++++++++
>>>>    include/media/v4l2-subdev.h           | 24 ++++++++++++++
>>>>    include/uapi/linux/v4l2-subdev.h      | 44 ++++++++++++++++++++++++++
>>>>    4 files changed, 137 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> index 6a5d1c6d11d6..f5732962753f 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> @@ -16,6 +16,7 @@
>>>>    #include <linux/kernel.h>
>>>>    #include <linux/version.h>
>>>>    
>>>> +#include <linux/v4l2-subdev.h>
>>>>    #include <linux/videodev2.h>
>>>>    
>>>>    #include <media/v4l2-common.h>
>>>> @@ -3108,6 +3109,21 @@ static int check_array_args(unsigned int cmd, void *parg, size_t *array_size,
>>>>    		ret = 1;
>>>>    		break;
>>>>    	}
>>>> +
>>>> +	case VIDIOC_SUBDEV_G_ROUTING:
>>>> +	case VIDIOC_SUBDEV_S_ROUTING: {
>>>> +		struct v4l2_subdev_routing *route = parg;
>>>
>>> s/route/routing/ to match the code below.
>>>
>>>> +
>>>> +		if (route->num_routes > 256)
>>>> +			return -EINVAL;
>>>> +
>>>> +		*user_ptr = u64_to_user_ptr(route->routes);
>>>> +		*kernel_ptr = (void **)&route->routes;
>>>> +		*array_size = sizeof(struct v4l2_subdev_route)
>>>> +			    * route->num_routes;
>>>> +		ret = 1;
>>>> +		break;
>>>> +	}
>>>>    	}
>>>>    
>>>>    	return ret;
>>>> @@ -3369,8 +3385,15 @@ video_usercopy(struct file *file, unsigned int orig_cmd, unsigned long arg,
>>>>    	/*
>>>>    	 * Some ioctls can return an error, but still have valid
>>>>    	 * results that must be returned.
>>>> +	 *
>>>> +	 * FIXME: subdev IOCTLS are partially handled here and partially in
>>>> +	 * v4l2-subdev.c and the 'always_copy' flag can only be set for IOCTLS
>>>> +	 * defined here as part of the 'v4l2_ioctls' array. As
>>>> +	 * VIDIOC_SUBDEV_G_ROUTING needs to return results to applications even
>>>> +	 * in case of failure, but it is not defined here as part of the
>>>> +	 * 'v4l2_ioctls' array, insert an ad-hoc check to address that.
>>>>    	 */
>>>> -	if (err < 0 && !always_copy)
>>>> +	if (err < 0 && !always_copy && cmd != VIDIOC_SUBDEV_G_ROUTING)
>>>>    		goto out;
>>>>    
>>>>    out_array_args:
>>>> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
>>>> index 956dafab43d4..95a4c3091fa6 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-subdev.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
>>>> @@ -681,6 +681,51 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg)
>>>>    	case VIDIOC_SUBDEV_QUERYSTD:
>>>>    		return v4l2_subdev_call(sd, video, querystd, arg);
>>>>    
>>>> +	case VIDIOC_SUBDEV_G_ROUTING: {
>>>> +		struct v4l2_subdev_routing *routing = arg;
>>>> +		struct v4l2_subdev_krouting krouting = {
>>>> +			.which = routing->which,
>>>> +			.num_routes = routing->num_routes,
>>>> +			.routes = (struct v4l2_subdev_route *)(uintptr_t)routing->routes,
>>>> +		};
>>>> +		int ret;
>>>> +
>>>> +		ret = v4l2_subdev_call(sd, pad, get_routing, &krouting);
>>>> +		if (ret)
>>>> +			return ret;
>>>> +
>>>> +		routing->num_routes = krouting.num_routes;
>>>
>>> Shouldn't num_routes be set even in case of errors ?
>>
>> Yes, you're right.
>>
>>>> +
>>>> +		return 0;
>>>> +	}
>>>> +
>>>> +	case VIDIOC_SUBDEV_S_ROUTING: {
>>>> +		struct v4l2_subdev_routing *routing = arg;
>>>> +		struct v4l2_subdev_route *route = (struct v4l2_subdev_route *)(uintptr_t)
>>>> +						  routing->routes;
>>>
>>> I'd name the variable routes as it points to an array.
>>>
>>>> +		struct v4l2_subdev_krouting krouting = {};
>>>> +		unsigned int i;
>>>> +
>>>> +		if (routing->which != V4L2_SUBDEV_FORMAT_TRY && ro_subdev)
>>>> +			return -EPERM;
>>>> +
>>>> +		for (i = 0; i < routing->num_routes; ++i) {
>>>> +			if (route[i].sink_pad >= sd->entity.num_pads ||
>>>> +			    route[i].source_pad >= sd->entity.num_pads)
>>>> +				return -EINVAL;
>>>> +
>>>> +			if (!(sd->entity.pads[route[i].sink_pad].flags & MEDIA_PAD_FL_SINK) ||
>>>> +			    !(sd->entity.pads[route[i].source_pad].flags & MEDIA_PAD_FL_SOURCE))
>>>> +				return -EINVAL;
>>>> +		}
>>>> +
>>>> +		krouting.which = routing->which;
>>>> +		krouting.num_routes = routing->num_routes;
>>>> +		krouting.routes = route;
>>>> +
>>>> +		return v4l2_subdev_call(sd, pad, set_routing, &krouting);
>>>> +	}
>>>> +
>>>>    	default:
>>>>    		return v4l2_subdev_call(sd, core, ioctl, cmd, arg);
>>>>    	}
>>>> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
>>>> index 436d0445aafd..3826ab918731 100644
>>>> --- a/include/media/v4l2-subdev.h
>>>> +++ b/include/media/v4l2-subdev.h
>>>> @@ -650,6 +650,22 @@ struct v4l2_subdev_pad_config {
>>>>    	struct v4l2_rect try_compose;
>>>>    };
>>>>    
>>>> +/**
>>>> + * struct v4l2_subdev_krouting - subdev routing table
>>>> + *
>>>> + * @which: format type (from enum v4l2_subdev_format_whence)
>>>> + * @routes: &struct v4l2_subdev_route
>>>> + * @num_routes: number of routes
>>>> + *
>>>> + * This structure is used to translate argument received from
>>>
>>> s/argument/the arguments/
>>>
>>>> + * VIDIOC_SUBDEV_G/S_ROUTING() ioctl to sudev device drivers operations.
>>>
>>> s/sudev/subdev/
>>>
>>>> + */
>>>> +struct v4l2_subdev_krouting {
>>>> +	u32 which;
>>>> +	struct v4l2_subdev_route *routes;
>>>> +	unsigned int num_routes;
>>>> +};
>>>> +
>>>>    /**
>>>>     * struct v4l2_subdev_pad_ops - v4l2-subdev pad level operations
>>>>     *
>>>> @@ -711,6 +727,10 @@ struct v4l2_subdev_pad_config {
>>>>     *		     applied to the hardware. The operation shall fail if the
>>>>     *		     pad index it has been called on is not valid or in case of
>>>>     *		     unrecoverable failures.
>>>> + *
>>>> + * @get_routing: get the subdevice routing table.
>>>> + * @set_routing: enable or disable data connection routes described in the
>>>> + *		 subdevice routing table.
>>>>     */
>>>>    struct v4l2_subdev_pad_ops {
>>>>    	int (*init_cfg)(struct v4l2_subdev *sd,
>>>> @@ -755,6 +775,10 @@ struct v4l2_subdev_pad_ops {
>>>>    			       struct v4l2_mbus_config *config);
>>>>    	int (*set_mbus_config)(struct v4l2_subdev *sd, unsigned int pad,
>>>>    			       struct v4l2_mbus_config *config);
>>>> +	int (*get_routing)(struct v4l2_subdev *sd,
>>>> +			   struct v4l2_subdev_krouting *route);
>>>> +	int (*set_routing)(struct v4l2_subdev *sd,
>>>> +			   struct v4l2_subdev_krouting *route);
>>>>    };
>>>>    
>>>>    /**
>>>> diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h
>>>> index 658106f5b5dc..f2a17cbd1e9a 100644
>>>> --- a/include/uapi/linux/v4l2-subdev.h
>>>> +++ b/include/uapi/linux/v4l2-subdev.h
>>>> @@ -188,6 +188,48 @@ struct v4l2_subdev_capability {
>>>>    /* The v4l2 sub-device video device node is registered in read-only mode. */
>>>>    #define V4L2_SUBDEV_CAP_RO_SUBDEV		0x00000001
>>>>    
>>>> +#define V4L2_SUBDEV_ROUTE_FL_ACTIVE		BIT(0)
>>>> +#define V4L2_SUBDEV_ROUTE_FL_IMMUTABLE		BIT(1)
>>>> +
>>>> +/**
>>>> + * struct v4l2_subdev_route - A route inside a subdev
>>>> + *
>>>> + * @sink_pad: the sink pad index
>>>> + * @sink_stream: the sink stream identifier
>>>> + * @source_pad: the source pad index
>>>> + * @source_stream: the source stream identifier
>>>> + * @flags: route flags:
>>>> + *
>>>> + *	V4L2_SUBDEV_ROUTE_FL_ACTIVE: Is the route active? An active
>>>> + *	route will start when streaming is enabled on a video node.
>>>> + *	Set by the user.
>>>> + *
>>>> + *	V4L2_SUBDEV_ROUTE_FL_IMMUTABLE: Is the route immutable, i.e.
>>>> + *	can it be activated and inactivated? Set by the driver.
>>>
>>> This should be moved to the macros.
>>>
>>>> + */
>>>> +struct v4l2_subdev_route {
>>>> +	__u32 sink_pad;
>>>> +	__u32 sink_stream;
>>>> +	__u32 source_pad;
>>>> +	__u32 source_stream;
>>>> +	__u32 flags;
>>>> +	__u32 reserved[5];
>>>> +};
>>>> +
>>>> +/**
>>>> + * struct v4l2_subdev_routing - Subdev routing information
>>>> + *
>>>> + * @which: format type (from enum v4l2_subdev_format_whence)
>>>
>>> v4l2_subdev_format_whence shoudn't have had "format" in its name :-S We
>>> can't do much about that, but should "format type" be changed to "routes
>>> type", or maybe "configuration type" ?
>>>
>>>> + * @routes: pointer to the routes array
>>>> + * @num_routes: the total number of routes in the routes array
>>>
>>> Won't kerneldoc complain about the undocumented reserved field ?
>>>
>>>> + */
>>>> +struct v4l2_subdev_routing {
>>>> +	__u32 which;
>>>> +	__u64 routes;
>>>> +	__u32 num_routes;
>>>> +	__u32 reserved[5];
>>>> +};
>>>> +
>>>>    /* Backwards compatibility define --- to be removed */
>>>>    #define v4l2_subdev_edid v4l2_edid
>>>>    
>>>> @@ -215,5 +257,7 @@ struct v4l2_subdev_capability {
>>>>    #define VIDIOC_SUBDEV_ENUM_DV_TIMINGS		_IOWR('V', 98, struct v4l2_enum_dv_timings)
>>>>    #define VIDIOC_SUBDEV_QUERY_DV_TIMINGS		_IOR('V', 99, struct v4l2_dv_timings)
>>>>    #define VIDIOC_SUBDEV_DV_TIMINGS_CAP		_IOWR('V', 100, struct v4l2_dv_timings_cap)
>>>> +#define VIDIOC_SUBDEV_G_ROUTING			_IOWR('V', 38, struct v4l2_subdev_routing)
>>>> +#define VIDIOC_SUBDEV_S_ROUTING			_IOWR('V', 39, struct v4l2_subdev_routing)
>>>
>>> I would keep the ioctls sorted by number.
>>
>> The ioctls are not sorted (even if it may look like it above =). I'm not
>> sure if there's a better place for them than the bottom.
> 
> Isn't the last block of ioctls (the ones reusing the V4L2 video node
> numbers) sorted ?

Oh, right, true. But, actually, these new ioctls shouldn't be here, as 
they are not "identical to the ioctls in videodev2.h". They should be 
moved above that comment, and the ioctls above that comment are not sorted.

  Tomi

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 11/24] media: entity: Skip link validation for pads to which there is no route to
  2021-04-20 11:41     ` Sakari Ailus
@ 2021-04-23 12:37       ` Tomi Valkeinen
  2021-04-29 12:06         ` Sakari Ailus
  0 siblings, 1 reply; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-23 12:37 UTC (permalink / raw)
  To: Sakari Ailus, Laurent Pinchart
  Cc: linux-media, Jacopo Mondi, niklas.soderlund+renesas,
	Mauro Carvalho Chehab, Hans Verkuil

On 20/04/2021 14:41, Sakari Ailus wrote:
> Hi Laurent,
> 
> On Sun, Apr 18, 2021 at 09:06:11PM +0300, Laurent Pinchart wrote:
>> Hi Tomi and Sakari,
>>
>> Thank you for the patch.
>>
>> There's an extra "to" in the subject line.
>>
>> On Thu, Apr 15, 2021 at 04:04:37PM +0300, Tomi Valkeinen wrote:
>>> From: Sakari Ailus <sakari.ailus@linux.intel.com>
>>>
>>> Links are validated along the pipeline which is about to start streaming.
>>> Not all the pads in entities that are traversed along that pipeline are
>>> part of the pipeline, however. Skip the link validation for such pads,
>>> and while at there rename "other_pad" to "local_pad" to convey the fact
>>> the route to be checked is internal to the entity.
>>
>> Both "pad" and "local_pad" are local. I would have kept the "other_pad"
> 
> The pad that in the remote entity is not local. The other one could be
> called remote_pad though.

I'm not sure what you mean here. Aren't both pad and local_pad pads of a 
single entity here? If so, I think Laurent's comment makes sense, and 
other_pad is a better name.

  Tomi

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 00/24] v4l: subdev internal routing
  2021-04-21 12:57   ` Tomi Valkeinen
@ 2021-04-29  1:27     ` Laurent Pinchart
  0 siblings, 0 replies; 72+ messages in thread
From: Laurent Pinchart @ 2021-04-29  1:27 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Tomi,

On Wed, Apr 21, 2021 at 03:57:07PM +0300, Tomi Valkeinen wrote:
> On 18/04/2021 20:32, Laurent Pinchart wrote:
> > On Thu, Apr 15, 2021 at 04:04:26PM +0300, Tomi Valkeinen wrote:
> >> Hi,
> >>
> >> This is an RFC for subdev internal routing which is needed for
> >> multiplexed streams support. I believe this is essentially a v5 of the
> >> series, the v4 posted here:
> >>
> >> https://lore.kernel.org/linux-media/20190328200608.9463-1-jacopo+renesas@jmondi.org/
> >>
> >> Most of the patches are not changed (aside from fixing rebase issues
> >> etc). The major changes in this version are:
> >>
> >> 1) Added 'which' field to the routing structs. It is currently not used,
> >> as implementing it is not trivial. However, I think it's good to add it
> >> to the uAPI now, and require the field to be set to
> >> V4L2_SUBDEV_FORMAT_ACTIVE for now. See this RFC for an idea how this
> >> could be implemented:
> >>
> >> https://lore.kernel.org/linux-media/20210409133659.389544-1-tomi.valkeinen@ideasonboard.com/
> > 
> > I've reviewed that, and I like it, but it's not straightforward to
> > understand from that patch how you envision TRY to be implemented.
> 
> To be honest I don't have much at all experience with TRY. But, afaics, 
> if we have means to store the TRY routes, and that is passed to the 
> relevant ioctls in the subdev's, isn't that enough? It's up to the 
> driver to implement the TRY functionality.

That should be fine yes.

> Although currently the S_ROUTING won't return anything to the userspace, 
> it's supposed to either accept the routing table or return an error, 
> whereas the S_FMT will do it's best to come up with a working setup and 
> return it.

That part is fine, as long as userspace has a way to figure out what
routes can be enabled, it's an good behaviour.

> >> 2) No hardcoded maximum number of routes. Defining a maximum is not
> >> possible, as there can be an arbitrary amount of routes per pad, and
> >> there can be an arbitrary amount of pads per subdev. This series
> >> allocates space for the routing table dynamically, which unfortunately
> >> leads to not-just-a-few allocs and frees.
> >>
> >> 3) When searching for a format for a stream, the v4 looked for a
> >> non-multiplexed pad only as far as the "other" side of the subdev. It
> >> wouldn't work for a subdev which has multiplexed sink and source pads.
> >> This series implements a "deep" get-format (v4l2_subdev_get_format_dir)
> >> which follows a stream either towards the original source or the final
> >> sink, while looking for a non-multiplexed pad with a format.
> >>
> >> Some thoughts:
> >>
> >> 1) Link validation and v4l2_subdev_get_format_dir need to look at the
> >> routing, and this leads to multiple allocs to get a copy of the routing
> >> table. There might be a possibility here to keep a table allocated and
> >> re-use it in consecutive get_routing calls.
> >>
> >> Or even better, perhaps the kAPI could be changed so that allocs are not
> >> needed. I thought about a kAPI where the subdev just returns a pointer
> >> to its routing table, but then we hit the life-cycle problem: how to
> >> ensure the table won't be freed or changed until the caller is done.
> > 
> > Storing the routing table in the v4l2_subdev_config (or
> > v4l2_subdev_state) would be one way to do so, and I'd like to explore
> > that direction. State lifetime is indeed an issue, and one simple option
> > would be to just take the graph lock to modify the routing.
> > 
> >> 2) The routing uAPI is a bit vague. There is no way for the userspace to
> >> figure out what kind of routing is allowed. Also, the existence of a
> >> route in the routing table already indicates that the route is active,
> >> but we also have V4L2_SUBDEV_ROUTE_FL_ACTIVE. I decided to keep
> >> V4L2_SUBDEV_ROUTE_FL_ACTIVE for now, even if it doesn't really provide
> >> any feature.
> > 
> > We can't report all possible routes if we take streams into account, but
> > maybe we could report all possible routes between pads ? This could go
> > through a separate ioctl.
> 
> That should be doable, but I wonder how much it helps. We should also 
> somehow indicate if, say, routes from two source pads can go to the same 
> sink pads, or can two streams from a single source pad go to separate 
> sink pads, and so on. Is it better just to document what the driver 
> supports for a specific hardware, than try to come up with a machine 
> readable representation of the possible routings.

I'd like userspace to have at least some level of genericity. I have two
devices in mind that have cross-bar switches between CSI-2 receivers and
DMA engines (or further processing pipelines), and I'd like that part to
be handled by userspace code that isn't device-specific. We can focus on
the API to configure routing first, and then see how discoverability
could be implemented.

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 12/24] media: entity: Add an iterator helper for connected pads
  2021-04-20 11:48     ` Sakari Ailus
@ 2021-04-29  1:33       ` Laurent Pinchart
  2021-04-29 11:56         ` Sakari Ailus
  0 siblings, 1 reply; 72+ messages in thread
From: Laurent Pinchart @ 2021-04-29  1:33 UTC (permalink / raw)
  To: Sakari Ailus
  Cc: Tomi Valkeinen, linux-media, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Sakari,

On Tue, Apr 20, 2021 at 02:48:25PM +0300, Sakari Ailus wrote:
> On Sun, Apr 18, 2021 at 09:20:03PM +0300, Laurent Pinchart wrote:
> > On Thu, Apr 15, 2021 at 04:04:38PM +0300, Tomi Valkeinen wrote:
> > > From: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > 
> > > Add a helper macro for iterating over pads that are connected through
> > > enabled routes. This can be used to find all the connected pads within an
> > > entity, for instance starting from the pad which has been obtained during
> > > the graph walk.
> > > 
> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > > 
> > > - Make __media_entity_next_routed_pad() return NULL and adjust the
> > >   iterator to handle that
> > > Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> > > ---
> > >  include/media/media-entity.h | 27 +++++++++++++++++++++++++++
> > >  1 file changed, 27 insertions(+)
> > > 
> > > diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> > > index 73de1c335e4e..edd6f60ed6b4 100644
> > > --- a/include/media/media-entity.h
> > > +++ b/include/media/media-entity.h
> > > @@ -916,6 +916,33 @@ __must_check int media_graph_walk_init(
> > >  bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
> > >  			    unsigned int pad1);
> > >  
> > > +static inline struct media_pad *__media_entity_next_routed_pad(
> > > +	struct media_pad *start, struct media_pad *iter)
> > > +{
> > > +	struct media_entity *entity = start->entity;
> > > +
> > > +	for (; iter < &entity->pads[entity->num_pads]; iter++)
> > > +		if (media_entity_has_route(entity, start->index, iter->index))
> > > +			return iter;
> > 
> > I'd use curly braces.
> > 
> > > +
> > > +	return NULL;
> > > +}
> > 
> > Does this need to be inlined ?
> 
> I guess it doesn't have to. It's used inside loops and it's rather small so
> I think it should be fine that way.

It may not be that small. I'd rather let the compiler decide whether to
inline it or not.

> > > +
> > > +/**
> > > + * media_entity_for_each_routed_pad - Iterate over entity pads connected by routes
> > 
> > "routed" sounds a bit weird. Would media_entity_for_each_connected_pad()
> > be a better name ?
> 
> "Connected" is often used in context of links. We're dealing with routes
> here, so I thought "routed" is appropriate to avoid confusion.

I understand the confusion, maybe we can find a better term that would
be different than "connected". "routed" really sounds weird in this
context.

> > > + *
> > > + * @start: The stating pad
> > 
> > s/stating/starting/
> > 
> > > + * @iter: The iterator pad
> > > + *
> > > + * Iterate over all pads connected through routes from a given pad
> > 
> > "from the @start pad"
> > 
> > > + * within an entity. The iteration will include the starting pad itself.
> > 
> > s/starting/@start/
> > 
> > I wonder if it wouldn't be more logical to not include the start pad.
> > That wouldn't match the current usage patterns, which would need to be
> > adapted accordingly, but I'm worried that including the start pad will
> > lead to annoying bugs in the future. Maybe I worry too much.
> 
> The aim here is to find all pads that are routed to another pad within the
> same entity. If you remove the start pad, it becomes a task harder than
> difficult.

Intuitively, "all pads that are routed to another pad" doesn't include
the "another pad". I'm not opposed to including the start pad as that's
what the current usage patterns need, but we should then rename the
macro accordingly as its current name is counter-intuitive.

> > And now that I reread the patch, I also wonder if "start" is a good
> > name, as it implies we start the enumeration from a given pad, while we
> > enumerate all pads connected to a given pad. I'm not sure what a better
> > name would be though, maybe just pad ?
> 
> There are two pads here. Therefore explicitly calling them something else
> makes sense IMO.

Makes sense, but "start" isn't a good name as we're not starting
anything.

> > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
> I agree on the comments I didn't reply to.
> 
> Thank you!
> 
> > > + */
> > > +#define media_entity_for_each_routed_pad(start, iter)			\
> > > +	for (iter = __media_entity_next_routed_pad(			\
> > > +		     start, (start)->entity->pads);			\
> > > +	     iter != NULL;						\
> > > +	     iter = __media_entity_next_routed_pad(start, iter + 1))
> > > +
> > >  /**
> > >   * media_graph_walk_cleanup - Release resources used by graph walk.
> > >   *

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 22/24] v4l: subdev: add v4l2_subdev_get_format_dir()
  2021-04-21 13:04     ` Tomi Valkeinen
@ 2021-04-29  1:43       ` Laurent Pinchart
  2021-05-04  6:49         ` Tomi Valkeinen
  0 siblings, 1 reply; 72+ messages in thread
From: Laurent Pinchart @ 2021-04-29  1:43 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Tomi,

On Wed, Apr 21, 2021 at 04:04:22PM +0300, Tomi Valkeinen wrote:
> On 18/04/2021 22:04, Laurent Pinchart wrote:
> > On Thu, Apr 15, 2021 at 04:04:48PM +0300, Tomi Valkeinen wrote:
> >> Add v4l2_subdev_get_format_dir() which can be used to find subdev format
> >> for a specific stream on a multiplexed pad. The function will follow the
> >> routes and links until it finds a non-multiplexed pad.
> >>
> >> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> >> ---
> >>   drivers/media/v4l2-core/v4l2-subdev.c | 96 +++++++++++++++++++++++++++
> >>   include/media/v4l2-subdev.h           | 26 ++++++++
> >>   2 files changed, 122 insertions(+)
> >>
> >> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> >> index 7a4f71d8c6c3..430dbdaab080 100644
> >> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> >> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> >> @@ -998,6 +998,102 @@ bool v4l2_subdev_has_route(struct v4l2_subdev_krouting *routing,
> >>   }
> >>   EXPORT_SYMBOL_GPL(v4l2_subdev_has_route);
> >>   
> >> +int v4l2_subdev_get_format_dir(struct media_pad *pad, u16 stream,
> >> +			       enum v4l2_direction dir,
> >> +			       struct v4l2_subdev_format *fmt)
> >> +{
> >> +	struct device *dev = pad->entity->graph_obj.mdev->dev;
> >> +	int ret;
> >> +	int i;
> >> +
> >> +	dev_dbg(dev, "%s '%s':%u:%u %s\n", __func__,
> >> +		pad->entity->name, pad->index, stream,
> >> +		dir == V4L2_DIR_SOURCEWARD ? "sourceward" : "sinkward");
> >> +
> >> +	while (true) {
> >> +		struct v4l2_subdev_krouting routing;
> >> +		struct v4l2_subdev_route *route;
> >> +
> >> +		if (pad->entity->obj_type != MEDIA_ENTITY_TYPE_V4L2_SUBDEV)
> >> +			return -EINVAL;
> >> +
> >> +		ret = v4l2_subdev_link_validate_get_format(pad, fmt);
> >> +		if (ret == 0)
> >> +			return 0;
> >> +		else if (ret != -ENOIOCTLCMD)
> >> +			return ret;
> >> +
> >> +		if (pad->flags &
> >> +		    (dir == V4L2_DIR_SINKWARD ? MEDIA_PAD_FL_SOURCE :
> >> +						MEDIA_PAD_FL_SINK)) {
> >> +			pad = media_entity_remote_pad(pad);
> >> +
> >> +			if (!pad)
> >> +				return -EINVAL;
> >> +
> >> +			if (pad->entity->obj_type != MEDIA_ENTITY_TYPE_V4L2_SUBDEV)
> >> +				return -EINVAL;
> >> +
> >> +			ret = v4l2_subdev_link_validate_get_format(pad, fmt);
> >> +			if (ret == 0)
> >> +				return 0;
> >> +			else if (ret != -ENOIOCTLCMD)
> >> +				return ret;
> >> +		}
> >> +
> >> +		ret = v4l2_subdev_get_krouting(media_entity_to_v4l2_subdev(pad->entity), &routing);
> >> +		if (ret)
> >> +			return ret;
> >> +
> >> +		route = NULL;
> >> +		for (i = 0; i < routing.num_routes; ++i) {
> >> +			u16 near_pad = dir == V4L2_DIR_SINKWARD ?
> >> +					       routing.routes[i].sink_pad :
> >> +					       routing.routes[i].source_pad;
> >> +			u16 near_stream = dir == V4L2_DIR_SINKWARD ?
> >> +						  routing.routes[i].sink_stream :
> >> +						  routing.routes[i].source_stream;
> >> +
> >> +			if (!(routing.routes[i].flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
> >> +				continue;
> >> +
> >> +			if (near_pad != pad->index)
> >> +				continue;
> >> +
> >> +			if (near_stream != stream)
> >> +				continue;
> >> +
> >> +			if (route) {
> >> +				dev_err(dev,
> >> +					"%s: '%s' has multiple active routes for stream %u\n",
> >> +					__func__, pad->entity->name, stream);
> >> +				v4l2_subdev_free_routing(&routing);
> >> +				return -EINVAL;
> >> +			}
> >> +
> >> +			route = &routing.routes[i];
> >> +		}
> >> +
> >> +		if (!route) {
> >> +			dev_err(dev, "%s: no route found in '%s' for stream %u\n",
> >> +				__func__, pad->entity->name, stream);
> >> +			v4l2_subdev_free_routing(&routing);
> >> +			return -EINVAL;
> >> +		}
> >> +
> >> +		if (dir == V4L2_DIR_SINKWARD) {
> >> +			pad = &pad->entity->pads[route->source_pad];
> >> +			stream = route->source_stream;
> >> +		} else {
> >> +			pad = &pad->entity->pads[route->sink_pad];
> >> +			stream = route->sink_stream;
> >> +		}
> >> +
> >> +		v4l2_subdev_free_routing(&routing);
> >> +	}
> >> +}
> >> +EXPORT_SYMBOL_GPL(v4l2_subdev_get_format_dir);
> >> +
> >>   int v4l2_subdev_link_validate(struct media_link *link)
> >>   {
> >>   	struct v4l2_subdev *sink;
> >> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> >> index 1843b77dd843..730631f9a091 100644
> >> --- a/include/media/v4l2-subdev.h
> >> +++ b/include/media/v4l2-subdev.h
> >> @@ -1239,4 +1239,30 @@ void v4l2_subdev_cpy_routing(struct v4l2_subdev_krouting *dst,
> >>   bool v4l2_subdev_has_route(struct v4l2_subdev_krouting *routing,
> >>   			   unsigned int pad0, unsigned int pad1);
> >>   
> >> +/**
> >> + * enum v4l2_direction - Direction either towards the source or the sink
> >> + *
> >> + * @V4L2_DIR_SOURCEWARD: Direction towards the source.
> >> + * @V4L2_DIR_SINKWARD: Direction towards the sink.
> >> + */
> >> +enum v4l2_direction {
> >> +	V4L2_DIR_SOURCEWARD,
> >> +	V4L2_DIR_SINKWARD,
> >> +};
> >> +
> >> +/**
> >> + * v4l2_subdev_get_format_dir() - Find format by following streams
> > 
> > The name is a bit cryptic, and the usage pattern error-prone. Can we do
> > better ?  In particular, if we could limit the usage of this function to
> > be called on a non-multiplexed pad, we could drop the stream argument.
> > Deducing the direction argument from the type of pad would also make the
> > API simpler.
> 
> Hmm, but that's not what the function does. It follows a specific 
> stream, from a multiplexed pad, so it has to get the stream as a parameter.
> 
> We can't deduct the direction from the type of the pad. We can of course 
> define that given a source pad this function goes sourceward. But if 
> that's not what the caller wants, then the caller needs to first follow 
> the stream either direction to get a sink pad, and then call this 
> function, which doesn't make sense.

What do the current callers need ? We don't have to implement something
that is more generic or featureful than our needs dictate, as this is a
very ad hoc function anyway. If we really need the full behaviour
implemented here, we should at the very least rename the function, but I
think it should be possible to do better overall, perhaps splitting the
operation in the caller into cleaner functions.

> >> + * @pad: The pad from which to start the search
> >> + * @stream: The stream for which we want to find the format
> >> + * @dir: The direction of the search
> >> + * @fmt: Pointer to &struct v4l2_subdev_format where the found format is stored
> >> + *
> >> + * This function attempts to find v4l2_subdev_format for a specific stream on a
> >> + * multiplexed pad by following the stream using routes and links to the specified
> >> + * direction, until a non-multiplexed pad is found.
> >> + */
> >> +int v4l2_subdev_get_format_dir(struct media_pad *pad, u16 stream,
> >> +			       enum v4l2_direction dir,
> >> +			       struct v4l2_subdev_format *fmt);
> >> +
> >>   #endif

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 12/24] media: entity: Add an iterator helper for connected pads
  2021-04-29  1:33       ` Laurent Pinchart
@ 2021-04-29 11:56         ` Sakari Ailus
  2021-04-29 12:04           ` Tomi Valkeinen
  0 siblings, 1 reply; 72+ messages in thread
From: Sakari Ailus @ 2021-04-29 11:56 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: Tomi Valkeinen, linux-media, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Laurent,

On Thu, Apr 29, 2021 at 04:33:48AM +0300, Laurent Pinchart wrote:
> Hi Sakari,
> 
> On Tue, Apr 20, 2021 at 02:48:25PM +0300, Sakari Ailus wrote:
> > On Sun, Apr 18, 2021 at 09:20:03PM +0300, Laurent Pinchart wrote:
> > > On Thu, Apr 15, 2021 at 04:04:38PM +0300, Tomi Valkeinen wrote:
> > > > From: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > 
> > > > Add a helper macro for iterating over pads that are connected through
> > > > enabled routes. This can be used to find all the connected pads within an
> > > > entity, for instance starting from the pad which has been obtained during
> > > > the graph walk.
> > > > 
> > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > > > 
> > > > - Make __media_entity_next_routed_pad() return NULL and adjust the
> > > >   iterator to handle that
> > > > Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> > > > ---
> > > >  include/media/media-entity.h | 27 +++++++++++++++++++++++++++
> > > >  1 file changed, 27 insertions(+)
> > > > 
> > > > diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> > > > index 73de1c335e4e..edd6f60ed6b4 100644
> > > > --- a/include/media/media-entity.h
> > > > +++ b/include/media/media-entity.h
> > > > @@ -916,6 +916,33 @@ __must_check int media_graph_walk_init(
> > > >  bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
> > > >  			    unsigned int pad1);
> > > >  
> > > > +static inline struct media_pad *__media_entity_next_routed_pad(
> > > > +	struct media_pad *start, struct media_pad *iter)
> > > > +{
> > > > +	struct media_entity *entity = start->entity;
> > > > +
> > > > +	for (; iter < &entity->pads[entity->num_pads]; iter++)
> > > > +		if (media_entity_has_route(entity, start->index, iter->index))
> > > > +			return iter;
> > > 
> > > I'd use curly braces.
> > > 
> > > > +
> > > > +	return NULL;
> > > > +}
> > > 
> > > Does this need to be inlined ?
> > 
> > I guess it doesn't have to. It's used inside loops and it's rather small so
> > I think it should be fine that way.
> 
> It may not be that small. I'd rather let the compiler decide whether to
> inline it or not.

Works for me.

> 
> > > > +
> > > > +/**
> > > > + * media_entity_for_each_routed_pad - Iterate over entity pads connected by routes
> > > 
> > > "routed" sounds a bit weird. Would media_entity_for_each_connected_pad()
> > > be a better name ?
> > 
> > "Connected" is often used in context of links. We're dealing with routes
> > here, so I thought "routed" is appropriate to avoid confusion.
> 
> I understand the confusion, maybe we can find a better term that would
> be different than "connected". "routed" really sounds weird in this
> context.

I'm fine with connected.

> 
> > > > + *
> > > > + * @start: The stating pad
> > > 
> > > s/stating/starting/
> > > 
> > > > + * @iter: The iterator pad
> > > > + *
> > > > + * Iterate over all pads connected through routes from a given pad
> > > 
> > > "from the @start pad"
> > > 
> > > > + * within an entity. The iteration will include the starting pad itself.
> > > 
> > > s/starting/@start/
> > > 
> > > I wonder if it wouldn't be more logical to not include the start pad.
> > > That wouldn't match the current usage patterns, which would need to be
> > > adapted accordingly, but I'm worried that including the start pad will
> > > lead to annoying bugs in the future. Maybe I worry too much.
> > 
> > The aim here is to find all pads that are routed to another pad within the
> > same entity. If you remove the start pad, it becomes a task harder than
> > difficult.
> 
> Intuitively, "all pads that are routed to another pad" doesn't include
> the "another pad". I'm not opposed to including the start pad as that's
> what the current usage patterns need, but we should then rename the
> macro accordingly as its current name is counter-intuitive.

I'm certainly not opposed to that. But it shouldn't be too much longer than
what's already there.

> 
> > > And now that I reread the patch, I also wonder if "start" is a good
> > > name, as it implies we start the enumeration from a given pad, while we
> > > enumerate all pads connected to a given pad. I'm not sure what a better
> > > name would be though, maybe just pad ?
> > 
> > There are two pads here. Therefore explicitly calling them something else
> > makes sense IMO.
> 
> Makes sense, but "start" isn't a good name as we're not starting
> anything.

"start" is not a verb here. It's where the iteration *starts*.

> 
> > > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > 
> > I agree on the comments I didn't reply to.
> > 
> > Thank you!
> > 
> > > > + */
> > > > +#define media_entity_for_each_routed_pad(start, iter)			\
> > > > +	for (iter = __media_entity_next_routed_pad(			\
> > > > +		     start, (start)->entity->pads);			\
> > > > +	     iter != NULL;						\
> > > > +	     iter = __media_entity_next_routed_pad(start, iter + 1))
> > > > +
> > > >  /**
> > > >   * media_graph_walk_cleanup - Release resources used by graph walk.
> > > >   *
> 

-- 
Regards,

Sakari Ailus

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 15/24] v4l: Add bus type to frame descriptors
  2021-04-22 12:30       ` Tomi Valkeinen
@ 2021-04-29 11:58         ` Sakari Ailus
  2021-04-29 14:09           ` Laurent Pinchart
  0 siblings, 1 reply; 72+ messages in thread
From: Sakari Ailus @ 2021-04-29 11:58 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: Laurent Pinchart, linux-media, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

On Thu, Apr 22, 2021 at 03:30:55PM +0300, Tomi Valkeinen wrote:
> On 20/04/2021 14:50, Sakari Ailus wrote:
> > Hi Laurent,
> > 
> > On Sun, Apr 18, 2021 at 10:23:35PM +0300, Laurent Pinchart wrote:
> > > Hi Tomi and Sakari,
> > > 
> > > Thank you for the patch.
> > > 
> > > On Thu, Apr 15, 2021 at 04:04:41PM +0300, Tomi Valkeinen wrote:
> > > > From: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > 
> > > > Add the media bus type to the frame descriptor. CSI-2 specific
> > > > information will be added in next patch to the frame descriptor.
> > > 
> > > I'd squash the next patch with this one.
> > > 
> > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > > > 
> > > > - Make the bus type a named enum
> > > > Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> > > > ---
> > > >   include/media/v4l2-subdev.h | 9 +++++++++
> > > >   1 file changed, 9 insertions(+)
> > > > 
> > > > diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> > > > index d0e9a5bdb08b..85977abbea46 100644
> > > > --- a/include/media/v4l2-subdev.h
> > > > +++ b/include/media/v4l2-subdev.h
> > > > @@ -340,12 +340,21 @@ struct v4l2_mbus_frame_desc_entry {
> > > >   #define V4L2_FRAME_DESC_ENTRY_MAX	4
> > > > +enum v4l2_mbus_frame_desc_type {
> > > > +	V4L2_MBUS_FRAME_DESC_TYPE_PLATFORM,
> > > > +	V4L2_MBUS_FRAME_DESC_TYPE_PARALLEL,
> > > > +	V4L2_MBUS_FRAME_DESC_TYPE_CCP2,
> > > > +	V4L2_MBUS_FRAME_DESC_TYPE_CSI2,
> > > > +};
> > > 
> > > This should be documented. In particular, I have no idea what
> > > V4L2_MBUS_FRAME_DESC_TYPE_PLATFORM is. I also wonder if we shouldn't
> > > drop CCP2 (at least for now), does anyone use that anymore ?
> > 
> > I guess we don't need one here, not now at least.
> > 
> > I agree on the documentation.
> 
> As it's the first one in the list, I think it really means "undefined", so
> that current users have a value there (I presume they initialize the struct
> to 0). Sakari?

I guess you could drop PLATFORM if there's no need for it now. In practice
PARALLEL is probably a good choice.

-- 
Sakari Ailus

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 12/24] media: entity: Add an iterator helper for connected pads
  2021-04-29 11:56         ` Sakari Ailus
@ 2021-04-29 12:04           ` Tomi Valkeinen
  2021-04-29 12:07             ` Sakari Ailus
  0 siblings, 1 reply; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-29 12:04 UTC (permalink / raw)
  To: Sakari Ailus, Laurent Pinchart
  Cc: linux-media, Jacopo Mondi, niklas.soderlund+renesas,
	Mauro Carvalho Chehab, Hans Verkuil

On 29/04/2021 14:56, Sakari Ailus wrote:
> Hi Laurent,
> 
> On Thu, Apr 29, 2021 at 04:33:48AM +0300, Laurent Pinchart wrote:
>> Hi Sakari,
>>
>> On Tue, Apr 20, 2021 at 02:48:25PM +0300, Sakari Ailus wrote:
>>> On Sun, Apr 18, 2021 at 09:20:03PM +0300, Laurent Pinchart wrote:
>>>> On Thu, Apr 15, 2021 at 04:04:38PM +0300, Tomi Valkeinen wrote:
>>>>> From: Sakari Ailus <sakari.ailus@linux.intel.com>
>>>>>
>>>>> Add a helper macro for iterating over pads that are connected through
>>>>> enabled routes. This can be used to find all the connected pads within an
>>>>> entity, for instance starting from the pad which has been obtained during
>>>>> the graph walk.
>>>>>
>>>>> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
>>>>> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
>>>>>
>>>>> - Make __media_entity_next_routed_pad() return NULL and adjust the
>>>>>    iterator to handle that
>>>>> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
>>>>> ---
>>>>>   include/media/media-entity.h | 27 +++++++++++++++++++++++++++
>>>>>   1 file changed, 27 insertions(+)
>>>>>
>>>>> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
>>>>> index 73de1c335e4e..edd6f60ed6b4 100644
>>>>> --- a/include/media/media-entity.h
>>>>> +++ b/include/media/media-entity.h
>>>>> @@ -916,6 +916,33 @@ __must_check int media_graph_walk_init(
>>>>>   bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
>>>>>   			    unsigned int pad1);
>>>>>   
>>>>> +static inline struct media_pad *__media_entity_next_routed_pad(
>>>>> +	struct media_pad *start, struct media_pad *iter)
>>>>> +{
>>>>> +	struct media_entity *entity = start->entity;
>>>>> +
>>>>> +	for (; iter < &entity->pads[entity->num_pads]; iter++)
>>>>> +		if (media_entity_has_route(entity, start->index, iter->index))
>>>>> +			return iter;
>>>>
>>>> I'd use curly braces.
>>>>
>>>>> +
>>>>> +	return NULL;
>>>>> +}
>>>>
>>>> Does this need to be inlined ?
>>>
>>> I guess it doesn't have to. It's used inside loops and it's rather small so
>>> I think it should be fine that way.
>>
>> It may not be that small. I'd rather let the compiler decide whether to
>> inline it or not.
> 
> Works for me.
> 
>>
>>>>> +
>>>>> +/**
>>>>> + * media_entity_for_each_routed_pad - Iterate over entity pads connected by routes
>>>>
>>>> "routed" sounds a bit weird. Would media_entity_for_each_connected_pad()
>>>> be a better name ?
>>>
>>> "Connected" is often used in context of links. We're dealing with routes
>>> here, so I thought "routed" is appropriate to avoid confusion.
>>
>> I understand the confusion, maybe we can find a better term that would
>> be different than "connected". "routed" really sounds weird in this
>> context.
> 
> I'm fine with connected.
> 
>>
>>>>> + *
>>>>> + * @start: The stating pad
>>>>
>>>> s/stating/starting/
>>>>
>>>>> + * @iter: The iterator pad
>>>>> + *
>>>>> + * Iterate over all pads connected through routes from a given pad
>>>>
>>>> "from the @start pad"
>>>>
>>>>> + * within an entity. The iteration will include the starting pad itself.
>>>>
>>>> s/starting/@start/
>>>>
>>>> I wonder if it wouldn't be more logical to not include the start pad.
>>>> That wouldn't match the current usage patterns, which would need to be
>>>> adapted accordingly, but I'm worried that including the start pad will
>>>> lead to annoying bugs in the future. Maybe I worry too much.
>>>
>>> The aim here is to find all pads that are routed to another pad within the
>>> same entity. If you remove the start pad, it becomes a task harder than
>>> difficult.
>>
>> Intuitively, "all pads that are routed to another pad" doesn't include
>> the "another pad". I'm not opposed to including the start pad as that's
>> what the current usage patterns need, but we should then rename the
>> macro accordingly as its current name is counter-intuitive.
> 
> I'm certainly not opposed to that. But it shouldn't be too much longer than
> what's already there.
> 
>>
>>>> And now that I reread the patch, I also wonder if "start" is a good
>>>> name, as it implies we start the enumeration from a given pad, while we
>>>> enumerate all pads connected to a given pad. I'm not sure what a better
>>>> name would be though, maybe just pad ?
>>>
>>> There are two pads here. Therefore explicitly calling them something else
>>> makes sense IMO.
>>
>> Makes sense, but "start" isn't a good name as we're not starting
>> anything.
> 
> "start" is not a verb here. It's where the iteration *starts*.

Hmm, no, the 'start' is a filter here, isn't it? The macro iterates over 
all pads which have a route to 'start'.

  Tomi

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 11/24] media: entity: Skip link validation for pads to which there is no route to
  2021-04-23 12:37       ` Tomi Valkeinen
@ 2021-04-29 12:06         ` Sakari Ailus
  2021-04-29 14:10           ` Laurent Pinchart
  0 siblings, 1 reply; 72+ messages in thread
From: Sakari Ailus @ 2021-04-29 12:06 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: Laurent Pinchart, linux-media, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Moi,

On Fri, Apr 23, 2021 at 03:37:03PM +0300, Tomi Valkeinen wrote:
> On 20/04/2021 14:41, Sakari Ailus wrote:
> > Hi Laurent,
> > 
> > On Sun, Apr 18, 2021 at 09:06:11PM +0300, Laurent Pinchart wrote:
> > > Hi Tomi and Sakari,
> > > 
> > > Thank you for the patch.
> > > 
> > > There's an extra "to" in the subject line.
> > > 
> > > On Thu, Apr 15, 2021 at 04:04:37PM +0300, Tomi Valkeinen wrote:
> > > > From: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > 
> > > > Links are validated along the pipeline which is about to start streaming.
> > > > Not all the pads in entities that are traversed along that pipeline are
> > > > part of the pipeline, however. Skip the link validation for such pads,
> > > > and while at there rename "other_pad" to "local_pad" to convey the fact
> > > > the route to be checked is internal to the entity.
> > > 
> > > Both "pad" and "local_pad" are local. I would have kept the "other_pad"
> > 
> > The pad that in the remote entity is not local. The other one could be
> > called remote_pad though.
> 
> I'm not sure what you mean here. Aren't both pad and local_pad pads of a
> single entity here? If so, I think Laurent's comment makes sense, and
> other_pad is a better name.

I guess you could argue for either. I'm fine dropping the patch.

-- 
Sakari Ailus

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 12/24] media: entity: Add an iterator helper for connected pads
  2021-04-29 12:04           ` Tomi Valkeinen
@ 2021-04-29 12:07             ` Sakari Ailus
  2021-04-29 12:14               ` Tomi Valkeinen
  0 siblings, 1 reply; 72+ messages in thread
From: Sakari Ailus @ 2021-04-29 12:07 UTC (permalink / raw)
  To: Tomi Valkeinen
  Cc: Laurent Pinchart, linux-media, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

On Thu, Apr 29, 2021 at 03:04:38PM +0300, Tomi Valkeinen wrote:
> On 29/04/2021 14:56, Sakari Ailus wrote:
> > Hi Laurent,
> > 
> > On Thu, Apr 29, 2021 at 04:33:48AM +0300, Laurent Pinchart wrote:
> > > Hi Sakari,
> > > 
> > > On Tue, Apr 20, 2021 at 02:48:25PM +0300, Sakari Ailus wrote:
> > > > On Sun, Apr 18, 2021 at 09:20:03PM +0300, Laurent Pinchart wrote:
> > > > > On Thu, Apr 15, 2021 at 04:04:38PM +0300, Tomi Valkeinen wrote:
> > > > > > From: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > > > 
> > > > > > Add a helper macro for iterating over pads that are connected through
> > > > > > enabled routes. This can be used to find all the connected pads within an
> > > > > > entity, for instance starting from the pad which has been obtained during
> > > > > > the graph walk.
> > > > > > 
> > > > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > > > Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > > > > > 
> > > > > > - Make __media_entity_next_routed_pad() return NULL and adjust the
> > > > > >    iterator to handle that
> > > > > > Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> > > > > > ---
> > > > > >   include/media/media-entity.h | 27 +++++++++++++++++++++++++++
> > > > > >   1 file changed, 27 insertions(+)
> > > > > > 
> > > > > > diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> > > > > > index 73de1c335e4e..edd6f60ed6b4 100644
> > > > > > --- a/include/media/media-entity.h
> > > > > > +++ b/include/media/media-entity.h
> > > > > > @@ -916,6 +916,33 @@ __must_check int media_graph_walk_init(
> > > > > >   bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
> > > > > >   			    unsigned int pad1);
> > > > > > +static inline struct media_pad *__media_entity_next_routed_pad(
> > > > > > +	struct media_pad *start, struct media_pad *iter)
> > > > > > +{
> > > > > > +	struct media_entity *entity = start->entity;
> > > > > > +
> > > > > > +	for (; iter < &entity->pads[entity->num_pads]; iter++)
> > > > > > +		if (media_entity_has_route(entity, start->index, iter->index))
> > > > > > +			return iter;
> > > > > 
> > > > > I'd use curly braces.
> > > > > 
> > > > > > +
> > > > > > +	return NULL;
> > > > > > +}
> > > > > 
> > > > > Does this need to be inlined ?
> > > > 
> > > > I guess it doesn't have to. It's used inside loops and it's rather small so
> > > > I think it should be fine that way.
> > > 
> > > It may not be that small. I'd rather let the compiler decide whether to
> > > inline it or not.
> > 
> > Works for me.
> > 
> > > 
> > > > > > +
> > > > > > +/**
> > > > > > + * media_entity_for_each_routed_pad - Iterate over entity pads connected by routes
> > > > > 
> > > > > "routed" sounds a bit weird. Would media_entity_for_each_connected_pad()
> > > > > be a better name ?
> > > > 
> > > > "Connected" is often used in context of links. We're dealing with routes
> > > > here, so I thought "routed" is appropriate to avoid confusion.
> > > 
> > > I understand the confusion, maybe we can find a better term that would
> > > be different than "connected". "routed" really sounds weird in this
> > > context.
> > 
> > I'm fine with connected.
> > 
> > > 
> > > > > > + *
> > > > > > + * @start: The stating pad
> > > > > 
> > > > > s/stating/starting/
> > > > > 
> > > > > > + * @iter: The iterator pad
> > > > > > + *
> > > > > > + * Iterate over all pads connected through routes from a given pad
> > > > > 
> > > > > "from the @start pad"
> > > > > 
> > > > > > + * within an entity. The iteration will include the starting pad itself.
> > > > > 
> > > > > s/starting/@start/
> > > > > 
> > > > > I wonder if it wouldn't be more logical to not include the start pad.
> > > > > That wouldn't match the current usage patterns, which would need to be
> > > > > adapted accordingly, but I'm worried that including the start pad will
> > > > > lead to annoying bugs in the future. Maybe I worry too much.
> > > > 
> > > > The aim here is to find all pads that are routed to another pad within the
> > > > same entity. If you remove the start pad, it becomes a task harder than
> > > > difficult.
> > > 
> > > Intuitively, "all pads that are routed to another pad" doesn't include
> > > the "another pad". I'm not opposed to including the start pad as that's
> > > what the current usage patterns need, but we should then rename the
> > > macro accordingly as its current name is counter-intuitive.
> > 
> > I'm certainly not opposed to that. But it shouldn't be too much longer than
> > what's already there.
> > 
> > > 
> > > > > And now that I reread the patch, I also wonder if "start" is a good
> > > > > name, as it implies we start the enumeration from a given pad, while we
> > > > > enumerate all pads connected to a given pad. I'm not sure what a better
> > > > > name would be though, maybe just pad ?
> > > > 
> > > > There are two pads here. Therefore explicitly calling them something else
> > > > makes sense IMO.
> > > 
> > > Makes sense, but "start" isn't a good name as we're not starting
> > > anything.
> > 
> > "start" is not a verb here. It's where the iteration *starts*.
> 
> Hmm, no, the 'start' is a filter here, isn't it? The macro iterates over all
> pads which have a route to 'start'.

The iteration starts from "start", but it does not return all pads, only
the connected ones.

But feel free to use another name if you have a better one.

-- 
Sakari Ailus

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 12/24] media: entity: Add an iterator helper for connected pads
  2021-04-29 12:07             ` Sakari Ailus
@ 2021-04-29 12:14               ` Tomi Valkeinen
  0 siblings, 0 replies; 72+ messages in thread
From: Tomi Valkeinen @ 2021-04-29 12:14 UTC (permalink / raw)
  To: Sakari Ailus
  Cc: Laurent Pinchart, linux-media, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

On 29/04/2021 15:07, Sakari Ailus wrote:
> On Thu, Apr 29, 2021 at 03:04:38PM +0300, Tomi Valkeinen wrote:
>> On 29/04/2021 14:56, Sakari Ailus wrote:
>>> Hi Laurent,
>>>
>>> On Thu, Apr 29, 2021 at 04:33:48AM +0300, Laurent Pinchart wrote:
>>>> Hi Sakari,
>>>>
>>>> On Tue, Apr 20, 2021 at 02:48:25PM +0300, Sakari Ailus wrote:
>>>>> On Sun, Apr 18, 2021 at 09:20:03PM +0300, Laurent Pinchart wrote:
>>>>>> On Thu, Apr 15, 2021 at 04:04:38PM +0300, Tomi Valkeinen wrote:
>>>>>>> From: Sakari Ailus <sakari.ailus@linux.intel.com>
>>>>>>>
>>>>>>> Add a helper macro for iterating over pads that are connected through
>>>>>>> enabled routes. This can be used to find all the connected pads within an
>>>>>>> entity, for instance starting from the pad which has been obtained during
>>>>>>> the graph walk.
>>>>>>>
>>>>>>> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
>>>>>>> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
>>>>>>>
>>>>>>> - Make __media_entity_next_routed_pad() return NULL and adjust the
>>>>>>>     iterator to handle that
>>>>>>> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
>>>>>>> ---
>>>>>>>    include/media/media-entity.h | 27 +++++++++++++++++++++++++++
>>>>>>>    1 file changed, 27 insertions(+)
>>>>>>>
>>>>>>> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
>>>>>>> index 73de1c335e4e..edd6f60ed6b4 100644
>>>>>>> --- a/include/media/media-entity.h
>>>>>>> +++ b/include/media/media-entity.h
>>>>>>> @@ -916,6 +916,33 @@ __must_check int media_graph_walk_init(
>>>>>>>    bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
>>>>>>>    			    unsigned int pad1);
>>>>>>> +static inline struct media_pad *__media_entity_next_routed_pad(
>>>>>>> +	struct media_pad *start, struct media_pad *iter)
>>>>>>> +{
>>>>>>> +	struct media_entity *entity = start->entity;
>>>>>>> +
>>>>>>> +	for (; iter < &entity->pads[entity->num_pads]; iter++)
>>>>>>> +		if (media_entity_has_route(entity, start->index, iter->index))
>>>>>>> +			return iter;
>>>>>>
>>>>>> I'd use curly braces.
>>>>>>
>>>>>>> +
>>>>>>> +	return NULL;
>>>>>>> +}
>>>>>>
>>>>>> Does this need to be inlined ?
>>>>>
>>>>> I guess it doesn't have to. It's used inside loops and it's rather small so
>>>>> I think it should be fine that way.
>>>>
>>>> It may not be that small. I'd rather let the compiler decide whether to
>>>> inline it or not.
>>>
>>> Works for me.
>>>
>>>>
>>>>>>> +
>>>>>>> +/**
>>>>>>> + * media_entity_for_each_routed_pad - Iterate over entity pads connected by routes
>>>>>>
>>>>>> "routed" sounds a bit weird. Would media_entity_for_each_connected_pad()
>>>>>> be a better name ?
>>>>>
>>>>> "Connected" is often used in context of links. We're dealing with routes
>>>>> here, so I thought "routed" is appropriate to avoid confusion.
>>>>
>>>> I understand the confusion, maybe we can find a better term that would
>>>> be different than "connected". "routed" really sounds weird in this
>>>> context.
>>>
>>> I'm fine with connected.
>>>
>>>>
>>>>>>> + *
>>>>>>> + * @start: The stating pad
>>>>>>
>>>>>> s/stating/starting/
>>>>>>
>>>>>>> + * @iter: The iterator pad
>>>>>>> + *
>>>>>>> + * Iterate over all pads connected through routes from a given pad
>>>>>>
>>>>>> "from the @start pad"
>>>>>>
>>>>>>> + * within an entity. The iteration will include the starting pad itself.
>>>>>>
>>>>>> s/starting/@start/
>>>>>>
>>>>>> I wonder if it wouldn't be more logical to not include the start pad.
>>>>>> That wouldn't match the current usage patterns, which would need to be
>>>>>> adapted accordingly, but I'm worried that including the start pad will
>>>>>> lead to annoying bugs in the future. Maybe I worry too much.
>>>>>
>>>>> The aim here is to find all pads that are routed to another pad within the
>>>>> same entity. If you remove the start pad, it becomes a task harder than
>>>>> difficult.
>>>>
>>>> Intuitively, "all pads that are routed to another pad" doesn't include
>>>> the "another pad". I'm not opposed to including the start pad as that's
>>>> what the current usage patterns need, but we should then rename the
>>>> macro accordingly as its current name is counter-intuitive.
>>>
>>> I'm certainly not opposed to that. But it shouldn't be too much longer than
>>> what's already there.
>>>
>>>>
>>>>>> And now that I reread the patch, I also wonder if "start" is a good
>>>>>> name, as it implies we start the enumeration from a given pad, while we
>>>>>> enumerate all pads connected to a given pad. I'm not sure what a better
>>>>>> name would be though, maybe just pad ?
>>>>>
>>>>> There are two pads here. Therefore explicitly calling them something else
>>>>> makes sense IMO.
>>>>
>>>> Makes sense, but "start" isn't a good name as we're not starting
>>>> anything.
>>>
>>> "start" is not a verb here. It's where the iteration *starts*.
>>
>> Hmm, no, the 'start' is a filter here, isn't it? The macro iterates over all
>> pads which have a route to 'start'.
> 
> The iteration starts from "start", but it does not return all pads, only
> the connected ones.

No, I don't think it does. It starts from the first pad, and if that is 
connected (has route to 'start') it returns that the first pad.

> But feel free to use another name if you have a better one.

I don't have one, even if I tried. "common_connected_pad" is a bit too 
long =).

Although 'start' hints that it would be the start for the iteration, 
which is misleading. So perhaps 'connected' is better, if only to remove 
the confusion.

  Tomi

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 15/24] v4l: Add bus type to frame descriptors
  2021-04-29 11:58         ` Sakari Ailus
@ 2021-04-29 14:09           ` Laurent Pinchart
  0 siblings, 0 replies; 72+ messages in thread
From: Laurent Pinchart @ 2021-04-29 14:09 UTC (permalink / raw)
  To: Sakari Ailus
  Cc: Tomi Valkeinen, linux-media, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Sakari,

On Thu, Apr 29, 2021 at 02:58:11PM +0300, Sakari Ailus wrote:
> On Thu, Apr 22, 2021 at 03:30:55PM +0300, Tomi Valkeinen wrote:
> > On 20/04/2021 14:50, Sakari Ailus wrote:
> > > On Sun, Apr 18, 2021 at 10:23:35PM +0300, Laurent Pinchart wrote:
> > > > On Thu, Apr 15, 2021 at 04:04:41PM +0300, Tomi Valkeinen wrote:
> > > > > From: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > > 
> > > > > Add the media bus type to the frame descriptor. CSI-2 specific
> > > > > information will be added in next patch to the frame descriptor.
> > > > 
> > > > I'd squash the next patch with this one.
> > > > 
> > > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > > Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > > > > 
> > > > > - Make the bus type a named enum
> > > > > Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> > > > > ---
> > > > >   include/media/v4l2-subdev.h | 9 +++++++++
> > > > >   1 file changed, 9 insertions(+)
> > > > > 
> > > > > diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> > > > > index d0e9a5bdb08b..85977abbea46 100644
> > > > > --- a/include/media/v4l2-subdev.h
> > > > > +++ b/include/media/v4l2-subdev.h
> > > > > @@ -340,12 +340,21 @@ struct v4l2_mbus_frame_desc_entry {
> > > > >   #define V4L2_FRAME_DESC_ENTRY_MAX	4
> > > > > +enum v4l2_mbus_frame_desc_type {
> > > > > +	V4L2_MBUS_FRAME_DESC_TYPE_PLATFORM,
> > > > > +	V4L2_MBUS_FRAME_DESC_TYPE_PARALLEL,
> > > > > +	V4L2_MBUS_FRAME_DESC_TYPE_CCP2,
> > > > > +	V4L2_MBUS_FRAME_DESC_TYPE_CSI2,
> > > > > +};
> > > > 
> > > > This should be documented. In particular, I have no idea what
> > > > V4L2_MBUS_FRAME_DESC_TYPE_PLATFORM is. I also wonder if we shouldn't
> > > > drop CCP2 (at least for now), does anyone use that anymore ?
> > > 
> > > I guess we don't need one here, not now at least.
> > > 
> > > I agree on the documentation.
> > 
> > As it's the first one in the list, I think it really means "undefined", so
> > that current users have a value there (I presume they initialize the struct
> > to 0). Sakari?
> 
> I guess you could drop PLATFORM if there's no need for it now. In practice
> PARALLEL is probably a good choice.

Works for me too.

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 11/24] media: entity: Skip link validation for pads to which there is no route to
  2021-04-29 12:06         ` Sakari Ailus
@ 2021-04-29 14:10           ` Laurent Pinchart
  0 siblings, 0 replies; 72+ messages in thread
From: Laurent Pinchart @ 2021-04-29 14:10 UTC (permalink / raw)
  To: Sakari Ailus
  Cc: Tomi Valkeinen, linux-media, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

Hi Sakari,

On Thu, Apr 29, 2021 at 03:06:12PM +0300, Sakari Ailus wrote:
> On Fri, Apr 23, 2021 at 03:37:03PM +0300, Tomi Valkeinen wrote:
> > On 20/04/2021 14:41, Sakari Ailus wrote:
> > > On Sun, Apr 18, 2021 at 09:06:11PM +0300, Laurent Pinchart wrote:
> > > > Hi Tomi and Sakari,
> > > > 
> > > > Thank you for the patch.
> > > > 
> > > > There's an extra "to" in the subject line.
> > > > 
> > > > On Thu, Apr 15, 2021 at 04:04:37PM +0300, Tomi Valkeinen wrote:
> > > > > From: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > > 
> > > > > Links are validated along the pipeline which is about to start streaming.
> > > > > Not all the pads in entities that are traversed along that pipeline are
> > > > > part of the pipeline, however. Skip the link validation for such pads,
> > > > > and while at there rename "other_pad" to "local_pad" to convey the fact
> > > > > the route to be checked is internal to the entity.
> > > > 
> > > > Both "pad" and "local_pad" are local. I would have kept the "other_pad"
> > > 
> > > The pad that in the remote entity is not local. The other one could be
> > > called remote_pad though.
> > 
> > I'm not sure what you mean here. Aren't both pad and local_pad pads of a
> > single entity here? If so, I think Laurent's comment makes sense, and
> > other_pad is a better name.
> 
> I guess you could argue for either. I'm fine dropping the patch.

I think the patch is good, it's just the extra rename that puzzled me.

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 72+ messages in thread

* Re: [PATCH v5 22/24] v4l: subdev: add v4l2_subdev_get_format_dir()
  2021-04-29  1:43       ` Laurent Pinchart
@ 2021-05-04  6:49         ` Tomi Valkeinen
  0 siblings, 0 replies; 72+ messages in thread
From: Tomi Valkeinen @ 2021-05-04  6:49 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: linux-media, sakari.ailus, Jacopo Mondi,
	niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil

On 29/04/2021 04:43, Laurent Pinchart wrote:
> Hi Tomi,
> 
> On Wed, Apr 21, 2021 at 04:04:22PM +0300, Tomi Valkeinen wrote:
>> On 18/04/2021 22:04, Laurent Pinchart wrote:
>>> On Thu, Apr 15, 2021 at 04:04:48PM +0300, Tomi Valkeinen wrote:
>>>> Add v4l2_subdev_get_format_dir() which can be used to find subdev format
>>>> for a specific stream on a multiplexed pad. The function will follow the
>>>> routes and links until it finds a non-multiplexed pad.
>>>>
>>>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>>>> ---
>>>>    drivers/media/v4l2-core/v4l2-subdev.c | 96 +++++++++++++++++++++++++++
>>>>    include/media/v4l2-subdev.h           | 26 ++++++++
>>>>    2 files changed, 122 insertions(+)
>>>>
>>>> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
>>>> index 7a4f71d8c6c3..430dbdaab080 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-subdev.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
>>>> @@ -998,6 +998,102 @@ bool v4l2_subdev_has_route(struct v4l2_subdev_krouting *routing,
>>>>    }
>>>>    EXPORT_SYMBOL_GPL(v4l2_subdev_has_route);
>>>>    
>>>> +int v4l2_subdev_get_format_dir(struct media_pad *pad, u16 stream,
>>>> +			       enum v4l2_direction dir,
>>>> +			       struct v4l2_subdev_format *fmt)
>>>> +{
>>>> +	struct device *dev = pad->entity->graph_obj.mdev->dev;
>>>> +	int ret;
>>>> +	int i;
>>>> +
>>>> +	dev_dbg(dev, "%s '%s':%u:%u %s\n", __func__,
>>>> +		pad->entity->name, pad->index, stream,
>>>> +		dir == V4L2_DIR_SOURCEWARD ? "sourceward" : "sinkward");
>>>> +
>>>> +	while (true) {
>>>> +		struct v4l2_subdev_krouting routing;
>>>> +		struct v4l2_subdev_route *route;
>>>> +
>>>> +		if (pad->entity->obj_type != MEDIA_ENTITY_TYPE_V4L2_SUBDEV)
>>>> +			return -EINVAL;
>>>> +
>>>> +		ret = v4l2_subdev_link_validate_get_format(pad, fmt);
>>>> +		if (ret == 0)
>>>> +			return 0;
>>>> +		else if (ret != -ENOIOCTLCMD)
>>>> +			return ret;
>>>> +
>>>> +		if (pad->flags &
>>>> +		    (dir == V4L2_DIR_SINKWARD ? MEDIA_PAD_FL_SOURCE :
>>>> +						MEDIA_PAD_FL_SINK)) {
>>>> +			pad = media_entity_remote_pad(pad);
>>>> +
>>>> +			if (!pad)
>>>> +				return -EINVAL;
>>>> +
>>>> +			if (pad->entity->obj_type != MEDIA_ENTITY_TYPE_V4L2_SUBDEV)
>>>> +				return -EINVAL;
>>>> +
>>>> +			ret = v4l2_subdev_link_validate_get_format(pad, fmt);
>>>> +			if (ret == 0)
>>>> +				return 0;
>>>> +			else if (ret != -ENOIOCTLCMD)
>>>> +				return ret;
>>>> +		}
>>>> +
>>>> +		ret = v4l2_subdev_get_krouting(media_entity_to_v4l2_subdev(pad->entity), &routing);
>>>> +		if (ret)
>>>> +			return ret;
>>>> +
>>>> +		route = NULL;
>>>> +		for (i = 0; i < routing.num_routes; ++i) {
>>>> +			u16 near_pad = dir == V4L2_DIR_SINKWARD ?
>>>> +					       routing.routes[i].sink_pad :
>>>> +					       routing.routes[i].source_pad;
>>>> +			u16 near_stream = dir == V4L2_DIR_SINKWARD ?
>>>> +						  routing.routes[i].sink_stream :
>>>> +						  routing.routes[i].source_stream;
>>>> +
>>>> +			if (!(routing.routes[i].flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
>>>> +				continue;
>>>> +
>>>> +			if (near_pad != pad->index)
>>>> +				continue;
>>>> +
>>>> +			if (near_stream != stream)
>>>> +				continue;
>>>> +
>>>> +			if (route) {
>>>> +				dev_err(dev,
>>>> +					"%s: '%s' has multiple active routes for stream %u\n",
>>>> +					__func__, pad->entity->name, stream);
>>>> +				v4l2_subdev_free_routing(&routing);
>>>> +				return -EINVAL;
>>>> +			}
>>>> +
>>>> +			route = &routing.routes[i];
>>>> +		}
>>>> +
>>>> +		if (!route) {
>>>> +			dev_err(dev, "%s: no route found in '%s' for stream %u\n",
>>>> +				__func__, pad->entity->name, stream);
>>>> +			v4l2_subdev_free_routing(&routing);
>>>> +			return -EINVAL;
>>>> +		}
>>>> +
>>>> +		if (dir == V4L2_DIR_SINKWARD) {
>>>> +			pad = &pad->entity->pads[route->source_pad];
>>>> +			stream = route->source_stream;
>>>> +		} else {
>>>> +			pad = &pad->entity->pads[route->sink_pad];
>>>> +			stream = route->sink_stream;
>>>> +		}
>>>> +
>>>> +		v4l2_subdev_free_routing(&routing);
>>>> +	}
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(v4l2_subdev_get_format_dir);
>>>> +
>>>>    int v4l2_subdev_link_validate(struct media_link *link)
>>>>    {
>>>>    	struct v4l2_subdev *sink;
>>>> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
>>>> index 1843b77dd843..730631f9a091 100644
>>>> --- a/include/media/v4l2-subdev.h
>>>> +++ b/include/media/v4l2-subdev.h
>>>> @@ -1239,4 +1239,30 @@ void v4l2_subdev_cpy_routing(struct v4l2_subdev_krouting *dst,
>>>>    bool v4l2_subdev_has_route(struct v4l2_subdev_krouting *routing,
>>>>    			   unsigned int pad0, unsigned int pad1);
>>>>    
>>>> +/**
>>>> + * enum v4l2_direction - Direction either towards the source or the sink
>>>> + *
>>>> + * @V4L2_DIR_SOURCEWARD: Direction towards the source.
>>>> + * @V4L2_DIR_SINKWARD: Direction towards the sink.
>>>> + */
>>>> +enum v4l2_direction {
>>>> +	V4L2_DIR_SOURCEWARD,
>>>> +	V4L2_DIR_SINKWARD,
>>>> +};
>>>> +
>>>> +/**
>>>> + * v4l2_subdev_get_format_dir() - Find format by following streams
>>>
>>> The name is a bit cryptic, and the usage pattern error-prone. Can we do
>>> better ?  In particular, if we could limit the usage of this function to
>>> be called on a non-multiplexed pad, we could drop the stream argument.
>>> Deducing the direction argument from the type of pad would also make the
>>> API simpler.
>>
>> Hmm, but that's not what the function does. It follows a specific
>> stream, from a multiplexed pad, so it has to get the stream as a parameter.
>>
>> We can't deduct the direction from the type of the pad. We can of course
>> define that given a source pad this function goes sourceward. But if
>> that's not what the caller wants, then the caller needs to first follow
>> the stream either direction to get a sink pad, and then call this
>> function, which doesn't make sense.
> 
> What do the current callers need ? We don't have to implement something
> that is more generic or featureful than our needs dictate, as this is a
> very ad hoc function anyway. If we really need the full behaviour
> implemented here, we should at the very least rename the function, but I
> think it should be possible to do better overall, perhaps splitting the
> operation in the caller into cleaner functions.

The link validation will call the function with both V4L2_DIR_SOURCEWARD 
and V4L2_DIR_SINKWARD. DS90UB960 driver also uses the function, with 
V4L2_DIR_SOURCEWARD.

The name may not be the most clear one, but it's not easy to invent a 
name for something like this =). Why do you think the usage pattern is 
error prone?

  Tomi

^ permalink raw reply	[flat|nested] 72+ messages in thread

end of thread, back to index

Thread overview: 72+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 01/24] media: entity: Use pad as a starting point for graph walk Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 02/24] media: entity: Use pads instead of entities in the media graph walk stack Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 03/24] media: entity: Walk the graph based on pads Tomi Valkeinen
2021-04-18 17:47   ` Laurent Pinchart
2021-04-20 11:30     ` Sakari Ailus
2021-04-15 13:04 ` [PATCH v5 04/24] v4l: mc: Start walk from a specific pad in use count calculation Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 05/24] media: entity: Add iterator helper for entity pads Tomi Valkeinen
2021-04-18 17:52   ` Laurent Pinchart
2021-04-22 12:04     ` Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 06/24] media: entity: Move the pipeline from entity to pads Tomi Valkeinen
2021-04-18 18:00   ` Laurent Pinchart
2021-04-20 11:38     ` Sakari Ailus
2021-04-15 13:04 ` [PATCH v5 07/24] media: entity: Use pad as the starting point for a pipeline Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 08/24] media: entity: Add has_route entity operation Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 09/24] media: entity: Add media_entity_has_route() function Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 10/24] media: entity: Use routing information during graph traversal Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 11/24] media: entity: Skip link validation for pads to which there is no route to Tomi Valkeinen
2021-04-18 18:06   ` Laurent Pinchart
2021-04-20 11:41     ` Sakari Ailus
2021-04-23 12:37       ` Tomi Valkeinen
2021-04-29 12:06         ` Sakari Ailus
2021-04-29 14:10           ` Laurent Pinchart
2021-04-15 13:04 ` [PATCH v5 12/24] media: entity: Add an iterator helper for connected pads Tomi Valkeinen
2021-04-18 18:20   ` Laurent Pinchart
2021-04-20 11:48     ` Sakari Ailus
2021-04-29  1:33       ` Laurent Pinchart
2021-04-29 11:56         ` Sakari Ailus
2021-04-29 12:04           ` Tomi Valkeinen
2021-04-29 12:07             ` Sakari Ailus
2021-04-29 12:14               ` Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 13/24] media: entity: Add only connected pads to the pipeline Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 14/24] media: entity: Add debug information in graph walk route check Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 15/24] v4l: Add bus type to frame descriptors Tomi Valkeinen
2021-04-18 19:23   ` Laurent Pinchart
2021-04-20 11:50     ` Sakari Ailus
2021-04-22 12:30       ` Tomi Valkeinen
2021-04-29 11:58         ` Sakari Ailus
2021-04-29 14:09           ` Laurent Pinchart
2021-04-15 13:04 ` [PATCH v5 16/24] v4l: Add CSI-2 bus configuration " Tomi Valkeinen
2021-04-18 19:24   ` Laurent Pinchart
2021-04-20 16:32     ` Sakari Ailus
2021-04-15 13:04 ` [PATCH v5 17/24] v4l: Add stream to frame descriptor Tomi Valkeinen
2021-04-18 19:27   ` Laurent Pinchart
2021-04-22 12:47     ` Tomi Valkeinen
2021-04-22 16:18       ` Laurent Pinchart
2021-04-15 13:04 ` [PATCH v5 18/24] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations Tomi Valkeinen
2021-04-18 18:32   ` Laurent Pinchart
2021-04-22 11:16     ` Tomi Valkeinen
2021-04-22 16:20       ` Laurent Pinchart
2021-04-22 16:58         ` Tomi Valkeinen
2021-04-20 16:35   ` Sakari Ailus
2021-04-15 13:04 ` [PATCH v5 19/24] media: Documentation: Add GS_ROUTING documentation Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 20/24] v4l: mc: Add an S_ROUTING helper function for power state changes Tomi Valkeinen
2021-04-18 18:55   ` Laurent Pinchart
2021-04-20 16:41     ` Sakari Ailus
2021-04-15 13:04 ` [PATCH v5 21/24] v4l: subdev: routing kernel helper functions Tomi Valkeinen
2021-04-18 19:18   ` Laurent Pinchart
2021-04-15 13:04 ` [PATCH v5 22/24] v4l: subdev: add v4l2_subdev_get_format_dir() Tomi Valkeinen
2021-04-18 19:04   ` Laurent Pinchart
2021-04-21 13:04     ` Tomi Valkeinen
2021-04-29  1:43       ` Laurent Pinchart
2021-05-04  6:49         ` Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 23/24] v4l: subdev: Take routing information into account in link validation Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 24/24] v4l: subdev: increase V4L2_FRAME_DESC_ENTRY_MAX to 8 Tomi Valkeinen
2021-04-18 19:06   ` Laurent Pinchart
2021-04-16  8:38 ` [PATCH v5 00/24] v4l: subdev internal routing Niklas Söderlund
2021-04-16  8:47   ` Tomi Valkeinen
2021-04-16  8:56     ` Niklas Söderlund
2021-04-18 17:32 ` Laurent Pinchart
2021-04-21 12:57   ` Tomi Valkeinen
2021-04-29  1:27     ` Laurent Pinchart

Linux-Media Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-media/0 linux-media/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-media linux-media/ https://lore.kernel.org/linux-media \
		linux-media@vger.kernel.org
	public-inbox-index linux-media

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-media


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git