* [PATCH v6 00/24] v4l: subdev internal routing
@ 2021-04-27 12:44 Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 01/24] media: entity: Use pad as a starting point for graph walk Tomi Valkeinen
` (24 more replies)
0 siblings, 25 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:44 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen
Hi,
This is v6 of the series. Previous one can be found from:
https://lore.kernel.org/linux-media/20210415130450.421168-1-tomi.valkeinen@ideasonboard.com/
Changes to v5:
- Dropped "v4l: mc: Add an S_ROUTING helper function for power state changes" as discussed
- Added MEDIA_PAD_FL_MULTIPLEXED flag to indicate that a pad works in multiplexed mode
- Documentation improvements
- Renamed struct v4l2_mbus_frame_desc_entry_csi2 fields
- Fixed setting routing->num_routes in G_ROUTING
- Many smaller cosmetic changes as per comments
One bigger topic I didn't change is the in-kernel API for get/set routing.
Currently the link validation needs to get the routing tables multiple times,
each leading to memory allocations (and frees). A different API would allow the
link validation to peek at the routing without any allocations, but I haven't
solved that yet.
A simple solution would be to require a lock to be held when accessing the
routing table, and making get_routing return a pointer to the driver's routing
table. But this kind of API would be quite different than, say, get_fmt.
Another would be to keep the API, but have a state object during link
validation, which would hold preallocated routing tables, so that link
validation wouldn't need to allocate new ones for each get_routing call.
Anything we do here is a kernel internal change, so it's not super critical to
solve it right away.
Tomi
Jacopo Mondi (2):
media: entity: Add iterator helper for entity pads
media: Documentation: Add GS_ROUTING documentation
Laurent Pinchart (3):
media: entity: Add has_route entity operation
media: entity: Add media_entity_has_route() function
media: entity: Use routing information during graph traversal
Sakari Ailus (9):
media: entity: Use pad as a starting point for graph walk
media: entity: Use pads instead of entities in the media graph walk
stack
v4l: mc: Start walk from a specific pad in use count calculation
media: entity: Move the pipeline from entity to pads
media: entity: Use pad as the starting point for a pipeline
media: entity: Skip link validation for pads to which there is no
route to
media: entity: Add only connected pads to the pipeline
media: entity: Add debug information in graph walk route check
v4l: Add stream to frame descriptor
Tomi Valkeinen (10):
media: entity: Walk the graph based on pads
media: entity: Add an iterator helper for connected pads
v4l: Add bus type to frame descriptors
v4l: Add CSI-2 bus configuration to frame descriptors
v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations
v4l: subdev: routing kernel helper functions
v4l: subdev: add v4l2_subdev_get_format_dir()
media: uapi: add MEDIA_PAD_FL_MULTIPLEXED flag
v4l: subdev: Take routing information into account in link validation
v4l: subdev: increase V4L2_FRAME_DESC_ENTRY_MAX to 8
Documentation/driver-api/media/mc-core.rst | 15 +-
.../media/mediactl/media-types.rst | 6 +
.../userspace-api/media/v4l/dev-subdev.rst | 125 ++++++
.../userspace-api/media/v4l/user-func.rst | 1 +
.../media/v4l/vidioc-subdev-g-routing.rst | 138 +++++++
drivers/media/mc/mc-device.c | 13 +-
drivers/media/mc/mc-entity.c | 241 ++++++-----
drivers/media/pci/intel/ipu3/ipu3-cio2-main.c | 6 +-
.../media/platform/exynos4-is/fimc-capture.c | 8 +-
.../platform/exynos4-is/fimc-isp-video.c | 8 +-
drivers/media/platform/exynos4-is/fimc-isp.c | 2 +-
drivers/media/platform/exynos4-is/fimc-lite.c | 10 +-
drivers/media/platform/exynos4-is/media-dev.c | 20 +-
drivers/media/platform/omap3isp/isp.c | 2 +-
drivers/media/platform/omap3isp/ispvideo.c | 25 +-
drivers/media/platform/omap3isp/ispvideo.h | 2 +-
.../media/platform/qcom/camss/camss-video.c | 6 +-
drivers/media/platform/rcar-vin/rcar-core.c | 16 +-
drivers/media/platform/rcar-vin/rcar-dma.c | 8 +-
.../platform/rockchip/rkisp1/rkisp1-capture.c | 6 +-
.../media/platform/s3c-camif/camif-capture.c | 6 +-
drivers/media/platform/stm32/stm32-dcmi.c | 6 +-
.../platform/sunxi/sun4i-csi/sun4i_dma.c | 6 +-
.../platform/sunxi/sun6i-csi/sun6i_video.c | 6 +-
drivers/media/platform/ti-vpe/cal-video.c | 6 +-
drivers/media/platform/vsp1/vsp1_video.c | 18 +-
drivers/media/platform/xilinx/xilinx-dma.c | 20 +-
drivers/media/platform/xilinx/xilinx-dma.h | 2 +-
.../media/test-drivers/vimc/vimc-capture.c | 6 +-
drivers/media/usb/au0828/au0828-core.c | 8 +-
drivers/media/v4l2-core/v4l2-ioctl.c | 25 +-
drivers/media/v4l2-core/v4l2-mc.c | 43 +-
drivers/media/v4l2-core/v4l2-subdev.c | 376 +++++++++++++++++-
drivers/staging/media/imx/imx-media-utils.c | 8 +-
drivers/staging/media/ipu3/ipu3-v4l2.c | 6 +-
drivers/staging/media/omap4iss/iss.c | 2 +-
drivers/staging/media/omap4iss/iss_video.c | 38 +-
drivers/staging/media/omap4iss/iss_video.h | 2 +-
drivers/staging/media/tegra-video/tegra210.c | 6 +-
include/media/media-entity.h | 144 +++++--
include/media/v4l2-subdev.h | 107 ++++-
include/uapi/linux/media.h | 1 +
include/uapi/linux/v4l2-subdev.h | 48 +++
43 files changed, 1246 insertions(+), 302 deletions(-)
create mode 100644 Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
--
2.25.1
^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v6 01/24] media: entity: Use pad as a starting point for graph walk
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 02/24] media: entity: Use pads instead of entities in the media graph walk stack Tomi Valkeinen
` (23 subsequent siblings)
24 siblings, 0 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
With the upcoming use of the recently added has_route() media entity op, all
the pads in an entity will no longer be considered interconnected. This has
an effect where the media graph is traversed: the starting pad does make a
difference.
Prepare for this change by using pad instead of the entity as an argument
for the graph walk operations. The actual graph traversal algorithm change
is in further patches.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
Documentation/driver-api/media/mc-core.rst | 2 +-
drivers/media/mc/mc-entity.c | 17 ++++++++---------
drivers/media/platform/exynos4-is/media-dev.c | 4 ++--
drivers/media/platform/omap3isp/ispvideo.c | 2 +-
drivers/media/platform/vsp1/vsp1_video.c | 2 +-
drivers/media/platform/xilinx/xilinx-dma.c | 2 +-
drivers/media/v4l2-core/v4l2-mc.c | 6 +++---
drivers/staging/media/omap4iss/iss_video.c | 4 ++--
include/media/media-entity.h | 10 ++++------
9 files changed, 23 insertions(+), 26 deletions(-)
diff --git a/Documentation/driver-api/media/mc-core.rst b/Documentation/driver-api/media/mc-core.rst
index 57b5bbba944e..ba0aee982124 100644
--- a/Documentation/driver-api/media/mc-core.rst
+++ b/Documentation/driver-api/media/mc-core.rst
@@ -167,7 +167,7 @@ Drivers initiate a graph traversal by calling
:c:func:`media_graph_walk_start()`
The graph structure, provided by the caller, is initialized to start graph
-traversal at the given entity.
+traversal at the given pad in an entity.
Drivers can then retrieve the next entity by calling
:c:func:`media_graph_walk_next()`
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 12b45e669bcc..459a5bb4ab41 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -291,17 +291,16 @@ void media_graph_walk_cleanup(struct media_graph *graph)
}
EXPORT_SYMBOL_GPL(media_graph_walk_cleanup);
-void media_graph_walk_start(struct media_graph *graph,
- struct media_entity *entity)
+void media_graph_walk_start(struct media_graph *graph, struct media_pad *pad)
{
media_entity_enum_zero(&graph->ent_enum);
- media_entity_enum_set(&graph->ent_enum, entity);
+ media_entity_enum_set(&graph->ent_enum, pad->entity);
graph->top = 0;
graph->stack[graph->top].entity = NULL;
- stack_push(graph, entity);
- dev_dbg(entity->graph_obj.mdev->dev,
- "begin graph walk at '%s'\n", entity->name);
+ stack_push(graph, pad->entity);
+ dev_dbg(pad->graph_obj.mdev->dev,
+ "begin graph walk at '%s':%u\n", pad->entity->name, pad->index);
}
EXPORT_SYMBOL_GPL(media_graph_walk_start);
@@ -419,7 +418,7 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
goto error_graph_walk_start;
}
- media_graph_walk_start(&pipe->graph, entity);
+ media_graph_walk_start(&pipe->graph, entity->pads);
while ((entity = media_graph_walk_next(graph))) {
DECLARE_BITMAP(active, MEDIA_ENTITY_MAX_PADS);
@@ -503,7 +502,7 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
* Link validation on graph failed. We revert what we did and
* return the error.
*/
- media_graph_walk_start(graph, entity_err);
+ media_graph_walk_start(graph, entity_err->pads);
while ((entity_err = media_graph_walk_next(graph))) {
/* Sanity check for negative stream_count */
@@ -554,7 +553,7 @@ void __media_pipeline_stop(struct media_entity *entity)
if (WARN_ON(!pipe))
return;
- media_graph_walk_start(graph, entity);
+ media_graph_walk_start(graph, entity->pads);
while ((entity = media_graph_walk_next(graph))) {
/* Sanity check for negative stream_count */
diff --git a/drivers/media/platform/exynos4-is/media-dev.c b/drivers/media/platform/exynos4-is/media-dev.c
index 13d192ba4aa6..d90663b65932 100644
--- a/drivers/media/platform/exynos4-is/media-dev.c
+++ b/drivers/media/platform/exynos4-is/media-dev.c
@@ -1175,7 +1175,7 @@ static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
* through active links. This is needed as we cannot power on/off the
* subdevs in random order.
*/
- media_graph_walk_start(graph, entity);
+ media_graph_walk_start(graph, entity->pads);
while ((entity = media_graph_walk_next(graph))) {
if (!is_media_entity_v4l2_video_device(entity))
@@ -1190,7 +1190,7 @@ static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
return 0;
err:
- media_graph_walk_start(graph, entity_err);
+ media_graph_walk_start(graph, entity_err->pads);
while ((entity_err = media_graph_walk_next(graph))) {
if (!is_media_entity_v4l2_video_device(entity_err))
diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
index 8811d6dd4ee7..3c1485d59404 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -234,7 +234,7 @@ static int isp_video_get_graph_data(struct isp_video *video,
return ret;
}
- media_graph_walk_start(&graph, entity);
+ media_graph_walk_start(&graph, entity->pads);
while ((entity = media_graph_walk_next(&graph))) {
struct isp_video *__video;
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index 044eb5778820..61e4fbaba7b7 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -569,7 +569,7 @@ static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
if (ret)
return ret;
- media_graph_walk_start(&graph, entity);
+ media_graph_walk_start(&graph, entity->pads);
while ((entity = media_graph_walk_next(&graph))) {
struct v4l2_subdev *subdev;
diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index 2a56201cb853..d64c3bee8b95 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -190,7 +190,7 @@ static int xvip_pipeline_validate(struct xvip_pipeline *pipe,
return ret;
}
- media_graph_walk_start(&graph, entity);
+ media_graph_walk_start(&graph, entity->pads);
while ((entity = media_graph_walk_next(&graph))) {
struct xvip_dma *dma;
diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
index b01474717dca..d215fe31b9a2 100644
--- a/drivers/media/v4l2-core/v4l2-mc.c
+++ b/drivers/media/v4l2-core/v4l2-mc.c
@@ -436,7 +436,7 @@ static int pipeline_pm_use_count(struct media_entity *entity,
{
int use = 0;
- media_graph_walk_start(graph, entity);
+ media_graph_walk_start(graph, entity->pads);
while ((entity = media_graph_walk_next(graph))) {
if (is_media_entity_v4l2_video_device(entity))
@@ -499,7 +499,7 @@ static int pipeline_pm_power(struct media_entity *entity, int change,
if (!change)
return 0;
- media_graph_walk_start(graph, entity);
+ media_graph_walk_start(graph, entity->pads);
while (!ret && (entity = media_graph_walk_next(graph)))
if (is_media_entity_v4l2_subdev(entity))
@@ -508,7 +508,7 @@ static int pipeline_pm_power(struct media_entity *entity, int change,
if (!ret)
return ret;
- media_graph_walk_start(graph, first);
+ media_graph_walk_start(graph, first->pads);
while ((first = media_graph_walk_next(graph))
&& first != entity)
diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
index 66975a37dc85..77bf1b8a56f7 100644
--- a/drivers/staging/media/omap4iss/iss_video.c
+++ b/drivers/staging/media/omap4iss/iss_video.c
@@ -217,7 +217,7 @@ iss_video_far_end(struct iss_video *video)
return NULL;
}
- media_graph_walk_start(&graph, entity);
+ media_graph_walk_start(&graph, entity->pads);
while ((entity = media_graph_walk_next(&graph))) {
if (entity == &video->video.entity)
@@ -890,7 +890,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
if (ret < 0)
goto err_media_pipeline_start;
- media_graph_walk_start(&graph, entity);
+ media_graph_walk_start(&graph, entity->pads);
while ((entity = media_graph_walk_next(&graph)))
media_entity_enum_set(&pipe->ent_enum, entity);
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index cbdfcb79d0d0..83c4fd93f349 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -897,22 +897,20 @@ __must_check int media_graph_walk_init(
void media_graph_walk_cleanup(struct media_graph *graph);
/**
- * media_graph_walk_start - Start walking the media graph at a
- * given entity
+ * media_graph_walk_start - Start walking the media graph at a given pad
*
* @graph: Media graph structure that will be used to walk the graph
- * @entity: Starting entity
+ * @pad: Starting pad
*
* Before using this function, media_graph_walk_init() must be
* used to allocate resources used for walking the graph. This
* function initializes the graph traversal structure to walk the
- * entities graph starting at the given entity. The traversal
+ * entities graph starting at the given pad. The traversal
* structure must not be modified by the caller during graph
* traversal. After the graph walk, the resources must be released
* using media_graph_walk_cleanup().
*/
-void media_graph_walk_start(struct media_graph *graph,
- struct media_entity *entity);
+void media_graph_walk_start(struct media_graph *graph, struct media_pad *pad);
/**
* media_graph_walk_next - Get the next entity in the graph
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 02/24] media: entity: Use pads instead of entities in the media graph walk stack
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 01/24] media: entity: Use pad as a starting point for graph walk Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 03/24] media: entity: Walk the graph based on pads Tomi Valkeinen
` (22 subsequent siblings)
24 siblings, 0 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
Change the media graph walk stack structure to use media pads instead of
using media entities. In addition to the entity, the pad contains the
information which pad in the entity are being dealt with.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/mc/mc-entity.c | 53 ++++++++++++++++++------------------
include/media/media-entity.h | 8 +++---
2 files changed, 30 insertions(+), 31 deletions(-)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 459a5bb4ab41..44a05806b589 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -228,40 +228,39 @@ EXPORT_SYMBOL_GPL(media_entity_pads_init);
* Graph traversal
*/
-static struct media_entity *
-media_entity_other(struct media_entity *entity, struct media_link *link)
+static struct media_pad *
+media_pad_other(struct media_pad *pad, struct media_link *link)
{
- if (link->source->entity == entity)
- return link->sink->entity;
+ if (link->source == pad)
+ return link->sink;
else
- return link->source->entity;
+ return link->source;
}
/* push an entity to traversal stack */
-static void stack_push(struct media_graph *graph,
- struct media_entity *entity)
+static void stack_push(struct media_graph *graph, struct media_pad *pad)
{
if (graph->top == MEDIA_ENTITY_ENUM_MAX_DEPTH - 1) {
WARN_ON(1);
return;
}
graph->top++;
- graph->stack[graph->top].link = entity->links.next;
- graph->stack[graph->top].entity = entity;
+ graph->stack[graph->top].link = pad->entity->links.next;
+ graph->stack[graph->top].pad = pad;
}
-static struct media_entity *stack_pop(struct media_graph *graph)
+static struct media_pad *stack_pop(struct media_graph *graph)
{
- struct media_entity *entity;
+ struct media_pad *pad;
- entity = graph->stack[graph->top].entity;
+ pad = graph->stack[graph->top].pad;
graph->top--;
- return entity;
+ return pad;
}
#define link_top(en) ((en)->stack[(en)->top].link)
-#define stack_top(en) ((en)->stack[(en)->top].entity)
+#define stack_top(en) ((en)->stack[(en)->top].pad)
/**
* media_graph_walk_init - Allocate resources for graph walk
@@ -297,8 +296,8 @@ void media_graph_walk_start(struct media_graph *graph, struct media_pad *pad)
media_entity_enum_set(&graph->ent_enum, pad->entity);
graph->top = 0;
- graph->stack[graph->top].entity = NULL;
- stack_push(graph, pad->entity);
+ graph->stack[graph->top].pad = NULL;
+ stack_push(graph, pad);
dev_dbg(pad->graph_obj.mdev->dev,
"begin graph walk at '%s':%u\n", pad->entity->name, pad->index);
}
@@ -306,16 +305,16 @@ EXPORT_SYMBOL_GPL(media_graph_walk_start);
static void media_graph_walk_iter(struct media_graph *graph)
{
- struct media_entity *entity = stack_top(graph);
+ struct media_pad *pad = stack_top(graph);
struct media_link *link;
- struct media_entity *next;
+ struct media_pad *next;
link = list_entry(link_top(graph), typeof(*link), list);
/* The link is not enabled so we do not follow. */
if (!(link->flags & MEDIA_LNK_FL_ENABLED)) {
link_top(graph) = link_top(graph)->next;
- dev_dbg(entity->graph_obj.mdev->dev,
+ dev_dbg(pad->graph_obj.mdev->dev,
"walk: skipping disabled link '%s':%u -> '%s':%u\n",
link->source->entity->name, link->source->index,
link->sink->entity->name, link->sink->index);
@@ -323,22 +322,22 @@ static void media_graph_walk_iter(struct media_graph *graph)
}
/* Get the entity in the other end of the link . */
- next = media_entity_other(entity, link);
+ next = media_pad_other(pad, link);
/* Has the entity already been visited? */
- if (media_entity_enum_test_and_set(&graph->ent_enum, next)) {
+ if (media_entity_enum_test_and_set(&graph->ent_enum, next->entity)) {
link_top(graph) = link_top(graph)->next;
- dev_dbg(entity->graph_obj.mdev->dev,
+ dev_dbg(pad->graph_obj.mdev->dev,
"walk: skipping entity '%s' (already seen)\n",
- next->name);
+ next->entity->name);
return;
}
/* Push the new entity to stack and start over. */
link_top(graph) = link_top(graph)->next;
stack_push(graph, next);
- dev_dbg(entity->graph_obj.mdev->dev, "walk: pushing '%s' on stack\n",
- next->name);
+ dev_dbg(next->graph_obj.mdev->dev, "walk: pushing '%s':%u on stack\n",
+ next->entity->name, next->index);
}
struct media_entity *media_graph_walk_next(struct media_graph *graph)
@@ -353,10 +352,10 @@ struct media_entity *media_graph_walk_next(struct media_graph *graph)
* top of the stack until no more entities on the level can be
* found.
*/
- while (link_top(graph) != &stack_top(graph)->links)
+ while (link_top(graph) != &stack_top(graph)->entity->links)
media_graph_walk_iter(graph);
- entity = stack_pop(graph);
+ entity = stack_pop(graph)->entity;
dev_dbg(entity->graph_obj.mdev->dev,
"walk: returning entity '%s'\n", entity->name);
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 83c4fd93f349..97b170cf38eb 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -78,16 +78,16 @@ struct media_entity_enum {
* struct media_graph - Media graph traversal state
*
* @stack: Graph traversal stack; the stack contains information
- * on the path the media entities to be walked and the
- * links through which they were reached.
- * @stack.entity: pointer to &struct media_entity at the graph.
+ * on the media pads to be walked and the links through
+ * which they were reached.
+ * @stack.pad: pointer to &struct media_pad at the graph.
* @stack.link: pointer to &struct list_head.
* @ent_enum: Visited entities
* @top: The top of the stack
*/
struct media_graph {
struct {
- struct media_entity *entity;
+ struct media_pad *pad;
struct list_head *link;
} stack[MEDIA_ENTITY_ENUM_MAX_DEPTH];
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 03/24] media: entity: Walk the graph based on pads
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 01/24] media: entity: Use pad as a starting point for graph walk Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 02/24] media: entity: Use pads instead of entities in the media graph walk stack Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 04/24] v4l: mc: Start walk from a specific pad in use count calculation Tomi Valkeinen
` (21 subsequent siblings)
24 siblings, 0 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen
Instead of iterating over graph entities during the walk, iterate the pads
through which the entity was first reached. This is required in order to
make the entity pipeline pad-based rather than entity based.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
Documentation/driver-api/media/mc-core.rst | 7 ++-
drivers/media/mc/mc-entity.c | 51 +++++++++++--------
drivers/media/platform/exynos4-is/media-dev.c | 20 ++++----
drivers/media/platform/omap3isp/ispvideo.c | 17 ++++---
drivers/media/platform/vsp1/vsp1_video.c | 12 ++---
drivers/media/platform/xilinx/xilinx-dma.c | 12 ++---
drivers/media/v4l2-core/v4l2-mc.c | 24 ++++-----
drivers/staging/media/omap4iss/iss_video.c | 34 +++++++------
include/media/media-entity.h | 7 +--
9 files changed, 100 insertions(+), 84 deletions(-)
diff --git a/Documentation/driver-api/media/mc-core.rst b/Documentation/driver-api/media/mc-core.rst
index ba0aee982124..8a13640bed56 100644
--- a/Documentation/driver-api/media/mc-core.rst
+++ b/Documentation/driver-api/media/mc-core.rst
@@ -169,8 +169,11 @@ Drivers initiate a graph traversal by calling
The graph structure, provided by the caller, is initialized to start graph
traversal at the given pad in an entity.
-Drivers can then retrieve the next entity by calling
-:c:func:`media_graph_walk_next()`
+Drivers can then retrieve the next pad by calling
+:c:func:`media_graph_walk_next()`. Only the pad through which the entity
+is first reached is returned. If the caller is interested in knowing which
+further pads would be connected, the :c:func:`media_entity_has_route()`
+function can be used for that.
When the graph traversal is complete the function will return ``NULL``.
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 44a05806b589..ac7c1dc0996e 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -340,9 +340,9 @@ static void media_graph_walk_iter(struct media_graph *graph)
next->entity->name, next->index);
}
-struct media_entity *media_graph_walk_next(struct media_graph *graph)
+struct media_pad *media_graph_walk_next(struct media_graph *graph)
{
- struct media_entity *entity;
+ struct media_pad *pad;
if (stack_top(graph) == NULL)
return NULL;
@@ -355,11 +355,11 @@ struct media_entity *media_graph_walk_next(struct media_graph *graph)
while (link_top(graph) != &stack_top(graph)->entity->links)
media_graph_walk_iter(graph);
- entity = stack_pop(graph)->entity;
- dev_dbg(entity->graph_obj.mdev->dev,
- "walk: returning entity '%s'\n", entity->name);
+ pad = stack_pop(graph);
+ dev_dbg(pad->graph_obj.mdev->dev,
+ "walk: returning pad '%s':%u\n", pad->entity->name, pad->index);
- return entity;
+ return pad;
}
EXPORT_SYMBOL_GPL(media_graph_walk_next);
@@ -407,7 +407,8 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
{
struct media_device *mdev = entity->graph_obj.mdev;
struct media_graph *graph = &pipe->graph;
- struct media_entity *entity_err = entity;
+ struct media_pad *pad = entity->pads;
+ struct media_pad *pad_err = pad;
struct media_link *link;
int ret;
@@ -417,9 +418,11 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
goto error_graph_walk_start;
}
- media_graph_walk_start(&pipe->graph, entity->pads);
+ media_graph_walk_start(&pipe->graph, pad);
+
+ while ((pad = media_graph_walk_next(graph))) {
+ struct media_entity *entity = pad->entity;
- while ((entity = media_graph_walk_next(graph))) {
DECLARE_BITMAP(active, MEDIA_ENTITY_MAX_PADS);
DECLARE_BITMAP(has_no_links, MEDIA_ENTITY_MAX_PADS);
@@ -428,7 +431,7 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
if (entity->pipe && entity->pipe != pipe) {
pr_err("Pipe active for %s. Can't start for %s\n",
entity->name,
- entity_err->name);
+ pad_err->entity->name);
ret = -EBUSY;
goto error;
}
@@ -446,26 +449,27 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
bitmap_fill(has_no_links, entity->num_pads);
list_for_each_entry(link, &entity->links, list) {
- struct media_pad *pad = link->sink->entity == entity
- ? link->sink : link->source;
+ struct media_pad *other_pad =
+ link->sink->entity == entity ?
+ link->sink : link->source;
/* Mark that a pad is connected by a link. */
- bitmap_clear(has_no_links, pad->index, 1);
+ bitmap_clear(has_no_links, other_pad->index, 1);
/*
* Pads that either do not need to connect or
* are connected through an enabled link are
* fine.
*/
- if (!(pad->flags & MEDIA_PAD_FL_MUST_CONNECT) ||
+ if (!(other_pad->flags & MEDIA_PAD_FL_MUST_CONNECT) ||
link->flags & MEDIA_LNK_FL_ENABLED)
- bitmap_set(active, pad->index, 1);
+ bitmap_set(active, other_pad->index, 1);
/*
* Link validation will only take place for
* sink ends of the link that are enabled.
*/
- if (link->sink != pad ||
+ if (link->sink != other_pad ||
!(link->flags & MEDIA_LNK_FL_ENABLED))
continue;
@@ -501,9 +505,11 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
* Link validation on graph failed. We revert what we did and
* return the error.
*/
- media_graph_walk_start(graph, entity_err->pads);
+ media_graph_walk_start(graph, pad_err);
+
+ while ((pad_err = media_graph_walk_next(graph))) {
+ struct media_entity *entity_err = pad_err->entity;
- while ((entity_err = media_graph_walk_next(graph))) {
/* Sanity check for negative stream_count */
if (!WARN_ON_ONCE(entity_err->stream_count <= 0)) {
entity_err->stream_count--;
@@ -515,7 +521,7 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
* We haven't increased stream_count further than this
* so we quit here.
*/
- if (entity_err == entity)
+ if (pad_err == pad)
break;
}
@@ -542,8 +548,9 @@ EXPORT_SYMBOL_GPL(media_pipeline_start);
void __media_pipeline_stop(struct media_entity *entity)
{
- struct media_graph *graph = &entity->pipe->graph;
struct media_pipeline *pipe = entity->pipe;
+ struct media_graph *graph = &pipe->graph;
+ struct media_pad *pad;
/*
* If the following check fails, the driver has performed an
@@ -554,7 +561,9 @@ void __media_pipeline_stop(struct media_entity *entity)
media_graph_walk_start(graph, entity->pads);
- while ((entity = media_graph_walk_next(graph))) {
+ while ((pad = media_graph_walk_next(graph))) {
+ struct media_entity *entity = pad->entity;
+
/* Sanity check for negative stream_count */
if (!WARN_ON_ONCE(entity->stream_count <= 0)) {
entity->stream_count--;
diff --git a/drivers/media/platform/exynos4-is/media-dev.c b/drivers/media/platform/exynos4-is/media-dev.c
index d90663b65932..b910a23b7e23 100644
--- a/drivers/media/platform/exynos4-is/media-dev.c
+++ b/drivers/media/platform/exynos4-is/media-dev.c
@@ -1166,7 +1166,7 @@ static int __fimc_md_modify_pipeline(struct media_entity *entity, bool enable)
static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
struct media_graph *graph)
{
- struct media_entity *entity_err = entity;
+ struct media_pad *pad, *pad_err = entity->pads;
int ret;
/*
@@ -1175,13 +1175,13 @@ static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
* through active links. This is needed as we cannot power on/off the
* subdevs in random order.
*/
- media_graph_walk_start(graph, entity->pads);
+ media_graph_walk_start(graph, pad_err);
- while ((entity = media_graph_walk_next(graph))) {
- if (!is_media_entity_v4l2_video_device(entity))
+ while ((pad = media_graph_walk_next(graph))) {
+ if (!is_media_entity_v4l2_video_device(pad->entity))
continue;
- ret = __fimc_md_modify_pipeline(entity, enable);
+ ret = __fimc_md_modify_pipeline(pad->entity, enable);
if (ret < 0)
goto err;
@@ -1190,15 +1190,15 @@ static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
return 0;
err:
- media_graph_walk_start(graph, entity_err->pads);
+ media_graph_walk_start(graph, pad_err);
- while ((entity_err = media_graph_walk_next(graph))) {
- if (!is_media_entity_v4l2_video_device(entity_err))
+ while ((pad_err = media_graph_walk_next(graph))) {
+ if (!is_media_entity_v4l2_video_device(pad_err->entity))
continue;
- __fimc_md_modify_pipeline(entity_err, !enable);
+ __fimc_md_modify_pipeline(pad_err->entity, !enable);
- if (entity_err == entity)
+ if (pad_err == pad)
break;
}
diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
index 3c1485d59404..5c1cbb1a9003 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -222,8 +222,8 @@ static int isp_video_get_graph_data(struct isp_video *video,
struct isp_pipeline *pipe)
{
struct media_graph graph;
- struct media_entity *entity = &video->video.entity;
- struct media_device *mdev = entity->graph_obj.mdev;
+ struct media_pad *pad = video->video.entity.pads;
+ struct media_device *mdev = video->video.entity.graph_obj.mdev;
struct isp_video *far_end = NULL;
int ret;
@@ -234,23 +234,24 @@ static int isp_video_get_graph_data(struct isp_video *video,
return ret;
}
- media_graph_walk_start(&graph, entity->pads);
+ media_graph_walk_start(&graph, pad);
- while ((entity = media_graph_walk_next(&graph))) {
+ while ((pad = media_graph_walk_next(&graph))) {
struct isp_video *__video;
- media_entity_enum_set(&pipe->ent_enum, entity);
+ media_entity_enum_set(&pipe->ent_enum, pad->entity);
if (far_end != NULL)
continue;
- if (entity == &video->video.entity)
+ if (pad == video->video.entity.pads)
continue;
- if (!is_media_entity_v4l2_video_device(entity))
+ if (!is_media_entity_v4l2_video_device(pad->entity))
continue;
- __video = to_isp_video(media_entity_to_video_device(entity));
+ __video = to_isp_video(media_entity_to_video_device(
+ pad->entity));
if (__video->type != video->type)
far_end = __video;
}
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index 61e4fbaba7b7..f2c36f2fdf53 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -559,8 +559,8 @@ static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
struct vsp1_video *video)
{
struct media_graph graph;
- struct media_entity *entity = &video->video.entity;
- struct media_device *mdev = entity->graph_obj.mdev;
+ struct media_pad *pad = video->video.entity.pads;
+ struct media_device *mdev = video->video.entity.graph_obj.mdev;
unsigned int i;
int ret;
@@ -569,17 +569,17 @@ static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
if (ret)
return ret;
- media_graph_walk_start(&graph, entity->pads);
+ media_graph_walk_start(&graph, pad);
- while ((entity = media_graph_walk_next(&graph))) {
+ while ((pad = media_graph_walk_next(&graph))) {
struct v4l2_subdev *subdev;
struct vsp1_rwpf *rwpf;
struct vsp1_entity *e;
- if (!is_media_entity_v4l2_subdev(entity))
+ if (!is_media_entity_v4l2_subdev(pad->entity))
continue;
- subdev = media_entity_to_v4l2_subdev(entity);
+ subdev = media_entity_to_v4l2_subdev(pad->entity);
e = to_vsp1_entity(subdev);
list_add_tail(&e->list_pipe, &pipe->entities);
e->pipe = pipe;
diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index d64c3bee8b95..8df3c43aecbe 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -175,8 +175,8 @@ static int xvip_pipeline_validate(struct xvip_pipeline *pipe,
struct xvip_dma *start)
{
struct media_graph graph;
- struct media_entity *entity = &start->video.entity;
- struct media_device *mdev = entity->graph_obj.mdev;
+ struct media_pad *pad = start->video.entity.pads;
+ struct media_device *mdev = pad->entity->graph_obj.mdev;
unsigned int num_inputs = 0;
unsigned int num_outputs = 0;
int ret;
@@ -190,15 +190,15 @@ static int xvip_pipeline_validate(struct xvip_pipeline *pipe,
return ret;
}
- media_graph_walk_start(&graph, entity->pads);
+ media_graph_walk_start(&graph, pad);
- while ((entity = media_graph_walk_next(&graph))) {
+ while ((pad = media_graph_walk_next(&graph))) {
struct xvip_dma *dma;
- if (entity->function != MEDIA_ENT_F_IO_V4L)
+ if (pad->entity->function != MEDIA_ENT_F_IO_V4L)
continue;
- dma = to_xvip_dma(media_entity_to_video_device(entity));
+ dma = to_xvip_dma(media_entity_to_video_device(pad->entity));
if (dma->pad.flags & MEDIA_PAD_FL_SINK) {
pipe->output = dma;
diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
index d215fe31b9a2..cbeb580c6754 100644
--- a/drivers/media/v4l2-core/v4l2-mc.c
+++ b/drivers/media/v4l2-core/v4l2-mc.c
@@ -434,13 +434,14 @@ EXPORT_SYMBOL_GPL(v4l2_create_fwnode_links);
static int pipeline_pm_use_count(struct media_entity *entity,
struct media_graph *graph)
{
+ struct media_pad *pad;
int use = 0;
media_graph_walk_start(graph, entity->pads);
- while ((entity = media_graph_walk_next(graph))) {
- if (is_media_entity_v4l2_video_device(entity))
- use += entity->use_count;
+ while ((pad = media_graph_walk_next(graph))) {
+ if (is_media_entity_v4l2_video_device(pad->entity))
+ use += pad->entity->use_count;
}
return use;
@@ -493,7 +494,7 @@ static int pipeline_pm_power_one(struct media_entity *entity, int change)
static int pipeline_pm_power(struct media_entity *entity, int change,
struct media_graph *graph)
{
- struct media_entity *first = entity;
+ struct media_pad *tmp_pad, *pad;
int ret = 0;
if (!change)
@@ -501,19 +502,18 @@ static int pipeline_pm_power(struct media_entity *entity, int change,
media_graph_walk_start(graph, entity->pads);
- while (!ret && (entity = media_graph_walk_next(graph)))
- if (is_media_entity_v4l2_subdev(entity))
- ret = pipeline_pm_power_one(entity, change);
+ while (!ret && (pad = media_graph_walk_next(graph)))
+ if (is_media_entity_v4l2_subdev(pad->entity))
+ ret = pipeline_pm_power_one(pad->entity, change);
if (!ret)
return ret;
- media_graph_walk_start(graph, first->pads);
+ media_graph_walk_start(graph, entity->pads);
- while ((first = media_graph_walk_next(graph))
- && first != entity)
- if (is_media_entity_v4l2_subdev(first))
- pipeline_pm_power_one(first, -change);
+ while ((tmp_pad = media_graph_walk_next(graph)) && tmp_pad != pad)
+ if (is_media_entity_v4l2_subdev(tmp_pad->entity))
+ pipeline_pm_power_one(tmp_pad->entity, -change);
return ret;
}
diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
index 77bf1b8a56f7..daf8e5d362d3 100644
--- a/drivers/staging/media/omap4iss/iss_video.c
+++ b/drivers/staging/media/omap4iss/iss_video.c
@@ -206,8 +206,8 @@ static struct iss_video *
iss_video_far_end(struct iss_video *video)
{
struct media_graph graph;
- struct media_entity *entity = &video->video.entity;
- struct media_device *mdev = entity->graph_obj.mdev;
+ struct media_pad *pad = video->video.entity.pads;
+ struct media_device *mdev = video->video.entity.graph_obj.mdev;
struct iss_video *far_end = NULL;
mutex_lock(&mdev->graph_mutex);
@@ -217,16 +217,17 @@ iss_video_far_end(struct iss_video *video)
return NULL;
}
- media_graph_walk_start(&graph, entity->pads);
+ media_graph_walk_start(&graph, pad);
- while ((entity = media_graph_walk_next(&graph))) {
- if (entity == &video->video.entity)
+ while ((pad = media_graph_walk_next(&graph))) {
+ if (pad->entity == &video->video.entity)
continue;
- if (!is_media_entity_v4l2_video_device(entity))
+ if (!is_media_entity_v4l2_video_device(pad->entity))
continue;
- far_end = to_iss_video(media_entity_to_video_device(entity));
+ far_end = to_iss_video(media_entity_to_video_device(
+ pad->entity));
if (far_end->type != video->type)
break;
@@ -853,7 +854,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
struct iss_video_fh *vfh = to_iss_video_fh(fh);
struct iss_video *video = video_drvdata(file);
struct media_graph graph;
- struct media_entity *entity = &video->video.entity;
+ struct media_pad *pad = video->video.entity.pads;
enum iss_pipeline_state state;
struct iss_pipeline *pipe;
struct iss_video *far_end;
@@ -869,30 +870,31 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
* Start streaming on the pipeline. No link touching an entity in the
* pipeline can be activated or deactivated once streaming is started.
*/
- pipe = entity->pipe
- ? to_iss_pipeline(entity) : &video->pipe;
+ pipe = pad->entity->pipe
+ ? to_iss_pipeline(pad->entity) : &video->pipe;
pipe->external = NULL;
pipe->external_rate = 0;
pipe->external_bpp = 0;
- ret = media_entity_enum_init(&pipe->ent_enum, entity->graph_obj.mdev);
+ ret = media_entity_enum_init(&pipe->ent_enum,
+ pad->entity->graph_obj.mdev);
if (ret)
goto err_graph_walk_init;
- ret = media_graph_walk_init(&graph, entity->graph_obj.mdev);
+ ret = media_graph_walk_init(&graph, pad->entity->graph_obj.mdev);
if (ret)
goto err_graph_walk_init;
if (video->iss->pdata->set_constraints)
video->iss->pdata->set_constraints(video->iss, true);
- ret = media_pipeline_start(entity, &pipe->pipe);
+ ret = media_pipeline_start(pad->entity, &pipe->pipe);
if (ret < 0)
goto err_media_pipeline_start;
- media_graph_walk_start(&graph, entity->pads);
- while ((entity = media_graph_walk_next(&graph)))
- media_entity_enum_set(&pipe->ent_enum, entity);
+ media_graph_walk_start(&graph, pad);
+ while ((pad = media_graph_walk_next(&graph)))
+ media_entity_enum_set(&pipe->ent_enum, pad->entity);
/*
* Verify that the currently configured format matches the output of
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 97b170cf38eb..e83d55e7f207 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -921,10 +921,11 @@ void media_graph_walk_start(struct media_graph *graph, struct media_pad *pad);
* The graph structure must have been previously initialized with a call to
* media_graph_walk_start().
*
- * Return: returns the next entity in the graph or %NULL if the whole graph
- * have been traversed.
+ * Return: returns the next entity in the graph, identified by the pad through
+ * which it has been reached. If the whole graph has been traversed, return
+ * %NULL.
*/
-struct media_entity *media_graph_walk_next(struct media_graph *graph);
+struct media_pad *media_graph_walk_next(struct media_graph *graph);
/**
* media_pipeline_start - Mark a pipeline as streaming
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 04/24] v4l: mc: Start walk from a specific pad in use count calculation
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (2 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 03/24] media: entity: Walk the graph based on pads Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 05/24] media: entity: Add iterator helper for entity pads Tomi Valkeinen
` (20 subsequent siblings)
24 siblings, 0 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
With the addition of the recent has_route() media entity op, the pads of a
media entity are no longer all interconnected. This has to be taken into
account in power management.
Prepare for the addition of a helper function supporting S_ROUTING.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/v4l2-core/v4l2-mc.c | 27 +++++++++++++--------------
1 file changed, 13 insertions(+), 14 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
index cbeb580c6754..35d18ed89fa5 100644
--- a/drivers/media/v4l2-core/v4l2-mc.c
+++ b/drivers/media/v4l2-core/v4l2-mc.c
@@ -427,17 +427,16 @@ EXPORT_SYMBOL_GPL(v4l2_create_fwnode_links);
/*
* pipeline_pm_use_count - Count the number of users of a pipeline
- * @entity: The entity
+ * @pad: Any pad along the pipeline
*
* Return the total number of users of all video device nodes in the pipeline.
*/
-static int pipeline_pm_use_count(struct media_entity *entity,
- struct media_graph *graph)
+static int pipeline_pm_use_count(struct media_pad *pad,
+ struct media_graph *graph)
{
- struct media_pad *pad;
int use = 0;
- media_graph_walk_start(graph, entity->pads);
+ media_graph_walk_start(graph, pad);
while ((pad = media_graph_walk_next(graph))) {
if (is_media_entity_v4l2_video_device(pad->entity))
@@ -483,7 +482,7 @@ static int pipeline_pm_power_one(struct media_entity *entity, int change)
/*
* pipeline_pm_power - Apply power change to all entities in a pipeline
- * @entity: The entity
+ * @pad: Any pad along the pipeline
* @change: Use count change
*
* Walk the pipeline to update the use count and the power state of all non-node
@@ -491,16 +490,16 @@ static int pipeline_pm_power_one(struct media_entity *entity, int change)
*
* Return 0 on success or a negative error code on failure.
*/
-static int pipeline_pm_power(struct media_entity *entity, int change,
- struct media_graph *graph)
+static int pipeline_pm_power(struct media_pad *pad, int change,
+ struct media_graph *graph)
{
- struct media_pad *tmp_pad, *pad;
+ struct media_pad *tmp_pad, *first = pad;
int ret = 0;
if (!change)
return 0;
- media_graph_walk_start(graph, entity->pads);
+ media_graph_walk_start(graph, pad);
while (!ret && (pad = media_graph_walk_next(graph)))
if (is_media_entity_v4l2_subdev(pad->entity))
@@ -509,7 +508,7 @@ static int pipeline_pm_power(struct media_entity *entity, int change,
if (!ret)
return ret;
- media_graph_walk_start(graph, entity->pads);
+ media_graph_walk_start(graph, first);
while ((tmp_pad = media_graph_walk_next(graph)) && tmp_pad != pad)
if (is_media_entity_v4l2_subdev(tmp_pad->entity))
@@ -531,7 +530,7 @@ static int v4l2_pipeline_pm_use(struct media_entity *entity, unsigned int use)
WARN_ON(entity->use_count < 0);
/* Apply power change to connected non-nodes. */
- ret = pipeline_pm_power(entity, change, &mdev->pm_count_walk);
+ ret = pipeline_pm_power(entity->pads, change, &mdev->pm_count_walk);
if (ret < 0)
entity->use_count -= change;
@@ -557,8 +556,8 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
unsigned int notification)
{
struct media_graph *graph = &link->graph_obj.mdev->pm_count_walk;
- struct media_entity *source = link->source->entity;
- struct media_entity *sink = link->sink->entity;
+ struct media_pad *source = link->source;
+ struct media_pad *sink = link->sink;
int source_use;
int sink_use;
int ret = 0;
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 05/24] media: entity: Add iterator helper for entity pads
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (3 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 04/24] v4l: mc: Start walk from a specific pad in use count calculation Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 06/24] media: entity: Move the pipeline from entity to pads Tomi Valkeinen
` (19 subsequent siblings)
24 siblings, 0 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen
From: Jacopo Mondi <jacopo+renesas@jmondi.org>
Add an iterator helper to easily cycle through all pads in an entity and
use it in media-entity and media-device code where appropriate.
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/mc/mc-device.c | 13 ++++++-------
drivers/media/mc/mc-entity.c | 11 ++++++-----
include/media/media-entity.h | 12 ++++++++++++
3 files changed, 24 insertions(+), 12 deletions(-)
diff --git a/drivers/media/mc/mc-device.c b/drivers/media/mc/mc-device.c
index 9e56d2ad6b94..704ef1360eba 100644
--- a/drivers/media/mc/mc-device.c
+++ b/drivers/media/mc/mc-device.c
@@ -581,7 +581,7 @@ static void __media_device_unregister_entity(struct media_entity *entity)
struct media_device *mdev = entity->graph_obj.mdev;
struct media_link *link, *tmp;
struct media_interface *intf;
- unsigned int i;
+ struct media_pad *iter;
ida_free(&mdev->entity_internal_idx, entity->internal_idx);
@@ -597,8 +597,8 @@ static void __media_device_unregister_entity(struct media_entity *entity)
__media_entity_remove_links(entity);
/* Remove all pads that belong to this entity */
- for (i = 0; i < entity->num_pads; i++)
- media_gobj_destroy(&entity->pads[i].graph_obj);
+ media_entity_for_each_pad(entity, iter)
+ media_gobj_destroy(&iter->graph_obj);
/* Remove the entity */
media_gobj_destroy(&entity->graph_obj);
@@ -617,7 +617,7 @@ int __must_check media_device_register_entity(struct media_device *mdev,
struct media_entity *entity)
{
struct media_entity_notify *notify, *next;
- unsigned int i;
+ struct media_pad *iter;
int ret;
if (entity->function == MEDIA_ENT_F_V4L2_SUBDEV_UNKNOWN ||
@@ -646,9 +646,8 @@ int __must_check media_device_register_entity(struct media_device *mdev,
media_gobj_create(mdev, MEDIA_GRAPH_ENTITY, &entity->graph_obj);
/* Initialize objects at the pads */
- for (i = 0; i < entity->num_pads; i++)
- media_gobj_create(mdev, MEDIA_GRAPH_PAD,
- &entity->pads[i].graph_obj);
+ media_entity_for_each_pad(entity, iter)
+ media_gobj_create(mdev, MEDIA_GRAPH_PAD, &iter->graph_obj);
/* invoke entity_notify callbacks */
list_for_each_entry_safe(notify, next, &mdev->entity_notify, list)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index ac7c1dc0996e..604904e13c5a 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -198,7 +198,8 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads,
struct media_pad *pads)
{
struct media_device *mdev = entity->graph_obj.mdev;
- unsigned int i;
+ struct media_pad *iter;
+ unsigned int i = 0;
if (num_pads >= MEDIA_ENTITY_MAX_PADS)
return -E2BIG;
@@ -209,12 +210,12 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads,
if (mdev)
mutex_lock(&mdev->graph_mutex);
- for (i = 0; i < num_pads; i++) {
- pads[i].entity = entity;
- pads[i].index = i;
+ media_entity_for_each_pad(entity, iter) {
+ iter->entity = entity;
+ iter->index = i++;
if (mdev)
media_gobj_create(mdev, MEDIA_GRAPH_PAD,
- &entity->pads[i].graph_obj);
+ &iter->graph_obj);
}
if (mdev)
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index e83d55e7f207..3d90b72f3858 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -1102,3 +1102,15 @@ void media_remove_intf_links(struct media_interface *intf);
(entity)->ops->operation((entity) , ##args) : -ENOIOCTLCMD)
#endif
+
+/**
+ * media_entity_for_each_pad - Iterate on all pads in an entity
+ * @entity: The entity the pads belong to
+ * @iter: The iterator pad
+ *
+ * Iterate on all pads in a media entity.
+ */
+#define media_entity_for_each_pad(entity, iter) \
+ for (iter = (entity)->pads; \
+ iter < &(entity)->pads[(entity)->num_pads]; \
+ ++iter)
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 06/24] media: entity: Move the pipeline from entity to pads
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (4 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 05/24] media: entity: Add iterator helper for entity pads Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 07/24] media: entity: Use pad as the starting point for a pipeline Tomi Valkeinen
` (18 subsequent siblings)
24 siblings, 0 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
This moves the pipe and stream_count fields from struct media_entity to
struct media_pad. Effectively streams become pad-specific rather than
being entity specific, allowing several independent streams to traverse a
single entity and an entity to be part of several streams.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
- Update documentation to use 'pads'
- Use the media pad iterator in media_entity.c
- Update rcar-dma.c to use the new per-pad stream count
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
- Fix cleanup in the error path of __media_pipeline_start()
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
drivers/media/mc/mc-entity.c | 68 +++++++++++--------
drivers/media/platform/exynos4-is/fimc-isp.c | 2 +-
drivers/media/platform/exynos4-is/fimc-lite.c | 2 +-
drivers/media/platform/omap3isp/isp.c | 2 +-
drivers/media/platform/omap3isp/ispvideo.c | 2 +-
drivers/media/platform/omap3isp/ispvideo.h | 2 +-
drivers/media/platform/rcar-vin/rcar-core.c | 16 +++--
drivers/media/platform/rcar-vin/rcar-dma.c | 2 +-
drivers/media/platform/xilinx/xilinx-dma.c | 2 +-
drivers/media/platform/xilinx/xilinx-dma.h | 2 +-
drivers/staging/media/imx/imx-media-utils.c | 2 +-
drivers/staging/media/omap4iss/iss.c | 2 +-
drivers/staging/media/omap4iss/iss_video.c | 2 +-
drivers/staging/media/omap4iss/iss_video.h | 2 +-
include/media/media-entity.h | 21 +++---
15 files changed, 73 insertions(+), 56 deletions(-)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 604904e13c5a..bc96d82ad10c 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -423,24 +423,28 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
while ((pad = media_graph_walk_next(graph))) {
struct media_entity *entity = pad->entity;
+ bool skip_validation = pad->pipe != NULL;
+ struct media_pad *iter;
DECLARE_BITMAP(active, MEDIA_ENTITY_MAX_PADS);
DECLARE_BITMAP(has_no_links, MEDIA_ENTITY_MAX_PADS);
- entity->stream_count++;
-
- if (entity->pipe && entity->pipe != pipe) {
- pr_err("Pipe active for %s. Can't start for %s\n",
- entity->name,
- pad_err->entity->name);
- ret = -EBUSY;
- goto error;
+ media_entity_for_each_pad(entity, iter) {
+ if (iter->pipe && iter->pipe != pipe) {
+ pr_err("Pipe active for %s. Can't start for %s\n",
+ entity->name, iter->entity->name);
+ ret = -EBUSY;
+ } else {
+ iter->pipe = pipe;
+ }
+ iter->stream_count++;
}
- entity->pipe = pipe;
+ if (ret)
+ goto error;
- /* Already streaming --- no need to check. */
- if (entity->stream_count > 1)
+ /* Already part of the pipeline, skip validation. */
+ if (skip_validation)
continue;
if (!entity->ops || !entity->ops->link_validate)
@@ -509,20 +513,23 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
media_graph_walk_start(graph, pad_err);
while ((pad_err = media_graph_walk_next(graph))) {
- struct media_entity *entity_err = pad_err->entity;
-
- /* Sanity check for negative stream_count */
- if (!WARN_ON_ONCE(entity_err->stream_count <= 0)) {
- entity_err->stream_count--;
- if (entity_err->stream_count == 0)
- entity_err->pipe = NULL;
+ struct media_entity *entity = pad_err->entity;
+ struct media_pad *iter;
+
+ media_entity_for_each_pad(entity, iter) {
+ /* Sanity check for negative stream_count */
+ if (!WARN_ON_ONCE(iter->stream_count <= 0)) {
+ --iter->stream_count;
+ if (iter->stream_count == 0)
+ iter->pipe = NULL;
+ }
}
/*
* We haven't increased stream_count further than this
* so we quit here.
*/
- if (pad_err == pad)
+ if (pad_err->entity == pad->entity)
break;
}
@@ -549,7 +556,7 @@ EXPORT_SYMBOL_GPL(media_pipeline_start);
void __media_pipeline_stop(struct media_entity *entity)
{
- struct media_pipeline *pipe = entity->pipe;
+ struct media_pipeline *pipe = entity->pads->pipe;
struct media_graph *graph = &pipe->graph;
struct media_pad *pad;
@@ -564,12 +571,15 @@ void __media_pipeline_stop(struct media_entity *entity)
while ((pad = media_graph_walk_next(graph))) {
struct media_entity *entity = pad->entity;
-
- /* Sanity check for negative stream_count */
- if (!WARN_ON_ONCE(entity->stream_count <= 0)) {
- entity->stream_count--;
- if (entity->stream_count == 0)
- entity->pipe = NULL;
+ struct media_pad *iter;
+
+ media_entity_for_each_pad(entity, iter) {
+ /* Sanity check for negative stream_count */
+ if (!WARN_ON_ONCE(iter->stream_count <= 0)) {
+ iter->stream_count--;
+ if (iter->stream_count == 0)
+ iter->pipe = NULL;
+ }
}
}
@@ -839,7 +849,7 @@ int __media_entity_setup_link(struct media_link *link, u32 flags)
{
const u32 mask = MEDIA_LNK_FL_ENABLED;
struct media_device *mdev;
- struct media_entity *source, *sink;
+ struct media_pad *source, *sink;
int ret = -EBUSY;
if (link == NULL)
@@ -855,8 +865,8 @@ int __media_entity_setup_link(struct media_link *link, u32 flags)
if (link->flags == flags)
return 0;
- source = link->source->entity;
- sink = link->sink->entity;
+ source = link->source;
+ sink = link->sink;
if (!(link->flags & MEDIA_LNK_FL_DYNAMIC) &&
(source->stream_count || sink->stream_count))
diff --git a/drivers/media/platform/exynos4-is/fimc-isp.c b/drivers/media/platform/exynos4-is/fimc-isp.c
index a77c49b18511..563d07f630bc 100644
--- a/drivers/media/platform/exynos4-is/fimc-isp.c
+++ b/drivers/media/platform/exynos4-is/fimc-isp.c
@@ -223,7 +223,7 @@ static int fimc_isp_subdev_set_fmt(struct v4l2_subdev *sd,
}
}
} else {
- if (sd->entity.stream_count == 0) {
+ if (sd->entity.pads->stream_count == 0) {
if (fmt->pad == FIMC_ISP_SD_PAD_SINK) {
struct v4l2_subdev_format format = *fmt;
diff --git a/drivers/media/platform/exynos4-is/fimc-lite.c b/drivers/media/platform/exynos4-is/fimc-lite.c
index fe20af3a7178..56773a0be8d9 100644
--- a/drivers/media/platform/exynos4-is/fimc-lite.c
+++ b/drivers/media/platform/exynos4-is/fimc-lite.c
@@ -1070,7 +1070,7 @@ static int fimc_lite_subdev_set_fmt(struct v4l2_subdev *sd,
mutex_lock(&fimc->lock);
if ((atomic_read(&fimc->out_path) == FIMC_IO_ISP &&
- sd->entity.stream_count > 0) ||
+ sd->entity.pads->stream_count > 0) ||
(atomic_read(&fimc->out_path) == FIMC_IO_DMA &&
vb2_is_busy(&fimc->vb_queue))) {
mutex_unlock(&fimc->lock);
diff --git a/drivers/media/platform/omap3isp/isp.c b/drivers/media/platform/omap3isp/isp.c
index a6bb7d9bf75f..4ef623f10a44 100644
--- a/drivers/media/platform/omap3isp/isp.c
+++ b/drivers/media/platform/omap3isp/isp.c
@@ -930,7 +930,7 @@ static int isp_pipeline_is_last(struct media_entity *me)
struct isp_pipeline *pipe;
struct media_pad *pad;
- if (!me->pipe)
+ if (!me->pads->pipe)
return 0;
pipe = to_isp_pipeline(me);
if (pipe->stream_state == ISP_PIPELINE_STREAM_STOPPED)
diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
index 5c1cbb1a9003..a8438040c4aa 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -1094,7 +1094,7 @@ isp_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
/* Start streaming on the pipeline. No link touching an entity in the
* pipeline can be activated or deactivated once streaming is started.
*/
- pipe = video->video.entity.pipe
+ pipe = video->video.entity.pads->pipe
? to_isp_pipeline(&video->video.entity) : &video->pipe;
ret = media_entity_enum_init(&pipe->ent_enum, &video->isp->media_dev);
diff --git a/drivers/media/platform/omap3isp/ispvideo.h b/drivers/media/platform/omap3isp/ispvideo.h
index a0908670c0cf..4c9c5b719ec5 100644
--- a/drivers/media/platform/omap3isp/ispvideo.h
+++ b/drivers/media/platform/omap3isp/ispvideo.h
@@ -100,7 +100,7 @@ struct isp_pipeline {
};
#define to_isp_pipeline(__e) \
- container_of((__e)->pipe, struct isp_pipeline, pipe)
+ container_of((__e)->pads->pipe, struct isp_pipeline, pipe)
static inline int isp_pipeline_ready(struct isp_pipeline *pipe)
{
diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index cb3025992817..07ec008aacc4 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -132,13 +132,17 @@ static int rvin_group_link_notify(struct media_link *link, u32 flags,
return 0;
/*
- * Don't allow link changes if any entity in the graph is
- * streaming, modifying the CHSEL register fields can disrupt
- * running streams.
+ * Don't allow link changes if any stream in the graph is active as
+ * modifying the CHSEL register fields can disrupt running streams.
*/
- media_device_for_each_entity(entity, &group->mdev)
- if (entity->stream_count)
- return -EBUSY;
+ media_device_for_each_entity(entity, &group->mdev) {
+ struct media_pad *iter;
+
+ media_entity_for_each_pad(entity, iter) {
+ if (iter->stream_count)
+ return -EBUSY;
+ }
+ }
mutex_lock(&group->lock);
diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index f30dafbdf61c..7994262c9b63 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -1231,7 +1231,7 @@ static int rvin_set_stream(struct rvin_dev *vin, int on)
*/
mdev = vin->vdev.entity.graph_obj.mdev;
mutex_lock(&mdev->graph_mutex);
- pipe = sd->entity.pipe ? sd->entity.pipe : &vin->vdev.pipe;
+ pipe = sd->entity.pads->pipe ? sd->entity.pads->pipe : &vin->vdev.pipe;
ret = __media_pipeline_start(&vin->vdev.entity, pipe);
mutex_unlock(&mdev->graph_mutex);
if (ret)
diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index 8df3c43aecbe..7fa0467dddde 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -403,7 +403,7 @@ static int xvip_dma_start_streaming(struct vb2_queue *vq, unsigned int count)
* Use the pipeline object embedded in the first DMA object that starts
* streaming.
*/
- pipe = dma->video.entity.pipe
+ pipe = dma->video.entity.pads->pipe
? to_xvip_pipeline(&dma->video.entity) : &dma->pipe;
ret = media_pipeline_start(&dma->video.entity, &pipe->pipe);
diff --git a/drivers/media/platform/xilinx/xilinx-dma.h b/drivers/media/platform/xilinx/xilinx-dma.h
index 2378bdae57ae..69ced71a5696 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.h
+++ b/drivers/media/platform/xilinx/xilinx-dma.h
@@ -47,7 +47,7 @@ struct xvip_pipeline {
static inline struct xvip_pipeline *to_xvip_pipeline(struct media_entity *e)
{
- return container_of(e->pipe, struct xvip_pipeline, pipe);
+ return container_of(e->pads->pipe, struct xvip_pipeline, pipe);
}
/**
diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
index 5128915a5d6f..04b7c6bdcd85 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -913,7 +913,7 @@ int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
__media_pipeline_stop(entity);
} else {
v4l2_subdev_call(sd, video, s_stream, 0);
- if (entity->pipe)
+ if (entity->pads->pipe)
__media_pipeline_stop(entity);
}
diff --git a/drivers/staging/media/omap4iss/iss.c b/drivers/staging/media/omap4iss/iss.c
index 085397045b36..4f4573219337 100644
--- a/drivers/staging/media/omap4iss/iss.c
+++ b/drivers/staging/media/omap4iss/iss.c
@@ -543,7 +543,7 @@ static int iss_pipeline_is_last(struct media_entity *me)
struct iss_pipeline *pipe;
struct media_pad *pad;
- if (!me->pipe)
+ if (!me->pads->pipe)
return 0;
pipe = to_iss_pipeline(me);
if (pipe->stream_state == ISS_PIPELINE_STREAM_STOPPED)
diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
index daf8e5d362d3..d781ed10253e 100644
--- a/drivers/staging/media/omap4iss/iss_video.c
+++ b/drivers/staging/media/omap4iss/iss_video.c
@@ -870,7 +870,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
* Start streaming on the pipeline. No link touching an entity in the
* pipeline can be activated or deactivated once streaming is started.
*/
- pipe = pad->entity->pipe
+ pipe = pad->pipe
? to_iss_pipeline(pad->entity) : &video->pipe;
pipe->external = NULL;
pipe->external_rate = 0;
diff --git a/drivers/staging/media/omap4iss/iss_video.h b/drivers/staging/media/omap4iss/iss_video.h
index 526281bf0051..9b8ec27bf87d 100644
--- a/drivers/staging/media/omap4iss/iss_video.h
+++ b/drivers/staging/media/omap4iss/iss_video.h
@@ -91,7 +91,7 @@ struct iss_pipeline {
};
#define to_iss_pipeline(__e) \
- container_of((__e)->pipe, struct iss_pipeline, pipe)
+ container_of((__e)->pads->pipe, struct iss_pipeline, pipe)
static inline int iss_pipeline_ready(struct iss_pipeline *pipe)
{
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 3d90b72f3858..3026a24cc591 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -180,15 +180,24 @@ enum media_pad_signal_type {
*
* @graph_obj: Embedded structure containing the media object common data
* @entity: Entity this pad belongs to
+ * @pipe: Pipeline this pad belongs to
+ * @stream_count: Stream count for the pad
* @index: Pad index in the entity pads array, numbered from 0 to n
* @sig_type: Type of the signal inside a media pad
* @flags: Pad flags, as defined in
* :ref:`include/uapi/linux/media.h <media_header>`
* (seek for ``MEDIA_PAD_FL_*``)
+ * .. note::
+ *
+ * @stream_count reference count must never be negative, but is a signed
+ * integer on purpose: a simple ``WARN_ON(<0)`` check can be used to
+ * detect reference count bugs that would make it negative.
*/
struct media_pad {
struct media_gobj graph_obj; /* must be first field in struct */
struct media_entity *entity;
+ struct media_pipeline *pipe;
+ int stream_count;
u16 index;
enum media_pad_signal_type sig_type;
unsigned long flags;
@@ -267,9 +276,7 @@ enum media_entity_type {
* @pads: Pads array with the size defined by @num_pads.
* @links: List of data links.
* @ops: Entity operations.
- * @stream_count: Stream count for the entity.
* @use_count: Use count for the entity.
- * @pipe: Pipeline this entity belongs to.
* @info: Union with devnode information. Kept just for backward
* compatibility.
* @info.dev: Contains device major and minor info.
@@ -282,10 +289,9 @@ enum media_entity_type {
*
* .. note::
*
- * @stream_count and @use_count reference counts must never be
- * negative, but are signed integers on purpose: a simple ``WARN_ON(<0)``
- * check can be used to detect reference count bugs that would make them
- * negative.
+ * @use_count reference count must never be negative, but is a signed
+ * integer on purpose: a simple ``WARN_ON(<0)`` check can be used to
+ * detect reference count bugs that would make it negative.
*/
struct media_entity {
struct media_gobj graph_obj; /* must be first field in struct */
@@ -304,11 +310,8 @@ struct media_entity {
const struct media_entity_operations *ops;
- int stream_count;
int use_count;
- struct media_pipeline *pipe;
-
union {
struct {
u32 major;
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 07/24] media: entity: Use pad as the starting point for a pipeline
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (5 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 06/24] media: entity: Move the pipeline from entity to pads Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 08/24] media: entity: Add has_route entity operation Tomi Valkeinen
` (17 subsequent siblings)
24 siblings, 0 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
The pipeline has been moved from the entity to the pads; reflect this in
the media pipeline function API.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
Documentation/driver-api/media/mc-core.rst | 6 ++--
drivers/media/mc/mc-entity.c | 24 ++++++-------
drivers/media/pci/intel/ipu3/ipu3-cio2-main.c | 6 ++--
.../media/platform/exynos4-is/fimc-capture.c | 8 ++---
.../platform/exynos4-is/fimc-isp-video.c | 8 ++---
drivers/media/platform/exynos4-is/fimc-lite.c | 8 ++---
drivers/media/platform/omap3isp/ispvideo.c | 6 ++--
.../media/platform/qcom/camss/camss-video.c | 6 ++--
drivers/media/platform/rcar-vin/rcar-dma.c | 6 ++--
.../platform/rockchip/rkisp1/rkisp1-capture.c | 6 ++--
.../media/platform/s3c-camif/camif-capture.c | 6 ++--
drivers/media/platform/stm32/stm32-dcmi.c | 6 ++--
.../platform/sunxi/sun4i-csi/sun4i_dma.c | 6 ++--
.../platform/sunxi/sun6i-csi/sun6i_video.c | 6 ++--
drivers/media/platform/ti-vpe/cal-video.c | 6 ++--
drivers/media/platform/vsp1/vsp1_video.c | 6 ++--
drivers/media/platform/xilinx/xilinx-dma.c | 6 ++--
.../media/test-drivers/vimc/vimc-capture.c | 6 ++--
drivers/media/usb/au0828/au0828-core.c | 8 ++---
drivers/staging/media/imx/imx-media-utils.c | 6 ++--
drivers/staging/media/ipu3/ipu3-v4l2.c | 6 ++--
drivers/staging/media/omap4iss/iss_video.c | 6 ++--
drivers/staging/media/tegra-video/tegra210.c | 6 ++--
include/media/media-entity.h | 34 +++++++++----------
24 files changed, 98 insertions(+), 100 deletions(-)
diff --git a/Documentation/driver-api/media/mc-core.rst b/Documentation/driver-api/media/mc-core.rst
index 8a13640bed56..69a64279a61f 100644
--- a/Documentation/driver-api/media/mc-core.rst
+++ b/Documentation/driver-api/media/mc-core.rst
@@ -213,11 +213,11 @@ When starting streaming, drivers must notify all entities in the pipeline to
prevent link states from being modified during streaming by calling
:c:func:`media_pipeline_start()`.
-The function will mark all entities connected to the given entity through
-enabled links, either directly or indirectly, as streaming.
+The function will mark all entities connected to the given pad through
+enabled routes and links, either directly or indirectly, as streaming.
The struct media_pipeline instance pointed to by
-the pipe argument will be stored in every entity in the pipeline.
+the pipe argument will be stored in every pad in the pipeline.
Drivers should embed the struct media_pipeline
in higher-level pipeline structures and can then access the
pipeline through the struct media_entity
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index bc96d82ad10c..c8c8cd75b6bf 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -403,12 +403,11 @@ EXPORT_SYMBOL_GPL(media_entity_get_fwnode_pad);
* Pipeline management
*/
-__must_check int __media_pipeline_start(struct media_entity *entity,
+__must_check int __media_pipeline_start(struct media_pad *pad,
struct media_pipeline *pipe)
{
- struct media_device *mdev = entity->graph_obj.mdev;
+ struct media_device *mdev = pad->graph_obj.mdev;
struct media_graph *graph = &pipe->graph;
- struct media_pad *pad = entity->pads;
struct media_pad *pad_err = pad;
struct media_link *link;
int ret;
@@ -541,24 +540,23 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
}
EXPORT_SYMBOL_GPL(__media_pipeline_start);
-__must_check int media_pipeline_start(struct media_entity *entity,
+__must_check int media_pipeline_start(struct media_pad *pad,
struct media_pipeline *pipe)
{
- struct media_device *mdev = entity->graph_obj.mdev;
+ struct media_device *mdev = pad->graph_obj.mdev;
int ret;
mutex_lock(&mdev->graph_mutex);
- ret = __media_pipeline_start(entity, pipe);
+ ret = __media_pipeline_start(pad, pipe);
mutex_unlock(&mdev->graph_mutex);
return ret;
}
EXPORT_SYMBOL_GPL(media_pipeline_start);
-void __media_pipeline_stop(struct media_entity *entity)
+void __media_pipeline_stop(struct media_pad *pad)
{
- struct media_pipeline *pipe = entity->pads->pipe;
+ struct media_pipeline *pipe = pad->pipe;
struct media_graph *graph = &pipe->graph;
- struct media_pad *pad;
/*
* If the following check fails, the driver has performed an
@@ -567,7 +565,7 @@ void __media_pipeline_stop(struct media_entity *entity)
if (WARN_ON(!pipe))
return;
- media_graph_walk_start(graph, entity->pads);
+ media_graph_walk_start(graph, pad);
while ((pad = media_graph_walk_next(graph))) {
struct media_entity *entity = pad->entity;
@@ -589,12 +587,12 @@ void __media_pipeline_stop(struct media_entity *entity)
}
EXPORT_SYMBOL_GPL(__media_pipeline_stop);
-void media_pipeline_stop(struct media_entity *entity)
+void media_pipeline_stop(struct media_pad *pad)
{
- struct media_device *mdev = entity->graph_obj.mdev;
+ struct media_device *mdev = pad->graph_obj.mdev;
mutex_lock(&mdev->graph_mutex);
- __media_pipeline_stop(entity);
+ __media_pipeline_stop(pad);
mutex_unlock(&mdev->graph_mutex);
}
EXPORT_SYMBOL_GPL(media_pipeline_stop);
diff --git a/drivers/media/pci/intel/ipu3/ipu3-cio2-main.c b/drivers/media/pci/intel/ipu3/ipu3-cio2-main.c
index 6e8c0c230e11..53c87e972048 100644
--- a/drivers/media/pci/intel/ipu3/ipu3-cio2-main.c
+++ b/drivers/media/pci/intel/ipu3/ipu3-cio2-main.c
@@ -982,7 +982,7 @@ static int cio2_vb2_start_streaming(struct vb2_queue *vq, unsigned int count)
return r;
}
- r = media_pipeline_start(&q->vdev.entity, &q->pipe);
+ r = media_pipeline_start(q->vdev.entity.pads, &q->pipe);
if (r)
goto fail_pipeline;
@@ -1002,7 +1002,7 @@ static int cio2_vb2_start_streaming(struct vb2_queue *vq, unsigned int count)
fail_csi2_subdev:
cio2_hw_exit(cio2, q);
fail_hw:
- media_pipeline_stop(&q->vdev.entity);
+ media_pipeline_stop(q->vdev.entity.pads);
fail_pipeline:
dev_dbg(&cio2->pci_dev->dev, "failed to start streaming (%d)\n", r);
cio2_vb2_return_all_buffers(q, VB2_BUF_STATE_QUEUED);
@@ -1023,7 +1023,7 @@ static void cio2_vb2_stop_streaming(struct vb2_queue *vq)
cio2_hw_exit(cio2, q);
synchronize_irq(cio2->pci_dev->irq);
cio2_vb2_return_all_buffers(q, VB2_BUF_STATE_ERROR);
- media_pipeline_stop(&q->vdev.entity);
+ media_pipeline_stop(q->vdev.entity.pads);
pm_runtime_put(&cio2->pci_dev->dev);
cio2->streaming = false;
}
diff --git a/drivers/media/platform/exynos4-is/fimc-capture.c b/drivers/media/platform/exynos4-is/fimc-capture.c
index 13c838d3f947..2f67415575db 100644
--- a/drivers/media/platform/exynos4-is/fimc-capture.c
+++ b/drivers/media/platform/exynos4-is/fimc-capture.c
@@ -526,7 +526,7 @@ static int fimc_capture_release(struct file *file)
mutex_lock(&fimc->lock);
if (close && vc->streaming) {
- media_pipeline_stop(&vc->ve.vdev.entity);
+ media_pipeline_stop(vc->ve.vdev.entity.pads);
vc->streaming = false;
}
@@ -1186,7 +1186,7 @@ static int fimc_cap_streamon(struct file *file, void *priv,
if (fimc_capture_active(fimc))
return -EBUSY;
- ret = media_pipeline_start(entity, &vc->ve.pipe->mp);
+ ret = media_pipeline_start(entity->pads, &vc->ve.pipe->mp);
if (ret < 0)
return ret;
@@ -1220,7 +1220,7 @@ static int fimc_cap_streamon(struct file *file, void *priv,
}
err_p_stop:
- media_pipeline_stop(entity);
+ media_pipeline_stop(entity->pads);
return ret;
}
@@ -1236,7 +1236,7 @@ static int fimc_cap_streamoff(struct file *file, void *priv,
return ret;
if (vc->streaming) {
- media_pipeline_stop(&vc->ve.vdev.entity);
+ media_pipeline_stop(vc->ve.vdev.entity.pads);
vc->streaming = false;
}
diff --git a/drivers/media/platform/exynos4-is/fimc-isp-video.c b/drivers/media/platform/exynos4-is/fimc-isp-video.c
index 612b9872afc8..d9289ff6949a 100644
--- a/drivers/media/platform/exynos4-is/fimc-isp-video.c
+++ b/drivers/media/platform/exynos4-is/fimc-isp-video.c
@@ -310,7 +310,7 @@ static int isp_video_release(struct file *file)
mutex_lock(&isp->video_lock);
if (v4l2_fh_is_singular_file(file) && ivc->streaming) {
- media_pipeline_stop(entity);
+ media_pipeline_stop(entity->pads);
ivc->streaming = 0;
}
@@ -491,7 +491,7 @@ static int isp_video_streamon(struct file *file, void *priv,
struct media_entity *me = &ve->vdev.entity;
int ret;
- ret = media_pipeline_start(me, &ve->pipe->mp);
+ ret = media_pipeline_start(me->pads, &ve->pipe->mp);
if (ret < 0)
return ret;
@@ -506,7 +506,7 @@ static int isp_video_streamon(struct file *file, void *priv,
isp->video_capture.streaming = 1;
return 0;
p_stop:
- media_pipeline_stop(me);
+ media_pipeline_stop(me->pads);
return ret;
}
@@ -521,7 +521,7 @@ static int isp_video_streamoff(struct file *file, void *priv,
if (ret < 0)
return ret;
- media_pipeline_stop(&video->ve.vdev.entity);
+ media_pipeline_stop(video->ve.vdev.entity.pads);
video->streaming = 0;
return 0;
}
diff --git a/drivers/media/platform/exynos4-is/fimc-lite.c b/drivers/media/platform/exynos4-is/fimc-lite.c
index 56773a0be8d9..a1570f6ac738 100644
--- a/drivers/media/platform/exynos4-is/fimc-lite.c
+++ b/drivers/media/platform/exynos4-is/fimc-lite.c
@@ -515,7 +515,7 @@ static int fimc_lite_release(struct file *file)
if (v4l2_fh_is_singular_file(file) &&
atomic_read(&fimc->out_path) == FIMC_IO_DMA) {
if (fimc->streaming) {
- media_pipeline_stop(entity);
+ media_pipeline_stop(entity->pads);
fimc->streaming = false;
}
fimc_lite_stop_capture(fimc, false);
@@ -819,7 +819,7 @@ static int fimc_lite_streamon(struct file *file, void *priv,
if (fimc_lite_active(fimc))
return -EBUSY;
- ret = media_pipeline_start(entity, &fimc->ve.pipe->mp);
+ ret = media_pipeline_start(entity->pads, &fimc->ve.pipe->mp);
if (ret < 0)
return ret;
@@ -836,7 +836,7 @@ static int fimc_lite_streamon(struct file *file, void *priv,
}
err_p_stop:
- media_pipeline_stop(entity);
+ media_pipeline_stop(entity->pads);
return 0;
}
@@ -850,7 +850,7 @@ static int fimc_lite_streamoff(struct file *file, void *priv,
if (ret < 0)
return ret;
- media_pipeline_stop(&fimc->ve.vdev.entity);
+ media_pipeline_stop(fimc->ve.vdev.entity.pads);
fimc->streaming = false;
return 0;
}
diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
index a8438040c4aa..87334477f223 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -1105,7 +1105,7 @@ isp_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
pipe->l3_ick = clk_get_rate(video->isp->clock[ISP_CLK_L3_ICK]);
pipe->max_rate = pipe->l3_ick;
- ret = media_pipeline_start(&video->video.entity, &pipe->pipe);
+ ret = media_pipeline_start(video->video.entity.pads, &pipe->pipe);
if (ret < 0)
goto err_pipeline_start;
@@ -1162,7 +1162,7 @@ isp_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
return 0;
err_check_format:
- media_pipeline_stop(&video->video.entity);
+ media_pipeline_stop(video->video.entity.pads);
err_pipeline_start:
/* TODO: Implement PM QoS */
/* The DMA queue must be emptied here, otherwise CCDC interrupts that
@@ -1229,7 +1229,7 @@ isp_video_streamoff(struct file *file, void *fh, enum v4l2_buf_type type)
video->error = false;
/* TODO: Implement PM QoS */
- media_pipeline_stop(&video->video.entity);
+ media_pipeline_stop(video->video.entity.pads);
media_entity_enum_cleanup(&pipe->ent_enum);
diff --git a/drivers/media/platform/qcom/camss/camss-video.c b/drivers/media/platform/qcom/camss/camss-video.c
index f282275af626..5cd494f17589 100644
--- a/drivers/media/platform/qcom/camss/camss-video.c
+++ b/drivers/media/platform/qcom/camss/camss-video.c
@@ -491,7 +491,7 @@ static int video_start_streaming(struct vb2_queue *q, unsigned int count)
struct v4l2_subdev *subdev;
int ret;
- ret = media_pipeline_start(&vdev->entity, &video->pipe);
+ ret = media_pipeline_start(vdev->entity.pads, &video->pipe);
if (ret < 0)
return ret;
@@ -520,7 +520,7 @@ static int video_start_streaming(struct vb2_queue *q, unsigned int count)
return 0;
error:
- media_pipeline_stop(&vdev->entity);
+ media_pipeline_stop(vdev->entity.pads);
video->ops->flush_buffers(video, VB2_BUF_STATE_QUEUED);
@@ -551,7 +551,7 @@ static void video_stop_streaming(struct vb2_queue *q)
v4l2_subdev_call(subdev, video, s_stream, 0);
}
- media_pipeline_stop(&vdev->entity);
+ media_pipeline_stop(vdev->entity.pads);
video->ops->flush_buffers(video, VB2_BUF_STATE_ERROR);
}
diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index 7994262c9b63..2d930e53c1e2 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -1215,7 +1215,7 @@ static int rvin_set_stream(struct rvin_dev *vin, int on)
sd = media_entity_to_v4l2_subdev(pad->entity);
if (!on) {
- media_pipeline_stop(&vin->vdev.entity);
+ media_pipeline_stop(vin->vdev.entity.pads);
return v4l2_subdev_call(sd, video, s_stream, 0);
}
@@ -1232,7 +1232,7 @@ static int rvin_set_stream(struct rvin_dev *vin, int on)
mdev = vin->vdev.entity.graph_obj.mdev;
mutex_lock(&mdev->graph_mutex);
pipe = sd->entity.pads->pipe ? sd->entity.pads->pipe : &vin->vdev.pipe;
- ret = __media_pipeline_start(&vin->vdev.entity, pipe);
+ ret = __media_pipeline_start(vin->vdev.entity.pads, pipe);
mutex_unlock(&mdev->graph_mutex);
if (ret)
return ret;
@@ -1241,7 +1241,7 @@ static int rvin_set_stream(struct rvin_dev *vin, int on)
if (ret == -ENOIOCTLCMD)
ret = 0;
if (ret)
- media_pipeline_stop(&vin->vdev.entity);
+ media_pipeline_stop(vin->vdev.entity.pads);
return ret;
}
diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-capture.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-capture.c
index 5f6c9d1623e4..c2a11fcc9709 100644
--- a/drivers/media/platform/rockchip/rkisp1/rkisp1-capture.c
+++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-capture.c
@@ -979,7 +979,7 @@ static void rkisp1_vb2_stop_streaming(struct vb2_queue *queue)
rkisp1_dummy_buf_destroy(cap);
- media_pipeline_stop(&node->vdev.entity);
+ media_pipeline_stop(node->vdev.entity.pads);
mutex_unlock(&cap->rkisp1->stream_lock);
}
@@ -993,7 +993,7 @@ rkisp1_vb2_start_streaming(struct vb2_queue *queue, unsigned int count)
mutex_lock(&cap->rkisp1->stream_lock);
- ret = media_pipeline_start(entity, &cap->rkisp1->pipe);
+ ret = media_pipeline_start(entity->pads, &cap->rkisp1->pipe);
if (ret) {
dev_err(cap->rkisp1->dev, "start pipeline failed %d\n", ret);
goto err_ret_buffers;
@@ -1030,7 +1030,7 @@ rkisp1_vb2_start_streaming(struct vb2_queue *queue, unsigned int count)
err_destroy_dummy:
rkisp1_dummy_buf_destroy(cap);
err_pipeline_stop:
- media_pipeline_stop(entity);
+ media_pipeline_stop(entity->pads);
err_ret_buffers:
rkisp1_return_all_buffers(cap, VB2_BUF_STATE_QUEUED);
mutex_unlock(&cap->rkisp1->stream_lock);
diff --git a/drivers/media/platform/s3c-camif/camif-capture.c b/drivers/media/platform/s3c-camif/camif-capture.c
index 9ca49af29542..22451bc2ef2d 100644
--- a/drivers/media/platform/s3c-camif/camif-capture.c
+++ b/drivers/media/platform/s3c-camif/camif-capture.c
@@ -848,13 +848,13 @@ static int s3c_camif_streamon(struct file *file, void *priv,
if (s3c_vp_active(vp))
return 0;
- ret = media_pipeline_start(sensor, camif->m_pipeline);
+ ret = media_pipeline_start(sensor->pads, camif->m_pipeline);
if (ret < 0)
return ret;
ret = camif_pipeline_validate(camif);
if (ret < 0) {
- media_pipeline_stop(sensor);
+ media_pipeline_stop(sensor->pads);
return ret;
}
@@ -878,7 +878,7 @@ static int s3c_camif_streamoff(struct file *file, void *priv,
ret = vb2_streamoff(&vp->vb_queue, type);
if (ret == 0)
- media_pipeline_stop(&camif->sensor.sd->entity);
+ media_pipeline_stop(camif->sensor.sd->entity.pads);
return ret;
}
diff --git a/drivers/media/platform/stm32/stm32-dcmi.c b/drivers/media/platform/stm32/stm32-dcmi.c
index bbcc2254fa2e..c68fdb6c766b 100644
--- a/drivers/media/platform/stm32/stm32-dcmi.c
+++ b/drivers/media/platform/stm32/stm32-dcmi.c
@@ -730,7 +730,7 @@ static int dcmi_start_streaming(struct vb2_queue *vq, unsigned int count)
goto err_pm_put;
}
- ret = media_pipeline_start(&dcmi->vdev->entity, &dcmi->pipeline);
+ ret = media_pipeline_start(dcmi->vdev->entity.pads, &dcmi->pipeline);
if (ret < 0) {
dev_err(dcmi->dev, "%s: Failed to start streaming, media pipeline start error (%d)\n",
__func__, ret);
@@ -844,7 +844,7 @@ static int dcmi_start_streaming(struct vb2_queue *vq, unsigned int count)
dcmi_pipeline_stop(dcmi);
err_media_pipeline_stop:
- media_pipeline_stop(&dcmi->vdev->entity);
+ media_pipeline_stop(dcmi->vdev->entity.pads);
err_pm_put:
pm_runtime_put(dcmi->dev);
@@ -870,7 +870,7 @@ static void dcmi_stop_streaming(struct vb2_queue *vq)
dcmi_pipeline_stop(dcmi);
- media_pipeline_stop(&dcmi->vdev->entity);
+ media_pipeline_stop(dcmi->vdev->entity.pads);
spin_lock_irq(&dcmi->irqlock);
diff --git a/drivers/media/platform/sunxi/sun4i-csi/sun4i_dma.c b/drivers/media/platform/sunxi/sun4i-csi/sun4i_dma.c
index 2c39cd7f2862..be0defdf74f1 100644
--- a/drivers/media/platform/sunxi/sun4i-csi/sun4i_dma.c
+++ b/drivers/media/platform/sunxi/sun4i-csi/sun4i_dma.c
@@ -266,7 +266,7 @@ static int sun4i_csi_start_streaming(struct vb2_queue *vq, unsigned int count)
goto err_clear_dma_queue;
}
- ret = media_pipeline_start(&csi->vdev.entity, &csi->vdev.pipe);
+ ret = media_pipeline_start(csi->vdev.entity.pads, &csi->vdev.pipe);
if (ret < 0)
goto err_free_scratch_buffer;
@@ -330,7 +330,7 @@ static int sun4i_csi_start_streaming(struct vb2_queue *vq, unsigned int count)
sun4i_csi_capture_stop(csi);
err_disable_pipeline:
- media_pipeline_stop(&csi->vdev.entity);
+ media_pipeline_stop(csi->vdev.entity.pads);
err_free_scratch_buffer:
dma_free_coherent(csi->dev, csi->scratch.size, csi->scratch.vaddr,
@@ -359,7 +359,7 @@ static void sun4i_csi_stop_streaming(struct vb2_queue *vq)
return_all_buffers(csi, VB2_BUF_STATE_ERROR);
spin_unlock_irqrestore(&csi->qlock, flags);
- media_pipeline_stop(&csi->vdev.entity);
+ media_pipeline_stop(csi->vdev.entity.pads);
dma_free_coherent(csi->dev, csi->scratch.size, csi->scratch.vaddr,
csi->scratch.paddr);
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
index 3181d0781b61..537057a75eaa 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
@@ -141,7 +141,7 @@ static int sun6i_video_start_streaming(struct vb2_queue *vq, unsigned int count)
video->sequence = 0;
- ret = media_pipeline_start(&video->vdev.entity, &video->vdev.pipe);
+ ret = media_pipeline_start(video->vdev.entity.pads, &video->vdev.pipe);
if (ret < 0)
goto clear_dma_queue;
@@ -207,7 +207,7 @@ static int sun6i_video_start_streaming(struct vb2_queue *vq, unsigned int count)
stop_csi_stream:
sun6i_csi_set_stream(video->csi, false);
stop_media_pipeline:
- media_pipeline_stop(&video->vdev.entity);
+ media_pipeline_stop(video->vdev.entity.pads);
clear_dma_queue:
spin_lock_irqsave(&video->dma_queue_lock, flags);
list_for_each_entry(buf, &video->dma_queue, list)
@@ -231,7 +231,7 @@ static void sun6i_video_stop_streaming(struct vb2_queue *vq)
sun6i_csi_set_stream(video->csi, false);
- media_pipeline_stop(&video->vdev.entity);
+ media_pipeline_stop(video->vdev.entity.pads);
/* Release all active buffers */
spin_lock_irqsave(&video->dma_queue_lock, flags);
diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 7b7436a355ee..ca75b54311a8 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -675,7 +675,7 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
dma_addr_t addr;
int ret;
- ret = media_pipeline_start(&ctx->vdev.entity, &ctx->phy->pipe);
+ ret = media_pipeline_start(ctx->vdev.entity.pads, &ctx->phy->pipe);
if (ret < 0) {
ctx_err(ctx, "Failed to start media pipeline: %d\n", ret);
goto error_release_buffers;
@@ -719,7 +719,7 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
pm_runtime_put_sync(ctx->cal->dev);
error_pipeline:
- media_pipeline_stop(&ctx->vdev.entity);
+ media_pipeline_stop(ctx->vdev.entity.pads);
error_release_buffers:
cal_release_buffers(ctx, VB2_BUF_STATE_QUEUED);
@@ -738,7 +738,7 @@ static void cal_stop_streaming(struct vb2_queue *vq)
cal_release_buffers(ctx, VB2_BUF_STATE_ERROR);
- media_pipeline_stop(&ctx->vdev.entity);
+ media_pipeline_stop(ctx->vdev.entity.pads);
}
static const struct vb2_ops cal_video_qops = {
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index f2c36f2fdf53..978f820b0f34 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -927,7 +927,7 @@ static void vsp1_video_stop_streaming(struct vb2_queue *vq)
}
mutex_unlock(&pipe->lock);
- media_pipeline_stop(&video->video.entity);
+ media_pipeline_stop(video->video.entity.pads);
vsp1_video_release_buffers(video);
vsp1_video_pipeline_put(pipe);
}
@@ -1048,7 +1048,7 @@ vsp1_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
return PTR_ERR(pipe);
}
- ret = __media_pipeline_start(&video->video.entity, &pipe->pipe);
+ ret = __media_pipeline_start(video->video.entity.pads, &pipe->pipe);
if (ret < 0) {
mutex_unlock(&mdev->graph_mutex);
goto err_pipe;
@@ -1072,7 +1072,7 @@ vsp1_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
return 0;
err_stop:
- media_pipeline_stop(&video->video.entity);
+ media_pipeline_stop(video->video.entity.pads);
err_pipe:
vsp1_video_pipeline_put(pipe);
return ret;
diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index 7fa0467dddde..07074eda5f70 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -406,7 +406,7 @@ static int xvip_dma_start_streaming(struct vb2_queue *vq, unsigned int count)
pipe = dma->video.entity.pads->pipe
? to_xvip_pipeline(&dma->video.entity) : &dma->pipe;
- ret = media_pipeline_start(&dma->video.entity, &pipe->pipe);
+ ret = media_pipeline_start(dma->video.entity.pads, &pipe->pipe);
if (ret < 0)
goto error;
@@ -432,7 +432,7 @@ static int xvip_dma_start_streaming(struct vb2_queue *vq, unsigned int count)
return 0;
error_stop:
- media_pipeline_stop(&dma->video.entity);
+ media_pipeline_stop(dma->video.entity.pads);
error:
/* Give back all queued buffers to videobuf2. */
@@ -460,7 +460,7 @@ static void xvip_dma_stop_streaming(struct vb2_queue *vq)
/* Cleanup the pipeline and mark it as being stopped. */
xvip_pipeline_cleanup(pipe);
- media_pipeline_stop(&dma->video.entity);
+ media_pipeline_stop(dma->video.entity.pads);
/* Give back all queued buffers to videobuf2. */
spin_lock_irq(&dma->queued_lock);
diff --git a/drivers/media/test-drivers/vimc/vimc-capture.c b/drivers/media/test-drivers/vimc/vimc-capture.c
index 5e9fd902cd37..10724b0a868c 100644
--- a/drivers/media/test-drivers/vimc/vimc-capture.c
+++ b/drivers/media/test-drivers/vimc/vimc-capture.c
@@ -246,7 +246,7 @@ static int vimc_cap_start_streaming(struct vb2_queue *vq, unsigned int count)
vcap->sequence = 0;
/* Start the media pipeline */
- ret = media_pipeline_start(entity, &vcap->stream.pipe);
+ ret = media_pipeline_start(entity->pads, &vcap->stream.pipe);
if (ret) {
vimc_cap_return_all_buffers(vcap, VB2_BUF_STATE_QUEUED);
return ret;
@@ -254,7 +254,7 @@ static int vimc_cap_start_streaming(struct vb2_queue *vq, unsigned int count)
ret = vimc_streamer_s_stream(&vcap->stream, &vcap->ved, 1);
if (ret) {
- media_pipeline_stop(entity);
+ media_pipeline_stop(entity->pads);
vimc_cap_return_all_buffers(vcap, VB2_BUF_STATE_QUEUED);
return ret;
}
@@ -273,7 +273,7 @@ static void vimc_cap_stop_streaming(struct vb2_queue *vq)
vimc_streamer_s_stream(&vcap->stream, &vcap->ved, 0);
/* Stop the media pipeline */
- media_pipeline_stop(&vcap->vdev.entity);
+ media_pipeline_stop(vcap->vdev.entity.pads);
/* Release all active buffers */
vimc_cap_return_all_buffers(vcap, VB2_BUF_STATE_ERROR);
diff --git a/drivers/media/usb/au0828/au0828-core.c b/drivers/media/usb/au0828/au0828-core.c
index a8a72d5fbd12..93dd7bb0ece0 100644
--- a/drivers/media/usb/au0828/au0828-core.c
+++ b/drivers/media/usb/au0828/au0828-core.c
@@ -410,7 +410,7 @@ static int au0828_enable_source(struct media_entity *entity,
goto end;
}
- ret = __media_pipeline_start(entity, pipe);
+ ret = __media_pipeline_start(entity->pads, pipe);
if (ret) {
pr_err("Start Pipeline: %s->%s Error %d\n",
source->name, entity->name, ret);
@@ -501,12 +501,12 @@ static void au0828_disable_source(struct media_entity *entity)
return;
/* stop pipeline */
- __media_pipeline_stop(dev->active_link_owner);
+ __media_pipeline_stop(dev->active_link_owner->pads);
pr_debug("Pipeline stop for %s\n",
dev->active_link_owner->name);
ret = __media_pipeline_start(
- dev->active_link_user,
+ dev->active_link_user->pads,
dev->active_link_user_pipe);
if (ret) {
pr_err("Start Pipeline: %s->%s %d\n",
@@ -532,7 +532,7 @@ static void au0828_disable_source(struct media_entity *entity)
return;
/* stop pipeline */
- __media_pipeline_stop(dev->active_link_owner);
+ __media_pipeline_stop(dev->active_link_owner->pads);
pr_debug("Pipeline stop for %s\n",
dev->active_link_owner->name);
diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
index 04b7c6bdcd85..b8cffb103f12 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -905,16 +905,16 @@ int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
mutex_lock(&imxmd->md.graph_mutex);
if (on) {
- ret = __media_pipeline_start(entity, &imxmd->pipe);
+ ret = __media_pipeline_start(entity->pads, &imxmd->pipe);
if (ret)
goto out;
ret = v4l2_subdev_call(sd, video, s_stream, 1);
if (ret)
- __media_pipeline_stop(entity);
+ __media_pipeline_stop(entity->pads);
} else {
v4l2_subdev_call(sd, video, s_stream, 0);
if (entity->pads->pipe)
- __media_pipeline_stop(entity);
+ __media_pipeline_stop(entity->pads);
}
out:
diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c b/drivers/staging/media/ipu3/ipu3-v4l2.c
index 60aa02eb7d2a..7a3e2737401d 100644
--- a/drivers/staging/media/ipu3/ipu3-v4l2.c
+++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
@@ -485,7 +485,7 @@ static int imgu_vb2_start_streaming(struct vb2_queue *vq, unsigned int count)
pipe = node->pipe;
imgu_pipe = &imgu->imgu_pipe[pipe];
- r = media_pipeline_start(&node->vdev.entity, &imgu_pipe->pipeline);
+ r = media_pipeline_start(node->vdev.entity.pads, &imgu_pipe->pipeline);
if (r < 0)
goto fail_return_bufs;
@@ -510,7 +510,7 @@ static int imgu_vb2_start_streaming(struct vb2_queue *vq, unsigned int count)
return 0;
fail_stop_pipeline:
- media_pipeline_stop(&node->vdev.entity);
+ media_pipeline_stop(node->vdev.entity.pads);
fail_return_bufs:
imgu_return_all_buffers(imgu, node, VB2_BUF_STATE_QUEUED);
@@ -550,7 +550,7 @@ static void imgu_vb2_stop_streaming(struct vb2_queue *vq)
imgu_return_all_buffers(imgu, node, VB2_BUF_STATE_ERROR);
mutex_unlock(&imgu->streaming_lock);
- media_pipeline_stop(&node->vdev.entity);
+ media_pipeline_stop(node->vdev.entity.pads);
}
/******************** v4l2_ioctl_ops ********************/
diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
index d781ed10253e..4b07d6682f6e 100644
--- a/drivers/staging/media/omap4iss/iss_video.c
+++ b/drivers/staging/media/omap4iss/iss_video.c
@@ -888,7 +888,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
if (video->iss->pdata->set_constraints)
video->iss->pdata->set_constraints(video->iss, true);
- ret = media_pipeline_start(pad->entity, &pipe->pipe);
+ ret = media_pipeline_start(pad, &pipe->pipe);
if (ret < 0)
goto err_media_pipeline_start;
@@ -977,7 +977,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
err_omap4iss_set_stream:
vb2_streamoff(&vfh->queue, type);
err_iss_video_check_format:
- media_pipeline_stop(&video->video.entity);
+ media_pipeline_stop(video->video.entity.pads);
err_media_pipeline_start:
if (video->iss->pdata->set_constraints)
video->iss->pdata->set_constraints(video->iss, false);
@@ -1031,7 +1031,7 @@ iss_video_streamoff(struct file *file, void *fh, enum v4l2_buf_type type)
if (video->iss->pdata->set_constraints)
video->iss->pdata->set_constraints(video->iss, false);
- media_pipeline_stop(&video->video.entity);
+ media_pipeline_stop(video->video.entity.pads);
done:
mutex_unlock(&video->stream_lock);
diff --git a/drivers/staging/media/tegra-video/tegra210.c b/drivers/staging/media/tegra-video/tegra210.c
index f10a041e3e6c..d2d7dd0e8624 100644
--- a/drivers/staging/media/tegra-video/tegra210.c
+++ b/drivers/staging/media/tegra-video/tegra210.c
@@ -547,7 +547,7 @@ static int tegra210_vi_start_streaming(struct vb2_queue *vq, u32 count)
VI_INCR_SYNCPT_NO_STALL);
/* start the pipeline */
- ret = media_pipeline_start(&chan->video.entity, pipe);
+ ret = media_pipeline_start(chan->video.entity.pads, pipe);
if (ret < 0)
goto error_pipeline_start;
@@ -595,7 +595,7 @@ static int tegra210_vi_start_streaming(struct vb2_queue *vq, u32 count)
error_kthread_start:
tegra_channel_set_stream(chan, false);
error_set_stream:
- media_pipeline_stop(&chan->video.entity);
+ media_pipeline_stop(chan->video.entity.pads);
error_pipeline_start:
tegra_channel_release_buffers(chan, VB2_BUF_STATE_QUEUED);
return ret;
@@ -617,7 +617,7 @@ static void tegra210_vi_stop_streaming(struct vb2_queue *vq)
tegra_channel_release_buffers(chan, VB2_BUF_STATE_ERROR);
tegra_channel_set_stream(chan, false);
- media_pipeline_stop(&chan->video.entity);
+ media_pipeline_stop(chan->video.entity.pads);
}
/*
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 3026a24cc591..17e460a50716 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -932,53 +932,53 @@ struct media_pad *media_graph_walk_next(struct media_graph *graph);
/**
* media_pipeline_start - Mark a pipeline as streaming
- * @entity: Starting entity
- * @pipe: Media pipeline to be assigned to all entities in the pipeline.
+ * @pad: Starting pad
+ * @pipe: Media pipeline to be assigned to all pads in the pipeline.
*
- * Mark all entities connected to a given entity through enabled links, either
- * directly or indirectly, as streaming. The given pipeline object is assigned
- * to every entity in the pipeline and stored in the media_entity pipe field.
+ * Mark all pads connected to a given pad through enabled routes or links,
+ * either directly or indirectly, as streaming. The given pipeline object is
+ * assigned to every pad in the pipeline and stored in the media_pad pipe
+ * field.
*
* Calls to this function can be nested, in which case the same number of
* media_pipeline_stop() calls will be required to stop streaming. The
* pipeline pointer must be identical for all nested calls to
* media_pipeline_start().
*/
-__must_check int media_pipeline_start(struct media_entity *entity,
+__must_check int media_pipeline_start(struct media_pad *pad,
struct media_pipeline *pipe);
/**
* __media_pipeline_start - Mark a pipeline as streaming
*
- * @entity: Starting entity
- * @pipe: Media pipeline to be assigned to all entities in the pipeline.
+ * @pad: Starting pad
+ * @pipe: Media pipeline to be assigned to all pads in the pipeline.
*
* ..note:: This is the non-locking version of media_pipeline_start()
*/
-__must_check int __media_pipeline_start(struct media_entity *entity,
+__must_check int __media_pipeline_start(struct media_pad *pad,
struct media_pipeline *pipe);
/**
* media_pipeline_stop - Mark a pipeline as not streaming
- * @entity: Starting entity
+ * @pad: Starting pad
*
- * Mark all entities connected to a given entity through enabled links, either
- * directly or indirectly, as not streaming. The media_entity pipe field is
- * reset to %NULL.
+ * Mark all pads connected to a given pad through enabled routes or links,
+ * either directly or indirectly, as not streaming.
*
* If multiple calls to media_pipeline_start() have been made, the same
* number of calls to this function are required to mark the pipeline as not
- * streaming.
+ * streaming and reset the media_pad pipe field to %NULL.
*/
-void media_pipeline_stop(struct media_entity *entity);
+void media_pipeline_stop(struct media_pad *pad);
/**
* __media_pipeline_stop - Mark a pipeline as not streaming
*
- * @entity: Starting entity
+ * @pad: Starting pad
*
* .. note:: This is the non-locking version of media_pipeline_stop()
*/
-void __media_pipeline_stop(struct media_entity *entity);
+void __media_pipeline_stop(struct media_pad *pad);
/**
* media_devnode_create() - creates and initializes a device node interface
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 08/24] media: entity: Add has_route entity operation
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (6 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 07/24] media: entity: Use pad as the starting point for a pipeline Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 09/24] media: entity: Add media_entity_has_route() function Tomi Valkeinen
` (16 subsequent siblings)
24 siblings, 0 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen, Michal Simek
From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The optional operation can be used by entities to report whether two
pads are internally connected.
While at there, fix a Sphinx compiler warning in a comment block a few
lines above.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
include/media/media-entity.h | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 17e460a50716..f262026bae1f 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -187,6 +187,7 @@ enum media_pad_signal_type {
* @flags: Pad flags, as defined in
* :ref:`include/uapi/linux/media.h <media_header>`
* (seek for ``MEDIA_PAD_FL_*``)
+ *
* .. note::
*
* @stream_count reference count must never be negative, but is a signed
@@ -214,6 +215,10 @@ struct media_pad {
* @link_validate: Return whether a link is valid from the entity point of
* view. The media_pipeline_start() function
* validates all links by calling this operation. Optional.
+ * @has_route: Return whether a route exists inside the entity between
+ * two given pads. Pads are passed to the operation ordered
+ * by index. Optional: If the operation isn't implemented
+ * all pads will be considered as connected.
*
* .. note::
*
@@ -227,6 +232,8 @@ struct media_entity_operations {
const struct media_pad *local,
const struct media_pad *remote, u32 flags);
int (*link_validate)(struct media_link *link);
+ bool (*has_route)(struct media_entity *entity, unsigned int pad0,
+ unsigned int pad1);
};
/**
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 09/24] media: entity: Add media_entity_has_route() function
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (7 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 08/24] media: entity: Add has_route entity operation Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 10/24] media: entity: Use routing information during graph traversal Tomi Valkeinen
` (15 subsequent siblings)
24 siblings, 0 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen, Michal Simek
From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
This is a wrapper around the media entity has_route operation.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/mc/mc-entity.c | 19 +++++++++++++++++++
include/media/media-entity.h | 17 +++++++++++++++++
2 files changed, 36 insertions(+)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index c8c8cd75b6bf..a33ca02ac42b 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -229,6 +229,25 @@ EXPORT_SYMBOL_GPL(media_entity_pads_init);
* Graph traversal
*/
+bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
+ unsigned int pad1)
+{
+ if (pad0 >= entity->num_pads || pad1 >= entity->num_pads)
+ return false;
+
+ if (pad0 == pad1)
+ return true;
+
+ if (!entity->ops || !entity->ops->has_route)
+ return true;
+
+ if (entity->pads[pad1].index < entity->pads[pad0].index)
+ swap(pad0, pad1);
+
+ return entity->ops->has_route(entity, pad0, pad1);
+}
+EXPORT_SYMBOL_GPL(media_entity_has_route);
+
static struct media_pad *
media_pad_other(struct media_pad *pad, struct media_link *link)
{
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index f262026bae1f..f8fa952fa38e 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -899,6 +899,23 @@ int media_entity_get_fwnode_pad(struct media_entity *entity,
__must_check int media_graph_walk_init(
struct media_graph *graph, struct media_device *mdev);
+/**
+ * media_entity_has_route - Check if two entity pads are connected internally
+ *
+ * @entity: The entity
+ * @pad0: The first pad index
+ * @pad1: The second pad index
+ *
+ * This function can be used to check whether two pads of an entity are
+ * connected internally in the entity.
+ *
+ * The caller must hold entity->graph_obj.mdev->mutex.
+ *
+ * Return: true if the pads are connected internally and false otherwise.
+ */
+bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
+ unsigned int pad1);
+
/**
* media_graph_walk_cleanup - Release resources used by graph walk.
*
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 10/24] media: entity: Use routing information during graph traversal
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (8 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 09/24] media: entity: Add media_entity_has_route() function Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 11/24] media: entity: Skip link validation for pads to which there is no route to Tomi Valkeinen
` (14 subsequent siblings)
24 siblings, 0 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen, Michal Simek
From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Take internal routing information as reported by the entity has_route
operation into account during graph traversal to avoid following
unrelated links.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/mc/mc-entity.c | 44 ++++++++++++++++++++++--------------
1 file changed, 27 insertions(+), 17 deletions(-)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index a33ca02ac42b..d00c7397e09a 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -248,15 +248,6 @@ bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
}
EXPORT_SYMBOL_GPL(media_entity_has_route);
-static struct media_pad *
-media_pad_other(struct media_pad *pad, struct media_link *link)
-{
- if (link->source == pad)
- return link->sink;
- else
- return link->source;
-}
-
/* push an entity to traversal stack */
static void stack_push(struct media_graph *graph, struct media_pad *pad)
{
@@ -327,7 +318,8 @@ static void media_graph_walk_iter(struct media_graph *graph)
{
struct media_pad *pad = stack_top(graph);
struct media_link *link;
- struct media_pad *next;
+ struct media_pad *remote;
+ struct media_pad *local;
link = list_entry(link_top(graph), typeof(*link), list);
@@ -341,23 +333,41 @@ static void media_graph_walk_iter(struct media_graph *graph)
return;
}
- /* Get the entity in the other end of the link . */
- next = media_pad_other(pad, link);
+ /*
+ * Get the local pad, the remote pad and the entity at the other
+ * end of the link.
+ */
+ if (link->source->entity == pad->entity) {
+ remote = link->sink;
+ local = link->source;
+ } else {
+ remote = link->source;
+ local = link->sink;
+ }
+
+ /*
+ * Are the local pad and the pad we came from connected
+ * internally in the entity ?
+ */
+ if (!media_entity_has_route(pad->entity, pad->index, local->index)) {
+ link_top(graph) = link_top(graph)->next;
+ return;
+ }
/* Has the entity already been visited? */
- if (media_entity_enum_test_and_set(&graph->ent_enum, next->entity)) {
+ if (media_entity_enum_test_and_set(&graph->ent_enum, remote->entity)) {
link_top(graph) = link_top(graph)->next;
dev_dbg(pad->graph_obj.mdev->dev,
"walk: skipping entity '%s' (already seen)\n",
- next->entity->name);
+ remote->entity->name);
return;
}
/* Push the new entity to stack and start over. */
link_top(graph) = link_top(graph)->next;
- stack_push(graph, next);
- dev_dbg(next->graph_obj.mdev->dev, "walk: pushing '%s':%u on stack\n",
- next->entity->name, next->index);
+ stack_push(graph, remote);
+ dev_dbg(remote->graph_obj.mdev->dev, "walk: pushing '%s':%u on stack\n",
+ remote->entity->name, remote->index);
}
struct media_pad *media_graph_walk_next(struct media_graph *graph)
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 11/24] media: entity: Skip link validation for pads to which there is no route to
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (9 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 10/24] media: entity: Use routing information during graph traversal Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-29 2:30 ` Laurent Pinchart
2021-04-27 12:45 ` [PATCH v6 12/24] media: entity: Add an iterator helper for connected pads Tomi Valkeinen
` (13 subsequent siblings)
24 siblings, 1 reply; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
Links are validated along the pipeline which is about to start streaming.
Not all the pads in entities that are traversed along that pipeline are
part of the pipeline, however. Skip the link validation for such pads,
and while at there rename "other_pad" to "local_pad" to convey the fact
the route to be checked is internal to the entity.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/mc/mc-entity.c | 15 ++++++++++-----
1 file changed, 10 insertions(+), 5 deletions(-)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index d00c7397e09a..a4cca53124b2 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -482,27 +482,32 @@ __must_check int __media_pipeline_start(struct media_pad *pad,
bitmap_fill(has_no_links, entity->num_pads);
list_for_each_entry(link, &entity->links, list) {
- struct media_pad *other_pad =
+ struct media_pad *local_pad =
link->sink->entity == entity ?
link->sink : link->source;
+ /* Ignore pads to which there is no route. */
+ if (!media_entity_has_route(entity, pad->index,
+ local_pad->index))
+ continue;
+
/* Mark that a pad is connected by a link. */
- bitmap_clear(has_no_links, other_pad->index, 1);
+ bitmap_clear(has_no_links, local_pad->index, 1);
/*
* Pads that either do not need to connect or
* are connected through an enabled link are
* fine.
*/
- if (!(other_pad->flags & MEDIA_PAD_FL_MUST_CONNECT) ||
+ if (!(local_pad->flags & MEDIA_PAD_FL_MUST_CONNECT) ||
link->flags & MEDIA_LNK_FL_ENABLED)
- bitmap_set(active, other_pad->index, 1);
+ bitmap_set(active, local_pad->index, 1);
/*
* Link validation will only take place for
* sink ends of the link that are enabled.
*/
- if (link->sink != other_pad ||
+ if (link->sink != local_pad ||
!(link->flags & MEDIA_LNK_FL_ENABLED))
continue;
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 12/24] media: entity: Add an iterator helper for connected pads
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (10 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 11/24] media: entity: Skip link validation for pads to which there is no route to Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-29 2:18 ` Laurent Pinchart
2021-04-27 12:45 ` [PATCH v6 13/24] media: entity: Add only connected pads to the pipeline Tomi Valkeinen
` (12 subsequent siblings)
24 siblings, 1 reply; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen
Add a helper macro for iterating over pads that are connected through
enabled routes. This can be used to find all the connected pads within an
entity, for instance starting from the pad which has been obtained during
the graph walk.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
- Make __media_entity_next_routed_pad() return NULL and adjust the
iterator to handle that
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
include/media/media-entity.h | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index f8fa952fa38e..42193d6c58e9 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -916,6 +916,34 @@ __must_check int media_graph_walk_init(
bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
unsigned int pad1);
+static inline struct media_pad *__media_entity_next_routed_pad(
+ struct media_pad *start, struct media_pad *iter)
+{
+ struct media_entity *entity = start->entity;
+
+ for (; iter < &entity->pads[entity->num_pads]; iter++) {
+ if (media_entity_has_route(entity, start->index, iter->index))
+ return iter;
+ }
+
+ return NULL;
+}
+
+/**
+ * media_entity_for_each_routed_pad - Iterate over entity pads connected by routes
+ *
+ * @start: The starting pad
+ * @iter: The iterator pad
+ *
+ * Iterate over all pads connected through routes from the @start pad
+ * within an entity. The iteration will include the @start pad itself.
+ */
+#define media_entity_for_each_routed_pad(start, iter) \
+ for (iter = __media_entity_next_routed_pad( \
+ start, (start)->entity->pads); \
+ iter != NULL; \
+ iter = __media_entity_next_routed_pad(start, iter + 1))
+
/**
* media_graph_walk_cleanup - Release resources used by graph walk.
*
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 13/24] media: entity: Add only connected pads to the pipeline
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (11 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 12/24] media: entity: Add an iterator helper for connected pads Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 14/24] media: entity: Add debug information in graph walk route check Tomi Valkeinen
` (11 subsequent siblings)
24 siblings, 0 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
A single entity may contain multiple pipelines. Only add pads that were
connected to the pad through which the entity was reached to the pipeline.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/mc/mc-entity.c | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index a4cca53124b2..2b70ef97c56c 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -457,7 +457,7 @@ __must_check int __media_pipeline_start(struct media_pad *pad,
DECLARE_BITMAP(active, MEDIA_ENTITY_MAX_PADS);
DECLARE_BITMAP(has_no_links, MEDIA_ENTITY_MAX_PADS);
- media_entity_for_each_pad(entity, iter) {
+ media_entity_for_each_routed_pad(pad, iter) {
if (iter->pipe && iter->pipe != pipe) {
pr_err("Pipe active for %s. Can't start for %s\n",
entity->name, iter->entity->name);
@@ -546,10 +546,9 @@ __must_check int __media_pipeline_start(struct media_pad *pad,
media_graph_walk_start(graph, pad_err);
while ((pad_err = media_graph_walk_next(graph))) {
- struct media_entity *entity = pad_err->entity;
struct media_pad *iter;
- media_entity_for_each_pad(entity, iter) {
+ media_entity_for_each_routed_pad(pad_err, iter) {
/* Sanity check for negative stream_count */
if (!WARN_ON_ONCE(iter->stream_count <= 0)) {
--iter->stream_count;
@@ -602,10 +601,9 @@ void __media_pipeline_stop(struct media_pad *pad)
media_graph_walk_start(graph, pad);
while ((pad = media_graph_walk_next(graph))) {
- struct media_entity *entity = pad->entity;
struct media_pad *iter;
- media_entity_for_each_pad(entity, iter) {
+ media_entity_for_each_routed_pad(pad, iter) {
/* Sanity check for negative stream_count */
if (!WARN_ON_ONCE(iter->stream_count <= 0)) {
iter->stream_count--;
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 14/24] media: entity: Add debug information in graph walk route check
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (12 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 13/24] media: entity: Add only connected pads to the pipeline Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 15/24] v4l: Add bus type to frame descriptors Tomi Valkeinen
` (10 subsequent siblings)
24 siblings, 0 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
Add debug printout in graph walk route check.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/mc/mc-entity.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 2b70ef97c56c..d62436cdd64f 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -351,6 +351,9 @@ static void media_graph_walk_iter(struct media_graph *graph)
*/
if (!media_entity_has_route(pad->entity, pad->index, local->index)) {
link_top(graph) = link_top(graph)->next;
+ dev_dbg(pad->graph_obj.mdev->dev,
+ "walk: skipping \"%s\":%u -> %u (no route)\n",
+ pad->entity->name, pad->index, local->index);
return;
}
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 15/24] v4l: Add bus type to frame descriptors
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (13 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 14/24] media: entity: Add debug information in graph walk route check Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-29 2:46 ` Laurent Pinchart
2021-04-27 12:45 ` [PATCH v6 16/24] v4l: Add CSI-2 bus configuration " Tomi Valkeinen
` (9 subsequent siblings)
24 siblings, 1 reply; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen
Add the media bus type to the frame descriptor. CSI-2 specific
information will be added in next patch to the frame descriptor.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
- Make the bus type a named enum
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
include/media/v4l2-subdev.h | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index d0e9a5bdb08b..ac531b752028 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -340,12 +340,31 @@ struct v4l2_mbus_frame_desc_entry {
#define V4L2_FRAME_DESC_ENTRY_MAX 4
+/**
+ * enum v4l2_mbus_frame_desc_type - media bus frame description type
+ *
+ * @V4L2_MBUS_FRAME_DESC_TYPE_PLATFORM:
+ * Platform specific frame desc type for backwards compatibility.
+ * @V4L2_MBUS_FRAME_DESC_TYPE_PARALLEL:
+ * Parallel media bus.
+ * @V4L2_MBUS_FRAME_DESC_TYPE_CSI2:
+ * CSI-2 media bus. Frame desc parameters must be set in
+ * &struct v4l2_mbus_frame_desc_entry->csi2.
+ */
+enum v4l2_mbus_frame_desc_type {
+ V4L2_MBUS_FRAME_DESC_TYPE_PLATFORM,
+ V4L2_MBUS_FRAME_DESC_TYPE_PARALLEL,
+ V4L2_MBUS_FRAME_DESC_TYPE_CSI2,
+};
+
/**
* struct v4l2_mbus_frame_desc - media bus data frame description
+ * @type: type of the bus (enum v4l2_mbus_frame_desc_type)
* @entry: frame descriptors array
* @num_entries: number of entries in @entry array
*/
struct v4l2_mbus_frame_desc {
+ enum v4l2_mbus_frame_desc_type type;
struct v4l2_mbus_frame_desc_entry entry[V4L2_FRAME_DESC_ENTRY_MAX];
unsigned short num_entries;
};
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 16/24] v4l: Add CSI-2 bus configuration to frame descriptors
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (14 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 15/24] v4l: Add bus type to frame descriptors Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 17/24] v4l: Add stream to frame descriptor Tomi Valkeinen
` (8 subsequent siblings)
24 siblings, 0 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen
Add CSI-2 bus specific configuration to the frame descriptors. This allows
obtaining the virtual channel and data type information for each stream
the transmitter is sending.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
include/media/v4l2-subdev.h | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index ac531b752028..695e97b6eb13 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -308,6 +308,17 @@ struct v4l2_subdev_audio_ops {
int (*s_stream)(struct v4l2_subdev *sd, int enable);
};
+/**
+ * struct v4l2_mbus_frame_desc_entry_csi2
+ *
+ * @vc: CSI-2 virtual channel
+ * @dt: CSI-2 data type ID
+ */
+struct v4l2_mbus_frame_desc_entry_csi2 {
+ u8 vc;
+ u8 dt;
+};
+
/**
* enum v4l2_mbus_frame_desc_flags - media bus frame description flags
*
@@ -331,11 +342,16 @@ enum v4l2_mbus_frame_desc_flags {
* %FRAME_DESC_FL_BLOB is not set.
* @length: number of octets per frame, valid if @flags
* %V4L2_MBUS_FRAME_DESC_FL_LEN_MAX is set.
+ * @bus: Bus-specific frame descriptor parameters
+ * @bus.csi2: CSI-2-specific bus configuration
*/
struct v4l2_mbus_frame_desc_entry {
enum v4l2_mbus_frame_desc_flags flags;
u32 pixelcode;
u32 length;
+ union {
+ struct v4l2_mbus_frame_desc_entry_csi2 csi2;
+ } bus;
};
#define V4L2_FRAME_DESC_ENTRY_MAX 4
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 17/24] v4l: Add stream to frame descriptor
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (15 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 16/24] v4l: Add CSI-2 bus configuration " Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-29 2:50 ` Laurent Pinchart
2021-04-27 12:45 ` [PATCH v6 18/24] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations Tomi Valkeinen
` (7 subsequent siblings)
24 siblings, 1 reply; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
The stream field identifies the stream this frame descriptor applies to in
routing configuration across a multiplexed link.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
include/media/v4l2-subdev.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 695e97b6eb13..70a21298020b 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -338,6 +338,7 @@ enum v4l2_mbus_frame_desc_flags {
* struct v4l2_mbus_frame_desc_entry - media bus frame description structure
*
* @flags: bitmask flags, as defined by &enum v4l2_mbus_frame_desc_flags.
+ * @stream: stream in routing configuration
* @pixelcode: media bus pixel code, valid if @flags
* %FRAME_DESC_FL_BLOB is not set.
* @length: number of octets per frame, valid if @flags
@@ -347,6 +348,7 @@ enum v4l2_mbus_frame_desc_flags {
*/
struct v4l2_mbus_frame_desc_entry {
enum v4l2_mbus_frame_desc_flags flags;
+ u32 stream;
u32 pixelcode;
u32 length;
union {
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 18/24] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (16 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 17/24] v4l: Add stream to frame descriptor Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-29 2:28 ` Laurent Pinchart
2021-04-27 12:45 ` [PATCH v6 19/24] media: Documentation: Add GS_ROUTING documentation Tomi Valkeinen
` (6 subsequent siblings)
24 siblings, 1 reply; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen, Michal Simek
Add support for subdev internal routing. A route is defined as a single
stream from a sink pad to a source pad.
The userspace can configure the routing via two new ioctls,
VIDIOC_SUBDEV_G_ROUTING and VIDIOC_SUBDEV_S_ROUTING, and subdevs can
implement the functionality with v4l2_subdev_pad_ops.get_routing() and
v4l2_subdev_pad_ops.set_routing().
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
- Add sink and source streams for multiplexed links
- Copy the argument back in case of an error. This is needed to let the
caller know the number of routes.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
- Expand and refine documentation.
- Make the 'routes' pointer a __u64 __user pointer so that a compat32
version of the ioctl is not required.
- Add struct v4l2_subdev_krouting to be used for subdevice operations.
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
- Fix typecasing warnings
- Check sink & source pad types
- Add 'which' field
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/v4l2-core/v4l2-ioctl.c | 25 +++++++++++++-
drivers/media/v4l2-core/v4l2-subdev.c | 46 +++++++++++++++++++++++++
include/media/v4l2-subdev.h | 24 ++++++++++++++
include/uapi/linux/v4l2-subdev.h | 48 +++++++++++++++++++++++++++
4 files changed, 142 insertions(+), 1 deletion(-)
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 6a5d1c6d11d6..7b1dabc23909 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -16,6 +16,7 @@
#include <linux/kernel.h>
#include <linux/version.h>
+#include <linux/v4l2-subdev.h>
#include <linux/videodev2.h>
#include <media/v4l2-common.h>
@@ -3108,6 +3109,21 @@ static int check_array_args(unsigned int cmd, void *parg, size_t *array_size,
ret = 1;
break;
}
+
+ case VIDIOC_SUBDEV_G_ROUTING:
+ case VIDIOC_SUBDEV_S_ROUTING: {
+ struct v4l2_subdev_routing *routing = parg;
+
+ if (routing->num_routes > 256)
+ return -EINVAL;
+
+ *user_ptr = u64_to_user_ptr(routing->routes);
+ *kernel_ptr = (void **)&routing->routes;
+ *array_size = sizeof(struct v4l2_subdev_route)
+ * routing->num_routes;
+ ret = 1;
+ break;
+ }
}
return ret;
@@ -3369,8 +3385,15 @@ video_usercopy(struct file *file, unsigned int orig_cmd, unsigned long arg,
/*
* Some ioctls can return an error, but still have valid
* results that must be returned.
+ *
+ * FIXME: subdev IOCTLS are partially handled here and partially in
+ * v4l2-subdev.c and the 'always_copy' flag can only be set for IOCTLS
+ * defined here as part of the 'v4l2_ioctls' array. As
+ * VIDIOC_SUBDEV_G_ROUTING needs to return results to applications even
+ * in case of failure, but it is not defined here as part of the
+ * 'v4l2_ioctls' array, insert an ad-hoc check to address that.
*/
- if (err < 0 && !always_copy)
+ if (err < 0 && !always_copy && cmd != VIDIOC_SUBDEV_G_ROUTING)
goto out;
out_array_args:
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 956dafab43d4..ad79ce121cee 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -681,6 +681,52 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg)
case VIDIOC_SUBDEV_QUERYSTD:
return v4l2_subdev_call(sd, video, querystd, arg);
+ case VIDIOC_SUBDEV_G_ROUTING: {
+ struct v4l2_subdev_routing *routing = arg;
+ struct v4l2_subdev_krouting krouting = {
+ .which = routing->which,
+ .num_routes = routing->num_routes,
+ .routes = (struct v4l2_subdev_route *)(uintptr_t)
+ routing->routes,
+ };
+ int ret;
+
+ ret = v4l2_subdev_call(sd, pad, get_routing, &krouting);
+
+ routing->num_routes = krouting.num_routes;
+
+ return ret;
+ }
+
+ case VIDIOC_SUBDEV_S_ROUTING: {
+ struct v4l2_subdev_routing *routing = arg;
+ struct v4l2_subdev_route *routes =
+ (struct v4l2_subdev_route *)(uintptr_t)routing->routes;
+ struct v4l2_subdev_krouting krouting = {};
+ unsigned int i;
+
+ if (routing->which != V4L2_SUBDEV_FORMAT_TRY && ro_subdev)
+ return -EPERM;
+
+ for (i = 0; i < routing->num_routes; ++i) {
+ if (routes[i].sink_pad >= sd->entity.num_pads ||
+ routes[i].source_pad >= sd->entity.num_pads)
+ return -EINVAL;
+
+ if (!(sd->entity.pads[routes[i].sink_pad].flags &
+ MEDIA_PAD_FL_SINK) ||
+ !(sd->entity.pads[routes[i].source_pad].flags &
+ MEDIA_PAD_FL_SOURCE))
+ return -EINVAL;
+ }
+
+ krouting.which = routing->which;
+ krouting.num_routes = routing->num_routes;
+ krouting.routes = routes;
+
+ return v4l2_subdev_call(sd, pad, set_routing, &krouting);
+ }
+
default:
return v4l2_subdev_call(sd, core, ioctl, cmd, arg);
}
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 70a21298020b..c45f5f0156c9 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -660,6 +660,22 @@ struct v4l2_subdev_pad_config {
struct v4l2_rect try_compose;
};
+/**
+ * struct v4l2_subdev_krouting - subdev routing table
+ *
+ * @which: format type (from enum v4l2_subdev_format_whence)
+ * @routes: &struct v4l2_subdev_route
+ * @num_routes: number of routes
+ *
+ * This structure is used to translate arguments received from
+ * VIDIOC_SUBDEV_G/S_ROUTING() ioctl to subdev device drivers operations.
+ */
+struct v4l2_subdev_krouting {
+ u32 which;
+ struct v4l2_subdev_route *routes;
+ unsigned int num_routes;
+};
+
/**
* struct v4l2_subdev_pad_ops - v4l2-subdev pad level operations
*
@@ -721,6 +737,10 @@ struct v4l2_subdev_pad_config {
* applied to the hardware. The operation shall fail if the
* pad index it has been called on is not valid or in case of
* unrecoverable failures.
+ *
+ * @get_routing: get the subdevice routing table.
+ * @set_routing: enable or disable data connection routes described in the
+ * subdevice routing table.
*/
struct v4l2_subdev_pad_ops {
int (*init_cfg)(struct v4l2_subdev *sd,
@@ -765,6 +785,10 @@ struct v4l2_subdev_pad_ops {
struct v4l2_mbus_config *config);
int (*set_mbus_config)(struct v4l2_subdev *sd, unsigned int pad,
struct v4l2_mbus_config *config);
+ int (*get_routing)(struct v4l2_subdev *sd,
+ struct v4l2_subdev_krouting *route);
+ int (*set_routing)(struct v4l2_subdev *sd,
+ struct v4l2_subdev_krouting *route);
};
/**
diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h
index 658106f5b5dc..3a360cdcacf4 100644
--- a/include/uapi/linux/v4l2-subdev.h
+++ b/include/uapi/linux/v4l2-subdev.h
@@ -188,6 +188,52 @@ struct v4l2_subdev_capability {
/* The v4l2 sub-device video device node is registered in read-only mode. */
#define V4L2_SUBDEV_CAP_RO_SUBDEV 0x00000001
+/*
+ * Is the route active? An active route will start when streaming is enabled
+ * on a video node.
+ */
+#define V4L2_SUBDEV_ROUTE_FL_ACTIVE BIT(0)
+
+/*
+ * Is the route immutable, i.e. can it be activated and inactivated?
+ * Set by the driver.
+ */
+#define V4L2_SUBDEV_ROUTE_FL_IMMUTABLE BIT(1)
+
+/**
+ * struct v4l2_subdev_route - A route inside a subdev
+ *
+ * @sink_pad: the sink pad index
+ * @sink_stream: the sink stream identifier
+ * @source_pad: the source pad index
+ * @source_stream: the source stream identifier
+ * @flags: route flags V4L2_SUBDEV_ROUTE_FL_*
+ * @reserved: drivers and applications must zero this array
+ */
+struct v4l2_subdev_route {
+ __u32 sink_pad;
+ __u32 sink_stream;
+ __u32 source_pad;
+ __u32 source_stream;
+ __u32 flags;
+ __u32 reserved[5];
+};
+
+/**
+ * struct v4l2_subdev_routing - Subdev routing information
+ *
+ * @which: configuration type (from enum v4l2_subdev_format_whence)
+ * @routes: pointer to the routes array
+ * @num_routes: the total number of routes in the routes array
+ * @reserved: drivers and applications must zero this array
+ */
+struct v4l2_subdev_routing {
+ __u32 which;
+ __u64 routes;
+ __u32 num_routes;
+ __u32 reserved[5];
+};
+
/* Backwards compatibility define --- to be removed */
#define v4l2_subdev_edid v4l2_edid
@@ -203,6 +249,8 @@ struct v4l2_subdev_capability {
#define VIDIOC_SUBDEV_S_CROP _IOWR('V', 60, struct v4l2_subdev_crop)
#define VIDIOC_SUBDEV_G_SELECTION _IOWR('V', 61, struct v4l2_subdev_selection)
#define VIDIOC_SUBDEV_S_SELECTION _IOWR('V', 62, struct v4l2_subdev_selection)
+#define VIDIOC_SUBDEV_G_ROUTING _IOWR('V', 38, struct v4l2_subdev_routing)
+#define VIDIOC_SUBDEV_S_ROUTING _IOWR('V', 39, struct v4l2_subdev_routing)
/* The following ioctls are identical to the ioctls in videodev2.h */
#define VIDIOC_SUBDEV_G_STD _IOR('V', 23, v4l2_std_id)
#define VIDIOC_SUBDEV_S_STD _IOW('V', 24, v4l2_std_id)
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 19/24] media: Documentation: Add GS_ROUTING documentation
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (17 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 18/24] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 20/24] v4l: subdev: routing kernel helper functions Tomi Valkeinen
` (5 subsequent siblings)
24 siblings, 0 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen
From: Jacopo Mondi <jacopo+renesas@jmondi.org>
Add documentation for VIDIOC_SUBDEV_G/S_ROUTING ioctl and add
description of multiplexed media pads and internal routing to the
V4L2-subdev documentation section.
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
.../userspace-api/media/v4l/dev-subdev.rst | 125 ++++++++++++++++
.../userspace-api/media/v4l/user-func.rst | 1 +
.../media/v4l/vidioc-subdev-g-routing.rst | 138 ++++++++++++++++++
3 files changed, 264 insertions(+)
create mode 100644 Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
index fd1de0a73a9f..ff922347f099 100644
--- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
+++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
@@ -29,6 +29,8 @@ will feature a character device node on which ioctls can be called to
- negotiate image formats on individual pads
+- inspect and modify internal data routing between pads of the same entity
+
Sub-device character device nodes, conventionally named
``/dev/v4l-subdev*``, use major number 81.
@@ -501,3 +503,126 @@ source pads.
:maxdepth: 1
subdev-formats
+
+
+Multiplexed media pads and internal routing
+-------------------------------------------
+
+Routing Table
+^^^^^^^^^^^^^
+
+Subdevice drivers may expose the internal connections between media pads of an
+entity by exposing a routing table that applications can inspect and manipulate.
+A routing table is described by a struct :c:type:`v4l2_subdev_routing`, which
+contains ``num_routes`` route entries, each one represented by a struct
+:c:type:`v4l2_subdev_route`.
+
+Data routes do not just connect one pad to another in an entity, but they refer
+instead to the ``streams`` a media pad provides. Streams are data connection
+endpoints in a media pad. Multiplexed media pads expose multiple ``streams``
+which represent, when the underlying hardware technology allows that, logical
+data flows transported over a single physical media bus.
+
+A noteworthy example of logical stream multiplexing techniques is represented
+by the data interleaving mechanism implemented by mean of Virtual Channels as
+defined by the MIPI CSI-2 media bus specifications. A subdevice that implements
+support for Virtual Channel data interleaving might expose up to 4 data
+``streams``, one for each available Virtual Channel, on the source media pad
+that represents a CSI-2 connection endpoint.
+
+A route is defined as a connection between a ``(sink_pad, sink_stream)`` pair
+and a ``(source_pad, source_stream)`` pair, where ``sink_pad`` and
+``source_pad`` are the indexes of two media pads part of the same media entity,
+and ``sink_stream`` and ``source_stream`` are the identifiers of the data
+streams to be connected in the media pads. The stream identifiers are arbitrary
+numbers, i.e. they have no relevance to the hardware, but they must be unique on
+a single pad, and the entity on the other side of the link must have a matching
+stream identifier.
+
+The current routes are reported to applications in a routing table which can be
+inspected using the :ref:`routing <VIDIOC_SUBDEV_G_ROUTING>` ioctl.
+
+Routes can be added or removed by adding or removing them to/from the routing
+table. Also, a route in the routing table can be activated and deactivated by
+setting or clearing the ``V4L2_SUBDEV_ROUTE_FL_ACTIVE`` flag in the ``flags``
+field of struct :c:type:`v4l2_subdev_route`.
+
+A subdev driver may create routes that cannot be modified by applications. Such
+routes are identified by the presence of the ``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE``
+flag in the ``flags`` field of struct :c:type:`v4l2_subdev_route`.
+
+As an example, a subdevice with two sink pads and one output pad has the pads
+defined as follows:
+
+.. flat-table::
+ :header-rows: 1
+
+ * - Pad Index
+ - Function
+ * - 0
+ - SINK
+ * - 1
+ - SINK
+ * - 2
+ - SOURCE
+
+A case where the subdevice would receive a single stream via each sink pad, and
+combine them to the source pad would result in a routing table as follows:
+
+.. flat-table:: routing table
+ :header-rows: 1
+
+ * - Sink Pad/Sink Stream
+ - Source Pad/Source Stream
+ * - 0/0
+ - 2/0
+ * - 1/0
+ - 2/1
+
+Whereas if the same subdev would receive two streams via each sink pad, and
+output the combined 4 streams would result in a routing table as follows:
+
+.. flat-table:: routing table
+ :header-rows: 1
+
+ * - Sink Pad/Sink Stream
+ - Source Pad/Source Stream
+ * - 0/0
+ - 2/0
+ * - 0/1
+ - 2/1
+ * - 1/0
+ - 2/2
+ * - 1/1
+ - 2/3
+
+Some subdevices may have immutable routes or a known set of routes which can be
+activated or deactivated. An example would be a sensor which produces pixel data
+and metadata via CSI-2 bus. The sensor would be modelled as two subdevices: a
+producer and a mux. The producer would have two source pads providing the pixel
+stream in the first pad and the metadata in the second pad. The mux subdevice
+would combine these two streams into a single source pad, and provide
+multiplexed streams. The mux subdevice would always have two routes.
+
+A subdevice that does not produce the data is another matter. Consider a device
+with two CSI-2 sink pads and two CSI-2 source pads, with the ability to route
+streams freely between the sink and source pads based on HW configuration. Each
+sink pad could receive streams for all four CSI-2 virtual channel. If we only
+consider the virtual channels, we would have maximum number of routes of 8.
+
+But CSI-2 also defines a datatype for each CSI-2 packet, allowing one to send,
+say, pixel data and metadata using the same virtual channel but different
+datatype. Now we would have a maximum number of routes of 16.
+
+Generally speaking, the concept of "stream" is very flexible. As a contrived
+example, you might even consider each line of a pixel frame to be a separate
+stream, if your hardware would support such a thing.
+
+Multiplexed Streams
+^^^^^^^^^^^^^^^^^^^
+
+When a subdevice exposes multiple streams in a single pad (multiplexed streams),
+the pad cannot be used to get or set the stream configuration with e.g.
+``VIDIOC_SUBDEV_S_FMT``. Such ioctls must return -ENOIOCTLCMD for pads with
+multiple streams. There must be a non-multiplexed source pad for each stream,
+and this pad must be used to configure the stream.
diff --git a/Documentation/userspace-api/media/v4l/user-func.rst b/Documentation/userspace-api/media/v4l/user-func.rst
index 53e604bd7d60..228c1521f190 100644
--- a/Documentation/userspace-api/media/v4l/user-func.rst
+++ b/Documentation/userspace-api/media/v4l/user-func.rst
@@ -70,6 +70,7 @@ Function Reference
vidioc-subdev-g-crop
vidioc-subdev-g-fmt
vidioc-subdev-g-frame-interval
+ vidioc-subdev-g-routing
vidioc-subdev-g-selection
vidioc-subdev-querycap
vidioc-subscribe-event
diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
new file mode 100644
index 000000000000..8828f7e49536
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
@@ -0,0 +1,138 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+.. c:namespace:: V4L
+
+.. _VIDIOC_SUBDEV_G_ROUTING:
+
+******************************************************
+ioctl VIDIOC_SUBDEV_G_ROUTING, VIDIOC_SUBDEV_S_ROUTING
+******************************************************
+
+Name
+====
+
+VIDIOC_SUBDEV_G_ROUTING - VIDIOC_SUBDEV_S_ROUTING - Get or set routing between streams of media pads in a media entity.
+
+
+Synopsis
+========
+
+.. c:function:: int ioctl( int fd, VIDIOC_SUBDEV_G_ROUTING, struct v4l2_subdev_routing *argp )
+ :name: VIDIOC_SUBDEV_G_ROUTING
+
+.. c:function:: int ioctl( int fd, VIDIOC_SUBDEV_S_ROUTING, struct v4l2_subdev_routing *argp )
+ :name: VIDIOC_SUBDEV_S_ROUTING
+
+
+Arguments
+=========
+
+``fd``
+ File descriptor returned by :ref:`open() <func-open>`.
+
+``argp``
+ Pointer to struct :c:type:`v4l2_subdev_routing`.
+
+
+Description
+===========
+
+These ioctls are used to get and set the routing in a media entity.
+The routing configuration determines the flows of data inside an entity.
+
+Drivers report their current routing tables using the
+``VIDIOC_SUBDEV_G_ROUTING`` ioctl and application may enable or disable routes
+with the VIDIOC_SUBDEV_S_ROUTING ioctl, by adding or removing routes and setting
+or clearing the ``V4L2_SUBDEV_ROUTE_FL_ACTIVE`` flag of the ``flags`` field of
+a struct :c:type:`v4l2_subdev_route`.
+
+When inspecting routes through VIDIOC_SUBDEV_G_ROUTING and the application
+provided ``num_routes`` is not big enough to contain all the available routes
+the subdevice exposes, drivers return the ENOSPC error code and adjust the
+value of the ``num_routes`` field. Application should then reserve enough memory
+for all the route entries and call VIDIOC_SUBDEV_G_ROUTING again.
+
+.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}|
+
+.. c:type:: v4l2_subdev_routing
+
+.. flat-table:: struct v4l2_subdev_routing
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - __u32
+ - ``which``
+ - Format to modified, from enum
+ :ref:`v4l2_subdev_format_whence <v4l2-subdev-format-whence>`.
+ * - struct :c:type:`v4l2_subdev_route`
+ - ``routes[]``
+ - Array of struct :c:type:`v4l2_subdev_route` entries
+ * - __u32
+ - ``num_routes``
+ - Number of entries of the routes array
+ * - __u32
+ - ``reserved``\ [5]
+ - Reserved for future extensions. Applications and drivers must set
+ the array to zero.
+
+.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}|
+
+.. c:type:: v4l2_subdev_route
+
+.. flat-table:: struct v4l2_subdev_route
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - __u32
+ - ``sink_pad``
+ - Sink pad number.
+ * - __u32
+ - ``sink_stream``
+ - Sink pad stream number.
+ * - __u32
+ - ``source_pad``
+ - Source pad number.
+ * - __u32
+ - ``source_stream``
+ - Source pad stream number.
+ * - __u32
+ - ``flags``
+ - Route enable/disable flags
+ :ref:`v4l2_subdev_routing_flags <v4l2-subdev-routing-flags>`.
+ * - __u32
+ - ``reserved``\ [5]
+ - Reserved for future extensions. Applications and drivers must set
+ the array to zero.
+
+.. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
+
+.. _v4l2-subdev-routing-flags:
+
+.. flat-table:: enum v4l2_subdev_routing_flags
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 3 1 4
+
+ * - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - 0
+ - The route is enabled. Set by applications.
+ * - V4L2_SUBDEV_ROUTE_FL_IMMUTABLE
+ - 1
+ - The route is immutable. Set by the driver.
+
+Return Value
+============
+
+On success 0 is returned, on error -1 and the ``errno`` variable is set
+appropriately. The generic error codes are described at the
+:ref:`Generic Error Codes <gen-errors>` chapter.
+
+ENOSPC
+ The number of provided route entries is less than the available ones.
+
+EINVAL
+ The sink or source pad identifiers reference a non-existing pad, or reference
+ pads of different types (ie. the sink_pad identifiers refers to a source pad)
+ or the sink or source stream identifiers reference a non-existing stream on
+ the sink or source pad.
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 20/24] v4l: subdev: routing kernel helper functions
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (18 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 19/24] media: Documentation: Add GS_ROUTING documentation Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-29 3:46 ` Laurent Pinchart
2021-04-27 12:45 ` [PATCH v6 21/24] v4l: subdev: add v4l2_subdev_get_format_dir() Tomi Valkeinen
` (4 subsequent siblings)
24 siblings, 1 reply; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen
Add helper functions for routing.
TODO: add docs.
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/v4l2-core/v4l2-subdev.c | 90 +++++++++++++++++++++++++++
include/media/v4l2-subdev.h | 14 +++++
2 files changed, 104 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index ad79ce121cee..31e279292ea6 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -910,6 +910,96 @@ v4l2_subdev_link_validate_get_format(struct media_pad *pad,
return -EINVAL;
}
+int v4l2_subdev_get_krouting(struct v4l2_subdev *sd,
+ struct v4l2_subdev_krouting *routing)
+{
+ int ret;
+
+ routing->which = V4L2_SUBDEV_FORMAT_ACTIVE;
+ routing->routes = NULL;
+ routing->num_routes = 0;
+
+ ret = v4l2_subdev_call(sd, pad, get_routing, routing);
+ if (ret == 0)
+ return 0;
+ if (ret != -ENOSPC)
+ return ret;
+
+ routing->routes = kvmalloc_array(routing->num_routes,
+ sizeof(*routing->routes), GFP_KERNEL);
+ if (!routing->routes)
+ return -ENOMEM;
+
+ ret = v4l2_subdev_call(sd, pad, get_routing, routing);
+ if (ret) {
+ kvfree(routing->routes);
+ return ret;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_get_krouting);
+
+void v4l2_subdev_free_routing(struct v4l2_subdev_krouting *routing)
+{
+ kvfree(routing->routes);
+ routing->routes = NULL;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_free_routing);
+
+void v4l2_subdev_cpy_routing(struct v4l2_subdev_krouting *dst,
+ const struct v4l2_subdev_krouting *src)
+{
+ memcpy(dst->routes, src->routes,
+ src->num_routes * sizeof(*src->routes));
+ dst->num_routes = src->num_routes;
+ dst->which = src->which;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_cpy_routing);
+
+int v4l2_subdev_dup_routing(struct v4l2_subdev_krouting *dst,
+ const struct v4l2_subdev_krouting *src)
+{
+ if (dst->routes)
+ kvfree(dst->routes);
+
+ if (src->num_routes == 0) {
+ dst->which = src->which;
+ dst->routes = NULL;
+ dst->num_routes = 0;
+ return 0;
+ }
+
+ dst->routes = kvmalloc_array(src->num_routes, sizeof(*src->routes),
+ GFP_KERNEL);
+ if (!dst->routes)
+ return -ENOMEM;
+
+ v4l2_subdev_cpy_routing(dst, src);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_dup_routing);
+
+bool v4l2_subdev_has_route(struct v4l2_subdev_krouting *routing,
+ unsigned int pad0, unsigned int pad1)
+{
+ unsigned int i;
+
+ for (i = 0; i < routing->num_routes; ++i) {
+ struct v4l2_subdev_route *route = &routing->routes[i];
+
+ if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+ continue;
+
+ if (route->sink_pad == pad0 && route->source_pad == pad1)
+ return true;
+ }
+
+ return false;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_has_route);
+
int v4l2_subdev_link_validate(struct media_link *link)
{
struct v4l2_subdev *sink;
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index c45f5f0156c9..1235d4832b76 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -1235,4 +1235,18 @@ extern const struct v4l2_subdev_ops v4l2_subdev_call_wrappers;
void v4l2_subdev_notify_event(struct v4l2_subdev *sd,
const struct v4l2_event *ev);
+int v4l2_subdev_get_krouting(struct v4l2_subdev *sd,
+ struct v4l2_subdev_krouting *routing);
+
+void v4l2_subdev_free_routing(struct v4l2_subdev_krouting *routing);
+
+int v4l2_subdev_dup_routing(struct v4l2_subdev_krouting *dst,
+ const struct v4l2_subdev_krouting *src);
+
+void v4l2_subdev_cpy_routing(struct v4l2_subdev_krouting *dst,
+ const struct v4l2_subdev_krouting *src);
+
+bool v4l2_subdev_has_route(struct v4l2_subdev_krouting *routing,
+ unsigned int pad0, unsigned int pad1);
+
#endif
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 21/24] v4l: subdev: add v4l2_subdev_get_format_dir()
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (19 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 20/24] v4l: subdev: routing kernel helper functions Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 22/24] media: uapi: add MEDIA_PAD_FL_MULTIPLEXED flag Tomi Valkeinen
` (3 subsequent siblings)
24 siblings, 0 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen
Add v4l2_subdev_get_format_dir() which can be used to find subdev format
for a specific stream on a multiplexed pad. The function will follow the
routes and links until it finds a non-multiplexed pad.
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/v4l2-core/v4l2-subdev.c | 96 +++++++++++++++++++++++++++
include/media/v4l2-subdev.h | 26 ++++++++
2 files changed, 122 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 31e279292ea6..f08847341a6b 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -1000,6 +1000,102 @@ bool v4l2_subdev_has_route(struct v4l2_subdev_krouting *routing,
}
EXPORT_SYMBOL_GPL(v4l2_subdev_has_route);
+int v4l2_subdev_get_format_dir(struct media_pad *pad, u16 stream,
+ enum v4l2_direction dir,
+ struct v4l2_subdev_format *fmt)
+{
+ struct device *dev = pad->entity->graph_obj.mdev->dev;
+ int ret;
+ int i;
+
+ dev_dbg(dev, "%s '%s':%u:%u %s\n", __func__,
+ pad->entity->name, pad->index, stream,
+ dir == V4L2_DIR_SOURCEWARD ? "sourceward" : "sinkward");
+
+ while (true) {
+ struct v4l2_subdev_krouting routing;
+ struct v4l2_subdev_route *route;
+
+ if (pad->entity->obj_type != MEDIA_ENTITY_TYPE_V4L2_SUBDEV)
+ return -EINVAL;
+
+ ret = v4l2_subdev_link_validate_get_format(pad, fmt);
+ if (ret == 0)
+ return 0;
+ else if (ret != -ENOIOCTLCMD)
+ return ret;
+
+ if (pad->flags &
+ (dir == V4L2_DIR_SINKWARD ? MEDIA_PAD_FL_SOURCE :
+ MEDIA_PAD_FL_SINK)) {
+ pad = media_entity_remote_pad(pad);
+
+ if (!pad)
+ return -EINVAL;
+
+ if (pad->entity->obj_type != MEDIA_ENTITY_TYPE_V4L2_SUBDEV)
+ return -EINVAL;
+
+ ret = v4l2_subdev_link_validate_get_format(pad, fmt);
+ if (ret == 0)
+ return 0;
+ else if (ret != -ENOIOCTLCMD)
+ return ret;
+ }
+
+ ret = v4l2_subdev_get_krouting(media_entity_to_v4l2_subdev(pad->entity), &routing);
+ if (ret)
+ return ret;
+
+ route = NULL;
+ for (i = 0; i < routing.num_routes; ++i) {
+ u16 near_pad = dir == V4L2_DIR_SINKWARD ?
+ routing.routes[i].sink_pad :
+ routing.routes[i].source_pad;
+ u16 near_stream = dir == V4L2_DIR_SINKWARD ?
+ routing.routes[i].sink_stream :
+ routing.routes[i].source_stream;
+
+ if (!(routing.routes[i].flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+ continue;
+
+ if (near_pad != pad->index)
+ continue;
+
+ if (near_stream != stream)
+ continue;
+
+ if (route) {
+ dev_err(dev,
+ "%s: '%s' has multiple active routes for stream %u\n",
+ __func__, pad->entity->name, stream);
+ v4l2_subdev_free_routing(&routing);
+ return -EINVAL;
+ }
+
+ route = &routing.routes[i];
+ }
+
+ if (!route) {
+ dev_err(dev, "%s: no route found in '%s' for stream %u\n",
+ __func__, pad->entity->name, stream);
+ v4l2_subdev_free_routing(&routing);
+ return -EINVAL;
+ }
+
+ if (dir == V4L2_DIR_SINKWARD) {
+ pad = &pad->entity->pads[route->source_pad];
+ stream = route->source_stream;
+ } else {
+ pad = &pad->entity->pads[route->sink_pad];
+ stream = route->sink_stream;
+ }
+
+ v4l2_subdev_free_routing(&routing);
+ }
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_get_format_dir);
+
int v4l2_subdev_link_validate(struct media_link *link)
{
struct v4l2_subdev *sink;
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 1235d4832b76..9025ef3a67f5 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -1249,4 +1249,30 @@ void v4l2_subdev_cpy_routing(struct v4l2_subdev_krouting *dst,
bool v4l2_subdev_has_route(struct v4l2_subdev_krouting *routing,
unsigned int pad0, unsigned int pad1);
+/**
+ * enum v4l2_direction - Direction either towards the source or the sink
+ *
+ * @V4L2_DIR_SOURCEWARD: Direction towards the source.
+ * @V4L2_DIR_SINKWARD: Direction towards the sink.
+ */
+enum v4l2_direction {
+ V4L2_DIR_SOURCEWARD,
+ V4L2_DIR_SINKWARD,
+};
+
+/**
+ * v4l2_subdev_get_format_dir() - Find format by following streams
+ * @pad: The pad from which to start the search
+ * @stream: The stream for which we want to find the format
+ * @dir: The direction of the search
+ * @fmt: Pointer to &struct v4l2_subdev_format where the found format is stored
+ *
+ * This function attempts to find v4l2_subdev_format for a specific stream on a
+ * multiplexed pad by following the stream using routes and links to the specified
+ * direction, until a non-multiplexed pad is found.
+ */
+int v4l2_subdev_get_format_dir(struct media_pad *pad, u16 stream,
+ enum v4l2_direction dir,
+ struct v4l2_subdev_format *fmt);
+
#endif
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 22/24] media: uapi: add MEDIA_PAD_FL_MULTIPLEXED flag
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (20 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 21/24] v4l: subdev: add v4l2_subdev_get_format_dir() Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-29 2:53 ` Laurent Pinchart
2021-04-27 12:45 ` [PATCH v6 23/24] v4l: subdev: Take routing information into account in link validation Tomi Valkeinen
` (2 subsequent siblings)
24 siblings, 1 reply; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen
Add a pad flag to indicate that the pad uses multiplexed streams. A pad
with multiplexed streams won't support pad format configuration and e.g.
VIDIOC_SUBDEV_S_FMT should return -ENOIOCTLCMD.
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
Documentation/userspace-api/media/mediactl/media-types.rst | 6 ++++++
include/uapi/linux/media.h | 1 +
2 files changed, 7 insertions(+)
diff --git a/Documentation/userspace-api/media/mediactl/media-types.rst b/Documentation/userspace-api/media/mediactl/media-types.rst
index 0a26397bd01d..d1dca50649f9 100644
--- a/Documentation/userspace-api/media/mediactl/media-types.rst
+++ b/Documentation/userspace-api/media/mediactl/media-types.rst
@@ -361,6 +361,7 @@ Types and flags used to represent the media graph elements
.. _MEDIA-PAD-FL-SINK:
.. _MEDIA-PAD-FL-SOURCE:
.. _MEDIA-PAD-FL-MUST-CONNECT:
+.. _MEDIA-PAD-FL-MULTIPLEXED:
.. flat-table:: Media pad flags
:header-rows: 0
@@ -382,6 +383,11 @@ Types and flags used to represent the media graph elements
when this flag isn't set; the absence of the flag doesn't imply
there is none.
+ * - ``MEDIA_PAD_FL_MULTIPLEXED``
+ - The pad has multiplexed streams. A pad with multiplexed streams
+ won't support pad format configuration and e.g. VIDIOC_SUBDEV_S_FMT
+ should return -ENOIOCTLCMD.
+
One and only one of ``MEDIA_PAD_FL_SINK`` and ``MEDIA_PAD_FL_SOURCE``
must be set for every pad.
diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
index 200fa8462b90..bdaadef80cbd 100644
--- a/include/uapi/linux/media.h
+++ b/include/uapi/linux/media.h
@@ -211,6 +211,7 @@ struct media_entity_desc {
#define MEDIA_PAD_FL_SINK (1 << 0)
#define MEDIA_PAD_FL_SOURCE (1 << 1)
#define MEDIA_PAD_FL_MUST_CONNECT (1 << 2)
+#define MEDIA_PAD_FL_MULTIPLEXED (1 << 3)
struct media_pad_desc {
__u32 entity; /* entity ID */
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 23/24] v4l: subdev: Take routing information into account in link validation
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (21 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 22/24] media: uapi: add MEDIA_PAD_FL_MULTIPLEXED flag Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 24/24] v4l: subdev: increase V4L2_FRAME_DESC_ENTRY_MAX to 8 Tomi Valkeinen
2021-04-29 10:28 ` [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
24 siblings, 0 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen
The routing information is essential in link validation for multiplexed
links: the pads at the ends of a multiplexed link have no single format
defined for them. Instead, the format is accessible in the sink (or
source) pads of the sub-devices at both ends of that link.
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/v4l2-core/v4l2-subdev.c | 144 +++++++++++++++++++++++++-
1 file changed, 142 insertions(+), 2 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index f08847341a6b..2dd78a293b03 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -1096,14 +1096,154 @@ int v4l2_subdev_get_format_dir(struct media_pad *pad, u16 stream,
}
EXPORT_SYMBOL_GPL(v4l2_subdev_get_format_dir);
+static int v4l2_subdev_link_validate_routing_stream(
+ struct media_link *link, struct media_pad *sink_pad, u16 sink_stream,
+ struct media_pad *source_pad, u16 source_stream)
+{
+ struct v4l2_subdev_format source_fmt;
+ struct v4l2_subdev_format sink_fmt;
+ struct v4l2_subdev *sink_sd;
+ int ret;
+
+ ret = v4l2_subdev_get_format_dir(sink_pad, sink_stream,
+ V4L2_DIR_SINKWARD, &sink_fmt);
+ if (ret)
+ return ret;
+
+ ret = v4l2_subdev_get_format_dir(source_pad, source_stream,
+ V4L2_DIR_SOURCEWARD, &source_fmt);
+ if (ret)
+ return ret;
+
+ sink_sd = media_entity_to_v4l2_subdev(sink_pad->entity);
+
+ ret = v4l2_subdev_call(sink_sd, pad, link_validate, link, &source_fmt,
+ &sink_fmt);
+ if (ret != -ENOIOCTLCMD)
+ return ret;
+
+ ret = v4l2_subdev_link_validate_default(sink_sd, link, &source_fmt,
+ &sink_fmt);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static int v4l2_subdev_link_validate_routing(struct media_link *link)
+{
+ int ret;
+ unsigned int i, j;
+
+ struct route_info {
+ struct v4l2_subdev_krouting routing;
+ struct media_pad *pad;
+ struct v4l2_subdev *subdev;
+ };
+
+ struct route_info source_route_info = {
+ .pad = link->source,
+ .subdev = media_entity_to_v4l2_subdev(link->source->entity),
+ };
+
+ struct route_info sink_route_info = {
+ .pad = link->sink,
+ .subdev = media_entity_to_v4l2_subdev(link->sink->entity),
+ };
+
+ struct device *dev = sink_route_info.subdev->entity.graph_obj.mdev->dev;
+
+ dev_dbg(dev, "validating link \"%s\":%u -> \"%s\":%u\n",
+ link->source->entity->name, link->source->index,
+ link->sink->entity->name, link->sink->index);
+
+ ret = v4l2_subdev_get_krouting(source_route_info.subdev,
+ &source_route_info.routing);
+ if (ret)
+ return ret;
+
+ ret = v4l2_subdev_get_krouting(sink_route_info.subdev,
+ &sink_route_info.routing);
+ if (ret) {
+ v4l2_subdev_free_routing(&source_route_info.routing);
+ return ret;
+ }
+
+ /*
+ * Every active sink route needs an active source route, but it's ok
+ * to have active source routes without matching sink route.
+ */
+ for (i = 0; i < sink_route_info.routing.num_routes; ++i) {
+ struct v4l2_subdev_route *sink_route =
+ &sink_route_info.routing.routes[i];
+
+ if (!(sink_route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+ continue;
+
+ if (sink_route->sink_pad != sink_route_info.pad->index)
+ continue;
+
+ for (j = 0; j < source_route_info.routing.num_routes; ++j) {
+ struct v4l2_subdev_route *source_route =
+ &source_route_info.routing.routes[j];
+
+ if (!(source_route->flags &
+ V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+ continue;
+
+ if (source_route->source_pad !=
+ source_route_info.pad->index)
+ continue;
+
+ if (source_route->source_stream !=
+ sink_route->sink_stream)
+ continue;
+
+ ret = v4l2_subdev_link_validate_routing_stream(
+ link,
+ &sink_route_info.pad->entity
+ ->pads[sink_route->sink_pad],
+ sink_route->sink_stream,
+ &source_route_info.pad->entity
+ ->pads[source_route->source_pad],
+ source_route->source_stream);
+ if (ret)
+ goto out;
+
+ break;
+ }
+
+ if (j == source_route_info.routing.num_routes) {
+ dev_err(dev,
+ "%s: no active source route found for sink route '%s':%u:%u\n",
+ __func__, sink_route_info.pad->entity->name,
+ sink_route->sink_pad, sink_route->sink_stream);
+ ret = -EINVAL;
+ goto out;
+ }
+ }
+
+out:
+ v4l2_subdev_free_routing(&source_route_info.routing);
+ v4l2_subdev_free_routing(&sink_route_info.routing);
+
+ return ret;
+}
+
int v4l2_subdev_link_validate(struct media_link *link)
{
struct v4l2_subdev *sink;
struct v4l2_subdev_format sink_fmt, source_fmt;
int rval;
- rval = v4l2_subdev_link_validate_get_format(
- link->source, &source_fmt);
+ if (WARN_ON((link->source->flags & MEDIA_PAD_FL_MULTIPLEXED) !=
+ (link->sink->flags & MEDIA_PAD_FL_MULTIPLEXED)))
+ return -EINVAL;
+
+ if (link->source->flags & MEDIA_PAD_FL_MULTIPLEXED)
+ return v4l2_subdev_link_validate_routing(link);
+
+ rval = v4l2_subdev_link_validate_get_format(link->source, &source_fmt);
if (rval < 0)
return 0;
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 24/24] v4l: subdev: increase V4L2_FRAME_DESC_ENTRY_MAX to 8
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (22 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 23/24] v4l: subdev: Take routing information into account in link validation Tomi Valkeinen
@ 2021-04-27 12:45 ` Tomi Valkeinen
2021-04-29 10:28 ` [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
24 siblings, 0 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-27 12:45 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil, Tomi Valkeinen
V4L2_FRAME_DESC_ENTRY_MAX is currently set to 4. In theory it's possible
to have an arbitrary amount of streams in a single pad, so preferably
there should be no hardcoded maximum number.
However, I believe a reasonable max is 8, which would cover a CSI-2 pad
with 4 streams of pixel data and 4 streams of metadata.
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
include/media/v4l2-subdev.h | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 9025ef3a67f5..8e2b990b06ff 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -356,7 +356,11 @@ struct v4l2_mbus_frame_desc_entry {
} bus;
};
-#define V4L2_FRAME_DESC_ENTRY_MAX 4
+ /*
+ * FIXME: If this number is too small, it should be dropped altogether and the
+ * API switched to a dynamic number of frame descriptor entries.
+ */
+#define V4L2_FRAME_DESC_ENTRY_MAX 8
/**
* enum v4l2_mbus_frame_desc_type - media bus frame description type
--
2.25.1
^ permalink raw reply related [flat|nested] 36+ messages in thread
* Re: [PATCH v6 12/24] media: entity: Add an iterator helper for connected pads
2021-04-27 12:45 ` [PATCH v6 12/24] media: entity: Add an iterator helper for connected pads Tomi Valkeinen
@ 2021-04-29 2:18 ` Laurent Pinchart
2021-05-04 6:53 ` Tomi Valkeinen
0 siblings, 1 reply; 36+ messages in thread
From: Laurent Pinchart @ 2021-04-29 2:18 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil
Hi Tomi,
Thank you for the patch.
On Tue, Apr 27, 2021 at 03:45:11PM +0300, Tomi Valkeinen wrote:
> Add a helper macro for iterating over pads that are connected through
> enabled routes. This can be used to find all the connected pads within an
> entity, for instance starting from the pad which has been obtained during
> the graph walk.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
>
> - Make __media_entity_next_routed_pad() return NULL and adjust the
> iterator to handle that
> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
> include/media/media-entity.h | 28 ++++++++++++++++++++++++++++
> 1 file changed, 28 insertions(+)
>
> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> index f8fa952fa38e..42193d6c58e9 100644
> --- a/include/media/media-entity.h
> +++ b/include/media/media-entity.h
> @@ -916,6 +916,34 @@ __must_check int media_graph_walk_init(
> bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
> unsigned int pad1);
>
> +static inline struct media_pad *__media_entity_next_routed_pad(
> + struct media_pad *start, struct media_pad *iter)
> +{
> + struct media_entity *entity = start->entity;
> +
> + for (; iter < &entity->pads[entity->num_pads]; iter++) {
> + if (media_entity_has_route(entity, start->index, iter->index))
> + return iter;
> + }
> +
> + return NULL;
> +}
> +
> +/**
> + * media_entity_for_each_routed_pad - Iterate over entity pads connected by routes
> + *
> + * @start: The starting pad
> + * @iter: The iterator pad
> + *
> + * Iterate over all pads connected through routes from the @start pad
> + * within an entity. The iteration will include the @start pad itself.
> + */
I still think a better name would be, well, better :-) Let's continue
the discussion in the v5 thread to avoid splitting it.
> +#define media_entity_for_each_routed_pad(start, iter) \
> + for (iter = __media_entity_next_routed_pad( \
> + start, (start)->entity->pads); \
> + iter != NULL; \
> + iter = __media_entity_next_routed_pad(start, iter + 1))
> +
> /**
> * media_graph_walk_cleanup - Release resources used by graph walk.
> *
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 18/24] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations
2021-04-27 12:45 ` [PATCH v6 18/24] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations Tomi Valkeinen
@ 2021-04-29 2:28 ` Laurent Pinchart
2021-04-29 6:15 ` Tomi Valkeinen
2021-05-04 7:04 ` Tomi Valkeinen
0 siblings, 2 replies; 36+ messages in thread
From: Laurent Pinchart @ 2021-04-29 2:28 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Michal Simek
Hi Tomi,
Thank you for the patch.
For some reason, it seems the original authorship got lost. It was fine
in v5. I haven't checked if other patches in the series are affected.
On Tue, Apr 27, 2021 at 03:45:17PM +0300, Tomi Valkeinen wrote:
> Add support for subdev internal routing. A route is defined as a single
> stream from a sink pad to a source pad.
>
> The userspace can configure the routing via two new ioctls,
> VIDIOC_SUBDEV_G_ROUTING and VIDIOC_SUBDEV_S_ROUTING, and subdevs can
> implement the functionality with v4l2_subdev_pad_ops.get_routing() and
> v4l2_subdev_pad_ops.set_routing().
>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Michal Simek <michal.simek@xilinx.com>
>
> - Add sink and source streams for multiplexed links
> - Copy the argument back in case of an error. This is needed to let the
> caller know the number of routes.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
>
> - Expand and refine documentation.
> - Make the 'routes' pointer a __u64 __user pointer so that a compat32
> version of the ioctl is not required.
> - Add struct v4l2_subdev_krouting to be used for subdevice operations.
>
> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
>
> - Fix typecasing warnings
> - Check sink & source pad types
> - Add 'which' field
>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
> drivers/media/v4l2-core/v4l2-ioctl.c | 25 +++++++++++++-
> drivers/media/v4l2-core/v4l2-subdev.c | 46 +++++++++++++++++++++++++
> include/media/v4l2-subdev.h | 24 ++++++++++++++
> include/uapi/linux/v4l2-subdev.h | 48 +++++++++++++++++++++++++++
> 4 files changed, 142 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 6a5d1c6d11d6..7b1dabc23909 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -16,6 +16,7 @@
> #include <linux/kernel.h>
> #include <linux/version.h>
>
> +#include <linux/v4l2-subdev.h>
> #include <linux/videodev2.h>
>
> #include <media/v4l2-common.h>
> @@ -3108,6 +3109,21 @@ static int check_array_args(unsigned int cmd, void *parg, size_t *array_size,
> ret = 1;
> break;
> }
> +
> + case VIDIOC_SUBDEV_G_ROUTING:
> + case VIDIOC_SUBDEV_S_ROUTING: {
> + struct v4l2_subdev_routing *routing = parg;
> +
> + if (routing->num_routes > 256)
> + return -EINVAL;
> +
> + *user_ptr = u64_to_user_ptr(routing->routes);
> + *kernel_ptr = (void **)&routing->routes;
> + *array_size = sizeof(struct v4l2_subdev_route)
> + * routing->num_routes;
> + ret = 1;
> + break;
> + }
> }
>
> return ret;
> @@ -3369,8 +3385,15 @@ video_usercopy(struct file *file, unsigned int orig_cmd, unsigned long arg,
> /*
> * Some ioctls can return an error, but still have valid
> * results that must be returned.
> + *
> + * FIXME: subdev IOCTLS are partially handled here and partially in
> + * v4l2-subdev.c and the 'always_copy' flag can only be set for IOCTLS
> + * defined here as part of the 'v4l2_ioctls' array. As
> + * VIDIOC_SUBDEV_G_ROUTING needs to return results to applications even
> + * in case of failure, but it is not defined here as part of the
> + * 'v4l2_ioctls' array, insert an ad-hoc check to address that.
> */
> - if (err < 0 && !always_copy)
> + if (err < 0 && !always_copy && cmd != VIDIOC_SUBDEV_G_ROUTING)
> goto out;
>
> out_array_args:
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 956dafab43d4..ad79ce121cee 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -681,6 +681,52 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg)
> case VIDIOC_SUBDEV_QUERYSTD:
> return v4l2_subdev_call(sd, video, querystd, arg);
>
> + case VIDIOC_SUBDEV_G_ROUTING: {
> + struct v4l2_subdev_routing *routing = arg;
> + struct v4l2_subdev_krouting krouting = {
> + .which = routing->which,
> + .num_routes = routing->num_routes,
> + .routes = (struct v4l2_subdev_route *)(uintptr_t)
> + routing->routes,
> + };
> + int ret;
> +
> + ret = v4l2_subdev_call(sd, pad, get_routing, &krouting);
> +
> + routing->num_routes = krouting.num_routes;
> +
> + return ret;
> + }
> +
> + case VIDIOC_SUBDEV_S_ROUTING: {
> + struct v4l2_subdev_routing *routing = arg;
> + struct v4l2_subdev_route *routes =
> + (struct v4l2_subdev_route *)(uintptr_t)routing->routes;
> + struct v4l2_subdev_krouting krouting = {};
> + unsigned int i;
> +
> + if (routing->which != V4L2_SUBDEV_FORMAT_TRY && ro_subdev)
> + return -EPERM;
> +
> + for (i = 0; i < routing->num_routes; ++i) {
> + if (routes[i].sink_pad >= sd->entity.num_pads ||
> + routes[i].source_pad >= sd->entity.num_pads)
> + return -EINVAL;
> +
> + if (!(sd->entity.pads[routes[i].sink_pad].flags &
> + MEDIA_PAD_FL_SINK) ||
> + !(sd->entity.pads[routes[i].source_pad].flags &
> + MEDIA_PAD_FL_SOURCE))
> + return -EINVAL;
> + }
> +
> + krouting.which = routing->which;
> + krouting.num_routes = routing->num_routes;
> + krouting.routes = routes;
> +
> + return v4l2_subdev_call(sd, pad, set_routing, &krouting);
> + }
> +
> default:
> return v4l2_subdev_call(sd, core, ioctl, cmd, arg);
> }
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index 70a21298020b..c45f5f0156c9 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -660,6 +660,22 @@ struct v4l2_subdev_pad_config {
> struct v4l2_rect try_compose;
> };
>
> +/**
> + * struct v4l2_subdev_krouting - subdev routing table
> + *
> + * @which: format type (from enum v4l2_subdev_format_whence)
> + * @routes: &struct v4l2_subdev_route
> + * @num_routes: number of routes
> + *
> + * This structure is used to translate arguments received from
> + * VIDIOC_SUBDEV_G/S_ROUTING() ioctl to subdev device drivers operations.
> + */
> +struct v4l2_subdev_krouting {
> + u32 which;
> + struct v4l2_subdev_route *routes;
> + unsigned int num_routes;
> +};
> +
> /**
> * struct v4l2_subdev_pad_ops - v4l2-subdev pad level operations
> *
> @@ -721,6 +737,10 @@ struct v4l2_subdev_pad_config {
> * applied to the hardware. The operation shall fail if the
> * pad index it has been called on is not valid or in case of
> * unrecoverable failures.
> + *
> + * @get_routing: get the subdevice routing table.
> + * @set_routing: enable or disable data connection routes described in the
> + * subdevice routing table.
> */
> struct v4l2_subdev_pad_ops {
> int (*init_cfg)(struct v4l2_subdev *sd,
> @@ -765,6 +785,10 @@ struct v4l2_subdev_pad_ops {
> struct v4l2_mbus_config *config);
> int (*set_mbus_config)(struct v4l2_subdev *sd, unsigned int pad,
> struct v4l2_mbus_config *config);
> + int (*get_routing)(struct v4l2_subdev *sd,
> + struct v4l2_subdev_krouting *route);
> + int (*set_routing)(struct v4l2_subdev *sd,
> + struct v4l2_subdev_krouting *route);
> };
>
> /**
> diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h
> index 658106f5b5dc..3a360cdcacf4 100644
> --- a/include/uapi/linux/v4l2-subdev.h
> +++ b/include/uapi/linux/v4l2-subdev.h
> @@ -188,6 +188,52 @@ struct v4l2_subdev_capability {
> /* The v4l2 sub-device video device node is registered in read-only mode. */
> #define V4L2_SUBDEV_CAP_RO_SUBDEV 0x00000001
>
> +/*
Shouldn't this be /** ?
> + * Is the route active? An active route will start when streaming is enabled
> + * on a video node.
Even with an a priori knowledge of the topic, I'm not sure to really
understand this, and I don't think it reflects the purpose of the flag.
> + */
> +#define V4L2_SUBDEV_ROUTE_FL_ACTIVE BIT(0)
> +
> +/*
> + * Is the route immutable, i.e. can it be activated and inactivated?
Maybe adding "An immutable route is always active".
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> + * Set by the driver.
> + */
> +#define V4L2_SUBDEV_ROUTE_FL_IMMUTABLE BIT(1)
> +
> +/**
> + * struct v4l2_subdev_route - A route inside a subdev
> + *
> + * @sink_pad: the sink pad index
> + * @sink_stream: the sink stream identifier
> + * @source_pad: the source pad index
> + * @source_stream: the source stream identifier
> + * @flags: route flags V4L2_SUBDEV_ROUTE_FL_*
> + * @reserved: drivers and applications must zero this array
> + */
> +struct v4l2_subdev_route {
> + __u32 sink_pad;
> + __u32 sink_stream;
> + __u32 source_pad;
> + __u32 source_stream;
> + __u32 flags;
> + __u32 reserved[5];
> +};
> +
> +/**
> + * struct v4l2_subdev_routing - Subdev routing information
> + *
> + * @which: configuration type (from enum v4l2_subdev_format_whence)
> + * @routes: pointer to the routes array
> + * @num_routes: the total number of routes in the routes array
> + * @reserved: drivers and applications must zero this array
> + */
> +struct v4l2_subdev_routing {
> + __u32 which;
> + __u64 routes;
> + __u32 num_routes;
> + __u32 reserved[5];
> +};
> +
> /* Backwards compatibility define --- to be removed */
> #define v4l2_subdev_edid v4l2_edid
>
> @@ -203,6 +249,8 @@ struct v4l2_subdev_capability {
> #define VIDIOC_SUBDEV_S_CROP _IOWR('V', 60, struct v4l2_subdev_crop)
> #define VIDIOC_SUBDEV_G_SELECTION _IOWR('V', 61, struct v4l2_subdev_selection)
> #define VIDIOC_SUBDEV_S_SELECTION _IOWR('V', 62, struct v4l2_subdev_selection)
> +#define VIDIOC_SUBDEV_G_ROUTING _IOWR('V', 38, struct v4l2_subdev_routing)
> +#define VIDIOC_SUBDEV_S_ROUTING _IOWR('V', 39, struct v4l2_subdev_routing)
> /* The following ioctls are identical to the ioctls in videodev2.h */
> #define VIDIOC_SUBDEV_G_STD _IOR('V', 23, v4l2_std_id)
> #define VIDIOC_SUBDEV_S_STD _IOW('V', 24, v4l2_std_id)
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 11/24] media: entity: Skip link validation for pads to which there is no route to
2021-04-27 12:45 ` [PATCH v6 11/24] media: entity: Skip link validation for pads to which there is no route to Tomi Valkeinen
@ 2021-04-29 2:30 ` Laurent Pinchart
0 siblings, 0 replies; 36+ messages in thread
From: Laurent Pinchart @ 2021-04-29 2:30 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil
Hi Tomi and Sakari,
Thank you for the patch.
On Tue, Apr 27, 2021 at 03:45:10PM +0300, Tomi Valkeinen wrote:
> From: Sakari Ailus <sakari.ailus@linux.intel.com>
>
> Links are validated along the pipeline which is about to start streaming.
> Not all the pads in entities that are traversed along that pipeline are
> part of the pipeline, however. Skip the link validation for such pads,
> and while at there rename "other_pad" to "local_pad" to convey the fact
> the route to be checked is internal to the entity.
After sorting out the "other_pad" name as discussed in v5,
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
> drivers/media/mc/mc-entity.c | 15 ++++++++++-----
> 1 file changed, 10 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
> index d00c7397e09a..a4cca53124b2 100644
> --- a/drivers/media/mc/mc-entity.c
> +++ b/drivers/media/mc/mc-entity.c
> @@ -482,27 +482,32 @@ __must_check int __media_pipeline_start(struct media_pad *pad,
> bitmap_fill(has_no_links, entity->num_pads);
>
> list_for_each_entry(link, &entity->links, list) {
> - struct media_pad *other_pad =
> + struct media_pad *local_pad =
> link->sink->entity == entity ?
> link->sink : link->source;
>
> + /* Ignore pads to which there is no route. */
> + if (!media_entity_has_route(entity, pad->index,
> + local_pad->index))
> + continue;
> +
> /* Mark that a pad is connected by a link. */
> - bitmap_clear(has_no_links, other_pad->index, 1);
> + bitmap_clear(has_no_links, local_pad->index, 1);
>
> /*
> * Pads that either do not need to connect or
> * are connected through an enabled link are
> * fine.
> */
> - if (!(other_pad->flags & MEDIA_PAD_FL_MUST_CONNECT) ||
> + if (!(local_pad->flags & MEDIA_PAD_FL_MUST_CONNECT) ||
> link->flags & MEDIA_LNK_FL_ENABLED)
> - bitmap_set(active, other_pad->index, 1);
> + bitmap_set(active, local_pad->index, 1);
>
> /*
> * Link validation will only take place for
> * sink ends of the link that are enabled.
> */
> - if (link->sink != other_pad ||
> + if (link->sink != local_pad ||
> !(link->flags & MEDIA_LNK_FL_ENABLED))
> continue;
>
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 15/24] v4l: Add bus type to frame descriptors
2021-04-27 12:45 ` [PATCH v6 15/24] v4l: Add bus type to frame descriptors Tomi Valkeinen
@ 2021-04-29 2:46 ` Laurent Pinchart
0 siblings, 0 replies; 36+ messages in thread
From: Laurent Pinchart @ 2021-04-29 2:46 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil
Hi Tomi,
Thank you for the patch.
Authorship lost here too it seems.
On Tue, Apr 27, 2021 at 03:45:14PM +0300, Tomi Valkeinen wrote:
> Add the media bus type to the frame descriptor. CSI-2 specific
> information will be added in next patch to the frame descriptor.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
>
> - Make the bus type a named enum
> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
> include/media/v4l2-subdev.h | 19 +++++++++++++++++++
> 1 file changed, 19 insertions(+)
>
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index d0e9a5bdb08b..ac531b752028 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -340,12 +340,31 @@ struct v4l2_mbus_frame_desc_entry {
>
> #define V4L2_FRAME_DESC_ENTRY_MAX 4
>
> +/**
> + * enum v4l2_mbus_frame_desc_type - media bus frame description type
> + *
> + * @V4L2_MBUS_FRAME_DESC_TYPE_PLATFORM:
> + * Platform specific frame desc type for backwards compatibility.
I'd have named it UNKNOWN, or possibly LEGACY or something similar, to
convey the fact that it shouldn't be used, but is only there so that
drivers that don't set the field can be told apart. Maybe the
documentation could capture this more clearly.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> + * @V4L2_MBUS_FRAME_DESC_TYPE_PARALLEL:
> + * Parallel media bus.
> + * @V4L2_MBUS_FRAME_DESC_TYPE_CSI2:
> + * CSI-2 media bus. Frame desc parameters must be set in
> + * &struct v4l2_mbus_frame_desc_entry->csi2.
> + */
> +enum v4l2_mbus_frame_desc_type {
> + V4L2_MBUS_FRAME_DESC_TYPE_PLATFORM,
> + V4L2_MBUS_FRAME_DESC_TYPE_PARALLEL,
> + V4L2_MBUS_FRAME_DESC_TYPE_CSI2,
> +};
> +
> /**
> * struct v4l2_mbus_frame_desc - media bus data frame description
> + * @type: type of the bus (enum v4l2_mbus_frame_desc_type)
> * @entry: frame descriptors array
> * @num_entries: number of entries in @entry array
> */
> struct v4l2_mbus_frame_desc {
> + enum v4l2_mbus_frame_desc_type type;
> struct v4l2_mbus_frame_desc_entry entry[V4L2_FRAME_DESC_ENTRY_MAX];
> unsigned short num_entries;
> };
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 17/24] v4l: Add stream to frame descriptor
2021-04-27 12:45 ` [PATCH v6 17/24] v4l: Add stream to frame descriptor Tomi Valkeinen
@ 2021-04-29 2:50 ` Laurent Pinchart
0 siblings, 0 replies; 36+ messages in thread
From: Laurent Pinchart @ 2021-04-29 2:50 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil
Hi Tomi,
Thank you for the patch.
On Tue, Apr 27, 2021 at 03:45:16PM +0300, Tomi Valkeinen wrote:
> From: Sakari Ailus <sakari.ailus@linux.intel.com>
>
> The stream field identifies the stream this frame descriptor applies to in
> routing configuration across a multiplexed link.
I'd still move this to the next series, to review it with code that uses
this new field.
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
> include/media/v4l2-subdev.h | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index 695e97b6eb13..70a21298020b 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -338,6 +338,7 @@ enum v4l2_mbus_frame_desc_flags {
> * struct v4l2_mbus_frame_desc_entry - media bus frame description structure
> *
> * @flags: bitmask flags, as defined by &enum v4l2_mbus_frame_desc_flags.
> + * @stream: stream in routing configuration
> * @pixelcode: media bus pixel code, valid if @flags
> * %FRAME_DESC_FL_BLOB is not set.
> * @length: number of octets per frame, valid if @flags
> @@ -347,6 +348,7 @@ enum v4l2_mbus_frame_desc_flags {
> */
> struct v4l2_mbus_frame_desc_entry {
> enum v4l2_mbus_frame_desc_flags flags;
> + u32 stream;
> u32 pixelcode;
> u32 length;
> union {
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 22/24] media: uapi: add MEDIA_PAD_FL_MULTIPLEXED flag
2021-04-27 12:45 ` [PATCH v6 22/24] media: uapi: add MEDIA_PAD_FL_MULTIPLEXED flag Tomi Valkeinen
@ 2021-04-29 2:53 ` Laurent Pinchart
0 siblings, 0 replies; 36+ messages in thread
From: Laurent Pinchart @ 2021-04-29 2:53 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil
Hi Tomi,
Thank you for the patch.
On Tue, Apr 27, 2021 at 03:45:21PM +0300, Tomi Valkeinen wrote:
> Add a pad flag to indicate that the pad uses multiplexed streams. A pad
> with multiplexed streams won't support pad format configuration and e.g.
> VIDIOC_SUBDEV_S_FMT should return -ENOIOCTLCMD.
>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
> Documentation/userspace-api/media/mediactl/media-types.rst | 6 ++++++
> include/uapi/linux/media.h | 1 +
> 2 files changed, 7 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/mediactl/media-types.rst b/Documentation/userspace-api/media/mediactl/media-types.rst
> index 0a26397bd01d..d1dca50649f9 100644
> --- a/Documentation/userspace-api/media/mediactl/media-types.rst
> +++ b/Documentation/userspace-api/media/mediactl/media-types.rst
> @@ -361,6 +361,7 @@ Types and flags used to represent the media graph elements
> .. _MEDIA-PAD-FL-SINK:
> .. _MEDIA-PAD-FL-SOURCE:
> .. _MEDIA-PAD-FL-MUST-CONNECT:
> +.. _MEDIA-PAD-FL-MULTIPLEXED:
>
> .. flat-table:: Media pad flags
> :header-rows: 0
> @@ -382,6 +383,11 @@ Types and flags used to represent the media graph elements
> when this flag isn't set; the absence of the flag doesn't imply
> there is none.
>
> + * - ``MEDIA_PAD_FL_MULTIPLEXED``
> + - The pad has multiplexed streams. A pad with multiplexed streams
> + won't support pad format configuration and e.g. VIDIOC_SUBDEV_S_FMT
> + should return -ENOIOCTLCMD.
This should also mention if the flag is set by drivers or by userspace,
and if it static or can change dynamically. If it can change
dynamically, the documentation should explain the circumstances under
which the flag can change.
Other than expanding the documentation, I see no issue here, this looks
good.
> +
>
> One and only one of ``MEDIA_PAD_FL_SINK`` and ``MEDIA_PAD_FL_SOURCE``
> must be set for every pad.
> diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
> index 200fa8462b90..bdaadef80cbd 100644
> --- a/include/uapi/linux/media.h
> +++ b/include/uapi/linux/media.h
> @@ -211,6 +211,7 @@ struct media_entity_desc {
> #define MEDIA_PAD_FL_SINK (1 << 0)
> #define MEDIA_PAD_FL_SOURCE (1 << 1)
> #define MEDIA_PAD_FL_MUST_CONNECT (1 << 2)
> +#define MEDIA_PAD_FL_MULTIPLEXED (1 << 3)
>
> struct media_pad_desc {
> __u32 entity; /* entity ID */
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 20/24] v4l: subdev: routing kernel helper functions
2021-04-27 12:45 ` [PATCH v6 20/24] v4l: subdev: routing kernel helper functions Tomi Valkeinen
@ 2021-04-29 3:46 ` Laurent Pinchart
0 siblings, 0 replies; 36+ messages in thread
From: Laurent Pinchart @ 2021-04-29 3:46 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil
Hi Tomi,
Thank you for the patch.
On Tue, Apr 27, 2021 at 03:45:19PM +0300, Tomi Valkeinen wrote:
> Add helper functions for routing.
>
> TODO: add docs.
>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
> drivers/media/v4l2-core/v4l2-subdev.c | 90 +++++++++++++++++++++++++++
> include/media/v4l2-subdev.h | 14 +++++
> 2 files changed, 104 insertions(+)
>
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index ad79ce121cee..31e279292ea6 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -910,6 +910,96 @@ v4l2_subdev_link_validate_get_format(struct media_pad *pad,
> return -EINVAL;
> }
>
> +int v4l2_subdev_get_krouting(struct v4l2_subdev *sd,
I wonder if we could call the function v4l2_subdev_get_routing(), the
functions below don't have a k prefix.
> + struct v4l2_subdev_krouting *routing)
> +{
> + int ret;
> +
> + routing->which = V4L2_SUBDEV_FORMAT_ACTIVE;
> + routing->routes = NULL;
> + routing->num_routes = 0;
> +
> + ret = v4l2_subdev_call(sd, pad, get_routing, routing);
> + if (ret == 0)
> + return 0;
> + if (ret != -ENOSPC)
> + return ret;
> +
> + routing->routes = kvmalloc_array(routing->num_routes,
> + sizeof(*routing->routes), GFP_KERNEL);
> + if (!routing->routes)
> + return -ENOMEM;
> +
> + ret = v4l2_subdev_call(sd, pad, get_routing, routing);
> + if (ret) {
> + kvfree(routing->routes);
Should we handle the case where the subdev configuration gets changed
between the two calls, and the number of routes increases ? If that can
happen, there's also a risk for the routing configuration to be stale
already when this function returns, so it's probably the caller that
needs to ensure this can't happen.
> + return ret;
> + }
> +
> + return 0;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_get_krouting);
> +
> +void v4l2_subdev_free_routing(struct v4l2_subdev_krouting *routing)
> +{
> + kvfree(routing->routes);
> + routing->routes = NULL;
I'd set num_routes to 0 too.
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_free_routing);
> +
> +void v4l2_subdev_cpy_routing(struct v4l2_subdev_krouting *dst,
> + const struct v4l2_subdev_krouting *src)
> +{
> + memcpy(dst->routes, src->routes,
> + src->num_routes * sizeof(*src->routes));
Is there a guarantee that dst has allocated enough entries in routes ?
It seems a bit fragile.
> + dst->num_routes = src->num_routes;
> + dst->which = src->which;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_cpy_routing);
> +
> +int v4l2_subdev_dup_routing(struct v4l2_subdev_krouting *dst,
> + const struct v4l2_subdev_krouting *src)
> +{
> + if (dst->routes)
> + kvfree(dst->routes);
I'd set routes to NULL and num_routes to 0 here, so that num_routes
would be 0 if the allocation fails below. The performance impact will be
negligible, and the data will be more coherent. We could set num_routes
to 0 in the allocation error path, but splitting the kvfree() and the
routes and num_routes assignments opens the door to more bugs when this
function will be extended/refactored.
> +
> + if (src->num_routes == 0) {
> + dst->which = src->which;
> + dst->routes = NULL;
> + dst->num_routes = 0;
> + return 0;
> + }
> +
> + dst->routes = kvmalloc_array(src->num_routes, sizeof(*src->routes),
> + GFP_KERNEL);
> + if (!dst->routes)
> + return -ENOMEM;
> +
> + v4l2_subdev_cpy_routing(dst, src);
> +
> + return 0;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_dup_routing);
> +
> +bool v4l2_subdev_has_route(struct v4l2_subdev_krouting *routing,
> + unsigned int pad0, unsigned int pad1)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < routing->num_routes; ++i) {
> + struct v4l2_subdev_route *route = &routing->routes[i];
> +
> + if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
> + continue;
> +
> + if (route->sink_pad == pad0 && route->source_pad == pad1)
pad0 and pad1 should be renamed to sink_pad and source_pad if that's how
they need to be used. Given that this function is called in the media
entity .has_route() operation, which only guarantees that pad0 < pad 1,
we probably want to support the case where pad0 is a source pad and pad1
a sink pad.
I'm trying to recall why the media entity operation was designed this
way, and I think it was meant to support testing if two sink pads or two
source pads are connected by routes, indirectly. For instance, if a
subdev has one sink pad A and two source pads B and C, active routes
from A to B and from A to C should result in .has_route(B, C) returning
true. If I remember correctly, this was needed by the graph traversal
code, but that would need to be double-checked.
> + return true;
> + }
> +
> + return false;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_has_route);
> +
> int v4l2_subdev_link_validate(struct media_link *link)
> {
> struct v4l2_subdev *sink;
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index c45f5f0156c9..1235d4832b76 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -1235,4 +1235,18 @@ extern const struct v4l2_subdev_ops v4l2_subdev_call_wrappers;
> void v4l2_subdev_notify_event(struct v4l2_subdev *sd,
> const struct v4l2_event *ev);
>
> +int v4l2_subdev_get_krouting(struct v4l2_subdev *sd,
> + struct v4l2_subdev_krouting *routing);
> +
> +void v4l2_subdev_free_routing(struct v4l2_subdev_krouting *routing);
> +
> +int v4l2_subdev_dup_routing(struct v4l2_subdev_krouting *dst,
> + const struct v4l2_subdev_krouting *src);
> +
> +void v4l2_subdev_cpy_routing(struct v4l2_subdev_krouting *dst,
> + const struct v4l2_subdev_krouting *src);
> +
> +bool v4l2_subdev_has_route(struct v4l2_subdev_krouting *routing,
> + unsigned int pad0, unsigned int pad1);
> +
> #endif
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 18/24] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations
2021-04-29 2:28 ` Laurent Pinchart
@ 2021-04-29 6:15 ` Tomi Valkeinen
2021-05-04 7:04 ` Tomi Valkeinen
1 sibling, 0 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-29 6:15 UTC (permalink / raw)
To: Laurent Pinchart
Cc: linux-media, sakari.ailus, Jacopo Mondi,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Michal Simek
On 29/04/2021 05:28, Laurent Pinchart wrote:
> Hi Tomi,
>
> Thank you for the patch.
>
> For some reason, it seems the original authorship got lost. It was fine
> in v5. I haven't checked if other patches in the series are affected.
>
Thanks for pointing this out. I found the issue in my tooling. I'll fix
the authorship information.
Tomi
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 00/24] v4l: subdev internal routing
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
` (23 preceding siblings ...)
2021-04-27 12:45 ` [PATCH v6 24/24] v4l: subdev: increase V4L2_FRAME_DESC_ENTRY_MAX to 8 Tomi Valkeinen
@ 2021-04-29 10:28 ` Tomi Valkeinen
24 siblings, 0 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-04-29 10:28 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas
Cc: Mauro Carvalho Chehab, Hans Verkuil
On 27/04/2021 15:44, Tomi Valkeinen wrote:
> Hi,
>
> This is v6 of the series. Previous one can be found from:
>
> https://lore.kernel.org/linux-media/20210415130450.421168-1-tomi.valkeinen@ideasonboard.com/
>
> Changes to v5:
> - Dropped "v4l: mc: Add an S_ROUTING helper function for power state changes" as discussed
> - Added MEDIA_PAD_FL_MULTIPLEXED flag to indicate that a pad works in multiplexed mode
> - Documentation improvements
> - Renamed struct v4l2_mbus_frame_desc_entry_csi2 fields
> - Fixed setting routing->num_routes in G_ROUTING
> - Many smaller cosmetic changes as per comments
>
> One bigger topic I didn't change is the in-kernel API for get/set routing.
> Currently the link validation needs to get the routing tables multiple times,
> each leading to memory allocations (and frees). A different API would allow the
> link validation to peek at the routing without any allocations, but I haven't
> solved that yet.
>
> A simple solution would be to require a lock to be held when accessing the
> routing table, and making get_routing return a pointer to the driver's routing
> table. But this kind of API would be quite different than, say, get_fmt.
>
> Another would be to keep the API, but have a state object during link
> validation, which would hold preallocated routing tables, so that link
> validation wouldn't need to allocate new ones for each get_routing call.
>
> Anything we do here is a kernel internal change, so it's not super critical to
> solve it right away.
I decided to test the other approach: changing subdev format ioctls to
work on a pad+stream tuple, instead of just on a pad. So essentially
adding a 'stream' field (taking one u32 from the reserved area) to
v4l2_subdev_format, v4l2_subdev_mbus_code_enum, and others. The current
userspace sets the field to 0, to it makes sense even if with older
userspace.
My branch is still quite messy, but this approach does give us some nice
benefits:
- I was able to drop the whole "follow-the-stream-to-get-format" code.
Link validation feels simpler.
- A sensor that provides pixel and metadata can be implemented in a
single subdev, with a single source pad.
- It makes the configuration of subdevs self-contained.
- A subdev can do operations (like, say, scaling) with a stream, instead
of having to split the subdev into multiple subdevs to get the
non-multiplexed pads.
- I don't need the cal_streams_api parameter (introduced in another
series for TI CAL driver), nor do I need the MEDIA_PAD_FL_MULTIPLEXED flag
- I can easily support a case where the subdev driver on one side of the
link supports multiplexed streams and the other driver doesn't (as long
as the multiplexed streams subdev's pad is configured for a single
stream). Well, this would probably be possible with the previous
approach also, but at last here it came kind of naturally.
Downsides:
- Need dynamic allocation for storing formats. It doesn't feel too bad,
though, and it, of course, only affects the drivers that support routing
and multiplexed streams. I do the reallocation in set_routing.
- TRY doesn't work, as there's no place to store the formats for
streams. This needs the proposed change to pass subdev state, instead of
just subdev pad state, to the various format ops.
- Possibly plenty of tooling changes to add support for streams
And I should perhaps add a subdev flag to indicate that a subdev
supports multiple streams. This could at least be used in the ioctl
handlers, rejecting stream field > 0 if the subdev does not support streams.
I'll continue working on this a bit to see how it goes, but I wanted to
share this now to hear if there are any clear NACKs for the approach.
Tomi
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 12/24] media: entity: Add an iterator helper for connected pads
2021-04-29 2:18 ` Laurent Pinchart
@ 2021-05-04 6:53 ` Tomi Valkeinen
0 siblings, 0 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-05-04 6:53 UTC (permalink / raw)
To: Laurent Pinchart
Cc: linux-media, sakari.ailus, Jacopo Mondi,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil
On 29/04/2021 05:18, Laurent Pinchart wrote:
> Hi Tomi,
>
> Thank you for the patch.
>
> On Tue, Apr 27, 2021 at 03:45:11PM +0300, Tomi Valkeinen wrote:
>> Add a helper macro for iterating over pads that are connected through
>> enabled routes. This can be used to find all the connected pads within an
>> entity, for instance starting from the pad which has been obtained during
>> the graph walk.
>>
>> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
>> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
>>
>> - Make __media_entity_next_routed_pad() return NULL and adjust the
>> iterator to handle that
>> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
>> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> ---
>> include/media/media-entity.h | 28 ++++++++++++++++++++++++++++
>> 1 file changed, 28 insertions(+)
>>
>> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
>> index f8fa952fa38e..42193d6c58e9 100644
>> --- a/include/media/media-entity.h
>> +++ b/include/media/media-entity.h
>> @@ -916,6 +916,34 @@ __must_check int media_graph_walk_init(
>> bool media_entity_has_route(struct media_entity *entity, unsigned int pad0,
>> unsigned int pad1);
>>
>> +static inline struct media_pad *__media_entity_next_routed_pad(
>> + struct media_pad *start, struct media_pad *iter)
>> +{
>> + struct media_entity *entity = start->entity;
>> +
>> + for (; iter < &entity->pads[entity->num_pads]; iter++) {
>> + if (media_entity_has_route(entity, start->index, iter->index))
>> + return iter;
>> + }
>> +
>> + return NULL;
>> +}
>> +
>> +/**
>> + * media_entity_for_each_routed_pad - Iterate over entity pads connected by routes
>> + *
>> + * @start: The starting pad
>> + * @iter: The iterator pad
>> + *
>> + * Iterate over all pads connected through routes from the @start pad
>> + * within an entity. The iteration will include the @start pad itself.
>> + */
>
> I still think a better name would be, well, better :-) Let's continue
> the discussion in the v5 thread to avoid splitting it.
I removed the inlining of __media_entity_next_routed_pad and renamed
'start' to 'root'. If anyone has a better name idea, let's hear it =).
But I think this at least removes the confusion of the parameter somehow
defining the start pad.
Tomi
>> +#define media_entity_for_each_routed_pad(start, iter) \
>> + for (iter = __media_entity_next_routed_pad( \
>> + start, (start)->entity->pads); \
>> + iter != NULL; \
>> + iter = __media_entity_next_routed_pad(start, iter + 1))
>> +
>> /**
>> * media_graph_walk_cleanup - Release resources used by graph walk.
>> *
>
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 18/24] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations
2021-04-29 2:28 ` Laurent Pinchart
2021-04-29 6:15 ` Tomi Valkeinen
@ 2021-05-04 7:04 ` Tomi Valkeinen
1 sibling, 0 replies; 36+ messages in thread
From: Tomi Valkeinen @ 2021-05-04 7:04 UTC (permalink / raw)
To: Laurent Pinchart
Cc: linux-media, sakari.ailus, Jacopo Mondi,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Michal Simek
On 29/04/2021 05:28, Laurent Pinchart wrote:
> Hi Tomi,
>
> Thank you for the patch.
>
> For some reason, it seems the original authorship got lost. It was fine
> in v5. I haven't checked if other patches in the series are affected.
>
> On Tue, Apr 27, 2021 at 03:45:17PM +0300, Tomi Valkeinen wrote:
>> Add support for subdev internal routing. A route is defined as a single
>> stream from a sink pad to a source pad.
>>
>> The userspace can configure the routing via two new ioctls,
>> VIDIOC_SUBDEV_G_ROUTING and VIDIOC_SUBDEV_S_ROUTING, and subdevs can
>> implement the functionality with v4l2_subdev_pad_ops.get_routing() and
>> v4l2_subdev_pad_ops.set_routing().
>>
>> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>> Signed-off-by: Michal Simek <michal.simek@xilinx.com>
>>
>> - Add sink and source streams for multiplexed links
>> - Copy the argument back in case of an error. This is needed to let the
>> caller know the number of routes.
>>
>> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
>> Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
>>
>> - Expand and refine documentation.
>> - Make the 'routes' pointer a __u64 __user pointer so that a compat32
>> version of the ioctl is not required.
>> - Add struct v4l2_subdev_krouting to be used for subdevice operations.
>>
>> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
>>
>> - Fix typecasing warnings
>> - Check sink & source pad types
>> - Add 'which' field
>>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> ---
>> drivers/media/v4l2-core/v4l2-ioctl.c | 25 +++++++++++++-
>> drivers/media/v4l2-core/v4l2-subdev.c | 46 +++++++++++++++++++++++++
>> include/media/v4l2-subdev.h | 24 ++++++++++++++
>> include/uapi/linux/v4l2-subdev.h | 48 +++++++++++++++++++++++++++
>> 4 files changed, 142 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>> index 6a5d1c6d11d6..7b1dabc23909 100644
>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>> @@ -16,6 +16,7 @@
>> #include <linux/kernel.h>
>> #include <linux/version.h>
>>
>> +#include <linux/v4l2-subdev.h>
>> #include <linux/videodev2.h>
>>
>> #include <media/v4l2-common.h>
>> @@ -3108,6 +3109,21 @@ static int check_array_args(unsigned int cmd, void *parg, size_t *array_size,
>> ret = 1;
>> break;
>> }
>> +
>> + case VIDIOC_SUBDEV_G_ROUTING:
>> + case VIDIOC_SUBDEV_S_ROUTING: {
>> + struct v4l2_subdev_routing *routing = parg;
>> +
>> + if (routing->num_routes > 256)
>> + return -EINVAL;
>> +
>> + *user_ptr = u64_to_user_ptr(routing->routes);
>> + *kernel_ptr = (void **)&routing->routes;
>> + *array_size = sizeof(struct v4l2_subdev_route)
>> + * routing->num_routes;
>> + ret = 1;
>> + break;
>> + }
>> }
>>
>> return ret;
>> @@ -3369,8 +3385,15 @@ video_usercopy(struct file *file, unsigned int orig_cmd, unsigned long arg,
>> /*
>> * Some ioctls can return an error, but still have valid
>> * results that must be returned.
>> + *
>> + * FIXME: subdev IOCTLS are partially handled here and partially in
>> + * v4l2-subdev.c and the 'always_copy' flag can only be set for IOCTLS
>> + * defined here as part of the 'v4l2_ioctls' array. As
>> + * VIDIOC_SUBDEV_G_ROUTING needs to return results to applications even
>> + * in case of failure, but it is not defined here as part of the
>> + * 'v4l2_ioctls' array, insert an ad-hoc check to address that.
>> */
>> - if (err < 0 && !always_copy)
>> + if (err < 0 && !always_copy && cmd != VIDIOC_SUBDEV_G_ROUTING)
>> goto out;
>>
>> out_array_args:
>> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
>> index 956dafab43d4..ad79ce121cee 100644
>> --- a/drivers/media/v4l2-core/v4l2-subdev.c
>> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
>> @@ -681,6 +681,52 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg)
>> case VIDIOC_SUBDEV_QUERYSTD:
>> return v4l2_subdev_call(sd, video, querystd, arg);
>>
>> + case VIDIOC_SUBDEV_G_ROUTING: {
>> + struct v4l2_subdev_routing *routing = arg;
>> + struct v4l2_subdev_krouting krouting = {
>> + .which = routing->which,
>> + .num_routes = routing->num_routes,
>> + .routes = (struct v4l2_subdev_route *)(uintptr_t)
>> + routing->routes,
>> + };
>> + int ret;
>> +
>> + ret = v4l2_subdev_call(sd, pad, get_routing, &krouting);
>> +
>> + routing->num_routes = krouting.num_routes;
>> +
>> + return ret;
>> + }
>> +
>> + case VIDIOC_SUBDEV_S_ROUTING: {
>> + struct v4l2_subdev_routing *routing = arg;
>> + struct v4l2_subdev_route *routes =
>> + (struct v4l2_subdev_route *)(uintptr_t)routing->routes;
>> + struct v4l2_subdev_krouting krouting = {};
>> + unsigned int i;
>> +
>> + if (routing->which != V4L2_SUBDEV_FORMAT_TRY && ro_subdev)
>> + return -EPERM;
>> +
>> + for (i = 0; i < routing->num_routes; ++i) {
>> + if (routes[i].sink_pad >= sd->entity.num_pads ||
>> + routes[i].source_pad >= sd->entity.num_pads)
>> + return -EINVAL;
>> +
>> + if (!(sd->entity.pads[routes[i].sink_pad].flags &
>> + MEDIA_PAD_FL_SINK) ||
>> + !(sd->entity.pads[routes[i].source_pad].flags &
>> + MEDIA_PAD_FL_SOURCE))
>> + return -EINVAL;
>> + }
>> +
>> + krouting.which = routing->which;
>> + krouting.num_routes = routing->num_routes;
>> + krouting.routes = routes;
>> +
>> + return v4l2_subdev_call(sd, pad, set_routing, &krouting);
>> + }
>> +
>> default:
>> return v4l2_subdev_call(sd, core, ioctl, cmd, arg);
>> }
>> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
>> index 70a21298020b..c45f5f0156c9 100644
>> --- a/include/media/v4l2-subdev.h
>> +++ b/include/media/v4l2-subdev.h
>> @@ -660,6 +660,22 @@ struct v4l2_subdev_pad_config {
>> struct v4l2_rect try_compose;
>> };
>>
>> +/**
>> + * struct v4l2_subdev_krouting - subdev routing table
>> + *
>> + * @which: format type (from enum v4l2_subdev_format_whence)
>> + * @routes: &struct v4l2_subdev_route
>> + * @num_routes: number of routes
>> + *
>> + * This structure is used to translate arguments received from
>> + * VIDIOC_SUBDEV_G/S_ROUTING() ioctl to subdev device drivers operations.
>> + */
>> +struct v4l2_subdev_krouting {
>> + u32 which;
>> + struct v4l2_subdev_route *routes;
>> + unsigned int num_routes;
>> +};
>> +
>> /**
>> * struct v4l2_subdev_pad_ops - v4l2-subdev pad level operations
>> *
>> @@ -721,6 +737,10 @@ struct v4l2_subdev_pad_config {
>> * applied to the hardware. The operation shall fail if the
>> * pad index it has been called on is not valid or in case of
>> * unrecoverable failures.
>> + *
>> + * @get_routing: get the subdevice routing table.
>> + * @set_routing: enable or disable data connection routes described in the
>> + * subdevice routing table.
>> */
>> struct v4l2_subdev_pad_ops {
>> int (*init_cfg)(struct v4l2_subdev *sd,
>> @@ -765,6 +785,10 @@ struct v4l2_subdev_pad_ops {
>> struct v4l2_mbus_config *config);
>> int (*set_mbus_config)(struct v4l2_subdev *sd, unsigned int pad,
>> struct v4l2_mbus_config *config);
>> + int (*get_routing)(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_krouting *route);
>> + int (*set_routing)(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_krouting *route);
>> };
>>
>> /**
>> diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h
>> index 658106f5b5dc..3a360cdcacf4 100644
>> --- a/include/uapi/linux/v4l2-subdev.h
>> +++ b/include/uapi/linux/v4l2-subdev.h
>> @@ -188,6 +188,52 @@ struct v4l2_subdev_capability {
>> /* The v4l2 sub-device video device node is registered in read-only mode. */
>> #define V4L2_SUBDEV_CAP_RO_SUBDEV 0x00000001
>>
>> +/*
>
> Shouldn't this be /** ?
>
>> + * Is the route active? An active route will start when streaming is enabled
>> + * on a video node.
>
> Even with an a priori knowledge of the topic, I'm not sure to really
> understand this, and I don't think it reflects the purpose of the flag.
>
>> + */
>> +#define V4L2_SUBDEV_ROUTE_FL_ACTIVE BIT(0)
>> +
>> +/*
>> + * Is the route immutable, i.e. can it be activated and inactivated?
>
> Maybe adding "An immutable route is always active".
Is it? I was thinking that immutable could be used as a way to have a
static routing table, but the routes could still be enabled and
disables. Say, a sensor could have pixel and metadata streams. They
would be immutable, you can't remove them, change them or add any other
streams. But you could still enable/disable them.
But the doc doesn't convey this. And I think we would still need a way
to describe what you mentioned, a route that's always there and enabled.
Tomi
^ permalink raw reply [flat|nested] 36+ messages in thread
end of thread, other threads:[~2021-05-04 7:04 UTC | newest]
Thread overview: 36+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-27 12:44 [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 01/24] media: entity: Use pad as a starting point for graph walk Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 02/24] media: entity: Use pads instead of entities in the media graph walk stack Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 03/24] media: entity: Walk the graph based on pads Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 04/24] v4l: mc: Start walk from a specific pad in use count calculation Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 05/24] media: entity: Add iterator helper for entity pads Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 06/24] media: entity: Move the pipeline from entity to pads Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 07/24] media: entity: Use pad as the starting point for a pipeline Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 08/24] media: entity: Add has_route entity operation Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 09/24] media: entity: Add media_entity_has_route() function Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 10/24] media: entity: Use routing information during graph traversal Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 11/24] media: entity: Skip link validation for pads to which there is no route to Tomi Valkeinen
2021-04-29 2:30 ` Laurent Pinchart
2021-04-27 12:45 ` [PATCH v6 12/24] media: entity: Add an iterator helper for connected pads Tomi Valkeinen
2021-04-29 2:18 ` Laurent Pinchart
2021-05-04 6:53 ` Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 13/24] media: entity: Add only connected pads to the pipeline Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 14/24] media: entity: Add debug information in graph walk route check Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 15/24] v4l: Add bus type to frame descriptors Tomi Valkeinen
2021-04-29 2:46 ` Laurent Pinchart
2021-04-27 12:45 ` [PATCH v6 16/24] v4l: Add CSI-2 bus configuration " Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 17/24] v4l: Add stream to frame descriptor Tomi Valkeinen
2021-04-29 2:50 ` Laurent Pinchart
2021-04-27 12:45 ` [PATCH v6 18/24] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations Tomi Valkeinen
2021-04-29 2:28 ` Laurent Pinchart
2021-04-29 6:15 ` Tomi Valkeinen
2021-05-04 7:04 ` Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 19/24] media: Documentation: Add GS_ROUTING documentation Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 20/24] v4l: subdev: routing kernel helper functions Tomi Valkeinen
2021-04-29 3:46 ` Laurent Pinchart
2021-04-27 12:45 ` [PATCH v6 21/24] v4l: subdev: add v4l2_subdev_get_format_dir() Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 22/24] media: uapi: add MEDIA_PAD_FL_MULTIPLEXED flag Tomi Valkeinen
2021-04-29 2:53 ` Laurent Pinchart
2021-04-27 12:45 ` [PATCH v6 23/24] v4l: subdev: Take routing information into account in link validation Tomi Valkeinen
2021-04-27 12:45 ` [PATCH v6 24/24] v4l: subdev: increase V4L2_FRAME_DESC_ENTRY_MAX to 8 Tomi Valkeinen
2021-04-29 10:28 ` [PATCH v6 00/24] v4l: subdev internal routing Tomi Valkeinen
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.