* [PATCH v11 00/36] v4l: routing and streams support
@ 2022-03-01 16:11 Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 01/36] media: entity: Use pad as a starting point for graph walk Tomi Valkeinen
` (36 more replies)
0 siblings, 37 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
Hi,
Here's v11 of the streams series (used to be "multiplexed streams").
v10 can be found from:
https://lore.kernel.org/all/20211130141536.891878-1-tomi.valkeinen@ideasonboard.com/
This series is based on the v5 of the subdev active state:
https://lore.kernel.org/all/20220301105548.305191-1-tomi.valkeinen@ideasonboard.com/
My work branch with additional drivers can be found from:
git://git.kernel.org/pub/scm/linux/kernel/git/tomba/linux.git streams/work-v11
And there's also the v4l-utils series to add support to v4l2-ctl and
media-ctl:
https://lore.kernel.org/all/20211130141815.892354-1-tomi.valkeinen@ideasonboard.com/
The main changes compared to v10:
- Rebased on the v5 of the active state series, which has different
locking system than what we had in v10 of this series.
- Moved code around in v4l2-subdev.[ch] so that the functions are inside
the correct ifdefs
- A better route validation helper v4l2_subdev_routing_validate
- Subdev enable/disable_streams ops (see "media: v4l2-subdev: Add subdev
.(enable|disable)_streams() operations")
There are still some comments I haven't addressed from the v10 review,
and I'd like to change the route flags a bit. We've also thought about
adding more support for drivers that don't need multiple streams but
would still use the new streams APIs to simplify the drivers.
However, as the v10 is already quite old, and I posted the v5 for the
active state, I thought it's better to also post the current version of
the streams series.
Tomi
Jacopo Mondi (3):
media: entity: Add iterator helper for entity pads
media: Documentation: Add GS_ROUTING documentation
media: subdev: Add for_each_active_route() macro
Laurent Pinchart (9):
media: entity: Add has_route entity operation
media: entity: Add media_entity_has_route() function
media: entity: Use routing information during graph traversal
media: subdev: Add [GS]_ROUTING subdev ioctls and operations
media: subdev: Fallback to pad config in v4l2_subdev_get_fmt()
media: subdev: add v4l2_subdev_routing_validate() helper
media: v4l2-subdev: Add v4l2_subdev_state_xlate_streams() helper
media: v4l2-subdev: Add subdev .(enable|disable)_streams() operations
media: v4l2-subdev: Add v4l2_subdev_s_stream_helper() function
Sakari Ailus (13):
media: entity: Use pad as a starting point for graph walk
media: entity: Use pads instead of entities in the media graph walk
stack
media: entity: Walk the graph based on pads
media: mc: Start walk from a specific pad in use count calculation
media: entity: Move the pipeline from entity to pads
media: entity: Use pad as the starting point for a pipeline
media: entity: Skip link validation for pads to which there is no
route
media: entity: Add an iterator helper for connected pads
media: entity: Add only connected pads to the pipeline
media: entity: Add debug information in graph walk route check
media: Add bus type to frame descriptors
media: Add CSI-2 bus configuration to frame descriptors
media: Add stream to frame descriptor
Tomi Valkeinen (11):
media: subdev: increase V4L2_FRAME_DESC_ENTRY_MAX to 8
media: add V4L2_SUBDEV_FL_MULTIPLEXED
media: add V4L2_SUBDEV_CAP_MPLEXED
media: subdev: add v4l2_subdev_has_route()
media: subdev: add v4l2_subdev_set_routing helper()
media: Documentation: add multiplexed streams documentation
media: subdev: add stream based configuration
media: subdev: use streams in v4l2_subdev_link_validate()
media: subdev: add "opposite" stream helper funcs
media: subdev: add v4l2_subdev_get_fmt() helper function
media: subdev: add v4l2_subdev_set_routing_with_fmt() helper
.clang-format | 1 +
Documentation/driver-api/media/mc-core.rst | 18 +-
.../driver-api/media/v4l2-subdev.rst | 8 +
.../userspace-api/media/v4l/dev-subdev.rst | 167 +++
.../userspace-api/media/v4l/user-func.rst | 1 +
.../v4l/vidioc-subdev-enum-frame-interval.rst | 5 +-
.../v4l/vidioc-subdev-enum-frame-size.rst | 5 +-
.../v4l/vidioc-subdev-enum-mbus-code.rst | 5 +-
.../media/v4l/vidioc-subdev-g-crop.rst | 5 +-
.../media/v4l/vidioc-subdev-g-fmt.rst | 5 +-
.../v4l/vidioc-subdev-g-frame-interval.rst | 5 +-
.../media/v4l/vidioc-subdev-g-routing.rst | 150 +++
.../media/v4l/vidioc-subdev-g-selection.rst | 5 +-
drivers/media/mc/mc-device.c | 13 +-
drivers/media/mc/mc-entity.c | 270 +++--
drivers/media/pci/intel/ipu3/ipu3-cio2-main.c | 6 +-
.../media/platform/exynos4-is/fimc-capture.c | 8 +-
.../platform/exynos4-is/fimc-isp-video.c | 8 +-
drivers/media/platform/exynos4-is/fimc-isp.c | 2 +-
drivers/media/platform/exynos4-is/fimc-lite.c | 10 +-
drivers/media/platform/exynos4-is/media-dev.c | 20 +-
drivers/media/platform/omap3isp/isp.c | 2 +-
drivers/media/platform/omap3isp/ispvideo.c | 25 +-
drivers/media/platform/omap3isp/ispvideo.h | 2 +-
.../media/platform/qcom/camss/camss-video.c | 6 +-
drivers/media/platform/rcar-vin/rcar-core.c | 16 +-
drivers/media/platform/rcar-vin/rcar-dma.c | 8 +-
.../platform/rockchip/rkisp1/rkisp1-capture.c | 6 +-
.../media/platform/s3c-camif/camif-capture.c | 6 +-
drivers/media/platform/stm32/stm32-dcmi.c | 6 +-
.../platform/sunxi/sun4i-csi/sun4i_dma.c | 6 +-
.../platform/sunxi/sun6i-csi/sun6i_video.c | 6 +-
drivers/media/platform/ti-vpe/cal-video.c | 6 +-
drivers/media/platform/vsp1/vsp1_video.c | 18 +-
drivers/media/platform/xilinx/xilinx-dma.c | 20 +-
drivers/media/platform/xilinx/xilinx-dma.h | 2 +-
.../media/test-drivers/vimc/vimc-capture.c | 6 +-
drivers/media/usb/au0828/au0828-core.c | 8 +-
drivers/media/v4l2-core/v4l2-ioctl.c | 25 +-
drivers/media/v4l2-core/v4l2-mc.c | 43 +-
drivers/media/v4l2-core/v4l2-subdev.c | 1002 ++++++++++++++++-
drivers/staging/media/imx/imx-media-utils.c | 8 +-
drivers/staging/media/ipu3/ipu3-v4l2.c | 6 +-
drivers/staging/media/omap4iss/iss.c | 2 +-
drivers/staging/media/omap4iss/iss_video.c | 40 +-
drivers/staging/media/omap4iss/iss_video.h | 2 +-
drivers/staging/media/tegra-video/tegra210.c | 6 +-
include/media/media-entity.h | 125 +-
include/media/v4l2-subdev.h | 403 ++++++-
include/uapi/linux/v4l2-subdev.h | 88 +-
50 files changed, 2273 insertions(+), 343 deletions(-)
create mode 100644 Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
--
2.25.1
^ permalink raw reply [flat|nested] 67+ messages in thread
* [PATCH v11 01/36] media: entity: Use pad as a starting point for graph walk
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 02/36] media: entity: Use pads instead of entities in the media graph walk stack Tomi Valkeinen
` (35 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
With the upcoming use of the recently added has_route() media entity op,
all the pads in an entity will no longer be considered interconnected.
This has an effect where the media graph is traversed: the starting pad
does make a difference.
Prepare for this change by using pad instead of the entity as an argument
for the graph walk operations. The actual graph traversal algorithm change
is in further patches.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
Documentation/driver-api/media/mc-core.rst | 2 +-
drivers/media/mc/mc-entity.c | 17 ++++++++---------
drivers/media/platform/exynos4-is/media-dev.c | 4 ++--
drivers/media/platform/omap3isp/ispvideo.c | 2 +-
drivers/media/platform/vsp1/vsp1_video.c | 2 +-
drivers/media/platform/xilinx/xilinx-dma.c | 2 +-
drivers/media/v4l2-core/v4l2-mc.c | 6 +++---
drivers/staging/media/omap4iss/iss_video.c | 4 ++--
include/media/media-entity.h | 10 ++++------
9 files changed, 23 insertions(+), 26 deletions(-)
diff --git a/Documentation/driver-api/media/mc-core.rst b/Documentation/driver-api/media/mc-core.rst
index 57b5bbba944e..ba0aee982124 100644
--- a/Documentation/driver-api/media/mc-core.rst
+++ b/Documentation/driver-api/media/mc-core.rst
@@ -167,7 +167,7 @@ Drivers initiate a graph traversal by calling
:c:func:`media_graph_walk_start()`
The graph structure, provided by the caller, is initialized to start graph
-traversal at the given entity.
+traversal at the given pad in an entity.
Drivers can then retrieve the next entity by calling
:c:func:`media_graph_walk_next()`
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index b411f9796191..6873dfc367d0 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -273,17 +273,16 @@ void media_graph_walk_cleanup(struct media_graph *graph)
}
EXPORT_SYMBOL_GPL(media_graph_walk_cleanup);
-void media_graph_walk_start(struct media_graph *graph,
- struct media_entity *entity)
+void media_graph_walk_start(struct media_graph *graph, struct media_pad *pad)
{
media_entity_enum_zero(&graph->ent_enum);
- media_entity_enum_set(&graph->ent_enum, entity);
+ media_entity_enum_set(&graph->ent_enum, pad->entity);
graph->top = 0;
graph->stack[graph->top].entity = NULL;
- stack_push(graph, entity);
- dev_dbg(entity->graph_obj.mdev->dev,
- "begin graph walk at '%s'\n", entity->name);
+ stack_push(graph, pad->entity);
+ dev_dbg(pad->graph_obj.mdev->dev,
+ "begin graph walk at '%s':%u\n", pad->entity->name, pad->index);
}
EXPORT_SYMBOL_GPL(media_graph_walk_start);
@@ -402,7 +401,7 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
goto error_graph_walk_start;
}
- media_graph_walk_start(&pipe->graph, entity);
+ media_graph_walk_start(&pipe->graph, entity->pads);
while ((entity = media_graph_walk_next(graph))) {
DECLARE_BITMAP(active, MEDIA_ENTITY_MAX_PADS);
@@ -486,7 +485,7 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
* Link validation on graph failed. We revert what we did and
* return the error.
*/
- media_graph_walk_start(graph, entity_err);
+ media_graph_walk_start(graph, entity_err->pads);
while ((entity_err = media_graph_walk_next(graph))) {
/* Sanity check for negative stream_count */
@@ -537,7 +536,7 @@ void __media_pipeline_stop(struct media_entity *entity)
if (WARN_ON(!pipe))
return;
- media_graph_walk_start(graph, entity);
+ media_graph_walk_start(graph, entity->pads);
while ((entity = media_graph_walk_next(graph))) {
/* Sanity check for negative stream_count */
diff --git a/drivers/media/platform/exynos4-is/media-dev.c b/drivers/media/platform/exynos4-is/media-dev.c
index 544b54e428c9..9a762779da49 100644
--- a/drivers/media/platform/exynos4-is/media-dev.c
+++ b/drivers/media/platform/exynos4-is/media-dev.c
@@ -1173,7 +1173,7 @@ static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
* through active links. This is needed as we cannot power on/off the
* subdevs in random order.
*/
- media_graph_walk_start(graph, entity);
+ media_graph_walk_start(graph, entity->pads);
while ((entity = media_graph_walk_next(graph))) {
if (!is_media_entity_v4l2_video_device(entity))
@@ -1188,7 +1188,7 @@ static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
return 0;
err:
- media_graph_walk_start(graph, entity_err);
+ media_graph_walk_start(graph, entity_err->pads);
while ((entity_err = media_graph_walk_next(graph))) {
if (!is_media_entity_v4l2_video_device(entity_err))
diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
index 8811d6dd4ee7..3c1485d59404 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -234,7 +234,7 @@ static int isp_video_get_graph_data(struct isp_video *video,
return ret;
}
- media_graph_walk_start(&graph, entity);
+ media_graph_walk_start(&graph, entity->pads);
while ((entity = media_graph_walk_next(&graph))) {
struct isp_video *__video;
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index 044eb5778820..61e4fbaba7b7 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -569,7 +569,7 @@ static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
if (ret)
return ret;
- media_graph_walk_start(&graph, entity);
+ media_graph_walk_start(&graph, entity->pads);
while ((entity = media_graph_walk_next(&graph))) {
struct v4l2_subdev *subdev;
diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index 338c3661d809..af0d55ab6c15 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -189,7 +189,7 @@ static int xvip_pipeline_validate(struct xvip_pipeline *pipe,
return ret;
}
- media_graph_walk_start(&graph, entity);
+ media_graph_walk_start(&graph, entity->pads);
while ((entity = media_graph_walk_next(&graph))) {
struct xvip_dma *dma;
diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
index b01474717dca..d215fe31b9a2 100644
--- a/drivers/media/v4l2-core/v4l2-mc.c
+++ b/drivers/media/v4l2-core/v4l2-mc.c
@@ -436,7 +436,7 @@ static int pipeline_pm_use_count(struct media_entity *entity,
{
int use = 0;
- media_graph_walk_start(graph, entity);
+ media_graph_walk_start(graph, entity->pads);
while ((entity = media_graph_walk_next(graph))) {
if (is_media_entity_v4l2_video_device(entity))
@@ -499,7 +499,7 @@ static int pipeline_pm_power(struct media_entity *entity, int change,
if (!change)
return 0;
- media_graph_walk_start(graph, entity);
+ media_graph_walk_start(graph, entity->pads);
while (!ret && (entity = media_graph_walk_next(graph)))
if (is_media_entity_v4l2_subdev(entity))
@@ -508,7 +508,7 @@ static int pipeline_pm_power(struct media_entity *entity, int change,
if (!ret)
return ret;
- media_graph_walk_start(graph, first);
+ media_graph_walk_start(graph, first->pads);
while ((first = media_graph_walk_next(graph))
&& first != entity)
diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
index d0da083deed5..760cd0ab1feb 100644
--- a/drivers/staging/media/omap4iss/iss_video.c
+++ b/drivers/staging/media/omap4iss/iss_video.c
@@ -217,7 +217,7 @@ iss_video_far_end(struct iss_video *video)
return NULL;
}
- media_graph_walk_start(&graph, entity);
+ media_graph_walk_start(&graph, entity->pads);
while ((entity = media_graph_walk_next(&graph))) {
if (entity == &video->video.entity)
@@ -892,7 +892,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
goto err_media_pipeline_start;
mutex_lock(&mdev->graph_mutex);
- media_graph_walk_start(&graph, entity);
+ media_graph_walk_start(&graph, entity->pads);
while ((entity = media_graph_walk_next(&graph)))
media_entity_enum_set(&pipe->ent_enum, entity);
mutex_unlock(&mdev->graph_mutex);
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index fea489f03d57..726dfef1d038 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -903,22 +903,20 @@ __must_check int media_graph_walk_init(
void media_graph_walk_cleanup(struct media_graph *graph);
/**
- * media_graph_walk_start - Start walking the media graph at a
- * given entity
+ * media_graph_walk_start - Start walking the media graph at a given pad
*
* @graph: Media graph structure that will be used to walk the graph
- * @entity: Starting entity
+ * @pad: Starting pad
*
* Before using this function, media_graph_walk_init() must be
* used to allocate resources used for walking the graph. This
* function initializes the graph traversal structure to walk the
- * entities graph starting at the given entity. The traversal
+ * entities graph starting at the given pad. The traversal
* structure must not be modified by the caller during graph
* traversal. After the graph walk, the resources must be released
* using media_graph_walk_cleanup().
*/
-void media_graph_walk_start(struct media_graph *graph,
- struct media_entity *entity);
+void media_graph_walk_start(struct media_graph *graph, struct media_pad *pad);
/**
* media_graph_walk_next - Get the next entity in the graph
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 02/36] media: entity: Use pads instead of entities in the media graph walk stack
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 01/36] media: entity: Use pad as a starting point for graph walk Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 03/36] media: entity: Walk the graph based on pads Tomi Valkeinen
` (34 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
Change the media graph walk stack structure to use media pads instead of
using media entities. In addition to the entity, the pad contains the
information which pad in the entity are being dealt with.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/mc/mc-entity.c | 55 ++++++++++++++++++------------------
include/media/media-entity.h | 8 +++---
2 files changed, 31 insertions(+), 32 deletions(-)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 6873dfc367d0..885918ffd762 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -210,40 +210,39 @@ EXPORT_SYMBOL_GPL(media_entity_pads_init);
* Graph traversal
*/
-static struct media_entity *
-media_entity_other(struct media_entity *entity, struct media_link *link)
+static struct media_pad *
+media_pad_other(struct media_pad *pad, struct media_link *link)
{
- if (link->source->entity == entity)
- return link->sink->entity;
+ if (link->source == pad)
+ return link->sink;
else
- return link->source->entity;
+ return link->source;
}
/* push an entity to traversal stack */
-static void stack_push(struct media_graph *graph,
- struct media_entity *entity)
+static void stack_push(struct media_graph *graph, struct media_pad *pad)
{
if (graph->top == MEDIA_ENTITY_ENUM_MAX_DEPTH - 1) {
WARN_ON(1);
return;
}
graph->top++;
- graph->stack[graph->top].link = entity->links.next;
- graph->stack[graph->top].entity = entity;
+ graph->stack[graph->top].link = pad->entity->links.next;
+ graph->stack[graph->top].pad = pad;
}
-static struct media_entity *stack_pop(struct media_graph *graph)
+static struct media_pad *stack_pop(struct media_graph *graph)
{
- struct media_entity *entity;
+ struct media_pad *pad;
- entity = graph->stack[graph->top].entity;
+ pad = graph->stack[graph->top].pad;
graph->top--;
- return entity;
+ return pad;
}
#define link_top(en) ((en)->stack[(en)->top].link)
-#define stack_top(en) ((en)->stack[(en)->top].entity)
+#define stack_top(en) ((en)->stack[(en)->top].pad)
/**
* media_graph_walk_init - Allocate resources for graph walk
@@ -279,8 +278,8 @@ void media_graph_walk_start(struct media_graph *graph, struct media_pad *pad)
media_entity_enum_set(&graph->ent_enum, pad->entity);
graph->top = 0;
- graph->stack[graph->top].entity = NULL;
- stack_push(graph, pad->entity);
+ graph->stack[graph->top].pad = NULL;
+ stack_push(graph, pad);
dev_dbg(pad->graph_obj.mdev->dev,
"begin graph walk at '%s':%u\n", pad->entity->name, pad->index);
}
@@ -288,16 +287,16 @@ EXPORT_SYMBOL_GPL(media_graph_walk_start);
static void media_graph_walk_iter(struct media_graph *graph)
{
- struct media_entity *entity = stack_top(graph);
+ struct media_pad *pad = stack_top(graph);
struct media_link *link;
- struct media_entity *next;
+ struct media_pad *next;
link = list_entry(link_top(graph), typeof(*link), list);
/* The link is not enabled so we do not follow. */
if (!(link->flags & MEDIA_LNK_FL_ENABLED)) {
link_top(graph) = link_top(graph)->next;
- dev_dbg(entity->graph_obj.mdev->dev,
+ dev_dbg(pad->graph_obj.mdev->dev,
"walk: skipping disabled link '%s':%u -> '%s':%u\n",
link->source->entity->name, link->source->index,
link->sink->entity->name, link->sink->index);
@@ -305,23 +304,23 @@ static void media_graph_walk_iter(struct media_graph *graph)
}
/* Get the entity at the other end of the link. */
- next = media_entity_other(entity, link);
+ next = media_pad_other(pad, link);
/* Has the entity already been visited? */
- if (media_entity_enum_test_and_set(&graph->ent_enum, next)) {
+ if (media_entity_enum_test_and_set(&graph->ent_enum, next->entity)) {
link_top(graph) = link_top(graph)->next;
- dev_dbg(entity->graph_obj.mdev->dev,
+ dev_dbg(pad->graph_obj.mdev->dev,
"walk: skipping entity '%s' (already seen)\n",
- next->name);
+ next->entity->name);
return;
}
/* Push the new entity to stack and start over. */
link_top(graph) = link_top(graph)->next;
stack_push(graph, next);
- dev_dbg(entity->graph_obj.mdev->dev, "walk: pushing '%s' on stack\n",
- next->name);
- lockdep_assert_held(&entity->graph_obj.mdev->graph_mutex);
+ dev_dbg(next->graph_obj.mdev->dev, "walk: pushing '%s':%u on stack\n",
+ next->entity->name, next->index);
+ lockdep_assert_held(&next->graph_obj.mdev->graph_mutex);
}
struct media_entity *media_graph_walk_next(struct media_graph *graph)
@@ -336,10 +335,10 @@ struct media_entity *media_graph_walk_next(struct media_graph *graph)
* top of the stack until no more entities on the level can be
* found.
*/
- while (link_top(graph) != &stack_top(graph)->links)
+ while (link_top(graph) != &stack_top(graph)->entity->links)
media_graph_walk_iter(graph);
- entity = stack_pop(graph);
+ entity = stack_pop(graph)->entity;
dev_dbg(entity->graph_obj.mdev->dev,
"walk: returning entity '%s'\n", entity->name);
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 726dfef1d038..56bd46dd6394 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -79,16 +79,16 @@ struct media_entity_enum {
* struct media_graph - Media graph traversal state
*
* @stack: Graph traversal stack; the stack contains information
- * on the path the media entities to be walked and the
- * links through which they were reached.
- * @stack.entity: pointer to &struct media_entity at the graph.
+ * on the media pads to be walked and the links through
+ * which they were reached.
+ * @stack.pad: pointer to &struct media_pad at the graph.
* @stack.link: pointer to &struct list_head.
* @ent_enum: Visited entities
* @top: The top of the stack
*/
struct media_graph {
struct {
- struct media_entity *entity;
+ struct media_pad *pad;
struct list_head *link;
} stack[MEDIA_ENTITY_ENUM_MAX_DEPTH];
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 03/36] media: entity: Walk the graph based on pads
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 01/36] media: entity: Use pad as a starting point for graph walk Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 02/36] media: entity: Use pads instead of entities in the media graph walk stack Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 04/36] media: mc: Start walk from a specific pad in use count calculation Tomi Valkeinen
` (33 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
Instead of iterating over graph entities during the walk, iterate the pads
through which the entity was first reached. This is required in order to
make the entity pipeline pad-based rather than entity based.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
Documentation/driver-api/media/mc-core.rst | 7 ++-
drivers/media/mc/mc-entity.c | 51 +++++++++++--------
drivers/media/platform/exynos4-is/media-dev.c | 20 ++++----
drivers/media/platform/omap3isp/ispvideo.c | 17 ++++---
drivers/media/platform/vsp1/vsp1_video.c | 12 ++---
drivers/media/platform/xilinx/xilinx-dma.c | 12 ++---
drivers/media/v4l2-core/v4l2-mc.c | 24 ++++-----
drivers/staging/media/omap4iss/iss_video.c | 36 ++++++-------
include/media/media-entity.h | 7 +--
9 files changed, 101 insertions(+), 85 deletions(-)
diff --git a/Documentation/driver-api/media/mc-core.rst b/Documentation/driver-api/media/mc-core.rst
index ba0aee982124..8a13640bed56 100644
--- a/Documentation/driver-api/media/mc-core.rst
+++ b/Documentation/driver-api/media/mc-core.rst
@@ -169,8 +169,11 @@ Drivers initiate a graph traversal by calling
The graph structure, provided by the caller, is initialized to start graph
traversal at the given pad in an entity.
-Drivers can then retrieve the next entity by calling
-:c:func:`media_graph_walk_next()`
+Drivers can then retrieve the next pad by calling
+:c:func:`media_graph_walk_next()`. Only the pad through which the entity
+is first reached is returned. If the caller is interested in knowing which
+further pads would be connected, the :c:func:`media_entity_has_route()`
+function can be used for that.
When the graph traversal is complete the function will return ``NULL``.
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 885918ffd762..a1ead81c1b0c 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -323,9 +323,9 @@ static void media_graph_walk_iter(struct media_graph *graph)
lockdep_assert_held(&next->graph_obj.mdev->graph_mutex);
}
-struct media_entity *media_graph_walk_next(struct media_graph *graph)
+struct media_pad *media_graph_walk_next(struct media_graph *graph)
{
- struct media_entity *entity;
+ struct media_pad *pad;
if (stack_top(graph) == NULL)
return NULL;
@@ -338,11 +338,11 @@ struct media_entity *media_graph_walk_next(struct media_graph *graph)
while (link_top(graph) != &stack_top(graph)->entity->links)
media_graph_walk_iter(graph);
- entity = stack_pop(graph)->entity;
- dev_dbg(entity->graph_obj.mdev->dev,
- "walk: returning entity '%s'\n", entity->name);
+ pad = stack_pop(graph);
+ dev_dbg(pad->graph_obj.mdev->dev,
+ "walk: returning pad '%s':%u\n", pad->entity->name, pad->index);
- return entity;
+ return pad;
}
EXPORT_SYMBOL_GPL(media_graph_walk_next);
@@ -390,7 +390,8 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
{
struct media_device *mdev = entity->graph_obj.mdev;
struct media_graph *graph = &pipe->graph;
- struct media_entity *entity_err = entity;
+ struct media_pad *pad = entity->pads;
+ struct media_pad *pad_err = pad;
struct media_link *link;
int ret;
@@ -400,9 +401,11 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
goto error_graph_walk_start;
}
- media_graph_walk_start(&pipe->graph, entity->pads);
+ media_graph_walk_start(&pipe->graph, pad);
+
+ while ((pad = media_graph_walk_next(graph))) {
+ struct media_entity *entity = pad->entity;
- while ((entity = media_graph_walk_next(graph))) {
DECLARE_BITMAP(active, MEDIA_ENTITY_MAX_PADS);
DECLARE_BITMAP(has_no_links, MEDIA_ENTITY_MAX_PADS);
@@ -411,7 +414,7 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
if (entity->pipe && entity->pipe != pipe) {
pr_err("Pipe active for %s. Can't start for %s\n",
entity->name,
- entity_err->name);
+ pad_err->entity->name);
ret = -EBUSY;
goto error;
}
@@ -429,26 +432,27 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
bitmap_fill(has_no_links, entity->num_pads);
list_for_each_entry(link, &entity->links, list) {
- struct media_pad *pad = link->sink->entity == entity
- ? link->sink : link->source;
+ struct media_pad *other_pad =
+ link->sink->entity == entity ?
+ link->sink : link->source;
/* Mark that a pad is connected by a link. */
- bitmap_clear(has_no_links, pad->index, 1);
+ bitmap_clear(has_no_links, other_pad->index, 1);
/*
* Pads that either do not need to connect or
* are connected through an enabled link are
* fine.
*/
- if (!(pad->flags & MEDIA_PAD_FL_MUST_CONNECT) ||
+ if (!(other_pad->flags & MEDIA_PAD_FL_MUST_CONNECT) ||
link->flags & MEDIA_LNK_FL_ENABLED)
- bitmap_set(active, pad->index, 1);
+ bitmap_set(active, other_pad->index, 1);
/*
* Link validation will only take place for
* sink ends of the link that are enabled.
*/
- if (link->sink != pad ||
+ if (link->sink != other_pad ||
!(link->flags & MEDIA_LNK_FL_ENABLED))
continue;
@@ -484,9 +488,11 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
* Link validation on graph failed. We revert what we did and
* return the error.
*/
- media_graph_walk_start(graph, entity_err->pads);
+ media_graph_walk_start(graph, pad_err);
+
+ while ((pad_err = media_graph_walk_next(graph))) {
+ struct media_entity *entity_err = pad_err->entity;
- while ((entity_err = media_graph_walk_next(graph))) {
/* Sanity check for negative stream_count */
if (!WARN_ON_ONCE(entity_err->stream_count <= 0)) {
entity_err->stream_count--;
@@ -498,7 +504,7 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
* We haven't increased stream_count further than this
* so we quit here.
*/
- if (entity_err == entity)
+ if (pad_err == pad)
break;
}
@@ -525,8 +531,9 @@ EXPORT_SYMBOL_GPL(media_pipeline_start);
void __media_pipeline_stop(struct media_entity *entity)
{
- struct media_graph *graph = &entity->pipe->graph;
struct media_pipeline *pipe = entity->pipe;
+ struct media_graph *graph = &pipe->graph;
+ struct media_pad *pad;
/*
* If the following check fails, the driver has performed an
@@ -537,7 +544,9 @@ void __media_pipeline_stop(struct media_entity *entity)
media_graph_walk_start(graph, entity->pads);
- while ((entity = media_graph_walk_next(graph))) {
+ while ((pad = media_graph_walk_next(graph))) {
+ struct media_entity *entity = pad->entity;
+
/* Sanity check for negative stream_count */
if (!WARN_ON_ONCE(entity->stream_count <= 0)) {
entity->stream_count--;
diff --git a/drivers/media/platform/exynos4-is/media-dev.c b/drivers/media/platform/exynos4-is/media-dev.c
index 9a762779da49..5dfba13cb35d 100644
--- a/drivers/media/platform/exynos4-is/media-dev.c
+++ b/drivers/media/platform/exynos4-is/media-dev.c
@@ -1164,7 +1164,7 @@ static int __fimc_md_modify_pipeline(struct media_entity *entity, bool enable)
static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
struct media_graph *graph)
{
- struct media_entity *entity_err = entity;
+ struct media_pad *pad, *pad_err = entity->pads;
int ret;
/*
@@ -1173,13 +1173,13 @@ static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
* through active links. This is needed as we cannot power on/off the
* subdevs in random order.
*/
- media_graph_walk_start(graph, entity->pads);
+ media_graph_walk_start(graph, pad_err);
- while ((entity = media_graph_walk_next(graph))) {
- if (!is_media_entity_v4l2_video_device(entity))
+ while ((pad = media_graph_walk_next(graph))) {
+ if (!is_media_entity_v4l2_video_device(pad->entity))
continue;
- ret = __fimc_md_modify_pipeline(entity, enable);
+ ret = __fimc_md_modify_pipeline(pad->entity, enable);
if (ret < 0)
goto err;
@@ -1188,15 +1188,15 @@ static int __fimc_md_modify_pipelines(struct media_entity *entity, bool enable,
return 0;
err:
- media_graph_walk_start(graph, entity_err->pads);
+ media_graph_walk_start(graph, pad_err);
- while ((entity_err = media_graph_walk_next(graph))) {
- if (!is_media_entity_v4l2_video_device(entity_err))
+ while ((pad_err = media_graph_walk_next(graph))) {
+ if (!is_media_entity_v4l2_video_device(pad_err->entity))
continue;
- __fimc_md_modify_pipeline(entity_err, !enable);
+ __fimc_md_modify_pipeline(pad_err->entity, !enable);
- if (entity_err == entity)
+ if (pad_err == pad)
break;
}
diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
index 3c1485d59404..5c1cbb1a9003 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -222,8 +222,8 @@ static int isp_video_get_graph_data(struct isp_video *video,
struct isp_pipeline *pipe)
{
struct media_graph graph;
- struct media_entity *entity = &video->video.entity;
- struct media_device *mdev = entity->graph_obj.mdev;
+ struct media_pad *pad = video->video.entity.pads;
+ struct media_device *mdev = video->video.entity.graph_obj.mdev;
struct isp_video *far_end = NULL;
int ret;
@@ -234,23 +234,24 @@ static int isp_video_get_graph_data(struct isp_video *video,
return ret;
}
- media_graph_walk_start(&graph, entity->pads);
+ media_graph_walk_start(&graph, pad);
- while ((entity = media_graph_walk_next(&graph))) {
+ while ((pad = media_graph_walk_next(&graph))) {
struct isp_video *__video;
- media_entity_enum_set(&pipe->ent_enum, entity);
+ media_entity_enum_set(&pipe->ent_enum, pad->entity);
if (far_end != NULL)
continue;
- if (entity == &video->video.entity)
+ if (pad == video->video.entity.pads)
continue;
- if (!is_media_entity_v4l2_video_device(entity))
+ if (!is_media_entity_v4l2_video_device(pad->entity))
continue;
- __video = to_isp_video(media_entity_to_video_device(entity));
+ __video = to_isp_video(media_entity_to_video_device(
+ pad->entity));
if (__video->type != video->type)
far_end = __video;
}
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index 61e4fbaba7b7..f2c36f2fdf53 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -559,8 +559,8 @@ static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
struct vsp1_video *video)
{
struct media_graph graph;
- struct media_entity *entity = &video->video.entity;
- struct media_device *mdev = entity->graph_obj.mdev;
+ struct media_pad *pad = video->video.entity.pads;
+ struct media_device *mdev = video->video.entity.graph_obj.mdev;
unsigned int i;
int ret;
@@ -569,17 +569,17 @@ static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
if (ret)
return ret;
- media_graph_walk_start(&graph, entity->pads);
+ media_graph_walk_start(&graph, pad);
- while ((entity = media_graph_walk_next(&graph))) {
+ while ((pad = media_graph_walk_next(&graph))) {
struct v4l2_subdev *subdev;
struct vsp1_rwpf *rwpf;
struct vsp1_entity *e;
- if (!is_media_entity_v4l2_subdev(entity))
+ if (!is_media_entity_v4l2_subdev(pad->entity))
continue;
- subdev = media_entity_to_v4l2_subdev(entity);
+ subdev = media_entity_to_v4l2_subdev(pad->entity);
e = to_vsp1_entity(subdev);
list_add_tail(&e->list_pipe, &pipe->entities);
e->pipe = pipe;
diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index af0d55ab6c15..d33f99c6ffa4 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -174,8 +174,8 @@ static int xvip_pipeline_validate(struct xvip_pipeline *pipe,
struct xvip_dma *start)
{
struct media_graph graph;
- struct media_entity *entity = &start->video.entity;
- struct media_device *mdev = entity->graph_obj.mdev;
+ struct media_pad *pad = start->video.entity.pads;
+ struct media_device *mdev = pad->entity->graph_obj.mdev;
unsigned int num_inputs = 0;
unsigned int num_outputs = 0;
int ret;
@@ -189,15 +189,15 @@ static int xvip_pipeline_validate(struct xvip_pipeline *pipe,
return ret;
}
- media_graph_walk_start(&graph, entity->pads);
+ media_graph_walk_start(&graph, pad);
- while ((entity = media_graph_walk_next(&graph))) {
+ while ((pad = media_graph_walk_next(&graph))) {
struct xvip_dma *dma;
- if (entity->function != MEDIA_ENT_F_IO_V4L)
+ if (pad->entity->function != MEDIA_ENT_F_IO_V4L)
continue;
- dma = to_xvip_dma(media_entity_to_video_device(entity));
+ dma = to_xvip_dma(media_entity_to_video_device(pad->entity));
if (dma->pad.flags & MEDIA_PAD_FL_SINK) {
pipe->output = dma;
diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
index d215fe31b9a2..cbeb580c6754 100644
--- a/drivers/media/v4l2-core/v4l2-mc.c
+++ b/drivers/media/v4l2-core/v4l2-mc.c
@@ -434,13 +434,14 @@ EXPORT_SYMBOL_GPL(v4l2_create_fwnode_links);
static int pipeline_pm_use_count(struct media_entity *entity,
struct media_graph *graph)
{
+ struct media_pad *pad;
int use = 0;
media_graph_walk_start(graph, entity->pads);
- while ((entity = media_graph_walk_next(graph))) {
- if (is_media_entity_v4l2_video_device(entity))
- use += entity->use_count;
+ while ((pad = media_graph_walk_next(graph))) {
+ if (is_media_entity_v4l2_video_device(pad->entity))
+ use += pad->entity->use_count;
}
return use;
@@ -493,7 +494,7 @@ static int pipeline_pm_power_one(struct media_entity *entity, int change)
static int pipeline_pm_power(struct media_entity *entity, int change,
struct media_graph *graph)
{
- struct media_entity *first = entity;
+ struct media_pad *tmp_pad, *pad;
int ret = 0;
if (!change)
@@ -501,19 +502,18 @@ static int pipeline_pm_power(struct media_entity *entity, int change,
media_graph_walk_start(graph, entity->pads);
- while (!ret && (entity = media_graph_walk_next(graph)))
- if (is_media_entity_v4l2_subdev(entity))
- ret = pipeline_pm_power_one(entity, change);
+ while (!ret && (pad = media_graph_walk_next(graph)))
+ if (is_media_entity_v4l2_subdev(pad->entity))
+ ret = pipeline_pm_power_one(pad->entity, change);
if (!ret)
return ret;
- media_graph_walk_start(graph, first->pads);
+ media_graph_walk_start(graph, entity->pads);
- while ((first = media_graph_walk_next(graph))
- && first != entity)
- if (is_media_entity_v4l2_subdev(first))
- pipeline_pm_power_one(first, -change);
+ while ((tmp_pad = media_graph_walk_next(graph)) && tmp_pad != pad)
+ if (is_media_entity_v4l2_subdev(tmp_pad->entity))
+ pipeline_pm_power_one(tmp_pad->entity, -change);
return ret;
}
diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
index 760cd0ab1feb..8c25ad73a81e 100644
--- a/drivers/staging/media/omap4iss/iss_video.c
+++ b/drivers/staging/media/omap4iss/iss_video.c
@@ -206,8 +206,8 @@ static struct iss_video *
iss_video_far_end(struct iss_video *video)
{
struct media_graph graph;
- struct media_entity *entity = &video->video.entity;
- struct media_device *mdev = entity->graph_obj.mdev;
+ struct media_pad *pad = video->video.entity.pads;
+ struct media_device *mdev = video->video.entity.graph_obj.mdev;
struct iss_video *far_end = NULL;
mutex_lock(&mdev->graph_mutex);
@@ -217,16 +217,17 @@ iss_video_far_end(struct iss_video *video)
return NULL;
}
- media_graph_walk_start(&graph, entity->pads);
+ media_graph_walk_start(&graph, pad);
- while ((entity = media_graph_walk_next(&graph))) {
- if (entity == &video->video.entity)
+ while ((pad = media_graph_walk_next(&graph))) {
+ if (pad->entity == &video->video.entity)
continue;
- if (!is_media_entity_v4l2_video_device(entity))
+ if (!is_media_entity_v4l2_video_device(pad->entity))
continue;
- far_end = to_iss_video(media_entity_to_video_device(entity));
+ far_end = to_iss_video(media_entity_to_video_device(
+ pad->entity));
if (far_end->type != video->type)
break;
@@ -853,8 +854,8 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
struct iss_video_fh *vfh = to_iss_video_fh(fh);
struct iss_video *video = video_drvdata(file);
struct media_graph graph;
- struct media_entity *entity = &video->video.entity;
- struct media_device *mdev = entity->graph_obj.mdev;
+ struct media_pad *pad = video->video.entity.pads;
+ struct media_device *mdev = video->video.entity.graph_obj.mdev;
enum iss_pipeline_state state;
struct iss_pipeline *pipe;
struct iss_video *far_end;
@@ -870,31 +871,32 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
* Start streaming on the pipeline. No link touching an entity in the
* pipeline can be activated or deactivated once streaming is started.
*/
- pipe = entity->pipe
- ? to_iss_pipeline(entity) : &video->pipe;
+ pipe = pad->entity->pipe
+ ? to_iss_pipeline(pad->entity) : &video->pipe;
pipe->external = NULL;
pipe->external_rate = 0;
pipe->external_bpp = 0;
- ret = media_entity_enum_init(&pipe->ent_enum, entity->graph_obj.mdev);
+ ret = media_entity_enum_init(&pipe->ent_enum,
+ pad->entity->graph_obj.mdev);
if (ret)
goto err_graph_walk_init;
- ret = media_graph_walk_init(&graph, entity->graph_obj.mdev);
+ ret = media_graph_walk_init(&graph, pad->entity->graph_obj.mdev);
if (ret)
goto err_graph_walk_init;
if (video->iss->pdata->set_constraints)
video->iss->pdata->set_constraints(video->iss, true);
- ret = media_pipeline_start(entity, &pipe->pipe);
+ ret = media_pipeline_start(pad->entity, &pipe->pipe);
if (ret < 0)
goto err_media_pipeline_start;
mutex_lock(&mdev->graph_mutex);
- media_graph_walk_start(&graph, entity->pads);
- while ((entity = media_graph_walk_next(&graph)))
- media_entity_enum_set(&pipe->ent_enum, entity);
+ media_graph_walk_start(&graph, pad);
+ while ((pad = media_graph_walk_next(&graph)))
+ media_entity_enum_set(&pipe->ent_enum, pad->entity);
mutex_unlock(&mdev->graph_mutex);
/*
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 56bd46dd6394..802459a5ad0a 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -927,10 +927,11 @@ void media_graph_walk_start(struct media_graph *graph, struct media_pad *pad);
* The graph structure must have been previously initialized with a call to
* media_graph_walk_start().
*
- * Return: returns the next entity in the graph or %NULL if the whole graph
- * have been traversed.
+ * Return: returns the next entity in the graph, identified by the pad through
+ * which it has been reached. If the whole graph has been traversed, return
+ * %NULL.
*/
-struct media_entity *media_graph_walk_next(struct media_graph *graph);
+struct media_pad *media_graph_walk_next(struct media_graph *graph);
/**
* media_pipeline_start - Mark a pipeline as streaming
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 04/36] media: mc: Start walk from a specific pad in use count calculation
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (2 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 03/36] media: entity: Walk the graph based on pads Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 05/36] media: entity: Add iterator helper for entity pads Tomi Valkeinen
` (32 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
With the addition of the recent has_route() media entity op, the pads of a
media entity are no longer all interconnected. This has to be taken into
account in power management.
Prepare for the addition of a helper function supporting S_ROUTING.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/v4l2-core/v4l2-mc.c | 27 +++++++++++++--------------
1 file changed, 13 insertions(+), 14 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
index cbeb580c6754..35d18ed89fa5 100644
--- a/drivers/media/v4l2-core/v4l2-mc.c
+++ b/drivers/media/v4l2-core/v4l2-mc.c
@@ -427,17 +427,16 @@ EXPORT_SYMBOL_GPL(v4l2_create_fwnode_links);
/*
* pipeline_pm_use_count - Count the number of users of a pipeline
- * @entity: The entity
+ * @pad: Any pad along the pipeline
*
* Return the total number of users of all video device nodes in the pipeline.
*/
-static int pipeline_pm_use_count(struct media_entity *entity,
- struct media_graph *graph)
+static int pipeline_pm_use_count(struct media_pad *pad,
+ struct media_graph *graph)
{
- struct media_pad *pad;
int use = 0;
- media_graph_walk_start(graph, entity->pads);
+ media_graph_walk_start(graph, pad);
while ((pad = media_graph_walk_next(graph))) {
if (is_media_entity_v4l2_video_device(pad->entity))
@@ -483,7 +482,7 @@ static int pipeline_pm_power_one(struct media_entity *entity, int change)
/*
* pipeline_pm_power - Apply power change to all entities in a pipeline
- * @entity: The entity
+ * @pad: Any pad along the pipeline
* @change: Use count change
*
* Walk the pipeline to update the use count and the power state of all non-node
@@ -491,16 +490,16 @@ static int pipeline_pm_power_one(struct media_entity *entity, int change)
*
* Return 0 on success or a negative error code on failure.
*/
-static int pipeline_pm_power(struct media_entity *entity, int change,
- struct media_graph *graph)
+static int pipeline_pm_power(struct media_pad *pad, int change,
+ struct media_graph *graph)
{
- struct media_pad *tmp_pad, *pad;
+ struct media_pad *tmp_pad, *first = pad;
int ret = 0;
if (!change)
return 0;
- media_graph_walk_start(graph, entity->pads);
+ media_graph_walk_start(graph, pad);
while (!ret && (pad = media_graph_walk_next(graph)))
if (is_media_entity_v4l2_subdev(pad->entity))
@@ -509,7 +508,7 @@ static int pipeline_pm_power(struct media_entity *entity, int change,
if (!ret)
return ret;
- media_graph_walk_start(graph, entity->pads);
+ media_graph_walk_start(graph, first);
while ((tmp_pad = media_graph_walk_next(graph)) && tmp_pad != pad)
if (is_media_entity_v4l2_subdev(tmp_pad->entity))
@@ -531,7 +530,7 @@ static int v4l2_pipeline_pm_use(struct media_entity *entity, unsigned int use)
WARN_ON(entity->use_count < 0);
/* Apply power change to connected non-nodes. */
- ret = pipeline_pm_power(entity, change, &mdev->pm_count_walk);
+ ret = pipeline_pm_power(entity->pads, change, &mdev->pm_count_walk);
if (ret < 0)
entity->use_count -= change;
@@ -557,8 +556,8 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
unsigned int notification)
{
struct media_graph *graph = &link->graph_obj.mdev->pm_count_walk;
- struct media_entity *source = link->source->entity;
- struct media_entity *sink = link->sink->entity;
+ struct media_pad *source = link->source;
+ struct media_pad *sink = link->sink;
int source_use;
int sink_use;
int ret = 0;
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 05/36] media: entity: Add iterator helper for entity pads
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (3 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 04/36] media: mc: Start walk from a specific pad in use count calculation Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 06/36] media: entity: Move the pipeline from entity to pads Tomi Valkeinen
` (31 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
From: Jacopo Mondi <jacopo+renesas@jmondi.org>
Add an iterator helper to easily cycle through all pads in an entity and
use it in media-entity and media-device code where appropriate.
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/mc/mc-device.c | 13 ++++++-------
drivers/media/mc/mc-entity.c | 11 ++++++-----
include/media/media-entity.h | 12 ++++++++++++
3 files changed, 24 insertions(+), 12 deletions(-)
diff --git a/drivers/media/mc/mc-device.c b/drivers/media/mc/mc-device.c
index cf5e459b1d96..cb569beab99e 100644
--- a/drivers/media/mc/mc-device.c
+++ b/drivers/media/mc/mc-device.c
@@ -581,7 +581,7 @@ static void __media_device_unregister_entity(struct media_entity *entity)
struct media_device *mdev = entity->graph_obj.mdev;
struct media_link *link, *tmp;
struct media_interface *intf;
- unsigned int i;
+ struct media_pad *iter;
ida_free(&mdev->entity_internal_idx, entity->internal_idx);
@@ -597,8 +597,8 @@ static void __media_device_unregister_entity(struct media_entity *entity)
__media_entity_remove_links(entity);
/* Remove all pads that belong to this entity */
- for (i = 0; i < entity->num_pads; i++)
- media_gobj_destroy(&entity->pads[i].graph_obj);
+ media_entity_for_each_pad(entity, iter)
+ media_gobj_destroy(&iter->graph_obj);
/* Remove the entity */
media_gobj_destroy(&entity->graph_obj);
@@ -617,7 +617,7 @@ int __must_check media_device_register_entity(struct media_device *mdev,
struct media_entity *entity)
{
struct media_entity_notify *notify, *next;
- unsigned int i;
+ struct media_pad *iter;
int ret;
if (entity->function == MEDIA_ENT_F_V4L2_SUBDEV_UNKNOWN ||
@@ -646,9 +646,8 @@ int __must_check media_device_register_entity(struct media_device *mdev,
media_gobj_create(mdev, MEDIA_GRAPH_ENTITY, &entity->graph_obj);
/* Initialize objects at the pads */
- for (i = 0; i < entity->num_pads; i++)
- media_gobj_create(mdev, MEDIA_GRAPH_PAD,
- &entity->pads[i].graph_obj);
+ media_entity_for_each_pad(entity, iter)
+ media_gobj_create(mdev, MEDIA_GRAPH_PAD, &iter->graph_obj);
/* invoke entity_notify callbacks */
list_for_each_entry_safe(notify, next, &mdev->entity_notify, list)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index a1ead81c1b0c..9115c21729f0 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -180,7 +180,8 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads,
struct media_pad *pads)
{
struct media_device *mdev = entity->graph_obj.mdev;
- unsigned int i;
+ struct media_pad *iter;
+ unsigned int i = 0;
if (num_pads >= MEDIA_ENTITY_MAX_PADS)
return -E2BIG;
@@ -191,12 +192,12 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads,
if (mdev)
mutex_lock(&mdev->graph_mutex);
- for (i = 0; i < num_pads; i++) {
- pads[i].entity = entity;
- pads[i].index = i;
+ media_entity_for_each_pad(entity, iter) {
+ iter->entity = entity;
+ iter->index = i++;
if (mdev)
media_gobj_create(mdev, MEDIA_GRAPH_PAD,
- &entity->pads[i].graph_obj);
+ &iter->graph_obj);
}
if (mdev)
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 802459a5ad0a..ff9f1181991e 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -1108,3 +1108,15 @@ void media_remove_intf_links(struct media_interface *intf);
(entity)->ops->operation((entity) , ##args) : -ENOIOCTLCMD)
#endif
+
+/**
+ * media_entity_for_each_pad - Iterate on all pads in an entity
+ * @entity: The entity the pads belong to
+ * @iter: The iterator pad
+ *
+ * Iterate on all pads in a media entity.
+ */
+#define media_entity_for_each_pad(entity, iter) \
+ for (iter = (entity)->pads; \
+ iter < &(entity)->pads[(entity)->num_pads]; \
+ ++iter)
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 06/36] media: entity: Move the pipeline from entity to pads
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (4 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 05/36] media: entity: Add iterator helper for entity pads Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 07/36] media: entity: Use pad as the starting point for a pipeline Tomi Valkeinen
` (30 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
This moves the pipe and stream_count fields from struct media_entity to
struct media_pad. Effectively streams become pad-specific rather than
being entity specific, allowing several independent streams to traverse a
single entity and an entity to be part of several streams.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
- Update documentation to use 'pads'
- Use the media pad iterator in media_entity.c
- Update rcar-dma.c to use the new per-pad stream count
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
- Fix cleanup in the error path of __media_pipeline_start()
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
drivers/media/mc/mc-entity.c | 68 +++++++++++--------
drivers/media/platform/exynos4-is/fimc-isp.c | 2 +-
drivers/media/platform/exynos4-is/fimc-lite.c | 2 +-
drivers/media/platform/omap3isp/isp.c | 2 +-
drivers/media/platform/omap3isp/ispvideo.c | 2 +-
drivers/media/platform/omap3isp/ispvideo.h | 2 +-
drivers/media/platform/rcar-vin/rcar-core.c | 16 +++--
drivers/media/platform/rcar-vin/rcar-dma.c | 2 +-
drivers/media/platform/xilinx/xilinx-dma.c | 2 +-
drivers/media/platform/xilinx/xilinx-dma.h | 2 +-
drivers/staging/media/imx/imx-media-utils.c | 2 +-
drivers/staging/media/omap4iss/iss.c | 2 +-
drivers/staging/media/omap4iss/iss_video.c | 2 +-
drivers/staging/media/omap4iss/iss_video.h | 2 +-
include/media/media-entity.h | 21 +++---
15 files changed, 74 insertions(+), 55 deletions(-)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 9115c21729f0..e9c508508e3e 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -406,24 +406,30 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
while ((pad = media_graph_walk_next(graph))) {
struct media_entity *entity = pad->entity;
+ bool skip_validation = pad->pipe != NULL;
+ struct media_pad *iter;
DECLARE_BITMAP(active, MEDIA_ENTITY_MAX_PADS);
DECLARE_BITMAP(has_no_links, MEDIA_ENTITY_MAX_PADS);
- entity->stream_count++;
+ ret = 0;
- if (entity->pipe && entity->pipe != pipe) {
- pr_err("Pipe active for %s. Can't start for %s\n",
- entity->name,
- pad_err->entity->name);
- ret = -EBUSY;
- goto error;
+ media_entity_for_each_pad(entity, iter) {
+ if (iter->pipe && iter->pipe != pipe) {
+ pr_err("Pipe active for %s. Can't start for %s\n",
+ entity->name, iter->entity->name);
+ ret = -EBUSY;
+ } else {
+ iter->pipe = pipe;
+ }
+ iter->stream_count++;
}
- entity->pipe = pipe;
+ if (ret)
+ goto error;
- /* Already streaming --- no need to check. */
- if (entity->stream_count > 1)
+ /* Already part of the pipeline, skip validation. */
+ if (skip_validation)
continue;
if (!entity->ops || !entity->ops->link_validate)
@@ -492,20 +498,23 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
media_graph_walk_start(graph, pad_err);
while ((pad_err = media_graph_walk_next(graph))) {
- struct media_entity *entity_err = pad_err->entity;
-
- /* Sanity check for negative stream_count */
- if (!WARN_ON_ONCE(entity_err->stream_count <= 0)) {
- entity_err->stream_count--;
- if (entity_err->stream_count == 0)
- entity_err->pipe = NULL;
+ struct media_entity *entity = pad_err->entity;
+ struct media_pad *iter;
+
+ media_entity_for_each_pad(entity, iter) {
+ /* Sanity check for negative stream_count */
+ if (!WARN_ON_ONCE(iter->stream_count <= 0)) {
+ --iter->stream_count;
+ if (iter->stream_count == 0)
+ iter->pipe = NULL;
+ }
}
/*
* We haven't increased stream_count further than this
* so we quit here.
*/
- if (pad_err == pad)
+ if (pad_err->entity == pad->entity)
break;
}
@@ -532,7 +541,7 @@ EXPORT_SYMBOL_GPL(media_pipeline_start);
void __media_pipeline_stop(struct media_entity *entity)
{
- struct media_pipeline *pipe = entity->pipe;
+ struct media_pipeline *pipe = entity->pads->pipe;
struct media_graph *graph = &pipe->graph;
struct media_pad *pad;
@@ -547,12 +556,15 @@ void __media_pipeline_stop(struct media_entity *entity)
while ((pad = media_graph_walk_next(graph))) {
struct media_entity *entity = pad->entity;
-
- /* Sanity check for negative stream_count */
- if (!WARN_ON_ONCE(entity->stream_count <= 0)) {
- entity->stream_count--;
- if (entity->stream_count == 0)
- entity->pipe = NULL;
+ struct media_pad *iter;
+
+ media_entity_for_each_pad(entity, iter) {
+ /* Sanity check for negative stream_count */
+ if (!WARN_ON_ONCE(iter->stream_count <= 0)) {
+ iter->stream_count--;
+ if (iter->stream_count == 0)
+ iter->pipe = NULL;
+ }
}
}
@@ -822,7 +834,7 @@ int __media_entity_setup_link(struct media_link *link, u32 flags)
{
const u32 mask = MEDIA_LNK_FL_ENABLED;
struct media_device *mdev;
- struct media_entity *source, *sink;
+ struct media_pad *source, *sink;
int ret = -EBUSY;
if (link == NULL)
@@ -838,8 +850,8 @@ int __media_entity_setup_link(struct media_link *link, u32 flags)
if (link->flags == flags)
return 0;
- source = link->source->entity;
- sink = link->sink->entity;
+ source = link->source;
+ sink = link->sink;
if (!(link->flags & MEDIA_LNK_FL_DYNAMIC) &&
(source->stream_count || sink->stream_count))
diff --git a/drivers/media/platform/exynos4-is/fimc-isp.c b/drivers/media/platform/exynos4-is/fimc-isp.c
index 855235bea46d..80274e29ccc5 100644
--- a/drivers/media/platform/exynos4-is/fimc-isp.c
+++ b/drivers/media/platform/exynos4-is/fimc-isp.c
@@ -226,7 +226,7 @@ static int fimc_isp_subdev_set_fmt(struct v4l2_subdev *sd,
}
}
} else {
- if (sd->entity.stream_count == 0) {
+ if (sd->entity.pads->stream_count == 0) {
if (fmt->pad == FIMC_ISP_SD_PAD_SINK) {
struct v4l2_subdev_format format = *fmt;
diff --git a/drivers/media/platform/exynos4-is/fimc-lite.c b/drivers/media/platform/exynos4-is/fimc-lite.c
index 9b7cc9564cf1..a9aa0d805736 100644
--- a/drivers/media/platform/exynos4-is/fimc-lite.c
+++ b/drivers/media/platform/exynos4-is/fimc-lite.c
@@ -1073,7 +1073,7 @@ static int fimc_lite_subdev_set_fmt(struct v4l2_subdev *sd,
mutex_lock(&fimc->lock);
if ((atomic_read(&fimc->out_path) == FIMC_IO_ISP &&
- sd->entity.stream_count > 0) ||
+ sd->entity.pads->stream_count > 0) ||
(atomic_read(&fimc->out_path) == FIMC_IO_DMA &&
vb2_is_busy(&fimc->vb_queue))) {
mutex_unlock(&fimc->lock);
diff --git a/drivers/media/platform/omap3isp/isp.c b/drivers/media/platform/omap3isp/isp.c
index 4c937f3f323e..80c0e20a0382 100644
--- a/drivers/media/platform/omap3isp/isp.c
+++ b/drivers/media/platform/omap3isp/isp.c
@@ -937,7 +937,7 @@ static int isp_pipeline_is_last(struct media_entity *me)
struct isp_pipeline *pipe;
struct media_pad *pad;
- if (!me->pipe)
+ if (!me->pads->pipe)
return 0;
pipe = to_isp_pipeline(me);
if (pipe->stream_state == ISP_PIPELINE_STREAM_STOPPED)
diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
index 5c1cbb1a9003..a8438040c4aa 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -1094,7 +1094,7 @@ isp_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
/* Start streaming on the pipeline. No link touching an entity in the
* pipeline can be activated or deactivated once streaming is started.
*/
- pipe = video->video.entity.pipe
+ pipe = video->video.entity.pads->pipe
? to_isp_pipeline(&video->video.entity) : &video->pipe;
ret = media_entity_enum_init(&pipe->ent_enum, &video->isp->media_dev);
diff --git a/drivers/media/platform/omap3isp/ispvideo.h b/drivers/media/platform/omap3isp/ispvideo.h
index a0908670c0cf..4c9c5b719ec5 100644
--- a/drivers/media/platform/omap3isp/ispvideo.h
+++ b/drivers/media/platform/omap3isp/ispvideo.h
@@ -100,7 +100,7 @@ struct isp_pipeline {
};
#define to_isp_pipeline(__e) \
- container_of((__e)->pipe, struct isp_pipeline, pipe)
+ container_of((__e)->pads->pipe, struct isp_pipeline, pipe)
static inline int isp_pipeline_ready(struct isp_pipeline *pipe)
{
diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c
index 0186ae235113..26c26d329e71 100644
--- a/drivers/media/platform/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/rcar-vin/rcar-core.c
@@ -811,13 +811,17 @@ static int rvin_csi2_link_notify(struct media_link *link, u32 flags,
return 0;
/*
- * Don't allow link changes if any entity in the graph is
- * streaming, modifying the CHSEL register fields can disrupt
- * running streams.
+ * Don't allow link changes if any stream in the graph is active as
+ * modifying the CHSEL register fields can disrupt running streams.
*/
- media_device_for_each_entity(entity, &group->mdev)
- if (entity->stream_count)
- return -EBUSY;
+ media_device_for_each_entity(entity, &group->mdev) {
+ struct media_pad *iter;
+
+ media_entity_for_each_pad(entity, iter) {
+ if (iter->stream_count)
+ return -EBUSY;
+ }
+ }
mutex_lock(&group->lock);
diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index 8136bc75e7c4..29525de50edc 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -1256,7 +1256,7 @@ static int rvin_set_stream(struct rvin_dev *vin, int on)
*/
mdev = vin->vdev.entity.graph_obj.mdev;
mutex_lock(&mdev->graph_mutex);
- pipe = sd->entity.pipe ? sd->entity.pipe : &vin->vdev.pipe;
+ pipe = sd->entity.pads->pipe ? sd->entity.pads->pipe : &vin->vdev.pipe;
ret = __media_pipeline_start(&vin->vdev.entity, pipe);
mutex_unlock(&mdev->graph_mutex);
if (ret)
diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index d33f99c6ffa4..03ee19d00041 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -402,7 +402,7 @@ static int xvip_dma_start_streaming(struct vb2_queue *vq, unsigned int count)
* Use the pipeline object embedded in the first DMA object that starts
* streaming.
*/
- pipe = dma->video.entity.pipe
+ pipe = dma->video.entity.pads->pipe
? to_xvip_pipeline(&dma->video.entity) : &dma->pipe;
ret = media_pipeline_start(&dma->video.entity, &pipe->pipe);
diff --git a/drivers/media/platform/xilinx/xilinx-dma.h b/drivers/media/platform/xilinx/xilinx-dma.h
index 2378bdae57ae..69ced71a5696 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.h
+++ b/drivers/media/platform/xilinx/xilinx-dma.h
@@ -47,7 +47,7 @@ struct xvip_pipeline {
static inline struct xvip_pipeline *to_xvip_pipeline(struct media_entity *e)
{
- return container_of(e->pipe, struct xvip_pipeline, pipe);
+ return container_of(e->pads->pipe, struct xvip_pipeline, pipe);
}
/**
diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
index 94bc866ca28c..479c88f3cd68 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -871,7 +871,7 @@ int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
__media_pipeline_stop(entity);
} else {
v4l2_subdev_call(sd, video, s_stream, 0);
- if (entity->pipe)
+ if (entity->pads->pipe)
__media_pipeline_stop(entity);
}
diff --git a/drivers/staging/media/omap4iss/iss.c b/drivers/staging/media/omap4iss/iss.c
index 68588e9dab0b..4c6f25aa8b57 100644
--- a/drivers/staging/media/omap4iss/iss.c
+++ b/drivers/staging/media/omap4iss/iss.c
@@ -548,7 +548,7 @@ static int iss_pipeline_is_last(struct media_entity *me)
struct iss_pipeline *pipe;
struct media_pad *pad;
- if (!me->pipe)
+ if (!me->pads->pipe)
return 0;
pipe = to_iss_pipeline(me);
if (pipe->stream_state == ISS_PIPELINE_STREAM_STOPPED)
diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
index 8c25ad73a81e..b74f7891711d 100644
--- a/drivers/staging/media/omap4iss/iss_video.c
+++ b/drivers/staging/media/omap4iss/iss_video.c
@@ -871,7 +871,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
* Start streaming on the pipeline. No link touching an entity in the
* pipeline can be activated or deactivated once streaming is started.
*/
- pipe = pad->entity->pipe
+ pipe = pad->pipe
? to_iss_pipeline(pad->entity) : &video->pipe;
pipe->external = NULL;
pipe->external_rate = 0;
diff --git a/drivers/staging/media/omap4iss/iss_video.h b/drivers/staging/media/omap4iss/iss_video.h
index 526281bf0051..9b8ec27bf87d 100644
--- a/drivers/staging/media/omap4iss/iss_video.h
+++ b/drivers/staging/media/omap4iss/iss_video.h
@@ -91,7 +91,7 @@ struct iss_pipeline {
};
#define to_iss_pipeline(__e) \
- container_of((__e)->pipe, struct iss_pipeline, pipe)
+ container_of((__e)->pads->pipe, struct iss_pipeline, pipe)
static inline int iss_pipeline_ready(struct iss_pipeline *pipe)
{
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index ff9f1181991e..e37074be3aa8 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -181,15 +181,24 @@ enum media_pad_signal_type {
*
* @graph_obj: Embedded structure containing the media object common data
* @entity: Entity this pad belongs to
+ * @pipe: Pipeline this pad belongs to
+ * @stream_count: Stream count for the pad
* @index: Pad index in the entity pads array, numbered from 0 to n
* @sig_type: Type of the signal inside a media pad
* @flags: Pad flags, as defined in
* :ref:`include/uapi/linux/media.h <media_header>`
* (seek for ``MEDIA_PAD_FL_*``)
+ * .. note::
+ *
+ * @stream_count reference count must never be negative, but is a signed
+ * integer on purpose: a simple ``WARN_ON(<0)`` check can be used to
+ * detect reference count bugs that would make it negative.
*/
struct media_pad {
struct media_gobj graph_obj; /* must be first field in struct */
struct media_entity *entity;
+ struct media_pipeline *pipe;
+ int stream_count;
u16 index;
enum media_pad_signal_type sig_type;
unsigned long flags;
@@ -268,9 +277,7 @@ enum media_entity_type {
* @pads: Pads array with the size defined by @num_pads.
* @links: List of data links.
* @ops: Entity operations.
- * @stream_count: Stream count for the entity.
* @use_count: Use count for the entity.
- * @pipe: Pipeline this entity belongs to.
* @info: Union with devnode information. Kept just for backward
* compatibility.
* @info.dev: Contains device major and minor info.
@@ -283,10 +290,9 @@ enum media_entity_type {
*
* .. note::
*
- * @stream_count and @use_count reference counts must never be
- * negative, but are signed integers on purpose: a simple ``WARN_ON(<0)``
- * check can be used to detect reference count bugs that would make them
- * negative.
+ * @use_count reference count must never be negative, but is a signed
+ * integer on purpose: a simple ``WARN_ON(<0)`` check can be used to
+ * detect reference count bugs that would make it negative.
*/
struct media_entity {
struct media_gobj graph_obj; /* must be first field in struct */
@@ -305,11 +311,8 @@ struct media_entity {
const struct media_entity_operations *ops;
- int stream_count;
int use_count;
- struct media_pipeline *pipe;
-
union {
struct {
u32 major;
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 07/36] media: entity: Use pad as the starting point for a pipeline
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (5 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 06/36] media: entity: Move the pipeline from entity to pads Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 08/36] media: entity: Add has_route entity operation Tomi Valkeinen
` (29 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
The pipeline has been moved from the entity to the pads; reflect this in
the media pipeline function API.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
Documentation/driver-api/media/mc-core.rst | 9 +++--
drivers/media/mc/mc-entity.c | 24 ++++++-------
drivers/media/pci/intel/ipu3/ipu3-cio2-main.c | 6 ++--
.../media/platform/exynos4-is/fimc-capture.c | 8 ++---
.../platform/exynos4-is/fimc-isp-video.c | 8 ++---
drivers/media/platform/exynos4-is/fimc-lite.c | 8 ++---
drivers/media/platform/omap3isp/ispvideo.c | 6 ++--
.../media/platform/qcom/camss/camss-video.c | 6 ++--
drivers/media/platform/rcar-vin/rcar-dma.c | 6 ++--
.../platform/rockchip/rkisp1/rkisp1-capture.c | 6 ++--
.../media/platform/s3c-camif/camif-capture.c | 6 ++--
drivers/media/platform/stm32/stm32-dcmi.c | 6 ++--
.../platform/sunxi/sun4i-csi/sun4i_dma.c | 6 ++--
.../platform/sunxi/sun6i-csi/sun6i_video.c | 6 ++--
drivers/media/platform/ti-vpe/cal-video.c | 6 ++--
drivers/media/platform/vsp1/vsp1_video.c | 6 ++--
drivers/media/platform/xilinx/xilinx-dma.c | 6 ++--
.../media/test-drivers/vimc/vimc-capture.c | 6 ++--
drivers/media/usb/au0828/au0828-core.c | 8 ++---
drivers/staging/media/imx/imx-media-utils.c | 6 ++--
drivers/staging/media/ipu3/ipu3-v4l2.c | 6 ++--
drivers/staging/media/omap4iss/iss_video.c | 6 ++--
drivers/staging/media/tegra-video/tegra210.c | 6 ++--
include/media/media-entity.h | 34 +++++++++----------
24 files changed, 99 insertions(+), 102 deletions(-)
diff --git a/Documentation/driver-api/media/mc-core.rst b/Documentation/driver-api/media/mc-core.rst
index 8a13640bed56..136047a22744 100644
--- a/Documentation/driver-api/media/mc-core.rst
+++ b/Documentation/driver-api/media/mc-core.rst
@@ -213,15 +213,14 @@ When starting streaming, drivers must notify all entities in the pipeline to
prevent link states from being modified during streaming by calling
:c:func:`media_pipeline_start()`.
-The function will mark all entities connected to the given entity through
-enabled links, either directly or indirectly, as streaming.
+The function will mark all the pads connected to the given pad through
+enabled routes and links, either directly or indirectly, as streaming.
The struct media_pipeline instance pointed to by
-the pipe argument will be stored in every entity in the pipeline.
+the pipe argument will be stored in every pad in the pipeline.
Drivers should embed the struct media_pipeline
in higher-level pipeline structures and can then access the
-pipeline through the struct media_entity
-pipe field.
+pipeline through the struct media_pad pipe field.
Calls to :c:func:`media_pipeline_start()` can be nested.
The pipeline pointer must be identical for all nested calls to the function.
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index e9c508508e3e..0a5a9feb26b8 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -386,12 +386,11 @@ EXPORT_SYMBOL_GPL(media_entity_get_fwnode_pad);
* Pipeline management
*/
-__must_check int __media_pipeline_start(struct media_entity *entity,
+__must_check int __media_pipeline_start(struct media_pad *pad,
struct media_pipeline *pipe)
{
- struct media_device *mdev = entity->graph_obj.mdev;
+ struct media_device *mdev = pad->graph_obj.mdev;
struct media_graph *graph = &pipe->graph;
- struct media_pad *pad = entity->pads;
struct media_pad *pad_err = pad;
struct media_link *link;
int ret;
@@ -526,24 +525,23 @@ __must_check int __media_pipeline_start(struct media_entity *entity,
}
EXPORT_SYMBOL_GPL(__media_pipeline_start);
-__must_check int media_pipeline_start(struct media_entity *entity,
+__must_check int media_pipeline_start(struct media_pad *pad,
struct media_pipeline *pipe)
{
- struct media_device *mdev = entity->graph_obj.mdev;
+ struct media_device *mdev = pad->graph_obj.mdev;
int ret;
mutex_lock(&mdev->graph_mutex);
- ret = __media_pipeline_start(entity, pipe);
+ ret = __media_pipeline_start(pad, pipe);
mutex_unlock(&mdev->graph_mutex);
return ret;
}
EXPORT_SYMBOL_GPL(media_pipeline_start);
-void __media_pipeline_stop(struct media_entity *entity)
+void __media_pipeline_stop(struct media_pad *pad)
{
- struct media_pipeline *pipe = entity->pads->pipe;
+ struct media_pipeline *pipe = pad->pipe;
struct media_graph *graph = &pipe->graph;
- struct media_pad *pad;
/*
* If the following check fails, the driver has performed an
@@ -552,7 +550,7 @@ void __media_pipeline_stop(struct media_entity *entity)
if (WARN_ON(!pipe))
return;
- media_graph_walk_start(graph, entity->pads);
+ media_graph_walk_start(graph, pad);
while ((pad = media_graph_walk_next(graph))) {
struct media_entity *entity = pad->entity;
@@ -574,12 +572,12 @@ void __media_pipeline_stop(struct media_entity *entity)
}
EXPORT_SYMBOL_GPL(__media_pipeline_stop);
-void media_pipeline_stop(struct media_entity *entity)
+void media_pipeline_stop(struct media_pad *pad)
{
- struct media_device *mdev = entity->graph_obj.mdev;
+ struct media_device *mdev = pad->graph_obj.mdev;
mutex_lock(&mdev->graph_mutex);
- __media_pipeline_stop(entity);
+ __media_pipeline_stop(pad);
mutex_unlock(&mdev->graph_mutex);
}
EXPORT_SYMBOL_GPL(media_pipeline_stop);
diff --git a/drivers/media/pci/intel/ipu3/ipu3-cio2-main.c b/drivers/media/pci/intel/ipu3/ipu3-cio2-main.c
index 0e9b0503b62a..075b08cbca71 100644
--- a/drivers/media/pci/intel/ipu3/ipu3-cio2-main.c
+++ b/drivers/media/pci/intel/ipu3/ipu3-cio2-main.c
@@ -984,7 +984,7 @@ static int cio2_vb2_start_streaming(struct vb2_queue *vq, unsigned int count)
return r;
}
- r = media_pipeline_start(&q->vdev.entity, &q->pipe);
+ r = media_pipeline_start(q->vdev.entity.pads, &q->pipe);
if (r)
goto fail_pipeline;
@@ -1004,7 +1004,7 @@ static int cio2_vb2_start_streaming(struct vb2_queue *vq, unsigned int count)
fail_csi2_subdev:
cio2_hw_exit(cio2, q);
fail_hw:
- media_pipeline_stop(&q->vdev.entity);
+ media_pipeline_stop(q->vdev.entity.pads);
fail_pipeline:
dev_dbg(dev, "failed to start streaming (%d)\n", r);
cio2_vb2_return_all_buffers(q, VB2_BUF_STATE_QUEUED);
@@ -1025,7 +1025,7 @@ static void cio2_vb2_stop_streaming(struct vb2_queue *vq)
cio2_hw_exit(cio2, q);
synchronize_irq(cio2->pci_dev->irq);
cio2_vb2_return_all_buffers(q, VB2_BUF_STATE_ERROR);
- media_pipeline_stop(&q->vdev.entity);
+ media_pipeline_stop(q->vdev.entity.pads);
pm_runtime_put(dev);
cio2->streaming = false;
}
diff --git a/drivers/media/platform/exynos4-is/fimc-capture.c b/drivers/media/platform/exynos4-is/fimc-capture.c
index 7ff4024003f4..eaac0ac4e406 100644
--- a/drivers/media/platform/exynos4-is/fimc-capture.c
+++ b/drivers/media/platform/exynos4-is/fimc-capture.c
@@ -524,7 +524,7 @@ static int fimc_capture_release(struct file *file)
mutex_lock(&fimc->lock);
if (close && vc->streaming) {
- media_pipeline_stop(&vc->ve.vdev.entity);
+ media_pipeline_stop(vc->ve.vdev.entity.pads);
vc->streaming = false;
}
@@ -1184,7 +1184,7 @@ static int fimc_cap_streamon(struct file *file, void *priv,
if (fimc_capture_active(fimc))
return -EBUSY;
- ret = media_pipeline_start(entity, &vc->ve.pipe->mp);
+ ret = media_pipeline_start(entity->pads, &vc->ve.pipe->mp);
if (ret < 0)
return ret;
@@ -1218,7 +1218,7 @@ static int fimc_cap_streamon(struct file *file, void *priv,
}
err_p_stop:
- media_pipeline_stop(entity);
+ media_pipeline_stop(entity->pads);
return ret;
}
@@ -1234,7 +1234,7 @@ static int fimc_cap_streamoff(struct file *file, void *priv,
return ret;
if (vc->streaming) {
- media_pipeline_stop(&vc->ve.vdev.entity);
+ media_pipeline_stop(vc->ve.vdev.entity.pads);
vc->streaming = false;
}
diff --git a/drivers/media/platform/exynos4-is/fimc-isp-video.c b/drivers/media/platform/exynos4-is/fimc-isp-video.c
index 83688a7982f7..e2862b3dcdfc 100644
--- a/drivers/media/platform/exynos4-is/fimc-isp-video.c
+++ b/drivers/media/platform/exynos4-is/fimc-isp-video.c
@@ -312,7 +312,7 @@ static int isp_video_release(struct file *file)
is_singular_file = v4l2_fh_is_singular_file(file);
if (is_singular_file && ivc->streaming) {
- media_pipeline_stop(entity);
+ media_pipeline_stop(entity->pads);
ivc->streaming = 0;
}
@@ -493,7 +493,7 @@ static int isp_video_streamon(struct file *file, void *priv,
struct media_entity *me = &ve->vdev.entity;
int ret;
- ret = media_pipeline_start(me, &ve->pipe->mp);
+ ret = media_pipeline_start(me->pads, &ve->pipe->mp);
if (ret < 0)
return ret;
@@ -508,7 +508,7 @@ static int isp_video_streamon(struct file *file, void *priv,
isp->video_capture.streaming = 1;
return 0;
p_stop:
- media_pipeline_stop(me);
+ media_pipeline_stop(me->pads);
return ret;
}
@@ -523,7 +523,7 @@ static int isp_video_streamoff(struct file *file, void *priv,
if (ret < 0)
return ret;
- media_pipeline_stop(&video->ve.vdev.entity);
+ media_pipeline_stop(video->ve.vdev.entity.pads);
video->streaming = 0;
return 0;
}
diff --git a/drivers/media/platform/exynos4-is/fimc-lite.c b/drivers/media/platform/exynos4-is/fimc-lite.c
index a9aa0d805736..7ce29bd60c91 100644
--- a/drivers/media/platform/exynos4-is/fimc-lite.c
+++ b/drivers/media/platform/exynos4-is/fimc-lite.c
@@ -516,7 +516,7 @@ static int fimc_lite_release(struct file *file)
if (v4l2_fh_is_singular_file(file) &&
atomic_read(&fimc->out_path) == FIMC_IO_DMA) {
if (fimc->streaming) {
- media_pipeline_stop(entity);
+ media_pipeline_stop(entity->pads);
fimc->streaming = false;
}
fimc_lite_stop_capture(fimc, false);
@@ -822,7 +822,7 @@ static int fimc_lite_streamon(struct file *file, void *priv,
if (fimc_lite_active(fimc))
return -EBUSY;
- ret = media_pipeline_start(entity, &fimc->ve.pipe->mp);
+ ret = media_pipeline_start(entity->pads, &fimc->ve.pipe->mp);
if (ret < 0)
return ret;
@@ -839,7 +839,7 @@ static int fimc_lite_streamon(struct file *file, void *priv,
}
err_p_stop:
- media_pipeline_stop(entity);
+ media_pipeline_stop(entity->pads);
return 0;
}
@@ -853,7 +853,7 @@ static int fimc_lite_streamoff(struct file *file, void *priv,
if (ret < 0)
return ret;
- media_pipeline_stop(&fimc->ve.vdev.entity);
+ media_pipeline_stop(fimc->ve.vdev.entity.pads);
fimc->streaming = false;
return 0;
}
diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
index a8438040c4aa..87334477f223 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -1105,7 +1105,7 @@ isp_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
pipe->l3_ick = clk_get_rate(video->isp->clock[ISP_CLK_L3_ICK]);
pipe->max_rate = pipe->l3_ick;
- ret = media_pipeline_start(&video->video.entity, &pipe->pipe);
+ ret = media_pipeline_start(video->video.entity.pads, &pipe->pipe);
if (ret < 0)
goto err_pipeline_start;
@@ -1162,7 +1162,7 @@ isp_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
return 0;
err_check_format:
- media_pipeline_stop(&video->video.entity);
+ media_pipeline_stop(video->video.entity.pads);
err_pipeline_start:
/* TODO: Implement PM QoS */
/* The DMA queue must be emptied here, otherwise CCDC interrupts that
@@ -1229,7 +1229,7 @@ isp_video_streamoff(struct file *file, void *fh, enum v4l2_buf_type type)
video->error = false;
/* TODO: Implement PM QoS */
- media_pipeline_stop(&video->video.entity);
+ media_pipeline_stop(video->video.entity.pads);
media_entity_enum_cleanup(&pipe->ent_enum);
diff --git a/drivers/media/platform/qcom/camss/camss-video.c b/drivers/media/platform/qcom/camss/camss-video.c
index 5dc1ddbe6d65..804b295bd237 100644
--- a/drivers/media/platform/qcom/camss/camss-video.c
+++ b/drivers/media/platform/qcom/camss/camss-video.c
@@ -493,7 +493,7 @@ static int video_start_streaming(struct vb2_queue *q, unsigned int count)
struct v4l2_subdev *subdev;
int ret;
- ret = media_pipeline_start(&vdev->entity, &video->pipe);
+ ret = media_pipeline_start(vdev->entity.pads, &video->pipe);
if (ret < 0)
return ret;
@@ -522,7 +522,7 @@ static int video_start_streaming(struct vb2_queue *q, unsigned int count)
return 0;
error:
- media_pipeline_stop(&vdev->entity);
+ media_pipeline_stop(vdev->entity.pads);
video->ops->flush_buffers(video, VB2_BUF_STATE_QUEUED);
@@ -553,7 +553,7 @@ static void video_stop_streaming(struct vb2_queue *q)
v4l2_subdev_call(subdev, video, s_stream, 0);
}
- media_pipeline_stop(&vdev->entity);
+ media_pipeline_stop(vdev->entity.pads);
video->ops->flush_buffers(video, VB2_BUF_STATE_ERROR);
}
diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c b/drivers/media/platform/rcar-vin/rcar-dma.c
index 29525de50edc..9f00ca4d719c 100644
--- a/drivers/media/platform/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/rcar-vin/rcar-dma.c
@@ -1240,7 +1240,7 @@ static int rvin_set_stream(struct rvin_dev *vin, int on)
sd = media_entity_to_v4l2_subdev(pad->entity);
if (!on) {
- media_pipeline_stop(&vin->vdev.entity);
+ media_pipeline_stop(vin->vdev.entity.pads);
return v4l2_subdev_call(sd, video, s_stream, 0);
}
@@ -1257,7 +1257,7 @@ static int rvin_set_stream(struct rvin_dev *vin, int on)
mdev = vin->vdev.entity.graph_obj.mdev;
mutex_lock(&mdev->graph_mutex);
pipe = sd->entity.pads->pipe ? sd->entity.pads->pipe : &vin->vdev.pipe;
- ret = __media_pipeline_start(&vin->vdev.entity, pipe);
+ ret = __media_pipeline_start(vin->vdev.entity.pads, pipe);
mutex_unlock(&mdev->graph_mutex);
if (ret)
return ret;
@@ -1266,7 +1266,7 @@ static int rvin_set_stream(struct rvin_dev *vin, int on)
if (ret == -ENOIOCTLCMD)
ret = 0;
if (ret)
- media_pipeline_stop(&vin->vdev.entity);
+ media_pipeline_stop(vin->vdev.entity.pads);
return ret;
}
diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-capture.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-capture.c
index fee2aaacb26b..311e0b406056 100644
--- a/drivers/media/platform/rockchip/rkisp1/rkisp1-capture.c
+++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-capture.c
@@ -1009,7 +1009,7 @@ static void rkisp1_vb2_stop_streaming(struct vb2_queue *queue)
rkisp1_dummy_buf_destroy(cap);
- media_pipeline_stop(&node->vdev.entity);
+ media_pipeline_stop(node->vdev.entity.pads);
mutex_unlock(&cap->rkisp1->stream_lock);
}
@@ -1023,7 +1023,7 @@ rkisp1_vb2_start_streaming(struct vb2_queue *queue, unsigned int count)
mutex_lock(&cap->rkisp1->stream_lock);
- ret = media_pipeline_start(entity, &cap->rkisp1->pipe);
+ ret = media_pipeline_start(entity->pads, &cap->rkisp1->pipe);
if (ret) {
dev_err(cap->rkisp1->dev, "start pipeline failed %d\n", ret);
goto err_ret_buffers;
@@ -1059,7 +1059,7 @@ rkisp1_vb2_start_streaming(struct vb2_queue *queue, unsigned int count)
err_destroy_dummy:
rkisp1_dummy_buf_destroy(cap);
err_pipeline_stop:
- media_pipeline_stop(entity);
+ media_pipeline_stop(entity->pads);
err_ret_buffers:
rkisp1_return_all_buffers(cap, VB2_BUF_STATE_QUEUED);
mutex_unlock(&cap->rkisp1->stream_lock);
diff --git a/drivers/media/platform/s3c-camif/camif-capture.c b/drivers/media/platform/s3c-camif/camif-capture.c
index 140854ab4dd8..0189b8a33032 100644
--- a/drivers/media/platform/s3c-camif/camif-capture.c
+++ b/drivers/media/platform/s3c-camif/camif-capture.c
@@ -848,13 +848,13 @@ static int s3c_camif_streamon(struct file *file, void *priv,
if (s3c_vp_active(vp))
return 0;
- ret = media_pipeline_start(sensor, camif->m_pipeline);
+ ret = media_pipeline_start(sensor->pads, camif->m_pipeline);
if (ret < 0)
return ret;
ret = camif_pipeline_validate(camif);
if (ret < 0) {
- media_pipeline_stop(sensor);
+ media_pipeline_stop(sensor->pads);
return ret;
}
@@ -878,7 +878,7 @@ static int s3c_camif_streamoff(struct file *file, void *priv,
ret = vb2_streamoff(&vp->vb_queue, type);
if (ret == 0)
- media_pipeline_stop(&camif->sensor.sd->entity);
+ media_pipeline_stop(camif->sensor.sd->entity.pads);
return ret;
}
diff --git a/drivers/media/platform/stm32/stm32-dcmi.c b/drivers/media/platform/stm32/stm32-dcmi.c
index c4c65d852525..8167fc99b62b 100644
--- a/drivers/media/platform/stm32/stm32-dcmi.c
+++ b/drivers/media/platform/stm32/stm32-dcmi.c
@@ -752,7 +752,7 @@ static int dcmi_start_streaming(struct vb2_queue *vq, unsigned int count)
goto err_unlocked;
}
- ret = media_pipeline_start(&dcmi->vdev->entity, &dcmi->pipeline);
+ ret = media_pipeline_start(dcmi->vdev->entity.pads, &dcmi->pipeline);
if (ret < 0) {
dev_err(dcmi->dev, "%s: Failed to start streaming, media pipeline start error (%d)\n",
__func__, ret);
@@ -866,7 +866,7 @@ static int dcmi_start_streaming(struct vb2_queue *vq, unsigned int count)
dcmi_pipeline_stop(dcmi);
err_media_pipeline_stop:
- media_pipeline_stop(&dcmi->vdev->entity);
+ media_pipeline_stop(dcmi->vdev->entity.pads);
err_pm_put:
pm_runtime_put(dcmi->dev);
@@ -893,7 +893,7 @@ static void dcmi_stop_streaming(struct vb2_queue *vq)
dcmi_pipeline_stop(dcmi);
- media_pipeline_stop(&dcmi->vdev->entity);
+ media_pipeline_stop(dcmi->vdev->entity.pads);
spin_lock_irq(&dcmi->irqlock);
diff --git a/drivers/media/platform/sunxi/sun4i-csi/sun4i_dma.c b/drivers/media/platform/sunxi/sun4i-csi/sun4i_dma.c
index 0912a1b6d525..412bf3e14611 100644
--- a/drivers/media/platform/sunxi/sun4i-csi/sun4i_dma.c
+++ b/drivers/media/platform/sunxi/sun4i-csi/sun4i_dma.c
@@ -266,7 +266,7 @@ static int sun4i_csi_start_streaming(struct vb2_queue *vq, unsigned int count)
goto err_clear_dma_queue;
}
- ret = media_pipeline_start(&csi->vdev.entity, &csi->vdev.pipe);
+ ret = media_pipeline_start(csi->vdev.entity.pads, &csi->vdev.pipe);
if (ret < 0)
goto err_free_scratch_buffer;
@@ -330,7 +330,7 @@ static int sun4i_csi_start_streaming(struct vb2_queue *vq, unsigned int count)
sun4i_csi_capture_stop(csi);
err_disable_pipeline:
- media_pipeline_stop(&csi->vdev.entity);
+ media_pipeline_stop(csi->vdev.entity.pads);
err_free_scratch_buffer:
dma_free_coherent(csi->dev, csi->scratch.size, csi->scratch.vaddr,
@@ -359,7 +359,7 @@ static void sun4i_csi_stop_streaming(struct vb2_queue *vq)
return_all_buffers(csi, VB2_BUF_STATE_ERROR);
spin_unlock_irqrestore(&csi->qlock, flags);
- media_pipeline_stop(&csi->vdev.entity);
+ media_pipeline_stop(csi->vdev.entity.pads);
dma_free_coherent(csi->dev, csi->scratch.size, csi->scratch.vaddr,
csi->scratch.paddr);
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
index 607a8d39fbe2..a4b998480d37 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c
@@ -141,7 +141,7 @@ static int sun6i_video_start_streaming(struct vb2_queue *vq, unsigned int count)
video->sequence = 0;
- ret = media_pipeline_start(&video->vdev.entity, &video->vdev.pipe);
+ ret = media_pipeline_start(video->vdev.entity.pads, &video->vdev.pipe);
if (ret < 0)
goto clear_dma_queue;
@@ -207,7 +207,7 @@ static int sun6i_video_start_streaming(struct vb2_queue *vq, unsigned int count)
stop_csi_stream:
sun6i_csi_set_stream(video->csi, false);
stop_media_pipeline:
- media_pipeline_stop(&video->vdev.entity);
+ media_pipeline_stop(video->vdev.entity.pads);
clear_dma_queue:
spin_lock_irqsave(&video->dma_queue_lock, flags);
list_for_each_entry(buf, &video->dma_queue, list)
@@ -231,7 +231,7 @@ static void sun6i_video_stop_streaming(struct vb2_queue *vq)
sun6i_csi_set_stream(video->csi, false);
- media_pipeline_stop(&video->vdev.entity);
+ media_pipeline_stop(video->vdev.entity.pads);
/* Release all active buffers */
spin_lock_irqsave(&video->dma_queue_lock, flags);
diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 3e936a2ca36c..0648e95371a9 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -711,7 +711,7 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
dma_addr_t addr;
int ret;
- ret = media_pipeline_start(&ctx->vdev.entity, &ctx->phy->pipe);
+ ret = media_pipeline_start(ctx->vdev.entity.pads, &ctx->phy->pipe);
if (ret < 0) {
ctx_err(ctx, "Failed to start media pipeline: %d\n", ret);
goto error_release_buffers;
@@ -764,7 +764,7 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
cal_ctx_unprepare(ctx);
error_pipeline:
- media_pipeline_stop(&ctx->vdev.entity);
+ media_pipeline_stop(ctx->vdev.entity.pads);
error_release_buffers:
cal_release_buffers(ctx, VB2_BUF_STATE_QUEUED);
@@ -785,7 +785,7 @@ static void cal_stop_streaming(struct vb2_queue *vq)
cal_release_buffers(ctx, VB2_BUF_STATE_ERROR);
- media_pipeline_stop(&ctx->vdev.entity);
+ media_pipeline_stop(ctx->vdev.entity.pads);
}
static const struct vb2_ops cal_video_qops = {
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index f2c36f2fdf53..978f820b0f34 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -927,7 +927,7 @@ static void vsp1_video_stop_streaming(struct vb2_queue *vq)
}
mutex_unlock(&pipe->lock);
- media_pipeline_stop(&video->video.entity);
+ media_pipeline_stop(video->video.entity.pads);
vsp1_video_release_buffers(video);
vsp1_video_pipeline_put(pipe);
}
@@ -1048,7 +1048,7 @@ vsp1_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
return PTR_ERR(pipe);
}
- ret = __media_pipeline_start(&video->video.entity, &pipe->pipe);
+ ret = __media_pipeline_start(video->video.entity.pads, &pipe->pipe);
if (ret < 0) {
mutex_unlock(&mdev->graph_mutex);
goto err_pipe;
@@ -1072,7 +1072,7 @@ vsp1_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
return 0;
err_stop:
- media_pipeline_stop(&video->video.entity);
+ media_pipeline_stop(video->video.entity.pads);
err_pipe:
vsp1_video_pipeline_put(pipe);
return ret;
diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index 03ee19d00041..f04b3d190562 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -405,7 +405,7 @@ static int xvip_dma_start_streaming(struct vb2_queue *vq, unsigned int count)
pipe = dma->video.entity.pads->pipe
? to_xvip_pipeline(&dma->video.entity) : &dma->pipe;
- ret = media_pipeline_start(&dma->video.entity, &pipe->pipe);
+ ret = media_pipeline_start(dma->video.entity.pads, &pipe->pipe);
if (ret < 0)
goto error;
@@ -431,7 +431,7 @@ static int xvip_dma_start_streaming(struct vb2_queue *vq, unsigned int count)
return 0;
error_stop:
- media_pipeline_stop(&dma->video.entity);
+ media_pipeline_stop(dma->video.entity.pads);
error:
/* Give back all queued buffers to videobuf2. */
@@ -459,7 +459,7 @@ static void xvip_dma_stop_streaming(struct vb2_queue *vq)
/* Cleanup the pipeline and mark it as being stopped. */
xvip_pipeline_cleanup(pipe);
- media_pipeline_stop(&dma->video.entity);
+ media_pipeline_stop(dma->video.entity.pads);
/* Give back all queued buffers to videobuf2. */
spin_lock_irq(&dma->queued_lock);
diff --git a/drivers/media/test-drivers/vimc/vimc-capture.c b/drivers/media/test-drivers/vimc/vimc-capture.c
index d1e2d0739c00..12abc3888824 100644
--- a/drivers/media/test-drivers/vimc/vimc-capture.c
+++ b/drivers/media/test-drivers/vimc/vimc-capture.c
@@ -247,7 +247,7 @@ static int vimc_cap_start_streaming(struct vb2_queue *vq, unsigned int count)
vcap->sequence = 0;
/* Start the media pipeline */
- ret = media_pipeline_start(entity, &vcap->stream.pipe);
+ ret = media_pipeline_start(entity->pads, &vcap->stream.pipe);
if (ret) {
vimc_cap_return_all_buffers(vcap, VB2_BUF_STATE_QUEUED);
return ret;
@@ -255,7 +255,7 @@ static int vimc_cap_start_streaming(struct vb2_queue *vq, unsigned int count)
ret = vimc_streamer_s_stream(&vcap->stream, &vcap->ved, 1);
if (ret) {
- media_pipeline_stop(entity);
+ media_pipeline_stop(entity->pads);
vimc_cap_return_all_buffers(vcap, VB2_BUF_STATE_QUEUED);
return ret;
}
@@ -274,7 +274,7 @@ static void vimc_cap_stop_streaming(struct vb2_queue *vq)
vimc_streamer_s_stream(&vcap->stream, &vcap->ved, 0);
/* Stop the media pipeline */
- media_pipeline_stop(&vcap->vdev.entity);
+ media_pipeline_stop(vcap->vdev.entity.pads);
/* Release all active buffers */
vimc_cap_return_all_buffers(vcap, VB2_BUF_STATE_ERROR);
diff --git a/drivers/media/usb/au0828/au0828-core.c b/drivers/media/usb/au0828/au0828-core.c
index caefac07af92..877e85a451cb 100644
--- a/drivers/media/usb/au0828/au0828-core.c
+++ b/drivers/media/usb/au0828/au0828-core.c
@@ -410,7 +410,7 @@ static int au0828_enable_source(struct media_entity *entity,
goto end;
}
- ret = __media_pipeline_start(entity, pipe);
+ ret = __media_pipeline_start(entity->pads, pipe);
if (ret) {
pr_err("Start Pipeline: %s->%s Error %d\n",
source->name, entity->name, ret);
@@ -501,12 +501,12 @@ static void au0828_disable_source(struct media_entity *entity)
return;
/* stop pipeline */
- __media_pipeline_stop(dev->active_link_owner);
+ __media_pipeline_stop(dev->active_link_owner->pads);
pr_debug("Pipeline stop for %s\n",
dev->active_link_owner->name);
ret = __media_pipeline_start(
- dev->active_link_user,
+ dev->active_link_user->pads,
dev->active_link_user_pipe);
if (ret) {
pr_err("Start Pipeline: %s->%s %d\n",
@@ -532,7 +532,7 @@ static void au0828_disable_source(struct media_entity *entity)
return;
/* stop pipeline */
- __media_pipeline_stop(dev->active_link_owner);
+ __media_pipeline_stop(dev->active_link_owner->pads);
pr_debug("Pipeline stop for %s\n",
dev->active_link_owner->name);
diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
index 479c88f3cd68..a785587636ea 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -863,16 +863,16 @@ int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
mutex_lock(&imxmd->md.graph_mutex);
if (on) {
- ret = __media_pipeline_start(entity, &imxmd->pipe);
+ ret = __media_pipeline_start(entity->pads, &imxmd->pipe);
if (ret)
goto out;
ret = v4l2_subdev_call(sd, video, s_stream, 1);
if (ret)
- __media_pipeline_stop(entity);
+ __media_pipeline_stop(entity->pads);
} else {
v4l2_subdev_call(sd, video, s_stream, 0);
if (entity->pads->pipe)
- __media_pipeline_stop(entity);
+ __media_pipeline_stop(entity->pads);
}
out:
diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c b/drivers/staging/media/ipu3/ipu3-v4l2.c
index 0473457b4e64..db38b061300e 100644
--- a/drivers/staging/media/ipu3/ipu3-v4l2.c
+++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
@@ -485,7 +485,7 @@ static int imgu_vb2_start_streaming(struct vb2_queue *vq, unsigned int count)
pipe = node->pipe;
imgu_pipe = &imgu->imgu_pipe[pipe];
- r = media_pipeline_start(&node->vdev.entity, &imgu_pipe->pipeline);
+ r = media_pipeline_start(node->vdev.entity.pads, &imgu_pipe->pipeline);
if (r < 0)
goto fail_return_bufs;
@@ -510,7 +510,7 @@ static int imgu_vb2_start_streaming(struct vb2_queue *vq, unsigned int count)
return 0;
fail_stop_pipeline:
- media_pipeline_stop(&node->vdev.entity);
+ media_pipeline_stop(node->vdev.entity.pads);
fail_return_bufs:
imgu_return_all_buffers(imgu, node, VB2_BUF_STATE_QUEUED);
@@ -550,7 +550,7 @@ static void imgu_vb2_stop_streaming(struct vb2_queue *vq)
imgu_return_all_buffers(imgu, node, VB2_BUF_STATE_ERROR);
mutex_unlock(&imgu->streaming_lock);
- media_pipeline_stop(&node->vdev.entity);
+ media_pipeline_stop(node->vdev.entity.pads);
}
/******************** v4l2_ioctl_ops ********************/
diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
index b74f7891711d..20fac40581c6 100644
--- a/drivers/staging/media/omap4iss/iss_video.c
+++ b/drivers/staging/media/omap4iss/iss_video.c
@@ -889,7 +889,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
if (video->iss->pdata->set_constraints)
video->iss->pdata->set_constraints(video->iss, true);
- ret = media_pipeline_start(pad->entity, &pipe->pipe);
+ ret = media_pipeline_start(pad, &pipe->pipe);
if (ret < 0)
goto err_media_pipeline_start;
@@ -980,7 +980,7 @@ iss_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
err_omap4iss_set_stream:
vb2_streamoff(&vfh->queue, type);
err_iss_video_check_format:
- media_pipeline_stop(&video->video.entity);
+ media_pipeline_stop(video->video.entity.pads);
err_media_pipeline_start:
if (video->iss->pdata->set_constraints)
video->iss->pdata->set_constraints(video->iss, false);
@@ -1034,7 +1034,7 @@ iss_video_streamoff(struct file *file, void *fh, enum v4l2_buf_type type)
if (video->iss->pdata->set_constraints)
video->iss->pdata->set_constraints(video->iss, false);
- media_pipeline_stop(&video->video.entity);
+ media_pipeline_stop(video->video.entity.pads);
done:
mutex_unlock(&video->stream_lock);
diff --git a/drivers/staging/media/tegra-video/tegra210.c b/drivers/staging/media/tegra-video/tegra210.c
index f10a041e3e6c..d2d7dd0e8624 100644
--- a/drivers/staging/media/tegra-video/tegra210.c
+++ b/drivers/staging/media/tegra-video/tegra210.c
@@ -547,7 +547,7 @@ static int tegra210_vi_start_streaming(struct vb2_queue *vq, u32 count)
VI_INCR_SYNCPT_NO_STALL);
/* start the pipeline */
- ret = media_pipeline_start(&chan->video.entity, pipe);
+ ret = media_pipeline_start(chan->video.entity.pads, pipe);
if (ret < 0)
goto error_pipeline_start;
@@ -595,7 +595,7 @@ static int tegra210_vi_start_streaming(struct vb2_queue *vq, u32 count)
error_kthread_start:
tegra_channel_set_stream(chan, false);
error_set_stream:
- media_pipeline_stop(&chan->video.entity);
+ media_pipeline_stop(chan->video.entity.pads);
error_pipeline_start:
tegra_channel_release_buffers(chan, VB2_BUF_STATE_QUEUED);
return ret;
@@ -617,7 +617,7 @@ static void tegra210_vi_stop_streaming(struct vb2_queue *vq)
tegra_channel_release_buffers(chan, VB2_BUF_STATE_ERROR);
tegra_channel_set_stream(chan, false);
- media_pipeline_stop(&chan->video.entity);
+ media_pipeline_stop(chan->video.entity.pads);
}
/*
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index e37074be3aa8..271603e93b2d 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -938,53 +938,53 @@ struct media_pad *media_graph_walk_next(struct media_graph *graph);
/**
* media_pipeline_start - Mark a pipeline as streaming
- * @entity: Starting entity
- * @pipe: Media pipeline to be assigned to all entities in the pipeline.
+ * @pad: Starting pad
+ * @pipe: Media pipeline to be assigned to all pads in the pipeline.
*
- * Mark all entities connected to a given entity through enabled links, either
- * directly or indirectly, as streaming. The given pipeline object is assigned
- * to every entity in the pipeline and stored in the media_entity pipe field.
+ * Mark all pads connected to a given pad through enabled routes or links,
+ * either directly or indirectly, as streaming. The given pipeline object is
+ * assigned to every pad in the pipeline and stored in the media_pad pipe
+ * field.
*
* Calls to this function can be nested, in which case the same number of
* media_pipeline_stop() calls will be required to stop streaming. The
* pipeline pointer must be identical for all nested calls to
* media_pipeline_start().
*/
-__must_check int media_pipeline_start(struct media_entity *entity,
+__must_check int media_pipeline_start(struct media_pad *pad,
struct media_pipeline *pipe);
/**
* __media_pipeline_start - Mark a pipeline as streaming
*
- * @entity: Starting entity
- * @pipe: Media pipeline to be assigned to all entities in the pipeline.
+ * @pad: Starting pad
+ * @pipe: Media pipeline to be assigned to all pads in the pipeline.
*
* ..note:: This is the non-locking version of media_pipeline_start()
*/
-__must_check int __media_pipeline_start(struct media_entity *entity,
+__must_check int __media_pipeline_start(struct media_pad *pad,
struct media_pipeline *pipe);
/**
* media_pipeline_stop - Mark a pipeline as not streaming
- * @entity: Starting entity
+ * @pad: Starting pad
*
- * Mark all entities connected to a given entity through enabled links, either
- * directly or indirectly, as not streaming. The media_entity pipe field is
- * reset to %NULL.
+ * Mark all pads connected to a given pad through enabled routes or links,
+ * either directly or indirectly, as not streaming.
*
* If multiple calls to media_pipeline_start() have been made, the same
* number of calls to this function are required to mark the pipeline as not
- * streaming.
+ * streaming and reset the media_pad pipe field to %NULL.
*/
-void media_pipeline_stop(struct media_entity *entity);
+void media_pipeline_stop(struct media_pad *pad);
/**
* __media_pipeline_stop - Mark a pipeline as not streaming
*
- * @entity: Starting entity
+ * @pad: Starting pad
*
* .. note:: This is the non-locking version of media_pipeline_stop()
*/
-void __media_pipeline_stop(struct media_entity *entity);
+void __media_pipeline_stop(struct media_pad *pad);
/**
* media_devnode_create() - creates and initializes a device node interface
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 08/36] media: entity: Add has_route entity operation
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (6 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 07/36] media: entity: Use pad as the starting point for a pipeline Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 09/36] media: entity: Add media_entity_has_route() function Tomi Valkeinen
` (28 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Michal Simek, Tomi Valkeinen
From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The optional operation can be used by entities to report whether two
pads are internally connected.
While at there, fix a Sphinx compiler warning in a comment block a few
lines above.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
include/media/media-entity.h | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 271603e93b2d..d74acd50361f 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -215,6 +215,10 @@ struct media_pad {
* @link_validate: Return whether a link is valid from the entity point of
* view. The media_pipeline_start() function
* validates all links by calling this operation. Optional.
+ * @has_route: Return whether a route exists inside the entity between
+ * two given pads. Pads are passed to the operation ordered
+ * by index. Optional: If the operation isn't implemented
+ * all pads will be considered as connected.
*
* .. note::
*
@@ -228,6 +232,8 @@ struct media_entity_operations {
const struct media_pad *local,
const struct media_pad *remote, u32 flags);
int (*link_validate)(struct media_link *link);
+ bool (*has_route)(struct media_entity *entity, unsigned int pad0,
+ unsigned int pad1);
};
/**
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 09/36] media: entity: Add media_entity_has_route() function
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (7 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 08/36] media: entity: Add has_route entity operation Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 10/36] media: entity: Use routing information during graph traversal Tomi Valkeinen
` (27 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Michal Simek, Tomi Valkeinen
From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
This is a wrapper around the media entity has_route operation.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/mc/mc-entity.c | 32 ++++++++++++++++++++++++++++++++
1 file changed, 32 insertions(+)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 0a5a9feb26b8..1ae9252b1b39 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -211,6 +211,38 @@ EXPORT_SYMBOL_GPL(media_entity_pads_init);
* Graph traversal
*/
+/**
+ * media_entity_has_route - Check if two entity pads are connected internally
+ *
+ * @entity: The entity
+ * @pad0: The first pad index
+ * @pad1: The second pad index
+ *
+ * This function can be used to check whether two pads of an entity are
+ * connected internally in the entity.
+ *
+ * The caller must hold entity->graph_obj.mdev->mutex.
+ *
+ * Return: true if the pads are connected internally and false otherwise.
+ */
+static bool media_entity_has_route(struct media_entity *entity,
+ unsigned int pad0, unsigned int pad1)
+{
+ if (pad0 >= entity->num_pads || pad1 >= entity->num_pads)
+ return false;
+
+ if (pad0 == pad1)
+ return true;
+
+ if (!entity->ops || !entity->ops->has_route)
+ return true;
+
+ if (entity->pads[pad1].index < entity->pads[pad0].index)
+ swap(pad0, pad1);
+
+ return entity->ops->has_route(entity, pad0, pad1);
+}
+
static struct media_pad *
media_pad_other(struct media_pad *pad, struct media_link *link)
{
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 10/36] media: entity: Use routing information during graph traversal
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (8 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 09/36] media: entity: Add media_entity_has_route() function Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 11/36] media: entity: Skip link validation for pads to which there is no route Tomi Valkeinen
` (26 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Michal Simek, Tomi Valkeinen
From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Take internal routing information as reported by the entity has_route
operation into account during graph traversal to avoid following
unrelated links.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/mc/mc-entity.c | 46 ++++++++++++++++++++++--------------
1 file changed, 28 insertions(+), 18 deletions(-)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 1ae9252b1b39..323e44d16f7b 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -243,15 +243,6 @@ static bool media_entity_has_route(struct media_entity *entity,
return entity->ops->has_route(entity, pad0, pad1);
}
-static struct media_pad *
-media_pad_other(struct media_pad *pad, struct media_link *link)
-{
- if (link->source == pad)
- return link->sink;
- else
- return link->source;
-}
-
/* push an entity to traversal stack */
static void stack_push(struct media_graph *graph, struct media_pad *pad)
{
@@ -322,7 +313,8 @@ static void media_graph_walk_iter(struct media_graph *graph)
{
struct media_pad *pad = stack_top(graph);
struct media_link *link;
- struct media_pad *next;
+ struct media_pad *remote;
+ struct media_pad *local;
link = list_entry(link_top(graph), typeof(*link), list);
@@ -336,24 +328,42 @@ static void media_graph_walk_iter(struct media_graph *graph)
return;
}
- /* Get the entity at the other end of the link. */
- next = media_pad_other(pad, link);
+ /*
+ * Get the local pad, the remote pad and the entity at the other
+ * end of the link.
+ */
+ if (link->source->entity == pad->entity) {
+ remote = link->sink;
+ local = link->source;
+ } else {
+ remote = link->source;
+ local = link->sink;
+ }
+
+ /*
+ * Are the local pad and the pad we came from connected
+ * internally in the entity ?
+ */
+ if (!media_entity_has_route(pad->entity, pad->index, local->index)) {
+ link_top(graph) = link_top(graph)->next;
+ return;
+ }
/* Has the entity already been visited? */
- if (media_entity_enum_test_and_set(&graph->ent_enum, next->entity)) {
+ if (media_entity_enum_test_and_set(&graph->ent_enum, remote->entity)) {
link_top(graph) = link_top(graph)->next;
dev_dbg(pad->graph_obj.mdev->dev,
"walk: skipping entity '%s' (already seen)\n",
- next->entity->name);
+ remote->entity->name);
return;
}
/* Push the new entity to stack and start over. */
link_top(graph) = link_top(graph)->next;
- stack_push(graph, next);
- dev_dbg(next->graph_obj.mdev->dev, "walk: pushing '%s':%u on stack\n",
- next->entity->name, next->index);
- lockdep_assert_held(&next->graph_obj.mdev->graph_mutex);
+ stack_push(graph, remote);
+ dev_dbg(remote->graph_obj.mdev->dev, "walk: pushing '%s':%u on stack\n",
+ remote->entity->name, remote->index);
+ lockdep_assert_held(&remote->graph_obj.mdev->graph_mutex);
}
struct media_pad *media_graph_walk_next(struct media_graph *graph)
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 11/36] media: entity: Skip link validation for pads to which there is no route
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (9 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 10/36] media: entity: Use routing information during graph traversal Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 12/36] media: entity: Add an iterator helper for connected pads Tomi Valkeinen
` (25 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
Links are validated along the pipeline which is about to start streaming.
Not all the pads in entities that are traversed along that pipeline are
part of the pipeline, however. Skip the link validation for such pads.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
drivers/media/mc/mc-entity.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 323e44d16f7b..6a4e1bbfd961 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -484,6 +484,11 @@ __must_check int __media_pipeline_start(struct media_pad *pad,
link->sink->entity == entity ?
link->sink : link->source;
+ /* Ignore pads to which there is no route. */
+ if (!media_entity_has_route(entity, pad->index,
+ other_pad->index))
+ continue;
+
/* Mark that a pad is connected by a link. */
bitmap_clear(has_no_links, other_pad->index, 1);
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 12/36] media: entity: Add an iterator helper for connected pads
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (10 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 11/36] media: entity: Skip link validation for pads to which there is no route Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 13/36] media: entity: Add only connected pads to the pipeline Tomi Valkeinen
` (24 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
Add a helper macro for iterating over pads that are connected through
enabled routes. This can be used to find all the connected pads within an
entity, for instance starting from the pad which has been obtained during
the graph walk.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
- Make __media_entity_next_routed_pad() return NULL and adjust the
iterator to handle that
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/mc/mc-entity.c | 14 ++++++++++++++
include/media/media-entity.h | 27 +++++++++++++++++++++++++++
2 files changed, 41 insertions(+)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 6a4e1bbfd961..4e1f4d3cbf6e 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -243,6 +243,20 @@ static bool media_entity_has_route(struct media_entity *entity,
return entity->ops->has_route(entity, pad0, pad1);
}
+struct media_pad *__media_entity_next_routed_pad(struct media_pad *root,
+ struct media_pad *iter)
+{
+ struct media_entity *entity = root->entity;
+
+ for (; iter < &entity->pads[entity->num_pads]; iter++) {
+ if (media_entity_has_route(entity, root->index, iter->index))
+ return iter;
+ }
+
+ return NULL;
+}
+EXPORT_SYMBOL_GPL(__media_entity_next_routed_pad);
+
/* push an entity to traversal stack */
static void stack_push(struct media_graph *graph, struct media_pad *pad)
{
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index d74acd50361f..b126b3f1c42b 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -904,6 +904,33 @@ int media_entity_get_fwnode_pad(struct media_entity *entity,
__must_check int media_graph_walk_init(
struct media_graph *graph, struct media_device *mdev);
+/**
+ * __media_entity_next_routed_pad - Get next pad connected to @root
+ *
+ * @root: The root pad to which the iterated pads have a route
+ * @iter: The iterator pad
+ *
+ * Get next pad which has a route to @root.
+ */
+struct media_pad *__media_entity_next_routed_pad(struct media_pad *root,
+ struct media_pad *iter);
+
+/**
+ * media_entity_for_each_routed_pad - Iterate over entity pads connected by
+ * routes
+ *
+ * @root: The root pad to which the iterated pads have a route
+ * @iter: The iterator pad
+ *
+ * Iterate over all pads of an entity which have an internal route to @root pad.
+ * The iteration will include the @root pad itself.
+ */
+#define media_entity_for_each_routed_pad(root, iter) \
+ for (iter = __media_entity_next_routed_pad(root, \
+ (root)->entity->pads); \
+ iter != NULL; \
+ iter = __media_entity_next_routed_pad(root, iter + 1))
+
/**
* media_graph_walk_cleanup - Release resources used by graph walk.
*
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 13/36] media: entity: Add only connected pads to the pipeline
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (11 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 12/36] media: entity: Add an iterator helper for connected pads Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 14/36] media: entity: Add debug information in graph walk route check Tomi Valkeinen
` (23 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
A single entity may contain multiple pipelines. Only add pads that were
connected to the pad through which the entity was reached to the pipeline.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/mc/mc-entity.c | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 4e1f4d3cbf6e..52cd2154e1bb 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -469,7 +469,7 @@ __must_check int __media_pipeline_start(struct media_pad *pad,
ret = 0;
- media_entity_for_each_pad(entity, iter) {
+ media_entity_for_each_routed_pad(pad, iter) {
if (iter->pipe && iter->pipe != pipe) {
pr_err("Pipe active for %s. Can't start for %s\n",
entity->name, iter->entity->name);
@@ -558,10 +558,9 @@ __must_check int __media_pipeline_start(struct media_pad *pad,
media_graph_walk_start(graph, pad_err);
while ((pad_err = media_graph_walk_next(graph))) {
- struct media_entity *entity = pad_err->entity;
struct media_pad *iter;
- media_entity_for_each_pad(entity, iter) {
+ media_entity_for_each_routed_pad(pad_err, iter) {
/* Sanity check for negative stream_count */
if (!WARN_ON_ONCE(iter->stream_count <= 0)) {
--iter->stream_count;
@@ -614,10 +613,9 @@ void __media_pipeline_stop(struct media_pad *pad)
media_graph_walk_start(graph, pad);
while ((pad = media_graph_walk_next(graph))) {
- struct media_entity *entity = pad->entity;
struct media_pad *iter;
- media_entity_for_each_pad(entity, iter) {
+ media_entity_for_each_routed_pad(pad, iter) {
/* Sanity check for negative stream_count */
if (!WARN_ON_ONCE(iter->stream_count <= 0)) {
iter->stream_count--;
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 14/36] media: entity: Add debug information in graph walk route check
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (12 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 13/36] media: entity: Add only connected pads to the pipeline Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 15/36] media: Add bus type to frame descriptors Tomi Valkeinen
` (22 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
Add debug printout in graph walk route check.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/mc/mc-entity.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 52cd2154e1bb..1c4f6cfcd3a1 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -360,6 +360,9 @@ static void media_graph_walk_iter(struct media_graph *graph)
*/
if (!media_entity_has_route(pad->entity, pad->index, local->index)) {
link_top(graph) = link_top(graph)->next;
+ dev_dbg(pad->graph_obj.mdev->dev,
+ "walk: skipping \"%s\":%u -> %u (no route)\n",
+ pad->entity->name, pad->index, local->index);
return;
}
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 15/36] media: Add bus type to frame descriptors
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (13 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 14/36] media: entity: Add debug information in graph walk route check Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 16/36] media: Add CSI-2 bus configuration " Tomi Valkeinen
` (21 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
Add the media bus type to the frame descriptor. CSI-2 specific
information will be added in next patch to the frame descriptor.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
- Make the bus type a named enum
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
include/media/v4l2-subdev.h | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 71d13d160d99..3e3d8cfa7df4 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -344,12 +344,32 @@ struct v4l2_mbus_frame_desc_entry {
#define V4L2_FRAME_DESC_ENTRY_MAX 4
+/**
+ * enum v4l2_mbus_frame_desc_type - media bus frame description type
+ *
+ * @V4L2_MBUS_FRAME_DESC_TYPE_UNDEFINED:
+ * Undefined frame desc type. Drivers should not use this, it is
+ * for backwards compatibility.
+ * @V4L2_MBUS_FRAME_DESC_TYPE_PARALLEL:
+ * Parallel media bus.
+ * @V4L2_MBUS_FRAME_DESC_TYPE_CSI2:
+ * CSI-2 media bus. Frame desc parameters must be set in
+ * &struct v4l2_mbus_frame_desc_entry->csi2.
+ */
+enum v4l2_mbus_frame_desc_type {
+ V4L2_MBUS_FRAME_DESC_TYPE_UNDEFINED = 0,
+ V4L2_MBUS_FRAME_DESC_TYPE_PARALLEL,
+ V4L2_MBUS_FRAME_DESC_TYPE_CSI2,
+};
+
/**
* struct v4l2_mbus_frame_desc - media bus data frame description
+ * @type: type of the bus (enum v4l2_mbus_frame_desc_type)
* @entry: frame descriptors array
* @num_entries: number of entries in @entry array
*/
struct v4l2_mbus_frame_desc {
+ enum v4l2_mbus_frame_desc_type type;
struct v4l2_mbus_frame_desc_entry entry[V4L2_FRAME_DESC_ENTRY_MAX];
unsigned short num_entries;
};
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 16/36] media: Add CSI-2 bus configuration to frame descriptors
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (14 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 15/36] media: Add bus type to frame descriptors Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 17/36] media: Add stream to frame descriptor Tomi Valkeinen
` (20 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
Add CSI-2 bus specific configuration to the frame descriptors. This allows
obtaining the virtual channel and data type information for each stream
the transmitter is sending.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
include/media/v4l2-subdev.h | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 3e3d8cfa7df4..4758dea60cfb 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -312,6 +312,17 @@ struct v4l2_subdev_audio_ops {
int (*s_stream)(struct v4l2_subdev *sd, int enable);
};
+/**
+ * struct v4l2_mbus_frame_desc_entry_csi2
+ *
+ * @vc: CSI-2 virtual channel
+ * @dt: CSI-2 data type ID
+ */
+struct v4l2_mbus_frame_desc_entry_csi2 {
+ u8 vc;
+ u8 dt;
+};
+
/**
* enum v4l2_mbus_frame_desc_flags - media bus frame description flags
*
@@ -335,11 +346,16 @@ enum v4l2_mbus_frame_desc_flags {
* %FRAME_DESC_FL_BLOB is not set.
* @length: number of octets per frame, valid if @flags
* %V4L2_MBUS_FRAME_DESC_FL_LEN_MAX is set.
+ * @bus: Bus-specific frame descriptor parameters
+ * @bus.csi2: CSI-2-specific bus configuration
*/
struct v4l2_mbus_frame_desc_entry {
enum v4l2_mbus_frame_desc_flags flags;
u32 pixelcode;
u32 length;
+ union {
+ struct v4l2_mbus_frame_desc_entry_csi2 csi2;
+ } bus;
};
#define V4L2_FRAME_DESC_ENTRY_MAX 4
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 17/36] media: Add stream to frame descriptor
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (15 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 16/36] media: Add CSI-2 bus configuration " Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 18/36] media: subdev: increase V4L2_FRAME_DESC_ENTRY_MAX to 8 Tomi Valkeinen
` (19 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
From: Sakari Ailus <sakari.ailus@linux.intel.com>
The stream field identifies the stream this frame descriptor applies to in
routing configuration across a multiplexed link.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
include/media/v4l2-subdev.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 4758dea60cfb..eb7433b27043 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -342,6 +342,7 @@ enum v4l2_mbus_frame_desc_flags {
* struct v4l2_mbus_frame_desc_entry - media bus frame description structure
*
* @flags: bitmask flags, as defined by &enum v4l2_mbus_frame_desc_flags.
+ * @stream: stream in routing configuration
* @pixelcode: media bus pixel code, valid if @flags
* %FRAME_DESC_FL_BLOB is not set.
* @length: number of octets per frame, valid if @flags
@@ -351,6 +352,7 @@ enum v4l2_mbus_frame_desc_flags {
*/
struct v4l2_mbus_frame_desc_entry {
enum v4l2_mbus_frame_desc_flags flags;
+ u32 stream;
u32 pixelcode;
u32 length;
union {
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 18/36] media: subdev: increase V4L2_FRAME_DESC_ENTRY_MAX to 8
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (16 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 17/36] media: Add stream to frame descriptor Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 19/36] media: add V4L2_SUBDEV_FL_MULTIPLEXED Tomi Valkeinen
` (18 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
V4L2_FRAME_DESC_ENTRY_MAX is currently set to 4. In theory it's possible
to have an arbitrary amount of streams in a single pad, so preferably
there should be no hardcoded maximum number.
However, I believe a reasonable max is 8, which would cover a CSI-2 pad
with 4 streams of pixel data and 4 streams of metadata.
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
---
include/media/v4l2-subdev.h | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index eb7433b27043..6c28b0fa4fe0 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -360,7 +360,11 @@ struct v4l2_mbus_frame_desc_entry {
} bus;
};
-#define V4L2_FRAME_DESC_ENTRY_MAX 4
+ /*
+ * FIXME: If this number is too small, it should be dropped altogether and the
+ * API switched to a dynamic number of frame descriptor entries.
+ */
+#define V4L2_FRAME_DESC_ENTRY_MAX 8
/**
* enum v4l2_mbus_frame_desc_type - media bus frame description type
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 19/36] media: add V4L2_SUBDEV_FL_MULTIPLEXED
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (17 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 18/36] media: subdev: increase V4L2_FRAME_DESC_ENTRY_MAX to 8 Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 20/36] media: add V4L2_SUBDEV_CAP_MPLEXED Tomi Valkeinen
` (17 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
Add subdev flag V4L2_SUBDEV_FL_MULTIPLEXED. It is used to indicate that
the subdev supports the new API with multiplexed streams (routing,
stream configs).
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
include/media/v4l2-subdev.h | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 6c28b0fa4fe0..ccb4ce9ea6ff 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -881,6 +881,17 @@ struct v4l2_subdev_internal_ops {
* should set this flag.
*/
#define V4L2_SUBDEV_FL_HAS_EVENTS (1U << 3)
+/*
+ * Set this flag if this subdev supports multiplexed streams. This means
+ * that the driver supports routing and handles the stream parameter in its
+ * v4l2_subdev_pad_ops handlers. More specifically, this means:
+ *
+ * - Centrally managed active state is enabled
+ * - Legacy pad config is _not_ supported (state->pads)
+ * - Routing ioctls are available
+ * - Multiple streams per pad are supported
+ */
+#define V4L2_SUBDEV_FL_MULTIPLEXED (1U << 4)
struct regulator_bulk_data;
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 20/36] media: add V4L2_SUBDEV_CAP_MPLEXED
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (18 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 19/36] media: add V4L2_SUBDEV_FL_MULTIPLEXED Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 21/36] media: Documentation: Add GS_ROUTING documentation Tomi Valkeinen
` (16 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
Add a subdev capability flag to expose to userspace if a subdev supports
multiplexed streams.
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/v4l2-core/v4l2-subdev.c | 4 +++-
include/uapi/linux/v4l2-subdev.h | 3 +++
2 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index fda0925afeb3..3ad24093abe9 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -399,7 +399,9 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
memset(cap->reserved, 0, sizeof(cap->reserved));
cap->version = LINUX_VERSION_CODE;
- cap->capabilities = ro_subdev ? V4L2_SUBDEV_CAP_RO_SUBDEV : 0;
+ cap->capabilities =
+ (ro_subdev ? V4L2_SUBDEV_CAP_RO_SUBDEV : 0) |
+ ((sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED) ? V4L2_SUBDEV_CAP_MPLEXED : 0);
return 0;
}
diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h
index 658106f5b5dc..d91ab6f22fa7 100644
--- a/include/uapi/linux/v4l2-subdev.h
+++ b/include/uapi/linux/v4l2-subdev.h
@@ -188,6 +188,9 @@ struct v4l2_subdev_capability {
/* The v4l2 sub-device video device node is registered in read-only mode. */
#define V4L2_SUBDEV_CAP_RO_SUBDEV 0x00000001
+/* The v4l2 sub-device supports multiplexed streams. */
+#define V4L2_SUBDEV_CAP_MPLEXED 0x00000002
+
/* Backwards compatibility define --- to be removed */
#define v4l2_subdev_edid v4l2_edid
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 21/36] media: Documentation: Add GS_ROUTING documentation
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (19 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 20/36] media: add V4L2_SUBDEV_CAP_MPLEXED Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 22/36] media: subdev: Add [GS]_ROUTING subdev ioctls and operations Tomi Valkeinen
` (15 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
From: Jacopo Mondi <jacopo+renesas@jmondi.org>
Add documentation for VIDIOC_SUBDEV_G/S_ROUTING ioctl and add
description of multiplexed media pads and internal routing to the
V4L2-subdev documentation section.
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
---
.../userspace-api/media/v4l/dev-subdev.rst | 2 +
.../userspace-api/media/v4l/user-func.rst | 1 +
.../media/v4l/vidioc-subdev-g-routing.rst | 150 ++++++++++++++++++
3 files changed, 153 insertions(+)
create mode 100644 Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
index fd1de0a73a9f..a67c2749089a 100644
--- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
+++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
@@ -29,6 +29,8 @@ will feature a character device node on which ioctls can be called to
- negotiate image formats on individual pads
+- inspect and modify internal data routing between pads of the same entity
+
Sub-device character device nodes, conventionally named
``/dev/v4l-subdev*``, use major number 81.
diff --git a/Documentation/userspace-api/media/v4l/user-func.rst b/Documentation/userspace-api/media/v4l/user-func.rst
index 53e604bd7d60..228c1521f190 100644
--- a/Documentation/userspace-api/media/v4l/user-func.rst
+++ b/Documentation/userspace-api/media/v4l/user-func.rst
@@ -70,6 +70,7 @@ Function Reference
vidioc-subdev-g-crop
vidioc-subdev-g-fmt
vidioc-subdev-g-frame-interval
+ vidioc-subdev-g-routing
vidioc-subdev-g-selection
vidioc-subdev-querycap
vidioc-subscribe-event
diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
new file mode 100644
index 000000000000..a0d9c79e162f
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
@@ -0,0 +1,150 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+.. c:namespace:: V4L
+
+.. _VIDIOC_SUBDEV_G_ROUTING:
+
+******************************************************
+ioctl VIDIOC_SUBDEV_G_ROUTING, VIDIOC_SUBDEV_S_ROUTING
+******************************************************
+
+Name
+====
+
+VIDIOC_SUBDEV_G_ROUTING - VIDIOC_SUBDEV_S_ROUTING - Get or set routing between streams of media pads in a media entity.
+
+
+Synopsis
+========
+
+.. c:function:: int ioctl( int fd, VIDIOC_SUBDEV_G_ROUTING, struct v4l2_subdev_routing *argp )
+ :name: VIDIOC_SUBDEV_G_ROUTING
+
+.. c:function:: int ioctl( int fd, VIDIOC_SUBDEV_S_ROUTING, struct v4l2_subdev_routing *argp )
+ :name: VIDIOC_SUBDEV_S_ROUTING
+
+
+Arguments
+=========
+
+``fd``
+ File descriptor returned by :ref:`open() <func-open>`.
+
+``argp``
+ Pointer to struct :c:type:`v4l2_subdev_routing`.
+
+
+Description
+===========
+
+These ioctls are used to get and set the routing in a media entity.
+The routing configuration determines the flows of data inside an entity.
+
+Drivers report their current routing tables using the
+``VIDIOC_SUBDEV_G_ROUTING`` ioctl and application may enable or disable routes
+with the ``VIDIOC_SUBDEV_S_ROUTING`` ioctl, by adding or removing routes and
+setting or clearing flags of the ``flags`` field of a
+struct :c:type:`v4l2_subdev_route`.
+
+All stream configurations are reset when ``VIDIOC_SUBDEV_S_ROUTING`` is called. This
+means that the userspace mut reconfigure all streams after calling the ioctl
+with e.g. ``VIDIOC_SUBDEV_S_FMT``.
+
+A special case for routing are routes marked with
+``V4L2_SUBDEV_ROUTE_FL_SOURCE`` flag. These routes are used to describe
+source endpoints on sensors and the sink fields are unused.
+
+When inspecting routes through ``VIDIOC_SUBDEV_G_ROUTING`` and the application
+provided ``num_routes`` is not big enough to contain all the available routes
+the subdevice exposes, drivers return the ENOSPC error code and adjust the
+value of the ``num_routes`` field. Application should then reserve enough memory
+for all the route entries and call ``VIDIOC_SUBDEV_G_ROUTING`` again.
+
+.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}|
+
+.. c:type:: v4l2_subdev_routing
+
+.. flat-table:: struct v4l2_subdev_routing
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - __u32
+ - ``which``
+ - Format to modified, from enum
+ :ref:`v4l2_subdev_format_whence <v4l2-subdev-format-whence>`.
+ * - struct :c:type:`v4l2_subdev_route`
+ - ``routes[]``
+ - Array of struct :c:type:`v4l2_subdev_route` entries
+ * - __u32
+ - ``num_routes``
+ - Number of entries of the routes array
+ * - __u32
+ - ``reserved``\ [5]
+ - Reserved for future extensions. Applications and drivers must set
+ the array to zero.
+
+.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}|
+
+.. c:type:: v4l2_subdev_route
+
+.. flat-table:: struct v4l2_subdev_route
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - __u32
+ - ``sink_pad``
+ - Sink pad number.
+ * - __u32
+ - ``sink_stream``
+ - Sink pad stream number.
+ * - __u32
+ - ``source_pad``
+ - Source pad number.
+ * - __u32
+ - ``source_stream``
+ - Source pad stream number.
+ * - __u32
+ - ``flags``
+ - Route enable/disable flags
+ :ref:`v4l2_subdev_routing_flags <v4l2-subdev-routing-flags>`.
+ * - __u32
+ - ``reserved``\ [5]
+ - Reserved for future extensions. Applications and drivers must set
+ the array to zero.
+
+.. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
+
+.. _v4l2-subdev-routing-flags:
+
+.. flat-table:: enum v4l2_subdev_routing_flags
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 3 1 4
+
+ * - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - 0
+ - The route is enabled. Set by applications.
+ * - V4L2_SUBDEV_ROUTE_FL_IMMUTABLE
+ - 1
+ - The route is immutable. Set by the driver.
+ * - V4L2_SUBDEV_ROUTE_FL_SOURCE
+ - 2
+ - The route is a source route, and the ``sink_pad`` and ``sink_stream``
+ fields are unused. Set by the driver.
+
+Return Value
+============
+
+On success 0 is returned, on error -1 and the ``errno`` variable is set
+appropriately. The generic error codes are described at the
+:ref:`Generic Error Codes <gen-errors>` chapter.
+
+ENOSPC
+ The number of provided route entries is less than the available ones.
+
+EINVAL
+ The sink or source pad identifiers reference a non-existing pad, or reference
+ pads of different types (ie. the sink_pad identifiers refers to a source pad)
+ or the sink or source stream identifiers reference a non-existing stream on
+ the sink or source pad.
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 22/36] media: subdev: Add [GS]_ROUTING subdev ioctls and operations
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (20 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 21/36] media: Documentation: Add GS_ROUTING documentation Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-07-07 9:15 ` Tomasz Figa
2022-03-01 16:11 ` [PATCH v11 23/36] media: subdev: add v4l2_subdev_has_route() Tomi Valkeinen
` (14 subsequent siblings)
36 siblings, 1 reply; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Michal Simek, Tomi Valkeinen
From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Add support for subdev internal routing. A route is defined as a single
stream from a sink pad to a source pad.
The userspace can configure the routing via two new ioctls,
VIDIOC_SUBDEV_G_ROUTING and VIDIOC_SUBDEV_S_ROUTING, and subdevs can
implement the functionality with v4l2_subdev_pad_ops.set_routing().
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
- Add sink and source streams for multiplexed links
- Copy the argument back in case of an error. This is needed to let the
caller know the number of routes.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
- Expand and refine documentation.
- Make the 'routes' pointer a __u64 __user pointer so that a compat32
version of the ioctl is not required.
- Add struct v4l2_subdev_krouting to be used for subdevice operations.
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
- Fix typecasing warnings
- Check sink & source pad types
- Add 'which' field
- Add V4L2_SUBDEV_ROUTE_FL_SOURCE
- Routing to subdev state
- Dropped get_routing subdev op
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/v4l2-core/v4l2-ioctl.c | 25 ++++++++-
drivers/media/v4l2-core/v4l2-subdev.c | 73 +++++++++++++++++++++++++++
include/media/v4l2-subdev.h | 22 ++++++++
include/uapi/linux/v4l2-subdev.h | 57 +++++++++++++++++++++
4 files changed, 176 insertions(+), 1 deletion(-)
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 642cb90f457c..add3b28d446e 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -16,6 +16,7 @@
#include <linux/kernel.h>
#include <linux/version.h>
+#include <linux/v4l2-subdev.h>
#include <linux/videodev2.h>
#include <media/v4l2-common.h>
@@ -3093,6 +3094,21 @@ static int check_array_args(unsigned int cmd, void *parg, size_t *array_size,
ret = 1;
break;
}
+
+ case VIDIOC_SUBDEV_G_ROUTING:
+ case VIDIOC_SUBDEV_S_ROUTING: {
+ struct v4l2_subdev_routing *routing = parg;
+
+ if (routing->num_routes > 256)
+ return -EINVAL;
+
+ *user_ptr = u64_to_user_ptr(routing->routes);
+ *kernel_ptr = (void **)&routing->routes;
+ *array_size = sizeof(struct v4l2_subdev_route)
+ * routing->num_routes;
+ ret = 1;
+ break;
+ }
}
return ret;
@@ -3356,8 +3372,15 @@ video_usercopy(struct file *file, unsigned int orig_cmd, unsigned long arg,
/*
* Some ioctls can return an error, but still have valid
* results that must be returned.
+ *
+ * FIXME: subdev IOCTLS are partially handled here and partially in
+ * v4l2-subdev.c and the 'always_copy' flag can only be set for IOCTLS
+ * defined here as part of the 'v4l2_ioctls' array. As
+ * VIDIOC_SUBDEV_G_ROUTING needs to return results to applications even
+ * in case of failure, but it is not defined here as part of the
+ * 'v4l2_ioctls' array, insert an ad-hoc check to address that.
*/
- if (err < 0 && !always_copy)
+ if (err < 0 && !always_copy && cmd != VIDIOC_SUBDEV_G_ROUTING)
goto out;
out_array_args:
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 3ad24093abe9..89c97bde4575 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -377,6 +377,10 @@ subdev_ioctl_get_state(struct v4l2_subdev *sd, struct v4l2_subdev_fh *subdev_fh,
case VIDIOC_SUBDEV_S_SELECTION:
which = ((struct v4l2_subdev_selection *)arg)->which;
break;
+ case VIDIOC_SUBDEV_G_ROUTING:
+ case VIDIOC_SUBDEV_S_ROUTING:
+ which = ((struct v4l2_subdev_routing *)arg)->which;
+ break;
}
return which == V4L2_SUBDEV_FORMAT_TRY ?
@@ -692,6 +696,74 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
case VIDIOC_SUBDEV_QUERYSTD:
return v4l2_subdev_call(sd, video, querystd, arg);
+ case VIDIOC_SUBDEV_G_ROUTING: {
+ struct v4l2_subdev_routing *routing = arg;
+ struct v4l2_subdev_krouting *krouting;
+
+ if (!(sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED))
+ return -ENOIOCTLCMD;
+
+ memset(routing->reserved, 0, sizeof(routing->reserved));
+
+ krouting = &state->routing;
+
+ if (routing->num_routes < krouting->num_routes) {
+ routing->num_routes = krouting->num_routes;
+ return -ENOSPC;
+ }
+
+ memcpy((struct v4l2_subdev_route *)(uintptr_t)routing->routes,
+ krouting->routes,
+ krouting->num_routes * sizeof(*krouting->routes));
+ routing->num_routes = krouting->num_routes;
+
+ return 0;
+ }
+
+ case VIDIOC_SUBDEV_S_ROUTING: {
+ struct v4l2_subdev_routing *routing = arg;
+ struct v4l2_subdev_route *routes =
+ (struct v4l2_subdev_route *)(uintptr_t)routing->routes;
+ struct v4l2_subdev_krouting krouting = {};
+ unsigned int i;
+
+ if (!(sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED))
+ return -ENOIOCTLCMD;
+
+ if (routing->which != V4L2_SUBDEV_FORMAT_TRY && ro_subdev)
+ return -EPERM;
+
+ memset(routing->reserved, 0, sizeof(routing->reserved));
+
+ for (i = 0; i < routing->num_routes; ++i) {
+ const struct v4l2_subdev_route *route = &routes[i];
+ const struct media_pad *pads = sd->entity.pads;
+
+ /* Do not check sink pad for source routes */
+ if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_SOURCE)) {
+ if (route->sink_pad >= sd->entity.num_pads)
+ return -EINVAL;
+
+ if (!(pads[route->sink_pad].flags &
+ MEDIA_PAD_FL_SINK))
+ return -EINVAL;
+ }
+
+ if (route->source_pad >= sd->entity.num_pads)
+ return -EINVAL;
+
+ if (!(pads[route->source_pad].flags &
+ MEDIA_PAD_FL_SOURCE))
+ return -EINVAL;
+ }
+
+ krouting.num_routes = routing->num_routes;
+ krouting.routes = routes;
+
+ return v4l2_subdev_call(sd, pad, set_routing, state,
+ routing->which, &krouting);
+ }
+
default:
return v4l2_subdev_call(sd, core, ioctl, cmd, arg);
}
@@ -979,6 +1051,7 @@ void __v4l2_subdev_state_free(struct v4l2_subdev_state *state)
mutex_destroy(&state->_lock);
+ kfree(state->routing.routes);
kvfree(state->pads);
kfree(state);
}
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index ccb4ce9ea6ff..2630bb4c0459 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -697,12 +697,26 @@ struct v4l2_subdev_pad_config {
struct v4l2_rect try_compose;
};
+/**
+ * struct v4l2_subdev_krouting - subdev routing table
+ *
+ * @num_routes: number of routes
+ * @routes: &struct v4l2_subdev_route
+ *
+ * This structure contains the routing table for a subdev.
+ */
+struct v4l2_subdev_krouting {
+ unsigned int num_routes;
+ struct v4l2_subdev_route *routes;
+};
+
/**
* struct v4l2_subdev_state - Used for storing subdev state information.
*
* @_lock: default for 'lock'
* @lock: mutex for the state. May be replaced by the user.
* @pads: &struct v4l2_subdev_pad_config array
+ * @routing: routing table for the subdev
*
* This structure only needs to be passed to the pad op if the 'which' field
* of the main argument is set to %V4L2_SUBDEV_FORMAT_TRY. For
@@ -713,6 +727,7 @@ struct v4l2_subdev_state {
struct mutex _lock;
struct mutex *lock;
struct v4l2_subdev_pad_config *pads;
+ struct v4l2_subdev_krouting routing;
};
/**
@@ -765,6 +780,9 @@ struct v4l2_subdev_state {
* this operation as close as possible to stream on time. The
* operation shall fail if the pad index it has been called on
* is not valid or in case of unrecoverable failures.
+ *
+ * @set_routing: enable or disable data connection routes described in the
+ * subdevice routing table.
*/
struct v4l2_subdev_pad_ops {
int (*init_cfg)(struct v4l2_subdev *sd,
@@ -807,6 +825,10 @@ struct v4l2_subdev_pad_ops {
struct v4l2_mbus_frame_desc *fd);
int (*get_mbus_config)(struct v4l2_subdev *sd, unsigned int pad,
struct v4l2_mbus_config *config);
+ int (*set_routing)(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ enum v4l2_subdev_format_whence which,
+ struct v4l2_subdev_krouting *route);
};
/**
diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h
index d91ab6f22fa7..1ec3141bf860 100644
--- a/include/uapi/linux/v4l2-subdev.h
+++ b/include/uapi/linux/v4l2-subdev.h
@@ -191,6 +191,61 @@ struct v4l2_subdev_capability {
/* The v4l2 sub-device supports multiplexed streams. */
#define V4L2_SUBDEV_CAP_MPLEXED 0x00000002
+/*
+ * Is the route active? An active route will start when streaming is enabled
+ * on a video node.
+ */
+#define V4L2_SUBDEV_ROUTE_FL_ACTIVE _BITUL(0)
+
+/*
+ * Is the route immutable, i.e. can it be activated and inactivated?
+ * Set by the driver.
+ */
+#define V4L2_SUBDEV_ROUTE_FL_IMMUTABLE _BITUL(1)
+
+/*
+ * Is the route a source endpoint? A source endpoint route refers to a stream
+ * generated internally by the subdevice (usually a sensor), and thus there
+ * is no sink-side endpoint for the route. The sink_pad and sink_stream
+ * fields are unused.
+ * Set by the driver.
+ */
+#define V4L2_SUBDEV_ROUTE_FL_SOURCE _BITUL(2)
+
+/**
+ * struct v4l2_subdev_route - A route inside a subdev
+ *
+ * @sink_pad: the sink pad index
+ * @sink_stream: the sink stream identifier
+ * @source_pad: the source pad index
+ * @source_stream: the source stream identifier
+ * @flags: route flags V4L2_SUBDEV_ROUTE_FL_*
+ * @reserved: drivers and applications must zero this array
+ */
+struct v4l2_subdev_route {
+ __u32 sink_pad;
+ __u32 sink_stream;
+ __u32 source_pad;
+ __u32 source_stream;
+ __u32 flags;
+ __u32 reserved[5];
+};
+
+/**
+ * struct v4l2_subdev_routing - Subdev routing information
+ *
+ * @which: configuration type (from enum v4l2_subdev_format_whence)
+ * @num_routes: the total number of routes in the routes array
+ * @routes: pointer to the routes array
+ * @reserved: drivers and applications must zero this array
+ */
+struct v4l2_subdev_routing {
+ __u32 which;
+ __u32 num_routes;
+ __u64 routes;
+ __u32 reserved[6];
+};
+
/* Backwards compatibility define --- to be removed */
#define v4l2_subdev_edid v4l2_edid
@@ -206,6 +261,8 @@ struct v4l2_subdev_capability {
#define VIDIOC_SUBDEV_S_CROP _IOWR('V', 60, struct v4l2_subdev_crop)
#define VIDIOC_SUBDEV_G_SELECTION _IOWR('V', 61, struct v4l2_subdev_selection)
#define VIDIOC_SUBDEV_S_SELECTION _IOWR('V', 62, struct v4l2_subdev_selection)
+#define VIDIOC_SUBDEV_G_ROUTING _IOWR('V', 38, struct v4l2_subdev_routing)
+#define VIDIOC_SUBDEV_S_ROUTING _IOWR('V', 39, struct v4l2_subdev_routing)
/* The following ioctls are identical to the ioctls in videodev2.h */
#define VIDIOC_SUBDEV_G_STD _IOR('V', 23, v4l2_std_id)
#define VIDIOC_SUBDEV_S_STD _IOW('V', 24, v4l2_std_id)
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 23/36] media: subdev: add v4l2_subdev_has_route()
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (21 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 22/36] media: subdev: Add [GS]_ROUTING subdev ioctls and operations Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 24/36] media: subdev: add v4l2_subdev_set_routing helper() Tomi Valkeinen
` (13 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
Add a v4l2_subdev_has_route() helper function which can be used for
media_entity_operations.has_route op.
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
---
drivers/media/v4l2-core/v4l2-subdev.c | 31 +++++++++++++++++++++++++++
include/media/v4l2-subdev.h | 16 ++++++++++++++
2 files changed, 47 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 89c97bde4575..d3047b932142 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -994,6 +994,37 @@ int v4l2_subdev_link_validate(struct media_link *link)
}
EXPORT_SYMBOL_GPL(v4l2_subdev_link_validate);
+bool v4l2_subdev_has_route(struct media_entity *entity, unsigned int pad0,
+ unsigned int pad1)
+{
+ struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+ struct v4l2_subdev_krouting *routing;
+ unsigned int i;
+ struct v4l2_subdev_state *state;
+
+ state = v4l2_subdev_lock_and_get_active_state(sd);
+
+ routing = &state->routing;
+
+ for (i = 0; i < routing->num_routes; ++i) {
+ struct v4l2_subdev_route *route = &routing->routes[i];
+
+ if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+ continue;
+
+ if ((route->sink_pad == pad0 && route->source_pad == pad1) ||
+ (route->source_pad == pad0 && route->sink_pad == pad1)) {
+ v4l2_subdev_unlock_state(state);
+ return true;
+ }
+ }
+
+ v4l2_subdev_unlock_state(state);
+
+ return false;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_has_route);
+
struct v4l2_subdev_state *
__v4l2_subdev_state_alloc(struct v4l2_subdev *sd, const char *lock_name,
struct lock_class_key *lock_key)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 2630bb4c0459..2a6880c71028 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -1224,6 +1224,22 @@ int v4l2_subdev_link_validate_default(struct v4l2_subdev *sd,
*/
int v4l2_subdev_link_validate(struct media_link *link);
+/**
+ * v4l2_subdev_has_route - MC has_route implementation for subdevs
+ *
+ * @entity: pointer to &struct media_entity
+ * @pad0: pad number for the first pad
+ * @pad1: pad number for the second pad
+ *
+ * This function looks at the routing in subdev's active state and returns if
+ * there is a route connecting pad0 and pad1.
+ *
+ * This function can be used as implementation for
+ * media_entity_operations.has_route.
+ */
+bool v4l2_subdev_has_route(struct media_entity *entity, unsigned int pad0,
+ unsigned int pad1);
+
/**
* __v4l2_subdev_state_alloc - allocate v4l2_subdev_state
*
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 24/36] media: subdev: add v4l2_subdev_set_routing helper()
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (22 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 23/36] media: subdev: add v4l2_subdev_has_route() Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 25/36] media: Documentation: add multiplexed streams documentation Tomi Valkeinen
` (12 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
Add a helper function to set the subdev routing. The helper can be used
from subdev driver's set_routing op to store the routing table.
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
---
drivers/media/v4l2-core/v4l2-subdev.c | 27 +++++++++++++++++++++++++++
include/media/v4l2-subdev.h | 16 ++++++++++++++++
2 files changed, 43 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index d3047b932142..1c836c2de86e 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -1110,6 +1110,33 @@ void v4l2_subdev_cleanup(struct v4l2_subdev *sd)
}
EXPORT_SYMBOL_GPL(v4l2_subdev_cleanup);
+int v4l2_subdev_set_routing(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_krouting *routing)
+{
+ struct v4l2_subdev_krouting *dst = &state->routing;
+ const struct v4l2_subdev_krouting *src = routing;
+
+ lockdep_assert_held(state->lock);
+
+ kfree(dst->routes);
+ dst->routes = NULL;
+ dst->num_routes = 0;
+
+ if (src->num_routes > 0) {
+ dst->routes = kmemdup(src->routes,
+ src->num_routes * sizeof(*src->routes),
+ GFP_KERNEL);
+ if (!dst->routes)
+ return -ENOMEM;
+
+ dst->num_routes = src->num_routes;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_set_routing);
+
#endif /* CONFIG_MEDIA_CONTROLLER */
void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 2a6880c71028..6323bae3860e 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -1381,6 +1381,22 @@ static inline void v4l2_subdev_unlock_state(struct v4l2_subdev_state *state)
mutex_unlock(state->lock);
}
+/**
+ * v4l2_subdev_set_routing() - Set given routing to subdev state
+ * @sd: The subdevice
+ * @state: The subdevice state
+ * @routing: Routing that will be copied to subdev state
+ *
+ * This will release old routing table (if any) from the state, allocate
+ * enough space for the given routing, and copy the routing.
+ *
+ * This can be used from the subdev driver's set_routing op, after validating
+ * the routing.
+ */
+int v4l2_subdev_set_routing(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_krouting *routing);
+
#endif /* CONFIG_MEDIA_CONTROLLER */
/**
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 25/36] media: Documentation: add multiplexed streams documentation
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (23 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 24/36] media: subdev: add v4l2_subdev_set_routing helper() Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-16 9:04 ` Jacopo Mondi
2022-03-01 16:11 ` [PATCH v11 26/36] media: subdev: add stream based configuration Tomi Valkeinen
` (11 subsequent siblings)
36 siblings, 1 reply; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
Add documentation related to multiplexed streams.
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
.../driver-api/media/v4l2-subdev.rst | 8 +
.../userspace-api/media/v4l/dev-subdev.rst | 165 ++++++++++++++++++
2 files changed, 173 insertions(+)
diff --git a/Documentation/driver-api/media/v4l2-subdev.rst b/Documentation/driver-api/media/v4l2-subdev.rst
index 115211cef4d1..80654f1bcac9 100644
--- a/Documentation/driver-api/media/v4l2-subdev.rst
+++ b/Documentation/driver-api/media/v4l2-subdev.rst
@@ -593,6 +593,14 @@ before calling v4l2_subdev_init_finalize():
This shares the driver's private mutex between the controls and the states.
+Streams, multiplexed media pads and internal routing
+----------------------------------------------------
+
+A subdevice driver can implement support for multiplexed streams by setting
+the V4L2_SUBDEV_FL_MULTIPLEXED subdev flag and implementing support for
+centrally managed subdev active state, routing and stream based
+configuration.
+
V4L2 sub-device functions and data structures
---------------------------------------------
diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
index a67c2749089a..fd042afeddd6 100644
--- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
+++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
@@ -503,3 +503,168 @@ source pads.
:maxdepth: 1
subdev-formats
+
+Streams, multiplexed media pads and internal routing
+----------------------------------------------------
+
+Commonly V4L2 subdevices support only separate video streams, that is, each
+link in the media graph and each pad in a subdevice pass through a single
+video stream. Thus each pad contains a format configuration for that single
+stream. In some cases a subdev can do stream processing and split a stream
+into two or compose two streams into one, but the inputs and outputs for the
+subdev are still a single stream per pad.
+
+Some hardware, e.g. MIPI CSI-2, support multiplexed streams, that is, a single
+bus carries multiple streams. Thus a camera could output two streams, a pixel
+stream and a metadata stream, and a bridge subdev could route the streams
+from multiple input pads into a single output pad.
+
+Subdevice drivers that support multiplexed streams are compatible with
+non-multiplexed subdev drivers, but, of course, require a routing configuration
+where the link between those two types of drivers contains only a single
+stream.
+
+Understanding streams
+^^^^^^^^^^^^^^^^^^^^^
+
+A stream is a stream of content (e.g. pixel data or metadata) flowing through
+the media pipeline from a source (e.g. a sensor) towards the final sink(e.g. a
+receiver and demultiplexer in a SoC). Each media link carries all the streams
+from one end of the link to the other, and subdevices have routing tables which
+describe how the incoming streams from sink pads are routed to the source
+pads.
+
+A stream ID (often just "stream") is a media link-local identifier for a
+stream. In other words, a configuration for a particular stream ID must exist
+on both sides of a media link, but another stream ID can be used for the same
+stream at the other side of the subdevice.
+
+A stream at a specific point in the media pipeline is identified with the
+subdev and a (pad, stream) pair. For subdevices that do not support
+multiplexed streams the 'stream' is always 0.
+
+Configuring streams
+^^^^^^^^^^^^^^^^^^^
+
+The configuration of the streams is done individually for each subdevice and
+the validity of the streams between subdevices is validated when the pipeline
+is started.
+
+There are three steps in configuring the streams:
+
+1) Set up links. Connect the pads between subdevices using the :ref:`Media
+Controller API <media_controller>`
+
+2) Routing. The routing table for the subdevice must be set with
+:ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl. Note that
+setting the routing table will reset all the stream configurations.
+
+3) Configure streams. Each route endpoint must be configured
+with :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>`.
+
+Multiplexed streams setup example
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+A simple example of a multiplexed stream setup might be as follows:
+
+- Two identical sensors (Sensor A and Sensor B). Each sensor has a single
+ source pad (pad 0), and outputs two streams, pixel data and metadata.
+
+- Multiplexer bridge (Bridge). The bridge has two sink pads, connected to the
+ sensors (pads 0, 1), and one source pad (pad 2), which outputs all 4
+ streams.
+
+- Receiver in the SoC (Receiver). The receiver has a single sink pad (pad 0),
+ connected to the bridge, and four source pads (pads 1-4), going to the DMA
+ engine. The receiver demultiplexes the incoming streams to the four source
+ pads.
+
+- Four DMA Engines in the SoC (DMA Engine). Each DMA engine is connected to a
+ single source pad in the receiver.
+
+The sensors, the bridge and the receiver are modeled as V4L2 subdevices,
+exposed to userspace via /dev/v4l-subdevX device nodes. The DMA engines are
+modeled as V4L2 devices, exposed to userspace via /dev/videoX nodes.
+
+To configure this pipeline, the userspace must take the following steps:
+
+1) Set up media links between entities: connect the sensors to the bridge,
+bridge to the receiver, and the receiver to the DMA engines. This step does
+not differ from normal non-multiplexed media controller setup.
+
+2) Configure routing.
+
+.. flat-table:: Sensor routing table (identical on both sensors)
+ :header-rows: 1
+
+ * - Sink Pad/Stream
+ - Source Pad/Stream
+ - Routing Flags
+ - Comments
+ * - 0/0 (unused)
+ - 0/0
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE | V4L2_SUBDEV_ROUTE_FL_SOURCE
+ - Pixel data stream. Source route, i.e. the sink fields are unused.
+ * - 0/0 (unused)
+ - 0/1
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE | V4L2_SUBDEV_ROUTE_FL_SOURCE
+ - Metadata stream. Source route, i.e. the sink fields are unused.
+
+.. flat-table:: Bridge routing table
+ :header-rows: 1
+
+ * - Sink Pad/Stream
+ - Source Pad/Stream
+ - Routing Flags
+ - Comments
+ * - 0/0
+ - 2/0
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Pixel data stream from Sensor A
+ * - 0/1
+ - 2/1
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Metadata stream from Sensor A
+ * - 1/0
+ - 2/2
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Pixel data stream from Sensor B
+ * - 1/1
+ - 2/3
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Metadata stream from Sensor B
+
+.. flat-table:: Receiver routing table
+ :header-rows: 1
+
+ * - Sink Pad/Stream
+ - Source Pad/Stream
+ - Routing Flags
+ - Comments
+ * - 0/0
+ - 1/0
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Pixel data stream from Sensor A
+ * - 0/1
+ - 2/0
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Metadata stream from Sensor A
+ * - 0/2
+ - 3/0
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Pixel data stream from Sensor B
+ * - 0/3
+ - 4/0
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Metadata stream from Sensor B
+
+3) Configure streams
+
+After configuring the routing table, the next step is configuring the streams.
+This step is similar to configuring the pads in a non-multiplexed streams
+setup, with the difference that we need to configure each (pad, stream) pair
+(i.e. route endpoint) instead of just a pad.
+
+A common way to accomplish this is to start from the sensors and propagate the
+configurations along the stream towards the receiver, using VIDIOC_SUBDEV_S_FMT
+to configure each stream endpoint in each subdev.
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 26/36] media: subdev: add stream based configuration
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (24 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 25/36] media: Documentation: add multiplexed streams documentation Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-16 9:59 ` Jacopo Mondi
2022-03-01 16:11 ` [PATCH v11 27/36] media: subdev: use streams in v4l2_subdev_link_validate() Tomi Valkeinen
` (10 subsequent siblings)
36 siblings, 1 reply; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
Add support to manage configurations (format, crop, compose) per stream,
instead of per pad. This is accomplished with data structures that hold
an array of all subdev's stream configurations.
The number of streams can vary at runtime based on routing. Every time
the routing is changed, the stream configurations need to be
re-initialized.
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
.../v4l/vidioc-subdev-enum-frame-interval.rst | 5 +-
.../v4l/vidioc-subdev-enum-frame-size.rst | 5 +-
.../v4l/vidioc-subdev-enum-mbus-code.rst | 5 +-
.../media/v4l/vidioc-subdev-g-crop.rst | 5 +-
.../media/v4l/vidioc-subdev-g-fmt.rst | 5 +-
.../v4l/vidioc-subdev-g-frame-interval.rst | 5 +-
.../media/v4l/vidioc-subdev-g-selection.rst | 5 +-
drivers/media/v4l2-core/v4l2-subdev.c | 129 ++++++++++++++++--
include/media/v4l2-subdev.h | 48 +++++++
include/uapi/linux/v4l2-subdev.h | 28 +++-
10 files changed, 218 insertions(+), 22 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-interval.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-interval.rst
index 3703943b412f..8def4c05d3da 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-interval.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-interval.rst
@@ -92,7 +92,10 @@ multiple pads of the same sub-device is not defined.
- Frame intervals to be enumerated, from enum
:ref:`v4l2_subdev_format_whence <v4l2-subdev-format-whence>`.
* - __u32
- - ``reserved``\ [8]
+ - ``stream``
+ - Stream identifier.
+ * - __u32
+ - ``reserved``\ [7]
- Reserved for future extensions. Applications and drivers must set
the array to zero.
diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-size.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-size.rst
index c25a9896df0e..3ef361c0dca7 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-size.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-size.rst
@@ -97,7 +97,10 @@ information about try formats.
- Frame sizes to be enumerated, from enum
:ref:`v4l2_subdev_format_whence <v4l2-subdev-format-whence>`.
* - __u32
- - ``reserved``\ [8]
+ - ``stream``
+ - Stream identifier.
+ * - __u32
+ - ``reserved``\ [7]
- Reserved for future extensions. Applications and drivers must set
the array to zero.
diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-mbus-code.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-mbus-code.rst
index 417f1a19bcc4..248f6f9ee7c5 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-mbus-code.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-mbus-code.rst
@@ -73,7 +73,10 @@ information about the try formats.
- ``flags``
- See :ref:`v4l2-subdev-mbus-code-flags`
* - __u32
- - ``reserved``\ [7]
+ - ``stream``
+ - Stream identifier.
+ * - __u32
+ - ``reserved``\ [6]
- Reserved for future extensions. Applications and drivers must set
the array to zero.
diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-crop.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-crop.rst
index bd15c0a5a66b..1d267f7e7991 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-crop.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-crop.rst
@@ -96,7 +96,10 @@ modified format should be as close as possible to the original request.
- ``rect``
- Crop rectangle boundaries, in pixels.
* - __u32
- - ``reserved``\ [8]
+ - ``stream``
+ - Stream identifier.
+ * - __u32
+ - ``reserved``\ [7]
- Reserved for future extensions. Applications and drivers must set
the array to zero.
diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-fmt.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-fmt.rst
index 7acdbb939d89..ed253a1e44b7 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-fmt.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-fmt.rst
@@ -102,7 +102,10 @@ should be as close as possible to the original request.
- Definition of an image format, see :c:type:`v4l2_mbus_framefmt` for
details.
* - __u32
- - ``reserved``\ [8]
+ - ``stream``
+ - Stream identifier.
+ * - __u32
+ - ``reserved``\ [7]
- Reserved for future extensions. Applications and drivers must set
the array to zero.
diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-frame-interval.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-frame-interval.rst
index d7fe7543c506..842f962d2aea 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-frame-interval.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-frame-interval.rst
@@ -90,7 +90,10 @@ the same sub-device is not defined.
- ``interval``
- Period, in seconds, between consecutive video frames.
* - __u32
- - ``reserved``\ [9]
+ - ``stream``
+ - Stream identifier.
+ * - __u32
+ - ``reserved``\ [8]
- Reserved for future extensions. Applications and drivers must set
the array to zero.
diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-selection.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-selection.rst
index f9172a42f036..6b629c19168c 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-selection.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-selection.rst
@@ -94,7 +94,10 @@ Selection targets and flags are documented in
- ``r``
- Selection rectangle, in pixels.
* - __u32
- - ``reserved``\ [8]
+ - ``stream``
+ - Stream identifier.
+ * - __u32
+ - ``reserved``\ [7]
- Reserved for future extensions. Applications and drivers must set
the array to zero.
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 1c836c2de86e..339d7b15e26c 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -149,14 +149,34 @@ static inline int check_pad(struct v4l2_subdev *sd, u32 pad)
return 0;
}
-static int check_state_pads(u32 which, struct v4l2_subdev_state *state)
+static int check_state_pads(struct v4l2_subdev *sd, u32 which,
+ struct v4l2_subdev_state *state)
{
+ if (sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED)
+ return 0;
+
if (which == V4L2_SUBDEV_FORMAT_TRY && (!state || !state->pads))
return -EINVAL;
return 0;
}
+static int check_state_pad_stream(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state, u32 pad,
+ u32 stream)
+{
+ struct v4l2_mbus_framefmt *fmt;
+
+ if (!(sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED))
+ return 0;
+
+ fmt = v4l2_subdev_state_get_stream_format(state, pad, stream);
+ if (!fmt)
+ return -EINVAL;
+
+ return 0;
+}
+
static inline int check_format(struct v4l2_subdev *sd,
struct v4l2_subdev_state *state,
struct v4l2_subdev_format *format)
@@ -165,7 +185,8 @@ static inline int check_format(struct v4l2_subdev *sd,
return -EINVAL;
return check_which(format->which) ? : check_pad(sd, format->pad) ? :
- check_state_pads(format->which, state);
+ check_state_pads(sd, format->which, state) ? :
+ check_state_pad_stream(sd, state, format->pad, format->stream);
}
static int call_get_fmt(struct v4l2_subdev *sd,
@@ -192,7 +213,8 @@ static int call_enum_mbus_code(struct v4l2_subdev *sd,
return -EINVAL;
return check_which(code->which) ? : check_pad(sd, code->pad) ? :
- check_state_pads(code->which, state) ? :
+ check_state_pads(sd, code->which, state) ? :
+ check_state_pad_stream(sd, state, code->pad, code->stream) ? :
sd->ops->pad->enum_mbus_code(sd, state, code);
}
@@ -204,7 +226,8 @@ static int call_enum_frame_size(struct v4l2_subdev *sd,
return -EINVAL;
return check_which(fse->which) ? : check_pad(sd, fse->pad) ? :
- check_state_pads(fse->which, state) ? :
+ check_state_pads(sd, fse->which, state) ? :
+ check_state_pad_stream(sd, state, fse->pad, fse->stream) ? :
sd->ops->pad->enum_frame_size(sd, state, fse);
}
@@ -239,7 +262,8 @@ static int call_enum_frame_interval(struct v4l2_subdev *sd,
return -EINVAL;
return check_which(fie->which) ? : check_pad(sd, fie->pad) ? :
- check_state_pads(fie->which, state) ? :
+ check_state_pads(sd, fie->which, state) ? :
+ check_state_pad_stream(sd, state, fie->pad, fie->stream) ? :
sd->ops->pad->enum_frame_interval(sd, state, fie);
}
@@ -251,7 +275,8 @@ static inline int check_selection(struct v4l2_subdev *sd,
return -EINVAL;
return check_which(sel->which) ? : check_pad(sd, sel->pad) ? :
- check_state_pads(sel->which, state);
+ check_state_pads(sd, sel->which, state) ? :
+ check_state_pad_stream(sd, state, sel->pad, sel->stream);
}
static int call_get_selection(struct v4l2_subdev *sd,
@@ -865,6 +890,71 @@ const struct v4l2_file_operations v4l2_subdev_fops = {
#ifdef CONFIG_MEDIA_CONTROLLER
+static int
+v4l2_subdev_init_stream_configs(struct v4l2_subdev_stream_configs *stream_configs,
+ const struct v4l2_subdev_krouting *routing)
+{
+ u32 num_configs = 0;
+ unsigned int i;
+ u32 format_idx = 0;
+
+ kvfree(stream_configs->configs);
+ stream_configs->configs = NULL;
+ stream_configs->num_configs = 0;
+
+ /* Count number of formats needed */
+ for (i = 0; i < routing->num_routes; ++i) {
+ struct v4l2_subdev_route *route = &routing->routes[i];
+
+ if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+ continue;
+
+ /*
+ * Each route needs a format on both ends of the route, except
+ * for source streams which only need one format.
+ */
+ num_configs +=
+ (route->flags & V4L2_SUBDEV_ROUTE_FL_SOURCE) ? 1 : 2;
+ }
+
+ if (num_configs) {
+ stream_configs->configs =
+ kvcalloc(num_configs, sizeof(*stream_configs->configs),
+ GFP_KERNEL);
+
+ if (!stream_configs->configs)
+ return -ENOMEM;
+
+ stream_configs->num_configs = num_configs;
+ }
+
+ /*
+ * Fill in the 'pad' and stream' value for each item in the array from
+ * the routing table
+ */
+ for (i = 0; i < routing->num_routes; ++i) {
+ struct v4l2_subdev_route *route = &routing->routes[i];
+ u32 idx;
+
+ if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+ continue;
+
+ if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_SOURCE)) {
+ idx = format_idx++;
+
+ stream_configs->configs[idx].pad = route->sink_pad;
+ stream_configs->configs[idx].stream = route->sink_stream;
+ }
+
+ idx = format_idx++;
+
+ stream_configs->configs[idx].pad = route->source_pad;
+ stream_configs->configs[idx].stream = route->source_stream;
+ }
+
+ return 0;
+}
+
int v4l2_subdev_get_fwnode_pad_1_to_1(struct media_entity *entity,
struct fwnode_endpoint *endpoint)
{
@@ -1042,7 +1132,8 @@ __v4l2_subdev_state_alloc(struct v4l2_subdev *sd, const char *lock_name,
else
state->lock = &state->_lock;
- if (sd->entity.num_pads) {
+ /* Drivers that support streams do not need the legacy pad config */
+ if (!(sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED) && sd->entity.num_pads) {
state->pads = kvmalloc_array(sd->entity.num_pads,
sizeof(*state->pads),
GFP_KERNEL | __GFP_ZERO);
@@ -1083,6 +1174,7 @@ void __v4l2_subdev_state_free(struct v4l2_subdev_state *state)
mutex_destroy(&state->_lock);
kfree(state->routing.routes);
+ kvfree(state->stream_configs.configs);
kvfree(state->pads);
kfree(state);
}
@@ -1133,10 +1225,31 @@ int v4l2_subdev_set_routing(struct v4l2_subdev *sd,
dst->num_routes = src->num_routes;
}
- return 0;
+ return v4l2_subdev_init_stream_configs(&state->stream_configs, dst);
}
EXPORT_SYMBOL_GPL(v4l2_subdev_set_routing);
+struct v4l2_mbus_framefmt *
+v4l2_subdev_state_get_stream_format(struct v4l2_subdev_state *state,
+ unsigned int pad, u32 stream)
+{
+ struct v4l2_subdev_stream_configs *stream_configs;
+ unsigned int i;
+
+ lockdep_assert_held(state->lock);
+
+ stream_configs = &state->stream_configs;
+
+ for (i = 0; i < stream_configs->num_configs; ++i) {
+ if (stream_configs->configs[i].pad == pad &&
+ stream_configs->configs[i].stream == stream)
+ return &stream_configs->configs[i].fmt;
+ }
+
+ return NULL;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_state_get_stream_format);
+
#endif /* CONFIG_MEDIA_CONTROLLER */
void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 6323bae3860e..2a40ad273cf8 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -697,6 +697,37 @@ struct v4l2_subdev_pad_config {
struct v4l2_rect try_compose;
};
+/**
+ * struct v4l2_subdev_stream_config - Used for storing stream configuration.
+ *
+ * @pad: pad number
+ * @stream: stream number
+ * @fmt: &struct v4l2_mbus_framefmt
+ * @crop: &struct v4l2_rect to be used for crop
+ * @compose: &struct v4l2_rect to be used for compose
+ *
+ * This structure stores configuration for a stream.
+ */
+struct v4l2_subdev_stream_config {
+ u32 pad;
+ u32 stream;
+
+ struct v4l2_mbus_framefmt fmt;
+ struct v4l2_rect crop;
+ struct v4l2_rect compose;
+};
+
+/**
+ * struct v4l2_subdev_stream_configs - A collection of stream configs.
+ *
+ * @num_configs: number of entries in @config.
+ * @configs: an array of &struct v4l2_subdev_stream_configs.
+ */
+struct v4l2_subdev_stream_configs {
+ u32 num_configs;
+ struct v4l2_subdev_stream_config *configs;
+};
+
/**
* struct v4l2_subdev_krouting - subdev routing table
*
@@ -717,6 +748,7 @@ struct v4l2_subdev_krouting {
* @lock: mutex for the state. May be replaced by the user.
* @pads: &struct v4l2_subdev_pad_config array
* @routing: routing table for the subdev
+ * @stream_configs: stream configurations (only for V4L2_SUBDEV_FL_MULTIPLEXED)
*
* This structure only needs to be passed to the pad op if the 'which' field
* of the main argument is set to %V4L2_SUBDEV_FORMAT_TRY. For
@@ -728,6 +760,7 @@ struct v4l2_subdev_state {
struct mutex *lock;
struct v4l2_subdev_pad_config *pads;
struct v4l2_subdev_krouting routing;
+ struct v4l2_subdev_stream_configs stream_configs;
};
/**
@@ -1397,6 +1430,21 @@ int v4l2_subdev_set_routing(struct v4l2_subdev *sd,
struct v4l2_subdev_state *state,
struct v4l2_subdev_krouting *routing);
+/**
+ * v4l2_subdev_state_get_stream_format() - Get pointer to a stream format
+ * @state: subdevice state
+ * @pad: pad id
+ * @stream: stream id
+ *
+ * This returns a pointer to &struct v4l2_mbus_framefmt for the given pad +
+ * stream in the subdev state.
+ *
+ * If the state does not contain the given pad + stream, NULL is returned.
+ */
+struct v4l2_mbus_framefmt *
+v4l2_subdev_state_get_stream_format(struct v4l2_subdev_state *state,
+ unsigned int pad, u32 stream);
+
#endif /* CONFIG_MEDIA_CONTROLLER */
/**
diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h
index 1ec3141bf860..480891dba193 100644
--- a/include/uapi/linux/v4l2-subdev.h
+++ b/include/uapi/linux/v4l2-subdev.h
@@ -44,13 +44,15 @@ enum v4l2_subdev_format_whence {
* @which: format type (from enum v4l2_subdev_format_whence)
* @pad: pad number, as reported by the media API
* @format: media bus format (format code and frame size)
+ * @stream: stream number, defined in subdev routing
* @reserved: drivers and applications must zero this array
*/
struct v4l2_subdev_format {
__u32 which;
__u32 pad;
struct v4l2_mbus_framefmt format;
- __u32 reserved[8];
+ __u32 stream;
+ __u32 reserved[7];
};
/**
@@ -58,13 +60,15 @@ struct v4l2_subdev_format {
* @which: format type (from enum v4l2_subdev_format_whence)
* @pad: pad number, as reported by the media API
* @rect: pad crop rectangle boundaries
+ * @stream: stream number, defined in subdev routing
* @reserved: drivers and applications must zero this array
*/
struct v4l2_subdev_crop {
__u32 which;
__u32 pad;
struct v4l2_rect rect;
- __u32 reserved[8];
+ __u32 stream;
+ __u32 reserved[7];
};
#define V4L2_SUBDEV_MBUS_CODE_CSC_COLORSPACE 0x00000001
@@ -80,6 +84,7 @@ struct v4l2_subdev_crop {
* @code: format code (MEDIA_BUS_FMT_ definitions)
* @which: format type (from enum v4l2_subdev_format_whence)
* @flags: flags set by the driver, (V4L2_SUBDEV_MBUS_CODE_*)
+ * @stream: stream number, defined in subdev routing
* @reserved: drivers and applications must zero this array
*/
struct v4l2_subdev_mbus_code_enum {
@@ -88,7 +93,8 @@ struct v4l2_subdev_mbus_code_enum {
__u32 code;
__u32 which;
__u32 flags;
- __u32 reserved[7];
+ __u32 stream;
+ __u32 reserved[6];
};
/**
@@ -101,6 +107,7 @@ struct v4l2_subdev_mbus_code_enum {
* @min_height: minimum frame height, in pixels
* @max_height: maximum frame height, in pixels
* @which: format type (from enum v4l2_subdev_format_whence)
+ * @stream: stream number, defined in subdev routing
* @reserved: drivers and applications must zero this array
*/
struct v4l2_subdev_frame_size_enum {
@@ -112,19 +119,22 @@ struct v4l2_subdev_frame_size_enum {
__u32 min_height;
__u32 max_height;
__u32 which;
- __u32 reserved[8];
+ __u32 stream;
+ __u32 reserved[7];
};
/**
* struct v4l2_subdev_frame_interval - Pad-level frame rate
* @pad: pad number, as reported by the media API
* @interval: frame interval in seconds
+ * @stream: stream number, defined in subdev routing
* @reserved: drivers and applications must zero this array
*/
struct v4l2_subdev_frame_interval {
__u32 pad;
struct v4l2_fract interval;
- __u32 reserved[9];
+ __u32 stream;
+ __u32 reserved[8];
};
/**
@@ -136,6 +146,7 @@ struct v4l2_subdev_frame_interval {
* @height: frame height in pixels
* @interval: frame interval in seconds
* @which: format type (from enum v4l2_subdev_format_whence)
+ * @stream: stream number, defined in subdev routing
* @reserved: drivers and applications must zero this array
*/
struct v4l2_subdev_frame_interval_enum {
@@ -146,7 +157,8 @@ struct v4l2_subdev_frame_interval_enum {
__u32 height;
struct v4l2_fract interval;
__u32 which;
- __u32 reserved[8];
+ __u32 stream;
+ __u32 reserved[7];
};
/**
@@ -158,6 +170,7 @@ struct v4l2_subdev_frame_interval_enum {
* defined in v4l2-common.h; V4L2_SEL_TGT_* .
* @flags: constraint flags, defined in v4l2-common.h; V4L2_SEL_FLAG_*.
* @r: coordinates of the selection window
+ * @stream: stream number, defined in subdev routing
* @reserved: for future use, set to zero for now
*
* Hardware may use multiple helper windows to process a video stream.
@@ -170,7 +183,8 @@ struct v4l2_subdev_selection {
__u32 target;
__u32 flags;
struct v4l2_rect r;
- __u32 reserved[8];
+ __u32 stream;
+ __u32 reserved[7];
};
/**
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 27/36] media: subdev: use streams in v4l2_subdev_link_validate()
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (25 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 26/36] media: subdev: add stream based configuration Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-16 10:47 ` Jacopo Mondi
2022-03-01 16:11 ` [PATCH v11 28/36] media: subdev: add "opposite" stream helper funcs Tomi Valkeinen
` (9 subsequent siblings)
36 siblings, 1 reply; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
Update v4l2_subdev_link_validate() to use routing and streams for
validation.
Instead of just looking at the format on the pad on both ends of the
link, the routing tables are used to collect all the streams going from
the source to the sink over the link, and the streams' formats on both
ends of the link are verified.
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/v4l2-core/v4l2-subdev.c | 245 ++++++++++++++++++++++++--
1 file changed, 227 insertions(+), 18 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 339d7b15e26c..091b854e00d0 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -16,6 +16,7 @@
#include <linux/videodev2.h>
#include <linux/export.h>
#include <linux/version.h>
+#include <linux/sort.h>
#include <media/v4l2-ctrls.h>
#include <media/v4l2-device.h>
@@ -1035,6 +1036,7 @@ EXPORT_SYMBOL_GPL(v4l2_subdev_link_validate_default);
static int
v4l2_subdev_link_validate_get_format(struct media_pad *pad,
+ u32 stream,
struct v4l2_subdev_format *fmt)
{
if (is_media_entity_v4l2_subdev(pad->entity)) {
@@ -1046,6 +1048,7 @@ v4l2_subdev_link_validate_get_format(struct media_pad *pad,
fmt->which = V4L2_SUBDEV_FORMAT_ACTIVE;
fmt->pad = pad->index;
+ fmt->stream = stream;
return v4l2_subdev_call(sd, pad, get_fmt, state, fmt);
}
@@ -1056,31 +1059,237 @@ v4l2_subdev_link_validate_get_format(struct media_pad *pad,
return -EINVAL;
}
-int v4l2_subdev_link_validate(struct media_link *link)
+static int cmp_u32(const void *a, const void *b)
{
- struct v4l2_subdev *sink;
- struct v4l2_subdev_format sink_fmt, source_fmt;
- int rval;
+ u32 a32 = *(u32 *)a;
+ u32 b32 = *(u32 *)b;
- rval = v4l2_subdev_link_validate_get_format(
- link->source, &source_fmt);
- if (rval < 0)
- return 0;
+ return a32 > b32 ? 1 : (a32 < b32 ? -1 : 0);
+}
+
+static int v4l2_link_validate_get_streams(struct media_link *link,
+ bool is_source, u32 *out_num_streams,
+ const u32 **out_streams,
+ bool *allocated)
+{
+ static const u32 default_streams[] = { 0 };
+ struct v4l2_subdev_krouting *routing;
+ struct v4l2_subdev *subdev;
+ u32 num_streams;
+ u32 *streams;
+ unsigned int i;
+ struct v4l2_subdev_state *state;
+
+ if (is_source)
+ subdev = media_entity_to_v4l2_subdev(link->source->entity);
+ else
+ subdev = media_entity_to_v4l2_subdev(link->sink->entity);
- rval = v4l2_subdev_link_validate_get_format(
- link->sink, &sink_fmt);
- if (rval < 0)
+ if (!(subdev->flags & V4L2_SUBDEV_FL_MULTIPLEXED)) {
+ *out_num_streams = 1;
+ *out_streams = default_streams;
+ *allocated = false;
return 0;
+ }
- sink = media_entity_to_v4l2_subdev(link->sink->entity);
+ state = v4l2_subdev_get_locked_active_state(subdev);
- rval = v4l2_subdev_call(sink, pad, link_validate, link,
- &source_fmt, &sink_fmt);
- if (rval != -ENOIOCTLCMD)
- return rval;
+ routing = &state->routing;
+
+ streams = kmalloc_array(routing->num_routes, sizeof(u32), GFP_KERNEL);
+ if (!streams)
+ return -ENOMEM;
+
+ num_streams = 0;
+
+ for (i = 0; i < routing->num_routes; ++i) {
+ struct v4l2_subdev_route *route = &routing->routes[i];
+ int j;
+ u32 route_pad;
+ u32 route_stream;
+ u32 link_pad;
+
+ if (is_source) {
+ route_pad = route->source_pad;
+ route_stream = route->source_stream;
+ link_pad = link->source->index;
+ } else {
+ route_pad = route->sink_pad;
+ route_stream = route->sink_stream;
+ link_pad = link->sink->index;
+ }
+
+ if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+ continue;
+
+ if (route_pad != link_pad)
+ continue;
+
+ /* look for duplicates... */
+ for (j = 0; j < num_streams; ++j) {
+ if (streams[j] == route_stream)
+ break;
+ }
+
+ /* ...and drop the stream if we already have it */
+ if (j != num_streams)
+ continue;
+
+ streams[num_streams++] = route_stream;
+ }
+
+ sort(streams, num_streams, sizeof(u32), &cmp_u32, NULL);
- return v4l2_subdev_link_validate_default(
- sink, link, &source_fmt, &sink_fmt);
+ *out_num_streams = num_streams;
+ *out_streams = streams;
+ *allocated = true;
+
+ return 0;
+}
+
+static int v4l2_subdev_link_validate_locked(struct media_link *link)
+{
+ struct v4l2_subdev *sink_subdev =
+ media_entity_to_v4l2_subdev(link->sink->entity);
+ struct device *dev = sink_subdev->entity.graph_obj.mdev->dev;
+ u32 num_source_streams;
+ const u32 *source_streams;
+ bool source_allocated;
+ u32 num_sink_streams;
+ const u32 *sink_streams;
+ bool sink_allocated;
+ unsigned int sink_idx;
+ unsigned int source_idx;
+ int ret;
+
+ dev_dbg(dev, "validating link \"%s\":%u -> \"%s\":%u\n",
+ link->source->entity->name, link->source->index,
+ link->sink->entity->name, link->sink->index);
+
+ ret = v4l2_link_validate_get_streams(link, true, &num_source_streams,
+ &source_streams,
+ &source_allocated);
+ if (ret)
+ return ret;
+
+ ret = v4l2_link_validate_get_streams(link, false, &num_sink_streams,
+ &sink_streams, &sink_allocated);
+ if (ret)
+ goto free_source;
+
+ /*
+ * It is ok to have more source streams than sink streams as extra
+ * source streams can just be ignored (i.e. they go nowhere), but extra
+ * sink streams is an error.
+ */
+ if (num_source_streams < num_sink_streams) {
+ dev_err(dev,
+ "Not enough source streams: %d < %d\n",
+ num_source_streams, num_sink_streams);
+ ret = -EINVAL;
+ goto out;
+ }
+
+ /* Validate source and sink stream formats */
+
+ source_idx = 0;
+
+ for (sink_idx = 0; sink_idx < num_sink_streams; ++sink_idx) {
+ struct v4l2_subdev_format sink_fmt, source_fmt;
+ u32 stream;
+
+ stream = sink_streams[sink_idx];
+
+ for (; source_idx < num_source_streams; ++source_idx) {
+ if (source_streams[source_idx] == stream)
+ break;
+ }
+
+ if (source_idx == num_source_streams) {
+ dev_err(dev, "No source stream for sink stream %u\n",
+ stream);
+ ret = -EINVAL;
+ goto out;
+ }
+
+ dev_dbg(dev, "validating stream \"%s\":%u:%u -> \"%s\":%u:%u\n",
+ link->source->entity->name, link->source->index, stream,
+ link->sink->entity->name, link->sink->index, stream);
+
+ ret = v4l2_subdev_link_validate_get_format(link->source, stream,
+ &source_fmt);
+ if (ret < 0) {
+ dev_dbg(dev, "Failed to get format for \"%s\":%u:%u (but that's ok)\n",
+ link->source->entity->name, link->source->index,
+ stream);
+ ret = 0;
+ continue;
+ }
+
+ ret = v4l2_subdev_link_validate_get_format(link->sink, stream,
+ &sink_fmt);
+ if (ret < 0) {
+ dev_dbg(dev, "Failed to get format for \"%s\":%u:%u (but that's ok)\n",
+ link->sink->entity->name, link->sink->index,
+ stream);
+ ret = 0;
+ continue;
+ }
+
+ /* TODO: add stream number to link_validate() */
+ ret = v4l2_subdev_call(sink_subdev, pad, link_validate, link,
+ &source_fmt, &sink_fmt);
+ if (!ret)
+ continue;
+
+ if (ret != -ENOIOCTLCMD)
+ goto out;
+
+ ret = v4l2_subdev_link_validate_default(sink_subdev, link,
+ &source_fmt, &sink_fmt);
+
+ if (ret)
+ goto out;
+ }
+
+out:
+ if (sink_allocated)
+ kfree(sink_streams);
+
+free_source:
+ if (source_allocated)
+ kfree(source_streams);
+
+ return ret;
+}
+
+int v4l2_subdev_link_validate(struct media_link *link)
+{
+ struct v4l2_subdev *source_sd, *sink_sd;
+ struct v4l2_subdev_state *source_state, *sink_state;
+ int ret;
+
+ sink_sd = media_entity_to_v4l2_subdev(link->sink->entity);
+ source_sd = media_entity_to_v4l2_subdev(link->source->entity);
+
+ sink_state = v4l2_subdev_get_active_state(sink_sd);
+ source_state = v4l2_subdev_get_active_state(source_sd);
+
+ if (sink_state)
+ v4l2_subdev_lock_state(sink_state);
+
+ if (source_state)
+ v4l2_subdev_lock_state(source_state);
+
+ ret = v4l2_subdev_link_validate_locked(link);
+
+ if (sink_state)
+ v4l2_subdev_unlock_state(sink_state);
+
+ if (source_state)
+ v4l2_subdev_unlock_state(source_state);
+
+ return ret;
}
EXPORT_SYMBOL_GPL(v4l2_subdev_link_validate);
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 28/36] media: subdev: add "opposite" stream helper funcs
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (26 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 27/36] media: subdev: use streams in v4l2_subdev_link_validate() Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-16 10:50 ` Jacopo Mondi
2022-03-01 16:11 ` [PATCH v11 29/36] media: subdev: add v4l2_subdev_get_fmt() helper function Tomi Valkeinen
` (8 subsequent siblings)
36 siblings, 1 reply; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
Add two helper functions to make dealing with streams easier:
v4l2_subdev_routing_find_opposite_end - given a routing table and a pad
+ stream, return the pad + stream on the opposite side of the subdev.
v4l2_subdev_state_get_opposite_stream_format - return a pointer to the
format on the pad + stream on the opposite side from the given pad +
stream.
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
---
drivers/media/v4l2-core/v4l2-subdev.c | 48 +++++++++++++++++++++++++++
include/media/v4l2-subdev.h | 36 ++++++++++++++++++++
2 files changed, 84 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 091b854e00d0..e65f802fe2aa 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -1459,6 +1459,54 @@ v4l2_subdev_state_get_stream_format(struct v4l2_subdev_state *state,
}
EXPORT_SYMBOL_GPL(v4l2_subdev_state_get_stream_format);
+int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
+ u32 pad, u32 stream, u32 *other_pad,
+ u32 *other_stream)
+{
+ unsigned int i;
+
+ for (i = 0; i < routing->num_routes; ++i) {
+ struct v4l2_subdev_route *route = &routing->routes[i];
+
+ if (route->source_pad == pad &&
+ route->source_stream == stream) {
+ if (other_pad)
+ *other_pad = route->sink_pad;
+ if (other_stream)
+ *other_stream = route->sink_stream;
+ return 0;
+ }
+
+ if (route->sink_pad == pad && route->sink_stream == stream) {
+ if (other_pad)
+ *other_pad = route->source_pad;
+ if (other_stream)
+ *other_stream = route->source_stream;
+ return 0;
+ }
+ }
+
+ return -EINVAL;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_routing_find_opposite_end);
+
+struct v4l2_mbus_framefmt *
+v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
+ u32 pad, u32 stream)
+{
+ u32 other_pad, other_stream;
+ int ret;
+
+ ret = v4l2_subdev_routing_find_opposite_end(&state->routing, pad, stream,
+ &other_pad, &other_stream);
+ if (ret)
+ return NULL;
+
+ return v4l2_subdev_state_get_stream_format(state, other_pad,
+ other_stream);
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_state_get_opposite_stream_format);
+
#endif /* CONFIG_MEDIA_CONTROLLER */
void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 2a40ad273cf8..ed3fe21637e6 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -1445,6 +1445,42 @@ struct v4l2_mbus_framefmt *
v4l2_subdev_state_get_stream_format(struct v4l2_subdev_state *state,
unsigned int pad, u32 stream);
+/**
+ * v4l2_subdev_routing_find_opposite_end() - Find the opposite stream
+ * @routing: routing used to find the opposite side
+ * @pad: pad id
+ * @stream: stream id
+ * @other_pad: pointer used to return the opposite pad
+ * @other_stream: pointer used to return the opposite stream
+ *
+ * This function uses the routing table to find the pad + stream which is
+ * opposite the given pad + stream.
+ *
+ * @other_pad and/or @other_stream can be NULL if the caller does not need the
+ * value.
+ *
+ * Returns 0 on success, or -EINVAL if no matching route is found.
+ */
+int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
+ u32 pad, u32 stream, u32 *other_pad,
+ u32 *other_stream);
+
+/**
+ * v4l2_subdev_state_get_opposite_stream_format() - Get pointer to opposite
+ * stream format
+ * @state: subdevice state
+ * @pad: pad id
+ * @stream: stream id
+ *
+ * This returns a pointer to &struct v4l2_mbus_framefmt for the pad + stream
+ * that is opposite the given pad + stream in the subdev state.
+ *
+ * If the state does not contain the given pad + stream, NULL is returned.
+ */
+struct v4l2_mbus_framefmt *
+v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
+ u32 pad, u32 stream);
+
#endif /* CONFIG_MEDIA_CONTROLLER */
/**
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 29/36] media: subdev: add v4l2_subdev_get_fmt() helper function
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (27 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 28/36] media: subdev: add "opposite" stream helper funcs Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-16 11:00 ` Jacopo Mondi
2022-03-01 16:11 ` [PATCH v11 30/36] media: subdev: Fallback to pad config in v4l2_subdev_get_fmt() Tomi Valkeinen
` (7 subsequent siblings)
36 siblings, 1 reply; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
Add v4l2_subdev_get_fmt() helper function which implements
v4l2_subdev_pad_ops.get_fmt using streams. Subdev drivers that do not
need to do anything special in their get_fmt op can use this helper
directly for v4l2_subdev_pad_ops.get_fmt.
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
---
drivers/media/v4l2-core/v4l2-subdev.c | 16 ++++++++++++++++
include/media/v4l2-subdev.h | 17 +++++++++++++++++
2 files changed, 33 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index e65f802fe2aa..c1cc9b91dba7 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -1507,6 +1507,22 @@ v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
}
EXPORT_SYMBOL_GPL(v4l2_subdev_state_get_opposite_stream_format);
+int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
+ struct v4l2_subdev_format *format)
+{
+ struct v4l2_mbus_framefmt *fmt;
+
+ fmt = v4l2_subdev_state_get_stream_format(state, format->pad,
+ format->stream);
+ if (!fmt)
+ return -EINVAL;
+
+ format->format = *fmt;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_get_fmt);
+
#endif /* CONFIG_MEDIA_CONTROLLER */
void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index ed3fe21637e6..a80830801a7f 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -1481,6 +1481,23 @@ struct v4l2_mbus_framefmt *
v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
u32 pad, u32 stream);
+/**
+ * v4l2_subdev_get_fmt() - Fill format based on state
+ * @sd: subdevice
+ * @state: subdevice state
+ * @format: pointer to &struct v4l2_subdev_format
+ *
+ * Fill @format based on the pad and stream given in the @format struct.
+ *
+ * This function can be used by the subdev drivers to implement
+ * v4l2_subdev_pad_ops.get_fmt if the subdev driver does not need to do
+ * anything special in their get_fmt op.
+ *
+ * Returns 0 on success, error value otherwise.
+ */
+int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
+ struct v4l2_subdev_format *format);
+
#endif /* CONFIG_MEDIA_CONTROLLER */
/**
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 30/36] media: subdev: Fallback to pad config in v4l2_subdev_get_fmt()
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (28 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 29/36] media: subdev: add v4l2_subdev_get_fmt() helper function Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-16 11:03 ` Jacopo Mondi
2022-03-01 16:11 ` [PATCH v11 31/36] media: subdev: add v4l2_subdev_set_routing_with_fmt() helper Tomi Valkeinen
` (6 subsequent siblings)
36 siblings, 1 reply; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
If the subdev doesn't implement routing support, fallback to pad config
as the storage for pad formats. This allows using the V4L2 subdev active
state API and the v4l2_subdev_get_fmt() helper in subdev drivers that
don't implement routing support.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
drivers/media/v4l2-core/v4l2-subdev.c | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index c1cc9b91dba7..7f50871054cd 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -1512,8 +1512,14 @@ int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
{
struct v4l2_mbus_framefmt *fmt;
- fmt = v4l2_subdev_state_get_stream_format(state, format->pad,
- format->stream);
+ if (sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED)
+ fmt = v4l2_subdev_state_get_stream_format(state, format->pad,
+ format->stream);
+ else if (format->pad < sd->entity.num_pads && format->stream == 0)
+ fmt = v4l2_subdev_get_try_format(sd, state, format->pad);
+ else
+ fmt = NULL;
+
if (!fmt)
return -EINVAL;
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 31/36] media: subdev: add v4l2_subdev_set_routing_with_fmt() helper
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (29 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 30/36] media: subdev: Fallback to pad config in v4l2_subdev_get_fmt() Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-16 11:04 ` Jacopo Mondi
2022-03-01 16:11 ` [PATCH v11 32/36] media: subdev: add v4l2_subdev_routing_validate() helper Tomi Valkeinen
` (5 subsequent siblings)
36 siblings, 1 reply; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
v4l2_subdev_set_routing_with_fmt() is the same as
v4l2_subdev_set_routing(), but additionally initializes all the streams
with the given format.
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
---
drivers/media/v4l2-core/v4l2-subdev.c | 22 ++++++++++++++++++++++
include/media/v4l2-subdev.h | 16 ++++++++++++++++
2 files changed, 38 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 7f50871054cd..1ceee8313246 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -1438,6 +1438,28 @@ int v4l2_subdev_set_routing(struct v4l2_subdev *sd,
}
EXPORT_SYMBOL_GPL(v4l2_subdev_set_routing);
+int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_krouting *routing,
+ const struct v4l2_mbus_framefmt *fmt)
+{
+ struct v4l2_subdev_stream_configs *stream_configs;
+ unsigned int i;
+ int ret;
+
+ ret = v4l2_subdev_set_routing(sd, state, routing);
+ if (ret)
+ return ret;
+
+ stream_configs = &state->stream_configs;
+
+ for (i = 0; i < stream_configs->num_configs; ++i)
+ stream_configs->configs[i].fmt = *fmt;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_set_routing_with_fmt);
+
struct v4l2_mbus_framefmt *
v4l2_subdev_state_get_stream_format(struct v4l2_subdev_state *state,
unsigned int pad, u32 stream)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index a80830801a7f..97db6dfc0b7a 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -1430,6 +1430,22 @@ int v4l2_subdev_set_routing(struct v4l2_subdev *sd,
struct v4l2_subdev_state *state,
struct v4l2_subdev_krouting *routing);
+/**
+ * v4l2_subdev_set_routing_with_fmt() - Set given routing and format to subdev
+ * state
+ * @sd: The subdevice
+ * @state: The subdevice state
+ * @routing: Routing that will be copied to subdev state
+ * @fmt: Format used to initialize all the streams
+ *
+ * This is the same as v4l2_subdev_set_routing, but additionally initializes
+ * all the streams using the given format.
+ */
+int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_krouting *routing,
+ const struct v4l2_mbus_framefmt *fmt);
+
/**
* v4l2_subdev_state_get_stream_format() - Get pointer to a stream format
* @state: subdevice state
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 32/36] media: subdev: add v4l2_subdev_routing_validate() helper
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (30 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 31/36] media: subdev: add v4l2_subdev_set_routing_with_fmt() helper Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-16 11:10 ` Jacopo Mondi
2022-03-01 16:11 ` [PATCH v11 33/36] media: subdev: Add for_each_active_route() macro Tomi Valkeinen
` (4 subsequent siblings)
36 siblings, 1 reply; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Add a v4l2_subdev_routing_validate() helper for verifying routing for
common cases like only allowing non-overlapping 1-to-1 streams.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/v4l2-core/v4l2-subdev.c | 102 ++++++++++++++++++++++++++
include/media/v4l2-subdev.h | 39 ++++++++++
2 files changed, 141 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 1ceee8313246..a19236cf11b8 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -1551,6 +1551,108 @@ int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
}
EXPORT_SYMBOL_GPL(v4l2_subdev_get_fmt);
+int v4l2_subdev_routing_validate(struct v4l2_subdev *sd,
+ const struct v4l2_subdev_krouting *routing,
+ enum v4l2_subdev_routing_restriction disallow)
+{
+ u32 *remote_pads = NULL;
+ unsigned int i, j;
+ int ret = -EINVAL;
+
+ if (disallow & V4L2_SUBDEV_ROUTING_NO_STREAM_MIX) {
+ remote_pads = kcalloc(sd->entity.num_pads, sizeof(*remote_pads),
+ GFP_KERNEL);
+ if (!remote_pads)
+ return -ENOMEM;
+
+ for (i = 0; i < sd->entity.num_pads; ++i)
+ remote_pads[i] = U32_MAX;
+ }
+
+ for (i = 0; i < routing->num_routes; ++i) {
+ const struct v4l2_subdev_route *route = &routing->routes[i];
+
+ /* Validate the sink and source pad numbers. */
+ if (route->sink_pad >= sd->entity.num_pads ||
+ !(sd->entity.pads[route->sink_pad].flags & MEDIA_PAD_FL_SINK)) {
+ dev_dbg(sd->dev, "route %u sink (%u) is not a sink pad\n",
+ i, route->sink_pad);
+ goto out;
+ }
+
+ if (route->source_pad >= sd->entity.num_pads ||
+ !(sd->entity.pads[route->source_pad].flags & MEDIA_PAD_FL_SOURCE)) {
+ dev_dbg(sd->dev, "route %u source (%u) is not a source pad\n",
+ i, route->source_pad);
+ goto out;
+ }
+
+ /*
+ * V4L2_SUBDEV_ROUTING_NO_STREAM_MIX: Streams on the same pad
+ * may not be routed to streams on different pads.
+ */
+ if (disallow & V4L2_SUBDEV_ROUTING_NO_STREAM_MIX) {
+ if (remote_pads[route->sink_pad] != U32_MAX &&
+ remote_pads[route->sink_pad] != route->source_pad) {
+ dev_dbg(sd->dev,
+ "route %u attempts to mix %s streams\n",
+ i, "sink");
+ goto out;
+ }
+
+ if (remote_pads[route->source_pad] != U32_MAX &&
+ remote_pads[route->source_pad] != route->sink_pad) {
+ dev_dbg(sd->dev,
+ "route %u attempts to mix %s streams\n",
+ i, "source");
+ goto out;
+ }
+
+ remote_pads[route->sink_pad] = route->source_pad;
+ remote_pads[route->source_pad] = route->sink_pad;
+ }
+
+ for (j = i + 1; j < routing->num_routes; ++j) {
+ const struct v4l2_subdev_route *r = &routing->routes[j];
+
+ /*
+ * V4L2_SUBDEV_ROUTING_NO_1_TO_N: No two routes can
+ * originate from the same (sink) stream.
+ */
+ if ((disallow & V4L2_SUBDEV_ROUTING_NO_1_TO_N) &&
+ route->sink_pad == r->sink_pad &&
+ route->sink_stream == r->sink_stream) {
+ dev_dbg(sd->dev,
+ "routes %u and %u originate from same sink (%u/%u)\n",
+ i, j, route->sink_pad,
+ route->sink_stream);
+ goto out;
+ }
+
+ /*
+ * V4L2_SUBDEV_ROUTING_NO_N_TO_1: No two routes can end
+ * at the same (source) stream.
+ */
+ if ((disallow & V4L2_SUBDEV_ROUTING_NO_N_TO_1) &&
+ route->source_pad == r->source_pad &&
+ route->source_stream == r->source_stream) {
+ dev_dbg(sd->dev,
+ "routes %u and %u end at same source (%u/%u)\n",
+ i, j, route->source_pad,
+ route->source_stream);
+ goto out;
+ }
+ }
+ }
+
+ ret = 0;
+
+out:
+ kfree(remote_pads);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_routing_validate);
+
#endif /* CONFIG_MEDIA_CONTROLLER */
void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 97db6dfc0b7a..f75e161d646b 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -1514,6 +1514,45 @@ v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
struct v4l2_subdev_format *format);
+/**
+ * enum v4l2_subdev_routing_restriction - Subdevice internal routing restrictions
+ *
+ * @V4L2_SUBDEV_ROUTING_NO_1_TO_N:
+ * an input stream may not be routed to multiple output streams (stream
+ * duplication)
+ * @V4L2_SUBDEV_ROUTING_NO_N_TO_1:
+ * multiple input streams may not be routed to the same output stream
+ * (stream merging)
+ * @V4L2_SUBDEV_ROUTING_NO_STREAM_MIX:
+ * streams on the same pad may not be routed to streams on different pads
+ * @V4L2_SUBDEV_ROUTING_ONLY_1_TO_1:
+ * only non-overlapping 1-to-1 stream routing is allowed (a combination of
+ * @V4L2_SUBDEV_ROUTING_NO_1_TO_N and @V4L2_SUBDEV_ROUTING_NO_N_TO_1)
+ */
+enum v4l2_subdev_routing_restriction {
+ V4L2_SUBDEV_ROUTING_NO_1_TO_N = BIT(0),
+ V4L2_SUBDEV_ROUTING_NO_N_TO_1 = BIT(1),
+ V4L2_SUBDEV_ROUTING_NO_STREAM_MIX = BIT(2),
+ V4L2_SUBDEV_ROUTING_ONLY_1_TO_1 =
+ V4L2_SUBDEV_ROUTING_NO_1_TO_N |
+ V4L2_SUBDEV_ROUTING_NO_N_TO_1,
+};
+
+/**
+ * v4l2_subdev_routing_validate() - Verify that routes comply with driver
+ * constraints
+ * @sd: The subdevice
+ * @routing: Routing to verify
+ * @disallow: Restrictions on routes
+ *
+ * This verifies that the given routing complies with the @disallow constraints.
+ *
+ * Returns 0 on success, error value otherwise.
+ */
+int v4l2_subdev_routing_validate(struct v4l2_subdev *sd,
+ const struct v4l2_subdev_krouting *routing,
+ enum v4l2_subdev_routing_restriction disallow);
+
#endif /* CONFIG_MEDIA_CONTROLLER */
/**
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 33/36] media: subdev: Add for_each_active_route() macro
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (31 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 32/36] media: subdev: add v4l2_subdev_routing_validate() helper Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 34/36] media: v4l2-subdev: Add v4l2_subdev_state_xlate_streams() helper Tomi Valkeinen
` (3 subsequent siblings)
36 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
From: Jacopo Mondi <jacopo+renesas@jmondi.org>
Add a for_each_active_route() macro to replace the repeated pattern
of iterating on the active routes of a routing table.
Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
.clang-format | 1 +
drivers/media/v4l2-core/v4l2-subdev.c | 20 ++++++++++++++++++++
include/media/v4l2-subdev.h | 13 +++++++++++++
3 files changed, 34 insertions(+)
diff --git a/.clang-format b/.clang-format
index fa959436bcfd..50b8bf89d519 100644
--- a/.clang-format
+++ b/.clang-format
@@ -140,6 +140,7 @@ ForEachMacros:
- 'for_each_active_dev_scope'
- 'for_each_active_drhd_unit'
- 'for_each_active_iommu'
+ - 'for_each_active_route'
- 'for_each_aggr_pgid'
- 'for_each_available_child_of_node'
- 'for_each_bio'
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index a19236cf11b8..e30338a53088 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -1653,6 +1653,26 @@ int v4l2_subdev_routing_validate(struct v4l2_subdev *sd,
}
EXPORT_SYMBOL_GPL(v4l2_subdev_routing_validate);
+struct v4l2_subdev_route *
+__v4l2_subdev_next_active_route(const struct v4l2_subdev_krouting *routing,
+ struct v4l2_subdev_route *route)
+{
+ if (route)
+ ++route;
+ else
+ route = &routing->routes[0];
+
+ for (; route < routing->routes + routing->num_routes; ++route) {
+ if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+ continue;
+
+ return route;
+ }
+
+ return NULL;
+}
+EXPORT_SYMBOL_GPL(__v4l2_subdev_next_active_route);
+
#endif /* CONFIG_MEDIA_CONTROLLER */
void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index f75e161d646b..1eced0f47296 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -1553,6 +1553,19 @@ int v4l2_subdev_routing_validate(struct v4l2_subdev *sd,
const struct v4l2_subdev_krouting *routing,
enum v4l2_subdev_routing_restriction disallow);
+struct v4l2_subdev_route *
+__v4l2_subdev_next_active_route(const struct v4l2_subdev_krouting *routing,
+ struct v4l2_subdev_route *route);
+
+/**
+ * for_each_active_route - iterate on all active routes of a routing table
+ * @routing: The routing table
+ * @route: The route iterator
+ */
+#define for_each_active_route(routing, route) \
+ for ((route) = NULL; \
+ ((route) = __v4l2_subdev_next_active_route((routing), (route)));)
+
#endif /* CONFIG_MEDIA_CONTROLLER */
/**
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 34/36] media: v4l2-subdev: Add v4l2_subdev_state_xlate_streams() helper
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (32 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 33/36] media: subdev: Add for_each_active_route() macro Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-16 11:26 ` Jacopo Mondi
2022-07-07 10:27 ` Tomasz Figa
2022-03-01 16:11 ` [PATCH v11 35/36] media: v4l2-subdev: Add subdev .(enable|disable)_streams() operations Tomi Valkeinen
` (2 subsequent siblings)
36 siblings, 2 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Add a helper function to translate streams between two pads of a subdev,
using the subdev's internal routing table.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/v4l2-core/v4l2-subdev.c | 25 +++++++++++++++++++++++++
include/media/v4l2-subdev.h | 23 +++++++++++++++++++++++
2 files changed, 48 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index e30338a53088..6a9fc62dacbf 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -1529,6 +1529,31 @@ v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
}
EXPORT_SYMBOL_GPL(v4l2_subdev_state_get_opposite_stream_format);
+u64 v4l2_subdev_state_xlate_streams(const struct v4l2_subdev_state *state,
+ u32 pad0, u32 pad1, u64 *streams)
+{
+ const struct v4l2_subdev_krouting *routing = &state->routing;
+ struct v4l2_subdev_route *route;
+ u64 streams0 = 0;
+ u64 streams1 = 0;
+
+ for_each_active_route(routing, route) {
+ if (route->sink_pad == pad0 && route->source_pad == pad1 &&
+ (*streams & BIT(route->sink_stream))) {
+ streams0 |= BIT(route->sink_stream);
+ streams1 |= BIT(route->source_stream);
+ }
+ if (route->source_pad == pad0 && route->sink_pad == pad1 &&
+ (*streams & BIT(route->source_stream))) {
+ streams0 |= BIT(route->source_stream);
+ streams1 |= BIT(route->sink_stream);
+ }
+ }
+
+ *streams = streams0;
+ return streams1;
+}
+
int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
struct v4l2_subdev_format *format)
{
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 1eced0f47296..992debe116ac 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -1497,6 +1497,29 @@ struct v4l2_mbus_framefmt *
v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
u32 pad, u32 stream);
+/**
+ * v4l2_subdev_state_xlate_streams() - Translate streams from one pad to another
+ *
+ * @state: Subdevice state
+ * @pad0: The first pad
+ * @pad1: The second pad
+ * @streams: Streams bitmask on the first pad
+ *
+ * Streams on sink pads of a subdev are routed to source pads as expressed in
+ * the subdev state routing table. Stream numbers don't necessarily match on
+ * the sink and source side of a route. This function translates stream numbers
+ * on @pad0, expressed as a bitmask in @streams, to the corresponding streams
+ * on @pad1 using the routing table from the @state. It returns the stream mask
+ * on @pad1, and updates @streams with the streams that have been found in the
+ * routing table.
+ *
+ * @pad0 and @pad1 must be a sink and a source, in any order.
+ *
+ * Return: The bitmask of streams of @pad1 that are routed to @streams on @pad0.
+ */
+u64 v4l2_subdev_state_xlate_streams(const struct v4l2_subdev_state *state,
+ u32 pad0, u32 pad1, u64 *streams);
+
/**
* v4l2_subdev_get_fmt() - Fill format based on state
* @sd: subdevice
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 35/36] media: v4l2-subdev: Add subdev .(enable|disable)_streams() operations
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (33 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 34/36] media: v4l2-subdev: Add v4l2_subdev_state_xlate_streams() helper Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-16 12:05 ` Jacopo Mondi
2022-03-01 16:11 ` [PATCH v11 36/36] media: v4l2-subdev: Add v4l2_subdev_s_stream_helper() function Tomi Valkeinen
2022-07-07 10:38 ` [PATCH v11 00/36] v4l: routing and streams support Tomasz Figa
36 siblings, 1 reply; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Add two new subdev pad operations, .enable_streams() and
.disable_streams(), to allow control of individual streams per pad. This
is a superset of what the video .s_stream() operation implements.
To help with handling of backward compatibility, add two wrapper
functions around those operations, and require their usage in drivers.
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
drivers/media/v4l2-core/v4l2-subdev.c | 216 ++++++++++++++++++++++++++
include/media/v4l2-subdev.h | 85 ++++++++++
2 files changed, 301 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 6a9fc62dacbf..f75a1995a70b 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -1698,6 +1698,222 @@ __v4l2_subdev_next_active_route(const struct v4l2_subdev_krouting *routing,
}
EXPORT_SYMBOL_GPL(__v4l2_subdev_next_active_route);
+static int v4l2_subdev_enable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
+ u64 streams_mask)
+{
+ unsigned int i;
+ int ret;
+
+ /*
+ * The subdev doesn't implement pad-based stream enable, fall back
+ * on the .s_stream() operation. This can only be done for subdevs that
+ * have a single source pad, as sd->enabled_streams is global to the
+ * subdev.
+ */
+ if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
+ return -EOPNOTSUPP;
+
+ for (i = 0; i < sd->entity.num_pads; ++i) {
+ if (i != pad && sd->entity.pads[i].flags & MEDIA_PAD_FL_SOURCE)
+ return -EOPNOTSUPP;
+ }
+
+ if (sd->enabled_streams & streams_mask)
+ return -EALREADY;
+
+ /* Start streaming when the first streams are enabled. */
+ if (!sd->enabled_streams) {
+ ret = v4l2_subdev_call(sd, video, s_stream, 1);
+ if (ret)
+ return ret;
+ }
+
+ sd->enabled_streams |= streams_mask;
+
+ return 0;
+}
+
+int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
+ u64 streams_mask)
+{
+ struct device *dev = sd->entity.graph_obj.mdev->dev;
+ struct v4l2_subdev_state *state;
+ u64 found_streams = 0;
+ unsigned int i;
+ int ret;
+
+ /* A few basic sanity checks first. */
+ if (pad >= sd->entity.num_pads)
+ return -EINVAL;
+
+ if (!streams_mask)
+ return 0;
+
+ /* Fallback on .s_stream() if .enable_streams() isn't available. */
+ if (!sd->ops->pad || !sd->ops->pad->enable_streams)
+ return v4l2_subdev_enable_streams_fallback(sd, pad,
+ streams_mask);
+
+ state = v4l2_subdev_lock_and_get_active_state(sd);
+
+ /*
+ * Verify that the requested streams exist and that they are not
+ * already enabled.
+ */
+ for (i = 0; i < state->stream_configs.num_configs; ++i) {
+ struct v4l2_subdev_stream_config *cfg =
+ &state->stream_configs.configs[i];
+
+ if (cfg->pad != pad || !(streams_mask & BIT(cfg->stream)))
+ continue;
+
+ found_streams |= BIT(cfg->stream);
+
+ if (cfg->enabled) {
+ dev_dbg(dev, "stream %u already enabled on %s/%u\n",
+ cfg->stream, sd->entity.name, pad);
+ ret = -EALREADY;
+ goto done;
+ }
+ }
+
+ if (found_streams != streams_mask) {
+ dev_dbg(dev, "streams 0x%llx not found on %s/%u\n",
+ streams_mask & ~found_streams, sd->entity.name, pad);
+ ret = -EINVAL;
+ goto done;
+ }
+
+ /* Call the .enable_streams() operation. */
+ ret = v4l2_subdev_call(sd, pad, enable_streams, state, pad,
+ streams_mask);
+ if (ret)
+ goto done;
+
+ /* Mark the streams as enabled. */
+ for (i = 0; i < state->stream_configs.num_configs; ++i) {
+ struct v4l2_subdev_stream_config *cfg =
+ &state->stream_configs.configs[i];
+
+ if (cfg->pad == pad && (streams_mask & BIT(cfg->stream)))
+ cfg->enabled = true;
+ }
+
+done:
+ v4l2_subdev_unlock_state(state);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_enable_streams);
+
+static int v4l2_subdev_disable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
+ u64 streams_mask)
+{
+ unsigned int i;
+ int ret;
+
+ /*
+ * If the subdev doesn't implement pad-based stream enable, fall back
+ * on the .s_stream() operation. This can only be done for subdevs that
+ * have a single source pad, as sd->enabled_streams is global to the
+ * subdev.
+ */
+ if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
+ return -EOPNOTSUPP;
+
+ for (i = 0; i < sd->entity.num_pads; ++i) {
+ if (i != pad && sd->entity.pads[i].flags & MEDIA_PAD_FL_SOURCE)
+ return -EOPNOTSUPP;
+ }
+
+ if ((sd->enabled_streams & streams_mask) != streams_mask)
+ return -EALREADY;
+
+ /* Stop streaming when the last streams are disabled. */
+ if (!(sd->enabled_streams & ~streams_mask)) {
+ ret = v4l2_subdev_call(sd, video, s_stream, 0);
+ if (ret)
+ return ret;
+ }
+
+ sd->enabled_streams &= ~streams_mask;
+
+ return 0;
+}
+
+int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
+ u64 streams_mask)
+{
+ struct device *dev = sd->entity.graph_obj.mdev->dev;
+ struct v4l2_subdev_state *state;
+ u64 found_streams = 0;
+ unsigned int i;
+ int ret;
+
+ /* A few basic sanity checks first. */
+ if (pad >= sd->entity.num_pads)
+ return -EINVAL;
+
+ if (!streams_mask)
+ return 0;
+
+ /* Fallback on .s_stream() if .disable_streams() isn't available. */
+ if (!sd->ops->pad || !sd->ops->pad->disable_streams)
+ return v4l2_subdev_disable_streams_fallback(sd, pad,
+ streams_mask);
+
+ state = v4l2_subdev_lock_and_get_active_state(sd);
+
+ /*
+ * Verify that the requested streams exist and that they are not
+ * already disabled.
+ */
+ for (i = 0; i < state->stream_configs.num_configs; ++i) {
+ struct v4l2_subdev_stream_config *cfg =
+ &state->stream_configs.configs[i];
+
+ if (cfg->pad != pad || !(streams_mask & BIT(cfg->stream)))
+ continue;
+
+ found_streams |= BIT(cfg->stream);
+
+ if (!cfg->enabled) {
+ dev_dbg(dev, "stream %u already disabled on %s/%u\n",
+ cfg->stream, sd->entity.name, pad);
+ ret = -EALREADY;
+ goto done;
+ }
+ }
+
+ if (found_streams != streams_mask) {
+ dev_dbg(dev, "streams 0x%llx not found on %s/%u\n",
+ streams_mask & ~found_streams, sd->entity.name, pad);
+ ret = -EINVAL;
+ goto done;
+ }
+
+ /* Call the .disable_streams() operation. */
+ ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
+ streams_mask);
+ if (ret)
+ goto done;
+
+ /* Mark the streams as disabled. */
+ for (i = 0; i < state->stream_configs.num_configs; ++i) {
+ struct v4l2_subdev_stream_config *cfg =
+ &state->stream_configs.configs[i];
+
+ if (cfg->pad == pad && (streams_mask & BIT(cfg->stream)))
+ cfg->enabled = false;
+ }
+
+done:
+ v4l2_subdev_unlock_state(state);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_disable_streams);
+
#endif /* CONFIG_MEDIA_CONTROLLER */
void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 992debe116ac..bb1713863973 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -702,6 +702,7 @@ struct v4l2_subdev_pad_config {
*
* @pad: pad number
* @stream: stream number
+ * @enabled: has the stream been enabled with v4l2_subdev_enable_stream()
* @fmt: &struct v4l2_mbus_framefmt
* @crop: &struct v4l2_rect to be used for crop
* @compose: &struct v4l2_rect to be used for compose
@@ -711,6 +712,7 @@ struct v4l2_subdev_pad_config {
struct v4l2_subdev_stream_config {
u32 pad;
u32 stream;
+ bool enabled;
struct v4l2_mbus_framefmt fmt;
struct v4l2_rect crop;
@@ -816,6 +818,18 @@ struct v4l2_subdev_state {
*
* @set_routing: enable or disable data connection routes described in the
* subdevice routing table.
+ *
+ * @enable_streams: Enable the streams defined in streams_mask on the given
+ * source pad. Subdevs that implement this operation must use the active
+ * state management provided by the subdev core (enabled through a call to
+ * v4l2_subdev_init_finalize() at initialization time). Do not call
+ * directly, use v4l2_subdev_enable_streams() instead.
+ *
+ * @disable_streams: Disable the streams defined in streams_mask on the given
+ * source pad. Subdevs that implement this operation must use the active
+ * state management provided by the subdev core (enabled through a call to
+ * v4l2_subdev_init_finalize() at initialization time). Do not call
+ * directly, use v4l2_subdev_disable_streams() instead.
*/
struct v4l2_subdev_pad_ops {
int (*init_cfg)(struct v4l2_subdev *sd,
@@ -862,6 +876,12 @@ struct v4l2_subdev_pad_ops {
struct v4l2_subdev_state *state,
enum v4l2_subdev_format_whence which,
struct v4l2_subdev_krouting *route);
+ int (*enable_streams)(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state, u32 pad,
+ u64 streams_mask);
+ int (*disable_streams)(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state, u32 pad,
+ u64 streams_mask);
};
/**
@@ -1007,6 +1027,10 @@ struct v4l2_subdev_platform_data {
* @active_state: Active state for the subdev (NULL for subdevs tracking the
* state internally). Initialized by calling
* v4l2_subdev_init_finalize().
+ * @enabled_streams: Bitmask of enabled streams used by
+ * v4l2_subdev_enable_streams() and
+ * v4l2_subdev_disable_streams() helper functions for fallback
+ * cases.
*
* Each instance of a subdev driver should create this struct, either
* stand-alone or embedded in a larger struct.
@@ -1052,6 +1076,7 @@ struct v4l2_subdev {
* doesn't support it.
*/
struct v4l2_subdev_state *active_state;
+ u64 enabled_streams;
};
@@ -1589,6 +1614,66 @@ __v4l2_subdev_next_active_route(const struct v4l2_subdev_krouting *routing,
for ((route) = NULL; \
((route) = __v4l2_subdev_next_active_route((routing), (route)));)
+/**
+ * v4l2_subdev_enable_streams() - Enable streams on a pad
+ * @sd: The subdevice
+ * @pad: The pad
+ * @streams_mask: Bitmask of streams to enable
+ *
+ * This function enables streams on a source @pad of a subdevice. The pad is
+ * identified by its index, while the streams are identified by the
+ * @streams_mask bitmask. This allows enabling multiple streams on a pad at
+ * once.
+ *
+ * Enabling a stream that is already enabled isn't allowed. If @streams_mask
+ * contains an already enabled stream, this function returns -EALREADY without
+ * performing any operation.
+ *
+ * Per-stream enable is only available for subdevs that implement the
+ * .enable_streams() and .disable_streams() operations. For other subdevs, this
+ * function implements a best-effort compatibility by calling the .s_stream()
+ * operation, limited to subdevs that have a single source pad.
+ *
+ * Return:
+ * * 0: Success
+ * * -EALREADY: One of the streams in streams_mask is already enabled
+ * * -EINVAL: The pad index is invalid, or doesn't correspond to a source pad
+ * * -EOPNOTSUPP: Falling back to the legacy .s_stream() operation is
+ * impossible because the subdev has multiple source pads
+ */
+int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
+ u64 streams_mask);
+
+/**
+ * v4l2_subdev_disable_streams() - Disable streams on a pad
+ * @sd: The subdevice
+ * @pad: The pad
+ * @streams_mask: Bitmask of streams to disable
+ *
+ * This function disables streams on a source @pad of a subdevice. The pad is
+ * identified by its index, while the streams are identified by the
+ * @streams_mask bitmask. This allows disabling multiple streams on a pad at
+ * once.
+ *
+ * Disabling a streams that is not enabled isn't allowed. If @streams_mask
+ * contains a disabled stream, this function returns -EALREADY without
+ * performing any operation.
+ *
+ * Per-stream disable is only available for subdevs that implement the
+ * .enable_streams() and .disable_streams() operations. For other subdevs, this
+ * function implements a best-effort compatibility by calling the .s_stream()
+ * operation, limited to subdevs that have a single source pad.
+ *
+ * Return:
+ * * 0: Success
+ * * -EALREADY: One of the streams in streams_mask is not enabled
+ * * -EINVAL: The pad index is invalid, or doesn't correspond to a source pad
+ * * -EOPNOTSUPP: Falling back to the legacy .s_stream() operation is
+ * impossible because the subdev has multiple source pads
+ */
+int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
+ u64 streams_mask);
+
#endif /* CONFIG_MEDIA_CONTROLLER */
/**
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* [PATCH v11 36/36] media: v4l2-subdev: Add v4l2_subdev_s_stream_helper() function
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (34 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 35/36] media: v4l2-subdev: Add subdev .(enable|disable)_streams() operations Tomi Valkeinen
@ 2022-03-01 16:11 ` Tomi Valkeinen
2022-03-16 12:10 ` Jacopo Mondi
2022-07-07 10:38 ` [PATCH v11 00/36] v4l: routing and streams support Tomasz Figa
36 siblings, 1 reply; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-01 16:11 UTC (permalink / raw)
To: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Cc: Tomi Valkeinen
From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The v4l2_subdev_s_stream_helper() helper can be used by subdevs that
implement the stream-aware .enable_streams() and .disable_streams()
operations to implement .s_stream(). This is limited to subdevs that
have a single source pad.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
drivers/media/v4l2-core/v4l2-subdev.c | 40 +++++++++++++++++++++++++++
include/media/v4l2-subdev.h | 17 ++++++++++++
2 files changed, 57 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index f75a1995a70b..270445821f06 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -1914,6 +1914,46 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
}
EXPORT_SYMBOL_GPL(v4l2_subdev_disable_streams);
+int v4l2_subdev_s_stream_helper(struct v4l2_subdev *sd, int enable)
+{
+ struct v4l2_subdev_state *state;
+ struct v4l2_subdev_route *route;
+ struct media_pad *pad;
+ u64 source_mask = 0;
+ int pad_index = -1;
+
+ /*
+ * Find the source pad. This helper is meant for subdevs that have a
+ * single source pad, so failures shouldn't happen, but catch them
+ * loudly nonetheless as they indicate a driver bug.
+ */
+ media_entity_for_each_pad(&sd->entity, pad) {
+ if (pad->flags & MEDIA_PAD_FL_SOURCE) {
+ pad_index = pad->index;
+ break;
+ }
+ }
+
+ if (WARN_ON(pad_index == -1))
+ return -EINVAL;
+
+ /*
+ * As there's a single source pad, just collect all the source streams.
+ */
+ state = v4l2_subdev_lock_and_get_active_state(sd);
+
+ for_each_active_route(&state->routing, route)
+ source_mask |= BIT(route->source_stream);
+
+ v4l2_subdev_unlock_state(state);
+
+ if (enable)
+ return v4l2_subdev_enable_streams(sd, pad_index, source_mask);
+ else
+ return v4l2_subdev_disable_streams(sd, pad_index, source_mask);
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_s_stream_helper);
+
#endif /* CONFIG_MEDIA_CONTROLLER */
void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index bb1713863973..817452ec30bb 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -1674,6 +1674,23 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
u64 streams_mask);
+/**
+ * v4l2_subdev_s_stream_helper() - Helper to implement the subdev s_stream
+ * operation using enable_streams and disable_streams
+ * @sd: The subdevice
+ * @enable: Enable to disable streaming
+ *
+ * Subdevice drivers that implement the streams-aware
+ * &v4l2_subdev_pad_ops.enable_streams and &v4l2_subdev_pad_ops.disable_streams
+ * operations can use this helper to implement the legacy
+ * &v4l2_subdev_video_ops.s_stream operation.
+ *
+ * This helper can only be used by subdevs that have a single source pad.
+ *
+ * Return: 0 on success, or a negative error code otherwise.
+ */
+int v4l2_subdev_s_stream_helper(struct v4l2_subdev *sd, int enable);
+
#endif /* CONFIG_MEDIA_CONTROLLER */
/**
--
2.25.1
^ permalink raw reply related [flat|nested] 67+ messages in thread
* Re: [PATCH v11 25/36] media: Documentation: add multiplexed streams documentation
2022-03-01 16:11 ` [PATCH v11 25/36] media: Documentation: add multiplexed streams documentation Tomi Valkeinen
@ 2022-03-16 9:04 ` Jacopo Mondi
2022-03-17 7:18 ` Tomi Valkeinen
0 siblings, 1 reply; 67+ messages in thread
From: Jacopo Mondi @ 2022-03-16 9:04 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Hi Tomi
On Tue, Mar 01, 2022 at 06:11:45PM +0200, Tomi Valkeinen wrote:
> Add documentation related to multiplexed streams.
>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
> .../driver-api/media/v4l2-subdev.rst | 8 +
> .../userspace-api/media/v4l/dev-subdev.rst | 165 ++++++++++++++++++
> 2 files changed, 173 insertions(+)
>
> diff --git a/Documentation/driver-api/media/v4l2-subdev.rst b/Documentation/driver-api/media/v4l2-subdev.rst
> index 115211cef4d1..80654f1bcac9 100644
> --- a/Documentation/driver-api/media/v4l2-subdev.rst
> +++ b/Documentation/driver-api/media/v4l2-subdev.rst
> @@ -593,6 +593,14 @@ before calling v4l2_subdev_init_finalize():
>
> This shares the driver's private mutex between the controls and the states.
>
> +Streams, multiplexed media pads and internal routing
> +----------------------------------------------------
> +
> +A subdevice driver can implement support for multiplexed streams by setting
Let me start by being picky with a minor thing: the rest of the
documentation seems to use "sub-device". Here you have "sub-device",
"subdevice" and "subdev". I think "sub-device" should be used
everywhere
> +the V4L2_SUBDEV_FL_MULTIPLEXED subdev flag and implementing support for
> +centrally managed subdev active state, routing and stream based
> +configuration.
> +
> V4L2 sub-device functions and data structures
> ---------------------------------------------
>
> diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> index a67c2749089a..fd042afeddd6 100644
> --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
> +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> @@ -503,3 +503,168 @@ source pads.
> :maxdepth: 1
>
> subdev-formats
> +
> +Streams, multiplexed media pads and internal routing
> +----------------------------------------------------
> +
> +Commonly V4L2 subdevices support only separate video streams, that is, each
> +link in the media graph and each pad in a subdevice pass through a single
> +video stream. Thus each pad contains a format configuration for that single
Isn't it the other way around ? A single video stream passes through a
media link.
> +stream. In some cases a subdev can do stream processing and split a stream
> +into two or compose two streams into one, but the inputs and outputs for the
> +subdev are still a single stream per pad.
> +
> +Some hardware, e.g. MIPI CSI-2, support multiplexed streams, that is, a single
> +bus carries multiple streams. Thus a camera could output two streams, a pixel
> +stream and a metadata stream, and a bridge subdev could route the streams
> +from multiple input pads into a single output pad.
I would use sink/source and not input/output and slightly rephrase
this, because I understand your "bridge subdev" in this example is a
CSI-2 transmitter but "bridge" usually refers to receiver drivers on
the SoC.
What about
Some hardware, e.g. MIPI CSI-2, support multiplexed streams, that is,
multiple data streams are transmitted on the same bus, which is
represented by a media link connecting a transmitter source pad with a
sink pad on the receiver. In example, a camera sensor can produce two
distinct streams, a pixel stream and a metadata stream, which are
transmitted on the multiplexed data bus, represented by a media link
which connects the single sensor's source pad with the receiver sink
pad. The stream-aware receiver will de-multiplex the streams received
on the its sink pad and allows to route them individually to one of
its source pads.
Sensor Receiver
+----------+ +--------------+
| Pixel | | ___________(1)
| \_(0) --------(0)/ |
| Meta / | | \___________(2)
| | | |
+----------+ +---------------
(not sure if makes sense to have a drawing here, but... )
> +
> +Subdevice drivers that support multiplexed streams are compatible with
> +non-multiplexed subdev drivers, but, of course, require a routing configuration
> +where the link between those two types of drivers contains only a single
> +stream.
> +
> +Understanding streams
> +^^^^^^^^^^^^^^^^^^^^^
> +
> +A stream is a stream of content (e.g. pixel data or metadata) flowing through
> +the media pipeline from a source (e.g. a sensor) towards the final sink(e.g. a
space between "sink("
> +receiver and demultiplexer in a SoC). Each media link carries all the streams
the enabled streams ?
> +from one end of the link to the other, and subdevices have routing tables which
> +describe how the incoming streams from sink pads are routed to the source
> +pads.
"... from sink pad", singular..
I would rather say "how the incoming streams from the multiplexed sink
pad are routed to source pads".
> +
> +A stream ID (often just "stream") is a media link-local identifier for a
> +stream. In other words, a configuration for a particular stream ID must exist
s/In other words//
Should the configuration on both ends of the link also be identical ?
> +on both sides of a media link, but another stream ID can be used for the same
> +stream at the other side of the subdevice.
I would
A stream ID (often just "stream") is a media link-local identifier for a
stream. The configuration for a stream ID must exist and be identical
on both ends of a media link connecting two multiplexed pads.
and leave the bits about routing out ?
> +
> +A stream at a specific point in the media pipeline is identified with the
> +subdev and a (pad, stream) pair. For subdevices that do not support
> +multiplexed streams the 'stream' is always 0.
> +
> +Configuring streams
> +^^^^^^^^^^^^^^^^^^^
> +
> +The configuration of the streams is done individually for each subdevice and
> +the validity of the streams between subdevices is validated when the pipeline
> +is started.
> +
> +There are three steps in configuring the streams:
> +
> +1) Set up links. Connect the pads between subdevices using the :ref:`Media
> +Controller API <media_controller>`
> +
> +2) Routing. The routing table for the subdevice must be set with
> +:ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl. Note that
> +setting the routing table will reset all the stream configurations.
"stream configurations in a media entity".
> +
> +3) Configure streams. Each route endpoint must be configured
> +with :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>`.
> +
> +Multiplexed streams setup example
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +A simple example of a multiplexed stream setup might be as follows:
> +
> +- Two identical sensors (Sensor A and Sensor B). Each sensor has a single
> + source pad (pad 0), and outputs two streams, pixel data and metadata.
a single source pad (pad 0) which carries two streams: a pixel data
stream and a metadata stream.
> +
> +- Multiplexer bridge (Bridge). The bridge has two sink pads, connected to the
> + sensors (pads 0, 1), and one source pad (pad 2), which outputs all 4
> + streams.
> +
> +- Receiver in the SoC (Receiver). The receiver has a single sink pad (pad 0),
> + connected to the bridge, and four source pads (pads 1-4), going to the DMA
> + engine. The receiver demultiplexes the incoming streams to the four source
> + pads.
> +
> +- Four DMA Engines in the SoC (DMA Engine). Each DMA engine is connected to a
> + single source pad in the receiver.
> +
> +The sensors, the bridge and the receiver are modeled as V4L2 subdevices,
> +exposed to userspace via /dev/v4l-subdevX device nodes. The DMA engines are
> +modeled as V4L2 devices, exposed to userspace via /dev/videoX nodes.
> +
> +To configure this pipeline, the userspace must take the following steps:
> +
> +1) Set up media links between entities: connect the sensors to the bridge,
> +bridge to the receiver, and the receiver to the DMA engines. This step does
> +not differ from normal non-multiplexed media controller setup.
> +
> +2) Configure routing.
> +
> +.. flat-table:: Sensor routing table (identical on both sensors)
> + :header-rows: 1
> +
> + * - Sink Pad/Stream
> + - Source Pad/Stream
> + - Routing Flags
> + - Comments
> + * - 0/0 (unused)
> + - 0/0
> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE | V4L2_SUBDEV_ROUTE_FL_SOURCE
> + - Pixel data stream. Source route, i.e. the sink fields are unused.
> + * - 0/0 (unused)
> + - 0/1
> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE | V4L2_SUBDEV_ROUTE_FL_SOURCE
> + - Metadata stream. Source route, i.e. the sink fields are unused.
> +
> +.. flat-table:: Bridge routing table
> + :header-rows: 1
> +
> + * - Sink Pad/Stream
> + - Source Pad/Stream
> + - Routing Flags
> + - Comments
> + * - 0/0
> + - 2/0
> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> + - Pixel data stream from Sensor A
> + * - 0/1
> + - 2/1
> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> + - Metadata stream from Sensor A
> + * - 1/0
> + - 2/2
> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> + - Pixel data stream from Sensor B
> + * - 1/1
> + - 2/3
> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> + - Metadata stream from Sensor B
> +
> +.. flat-table:: Receiver routing table
> + :header-rows: 1
> +
> + * - Sink Pad/Stream
> + - Source Pad/Stream
> + - Routing Flags
> + - Comments
> + * - 0/0
> + - 1/0
> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> + - Pixel data stream from Sensor A
> + * - 0/1
> + - 2/0
> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> + - Metadata stream from Sensor A
> + * - 0/2
> + - 3/0
> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> + - Pixel data stream from Sensor B
> + * - 0/3
> + - 4/0
> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> + - Metadata stream from Sensor B
> +
> +3) Configure streams
> +
> +After configuring the routing table, the next step is configuring the streams.
> +This step is similar to configuring the pads in a non-multiplexed streams
> +setup, with the difference that we need to configure each (pad, stream) pair
> +(i.e. route endpoint) instead of just a pad.
> +
> +A common way to accomplish this is to start from the sensors and propagate the
> +configurations along the stream towards the receiver, using VIDIOC_SUBDEV_S_FMT
using the :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls
> +to configure each stream endpoint in each subdev.
All minors or just suggestions:
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Thanks
j
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 26/36] media: subdev: add stream based configuration
2022-03-01 16:11 ` [PATCH v11 26/36] media: subdev: add stream based configuration Tomi Valkeinen
@ 2022-03-16 9:59 ` Jacopo Mondi
2022-03-17 8:01 ` Tomi Valkeinen
0 siblings, 1 reply; 67+ messages in thread
From: Jacopo Mondi @ 2022-03-16 9:59 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Hi Tomi
On Tue, Mar 01, 2022 at 06:11:46PM +0200, Tomi Valkeinen wrote:
> Add support to manage configurations (format, crop, compose) per stream,
> instead of per pad. This is accomplished with data structures that hold
> an array of all subdev's stream configurations.
>
> The number of streams can vary at runtime based on routing. Every time
> the routing is changed, the stream configurations need to be
> re-initialized.
>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
> .../v4l/vidioc-subdev-enum-frame-interval.rst | 5 +-
> .../v4l/vidioc-subdev-enum-frame-size.rst | 5 +-
> .../v4l/vidioc-subdev-enum-mbus-code.rst | 5 +-
> .../media/v4l/vidioc-subdev-g-crop.rst | 5 +-
> .../media/v4l/vidioc-subdev-g-fmt.rst | 5 +-
> .../v4l/vidioc-subdev-g-frame-interval.rst | 5 +-
> .../media/v4l/vidioc-subdev-g-selection.rst | 5 +-
> drivers/media/v4l2-core/v4l2-subdev.c | 129 ++++++++++++++++--
> include/media/v4l2-subdev.h | 48 +++++++
> include/uapi/linux/v4l2-subdev.h | 28 +++-
> 10 files changed, 218 insertions(+), 22 deletions(-)
>
> diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-interval.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-interval.rst
> index 3703943b412f..8def4c05d3da 100644
> --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-interval.rst
> +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-interval.rst
> @@ -92,7 +92,10 @@ multiple pads of the same sub-device is not defined.
> - Frame intervals to be enumerated, from enum
> :ref:`v4l2_subdev_format_whence <v4l2-subdev-format-whence>`.
> * - __u32
> - - ``reserved``\ [8]
> + - ``stream``
> + - Stream identifier.
> + * - __u32
> + - ``reserved``\ [7]
Does VIDIOC_SUBDEV_ENUM_FRAME_INTERVAL (as well as
VIDIOC_SUBDEV_G_FRAME_INTERVAL) need to be stream-aware ?
What is the semantic of the stream identifiers for IOCTLs that seem to
control a paramter which is global to the subdev ? Isn't the stream semantic
required to be specified in the IOCTL documentation and not just added
to the list of fields ?
> - Reserved for future extensions. Applications and drivers must set
> the array to zero.
>
> diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-size.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-size.rst
> index c25a9896df0e..3ef361c0dca7 100644
> --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-size.rst
> +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-size.rst
> @@ -97,7 +97,10 @@ information about try formats.
> - Frame sizes to be enumerated, from enum
> :ref:`v4l2_subdev_format_whence <v4l2-subdev-format-whence>`.
> * - __u32
> - - ``reserved``\ [8]
> + - ``stream``
> + - Stream identifier.
> + * - __u32
> + - ``reserved``\ [7]
> - Reserved for future extensions. Applications and drivers must set
> the array to zero.
>
> diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-mbus-code.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-mbus-code.rst
> index 417f1a19bcc4..248f6f9ee7c5 100644
> --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-mbus-code.rst
> +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-mbus-code.rst
> @@ -73,7 +73,10 @@ information about the try formats.
> - ``flags``
> - See :ref:`v4l2-subdev-mbus-code-flags`
> * - __u32
> - - ``reserved``\ [7]
> + - ``stream``
> + - Stream identifier.
> + * - __u32
> + - ``reserved``\ [6]
> - Reserved for future extensions. Applications and drivers must set
> the array to zero.
>
> diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-crop.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-crop.rst
> index bd15c0a5a66b..1d267f7e7991 100644
> --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-crop.rst
> +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-crop.rst
> @@ -96,7 +96,10 @@ modified format should be as close as possible to the original request.
> - ``rect``
> - Crop rectangle boundaries, in pixels.
> * - __u32
> - - ``reserved``\ [8]
> + - ``stream``
> + - Stream identifier.
> + * - __u32
> + - ``reserved``\ [7]
> - Reserved for future extensions. Applications and drivers must set
> the array to zero.
>
> diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-fmt.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-fmt.rst
> index 7acdbb939d89..ed253a1e44b7 100644
> --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-fmt.rst
> +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-fmt.rst
> @@ -102,7 +102,10 @@ should be as close as possible to the original request.
> - Definition of an image format, see :c:type:`v4l2_mbus_framefmt` for
> details.
> * - __u32
> - - ``reserved``\ [8]
> + - ``stream``
> + - Stream identifier.
> + * - __u32
> + - ``reserved``\ [7]
> - Reserved for future extensions. Applications and drivers must set
> the array to zero.
>
> diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-frame-interval.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-frame-interval.rst
> index d7fe7543c506..842f962d2aea 100644
> --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-frame-interval.rst
> +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-frame-interval.rst
> @@ -90,7 +90,10 @@ the same sub-device is not defined.
> - ``interval``
> - Period, in seconds, between consecutive video frames.
> * - __u32
> - - ``reserved``\ [9]
> + - ``stream``
> + - Stream identifier.
> + * - __u32
> + - ``reserved``\ [8]
> - Reserved for future extensions. Applications and drivers must set
> the array to zero.
>
> diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-selection.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-selection.rst
> index f9172a42f036..6b629c19168c 100644
> --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-selection.rst
> +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-selection.rst
> @@ -94,7 +94,10 @@ Selection targets and flags are documented in
> - ``r``
> - Selection rectangle, in pixels.
> * - __u32
> - - ``reserved``\ [8]
> + - ``stream``
> + - Stream identifier.
> + * - __u32
> + - ``reserved``\ [7]
> - Reserved for future extensions. Applications and drivers must set
> the array to zero.
>
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 1c836c2de86e..339d7b15e26c 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -149,14 +149,34 @@ static inline int check_pad(struct v4l2_subdev *sd, u32 pad)
> return 0;
> }
To be honest, the only IOCTL for which I have a clear idea of the
stream paramter semantic is s/g_format.
>
> -static int check_state_pads(u32 which, struct v4l2_subdev_state *state)
> +static int check_state_pads(struct v4l2_subdev *sd, u32 which,
> + struct v4l2_subdev_state *state)
> {
> + if (sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED)
> + return 0;
> +
> if (which == V4L2_SUBDEV_FORMAT_TRY && (!state || !state->pads))
> return -EINVAL;
>
> return 0;
> }
>
> +static int check_state_pad_stream(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state, u32 pad,
> + u32 stream)
> +{
> + struct v4l2_mbus_framefmt *fmt;
> +
> + if (!(sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED))
> + return 0;
> +
> + fmt = v4l2_subdev_state_get_stream_format(state, pad, stream);
> + if (!fmt)
> + return -EINVAL;
> +
> + return 0;
> +}
> +
check_state_pads() is always called in conjunction with
check_state_pad_stream(). I would have made a check_state() that
handles the multiplexed and non-multiplexed case. But that's an
implementation detail, so up to you.
> static inline int check_format(struct v4l2_subdev *sd,
> struct v4l2_subdev_state *state,
> struct v4l2_subdev_format *format)
> @@ -165,7 +185,8 @@ static inline int check_format(struct v4l2_subdev *sd,
> return -EINVAL;
>
> return check_which(format->which) ? : check_pad(sd, format->pad) ? :
> - check_state_pads(format->which, state);
> + check_state_pads(sd, format->which, state) ? :
> + check_state_pad_stream(sd, state, format->pad, format->stream);
> }
>
> static int call_get_fmt(struct v4l2_subdev *sd,
> @@ -192,7 +213,8 @@ static int call_enum_mbus_code(struct v4l2_subdev *sd,
> return -EINVAL;
>
> return check_which(code->which) ? : check_pad(sd, code->pad) ? :
> - check_state_pads(code->which, state) ? :
> + check_state_pads(sd, code->which, state) ? :
> + check_state_pad_stream(sd, state, code->pad, code->stream) ? :
> sd->ops->pad->enum_mbus_code(sd, state, code);
> }
>
> @@ -204,7 +226,8 @@ static int call_enum_frame_size(struct v4l2_subdev *sd,
> return -EINVAL;
>
> return check_which(fse->which) ? : check_pad(sd, fse->pad) ? :
> - check_state_pads(fse->which, state) ? :
> + check_state_pads(sd, fse->which, state) ? :
> + check_state_pad_stream(sd, state, fse->pad, fse->stream) ? :
> sd->ops->pad->enum_frame_size(sd, state, fse);
> }
>
> @@ -239,7 +262,8 @@ static int call_enum_frame_interval(struct v4l2_subdev *sd,
> return -EINVAL;
>
> return check_which(fie->which) ? : check_pad(sd, fie->pad) ? :
> - check_state_pads(fie->which, state) ? :
> + check_state_pads(sd, fie->which, state) ? :
> + check_state_pad_stream(sd, state, fie->pad, fie->stream) ? :
> sd->ops->pad->enum_frame_interval(sd, state, fie);
call_g_frame_interval and call_s_frame_interval do accept a struct
v4l2_subdev_frame_interval paramter now. Should the validity of
streams be checked there too ?
> }
>
> @@ -251,7 +275,8 @@ static inline int check_selection(struct v4l2_subdev *sd,
> return -EINVAL;
>
> return check_which(sel->which) ? : check_pad(sd, sel->pad) ? :
> - check_state_pads(sel->which, state);
> + check_state_pads(sd, sel->which, state) ? :
> + check_state_pad_stream(sd, state, sel->pad, sel->stream);
> }
>
> static int call_get_selection(struct v4l2_subdev *sd,
> @@ -865,6 +890,71 @@ const struct v4l2_file_operations v4l2_subdev_fops = {
>
> #ifdef CONFIG_MEDIA_CONTROLLER
>
> +static int
> +v4l2_subdev_init_stream_configs(struct v4l2_subdev_stream_configs *stream_configs,
> + const struct v4l2_subdev_krouting *routing)
> +{
> + u32 num_configs = 0;
> + unsigned int i;
> + u32 format_idx = 0;
> +
> + kvfree(stream_configs->configs);
> + stream_configs->configs = NULL;
> + stream_configs->num_configs = 0;
> +
> + /* Count number of formats needed */
> + for (i = 0; i < routing->num_routes; ++i) {
> + struct v4l2_subdev_route *route = &routing->routes[i];
This is a good candidate for for_each_active_route()
> +
> + if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
> + continue;
> +
> + /*
> + * Each route needs a format on both ends of the route, except
> + * for source streams which only need one format.
> + */
> + num_configs +=
> + (route->flags & V4L2_SUBDEV_ROUTE_FL_SOURCE) ? 1 : 2;
> + }
> +
> + if (num_configs) {
> + stream_configs->configs =
> + kvcalloc(num_configs, sizeof(*stream_configs->configs),
> + GFP_KERNEL);
> +
> + if (!stream_configs->configs)
> + return -ENOMEM;
> +
> + stream_configs->num_configs = num_configs;
> + }
> +
> + /*
> + * Fill in the 'pad' and stream' value for each item in the array from
> + * the routing table
> + */
> + for (i = 0; i < routing->num_routes; ++i) {
> + struct v4l2_subdev_route *route = &routing->routes[i];
> + u32 idx;
> +
> + if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
> + continue;
you iterate only active routes again. Which makes me think that you
could return after the first loop if (!num_configs) ?
> +
> + if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_SOURCE)) {
> + idx = format_idx++;
> +
> + stream_configs->configs[idx].pad = route->sink_pad;
> + stream_configs->configs[idx].stream = route->sink_stream;
> + }
> +
> + idx = format_idx++;
> +
> + stream_configs->configs[idx].pad = route->source_pad;
> + stream_configs->configs[idx].stream = route->source_stream;
> + }
> +
> + return 0;
> +}
> +
> int v4l2_subdev_get_fwnode_pad_1_to_1(struct media_entity *entity,
> struct fwnode_endpoint *endpoint)
> {
> @@ -1042,7 +1132,8 @@ __v4l2_subdev_state_alloc(struct v4l2_subdev *sd, const char *lock_name,
> else
> state->lock = &state->_lock;
>
> - if (sd->entity.num_pads) {
> + /* Drivers that support streams do not need the legacy pad config */
> + if (!(sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED) && sd->entity.num_pads) {
> state->pads = kvmalloc_array(sd->entity.num_pads,
> sizeof(*state->pads),
> GFP_KERNEL | __GFP_ZERO);
> @@ -1083,6 +1174,7 @@ void __v4l2_subdev_state_free(struct v4l2_subdev_state *state)
> mutex_destroy(&state->_lock);
>
> kfree(state->routing.routes);
> + kvfree(state->stream_configs.configs);
> kvfree(state->pads);
> kfree(state);
> }
> @@ -1133,10 +1225,31 @@ int v4l2_subdev_set_routing(struct v4l2_subdev *sd,
> dst->num_routes = src->num_routes;
> }
>
> - return 0;
> + return v4l2_subdev_init_stream_configs(&state->stream_configs, dst);
> }
> EXPORT_SYMBOL_GPL(v4l2_subdev_set_routing);
>
> +struct v4l2_mbus_framefmt *
> +v4l2_subdev_state_get_stream_format(struct v4l2_subdev_state *state,
> + unsigned int pad, u32 stream)
> +{
> + struct v4l2_subdev_stream_configs *stream_configs;
> + unsigned int i;
> +
> + lockdep_assert_held(state->lock);
This function is only called by check_state_pad_stream(). Does it
locks the state ? I understand this is temporary only as other users
introduced later will lock the state ?
Thanks
j
> +
> + stream_configs = &state->stream_configs;
> +
> + for (i = 0; i < stream_configs->num_configs; ++i) {
> + if (stream_configs->configs[i].pad == pad &&
> + stream_configs->configs[i].stream == stream)
> + return &stream_configs->configs[i].fmt;
> + }
> +
> + return NULL;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_state_get_stream_format);
> +
> #endif /* CONFIG_MEDIA_CONTROLLER */
>
> void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index 6323bae3860e..2a40ad273cf8 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -697,6 +697,37 @@ struct v4l2_subdev_pad_config {
> struct v4l2_rect try_compose;
> };
>
> +/**
> + * struct v4l2_subdev_stream_config - Used for storing stream configuration.
> + *
> + * @pad: pad number
> + * @stream: stream number
> + * @fmt: &struct v4l2_mbus_framefmt
> + * @crop: &struct v4l2_rect to be used for crop
> + * @compose: &struct v4l2_rect to be used for compose
> + *
> + * This structure stores configuration for a stream.
> + */
> +struct v4l2_subdev_stream_config {
> + u32 pad;
> + u32 stream;
> +
> + struct v4l2_mbus_framefmt fmt;
> + struct v4l2_rect crop;
> + struct v4l2_rect compose;
> +};
> +
> +/**
> + * struct v4l2_subdev_stream_configs - A collection of stream configs.
> + *
> + * @num_configs: number of entries in @config.
> + * @configs: an array of &struct v4l2_subdev_stream_configs.
> + */
> +struct v4l2_subdev_stream_configs {
> + u32 num_configs;
> + struct v4l2_subdev_stream_config *configs;
> +};
> +
> /**
> * struct v4l2_subdev_krouting - subdev routing table
> *
> @@ -717,6 +748,7 @@ struct v4l2_subdev_krouting {
> * @lock: mutex for the state. May be replaced by the user.
> * @pads: &struct v4l2_subdev_pad_config array
> * @routing: routing table for the subdev
> + * @stream_configs: stream configurations (only for V4L2_SUBDEV_FL_MULTIPLEXED)
> *
> * This structure only needs to be passed to the pad op if the 'which' field
> * of the main argument is set to %V4L2_SUBDEV_FORMAT_TRY. For
> @@ -728,6 +760,7 @@ struct v4l2_subdev_state {
> struct mutex *lock;
> struct v4l2_subdev_pad_config *pads;
> struct v4l2_subdev_krouting routing;
> + struct v4l2_subdev_stream_configs stream_configs;
> };
>
> /**
> @@ -1397,6 +1430,21 @@ int v4l2_subdev_set_routing(struct v4l2_subdev *sd,
> struct v4l2_subdev_state *state,
> struct v4l2_subdev_krouting *routing);
>
> +/**
> + * v4l2_subdev_state_get_stream_format() - Get pointer to a stream format
> + * @state: subdevice state
> + * @pad: pad id
> + * @stream: stream id
> + *
> + * This returns a pointer to &struct v4l2_mbus_framefmt for the given pad +
> + * stream in the subdev state.
for a given (pad, stream) ?
> + *
> + * If the state does not contain the given pad + stream, NULL is returned.
> + */
> +struct v4l2_mbus_framefmt *
> +v4l2_subdev_state_get_stream_format(struct v4l2_subdev_state *state,
> + unsigned int pad, u32 stream);
> +
> #endif /* CONFIG_MEDIA_CONTROLLER */
>
> /**
> diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h
> index 1ec3141bf860..480891dba193 100644
> --- a/include/uapi/linux/v4l2-subdev.h
> +++ b/include/uapi/linux/v4l2-subdev.h
> @@ -44,13 +44,15 @@ enum v4l2_subdev_format_whence {
> * @which: format type (from enum v4l2_subdev_format_whence)
> * @pad: pad number, as reported by the media API
> * @format: media bus format (format code and frame size)
> + * @stream: stream number, defined in subdev routing
@stream: stream identifier
what does "defined in subdev routing" mean ?
> * @reserved: drivers and applications must zero this array
> */
> struct v4l2_subdev_format {
> __u32 which;
> __u32 pad;
> struct v4l2_mbus_framefmt format;
> - __u32 reserved[8];
> + __u32 stream;
> + __u32 reserved[7];
> };
>
> /**
> @@ -58,13 +60,15 @@ struct v4l2_subdev_format {
> * @which: format type (from enum v4l2_subdev_format_whence)
> * @pad: pad number, as reported by the media API
> * @rect: pad crop rectangle boundaries
> + * @stream: stream number, defined in subdev routing
> * @reserved: drivers and applications must zero this array
> */
> struct v4l2_subdev_crop {
> __u32 which;
> __u32 pad;
> struct v4l2_rect rect;
> - __u32 reserved[8];
> + __u32 stream;
> + __u32 reserved[7];
> };
>
> #define V4L2_SUBDEV_MBUS_CODE_CSC_COLORSPACE 0x00000001
> @@ -80,6 +84,7 @@ struct v4l2_subdev_crop {
> * @code: format code (MEDIA_BUS_FMT_ definitions)
> * @which: format type (from enum v4l2_subdev_format_whence)
> * @flags: flags set by the driver, (V4L2_SUBDEV_MBUS_CODE_*)
> + * @stream: stream number, defined in subdev routing
> * @reserved: drivers and applications must zero this array
> */
> struct v4l2_subdev_mbus_code_enum {
> @@ -88,7 +93,8 @@ struct v4l2_subdev_mbus_code_enum {
> __u32 code;
> __u32 which;
> __u32 flags;
> - __u32 reserved[7];
> + __u32 stream;
> + __u32 reserved[6];
> };
>
> /**
> @@ -101,6 +107,7 @@ struct v4l2_subdev_mbus_code_enum {
> * @min_height: minimum frame height, in pixels
> * @max_height: maximum frame height, in pixels
> * @which: format type (from enum v4l2_subdev_format_whence)
> + * @stream: stream number, defined in subdev routing
> * @reserved: drivers and applications must zero this array
> */
> struct v4l2_subdev_frame_size_enum {
> @@ -112,19 +119,22 @@ struct v4l2_subdev_frame_size_enum {
> __u32 min_height;
> __u32 max_height;
> __u32 which;
> - __u32 reserved[8];
> + __u32 stream;
> + __u32 reserved[7];
> };
>
> /**
> * struct v4l2_subdev_frame_interval - Pad-level frame rate
> * @pad: pad number, as reported by the media API
> * @interval: frame interval in seconds
> + * @stream: stream number, defined in subdev routing
> * @reserved: drivers and applications must zero this array
> */
> struct v4l2_subdev_frame_interval {
> __u32 pad;
> struct v4l2_fract interval;
> - __u32 reserved[9];
> + __u32 stream;
> + __u32 reserved[8];
> };
>
> /**
> @@ -136,6 +146,7 @@ struct v4l2_subdev_frame_interval {
> * @height: frame height in pixels
> * @interval: frame interval in seconds
> * @which: format type (from enum v4l2_subdev_format_whence)
> + * @stream: stream number, defined in subdev routing
> * @reserved: drivers and applications must zero this array
> */
> struct v4l2_subdev_frame_interval_enum {
> @@ -146,7 +157,8 @@ struct v4l2_subdev_frame_interval_enum {
> __u32 height;
> struct v4l2_fract interval;
> __u32 which;
> - __u32 reserved[8];
> + __u32 stream;
> + __u32 reserved[7];
> };
>
> /**
> @@ -158,6 +170,7 @@ struct v4l2_subdev_frame_interval_enum {
> * defined in v4l2-common.h; V4L2_SEL_TGT_* .
> * @flags: constraint flags, defined in v4l2-common.h; V4L2_SEL_FLAG_*.
> * @r: coordinates of the selection window
> + * @stream: stream number, defined in subdev routing
> * @reserved: for future use, set to zero for now
> *
> * Hardware may use multiple helper windows to process a video stream.
> @@ -170,7 +183,8 @@ struct v4l2_subdev_selection {
> __u32 target;
> __u32 flags;
> struct v4l2_rect r;
> - __u32 reserved[8];
> + __u32 stream;
> + __u32 reserved[7];
> };
>
> /**
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 27/36] media: subdev: use streams in v4l2_subdev_link_validate()
2022-03-01 16:11 ` [PATCH v11 27/36] media: subdev: use streams in v4l2_subdev_link_validate() Tomi Valkeinen
@ 2022-03-16 10:47 ` Jacopo Mondi
2022-03-17 8:39 ` Tomi Valkeinen
0 siblings, 1 reply; 67+ messages in thread
From: Jacopo Mondi @ 2022-03-16 10:47 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Hi Tomi
On Tue, Mar 01, 2022 at 06:11:47PM +0200, Tomi Valkeinen wrote:
> Update v4l2_subdev_link_validate() to use routing and streams for
> validation.
>
> Instead of just looking at the format on the pad on both ends of the
> link, the routing tables are used to collect all the streams going from
> the source to the sink over the link, and the streams' formats on both
> ends of the link are verified.
>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
> drivers/media/v4l2-core/v4l2-subdev.c | 245 ++++++++++++++++++++++++--
> 1 file changed, 227 insertions(+), 18 deletions(-)
>
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 339d7b15e26c..091b854e00d0 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -16,6 +16,7 @@
> #include <linux/videodev2.h>
> #include <linux/export.h>
> #include <linux/version.h>
> +#include <linux/sort.h>
Not sorted, but it was already like this
>
> #include <media/v4l2-ctrls.h>
> #include <media/v4l2-device.h>
> @@ -1035,6 +1036,7 @@ EXPORT_SYMBOL_GPL(v4l2_subdev_link_validate_default);
>
> static int
> v4l2_subdev_link_validate_get_format(struct media_pad *pad,
> + u32 stream,
fits on the previous line
> struct v4l2_subdev_format *fmt)
> {
> if (is_media_entity_v4l2_subdev(pad->entity)) {
> @@ -1046,6 +1048,7 @@ v4l2_subdev_link_validate_get_format(struct media_pad *pad,
>
> fmt->which = V4L2_SUBDEV_FORMAT_ACTIVE;
> fmt->pad = pad->index;
> + fmt->stream = stream;
> return v4l2_subdev_call(sd, pad, get_fmt, state, fmt);
> }
>
> @@ -1056,31 +1059,237 @@ v4l2_subdev_link_validate_get_format(struct media_pad *pad,
> return -EINVAL;
> }
>
> -int v4l2_subdev_link_validate(struct media_link *link)
> +static int cmp_u32(const void *a, const void *b)
> {
> - struct v4l2_subdev *sink;
> - struct v4l2_subdev_format sink_fmt, source_fmt;
> - int rval;
> + u32 a32 = *(u32 *)a;
> + u32 b32 = *(u32 *)b;
>
> - rval = v4l2_subdev_link_validate_get_format(
> - link->source, &source_fmt);
> - if (rval < 0)
> - return 0;
> + return a32 > b32 ? 1 : (a32 < b32 ? -1 : 0);
> +}
> +
> +static int v4l2_link_validate_get_streams(struct media_link *link,
> + bool is_source, u32 *out_num_streams,
> + const u32 **out_streams,
> + bool *allocated)
> +{
> + static const u32 default_streams[] = { 0 };
> + struct v4l2_subdev_krouting *routing;
> + struct v4l2_subdev *subdev;
> + u32 num_streams;
> + u32 *streams;
> + unsigned int i;
> + struct v4l2_subdev_state *state;
> +
> + if (is_source)
> + subdev = media_entity_to_v4l2_subdev(link->source->entity);
> + else
> + subdev = media_entity_to_v4l2_subdev(link->sink->entity);
>
> - rval = v4l2_subdev_link_validate_get_format(
> - link->sink, &sink_fmt);
> - if (rval < 0)
> + if (!(subdev->flags & V4L2_SUBDEV_FL_MULTIPLEXED)) {
> + *out_num_streams = 1;
> + *out_streams = default_streams;
> + *allocated = false;
> return 0;
> + }
>
> - sink = media_entity_to_v4l2_subdev(link->sink->entity);
> + state = v4l2_subdev_get_locked_active_state(subdev);
>
> - rval = v4l2_subdev_call(sink, pad, link_validate, link,
> - &source_fmt, &sink_fmt);
> - if (rval != -ENOIOCTLCMD)
> - return rval;
> + routing = &state->routing;
> +
> + streams = kmalloc_array(routing->num_routes, sizeof(u32), GFP_KERNEL);
> + if (!streams)
> + return -ENOMEM;
> +
> + num_streams = 0;
> +
> + for (i = 0; i < routing->num_routes; ++i) {
> + struct v4l2_subdev_route *route = &routing->routes[i];
> + int j;
> + u32 route_pad;
> + u32 route_stream;
> + u32 link_pad;
> +
> + if (is_source) {
> + route_pad = route->source_pad;
> + route_stream = route->source_stream;
> + link_pad = link->source->index;
> + } else {
> + route_pad = route->sink_pad;
> + route_stream = route->sink_stream;
> + link_pad = link->sink->index;
> + }
> +
> + if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
> + continue;
Should this check be done before the previous if/else block ?
> +
> + if (route_pad != link_pad)
> + continue;
> +
> + /* look for duplicates... */
> + for (j = 0; j < num_streams; ++j) {
> + if (streams[j] == route_stream)
> + break;
> + }
> +
> + /* ...and drop the stream if we already have it */
> + if (j != num_streams)
> + continue;
Is this an error ? can the same stream be routed to multiple ones on a
media link ?
> +
> + streams[num_streams++] = route_stream;
> + }
> +
> + sort(streams, num_streams, sizeof(u32), &cmp_u32, NULL);
>
> - return v4l2_subdev_link_validate_default(
> - sink, link, &source_fmt, &sink_fmt);
> + *out_num_streams = num_streams;
> + *out_streams = streams;
> + *allocated = true;
> +
> + return 0;
> +}
> +
> +static int v4l2_subdev_link_validate_locked(struct media_link *link)
> +{
> + struct v4l2_subdev *sink_subdev =
> + media_entity_to_v4l2_subdev(link->sink->entity);
> + struct device *dev = sink_subdev->entity.graph_obj.mdev->dev;
> + u32 num_source_streams;
> + const u32 *source_streams;
> + bool source_allocated;
> + u32 num_sink_streams;
> + const u32 *sink_streams;
> + bool sink_allocated;
> + unsigned int sink_idx;
> + unsigned int source_idx;
> + int ret;
> +
> + dev_dbg(dev, "validating link \"%s\":%u -> \"%s\":%u\n",
> + link->source->entity->name, link->source->index,
> + link->sink->entity->name, link->sink->index);
> +
> + ret = v4l2_link_validate_get_streams(link, true, &num_source_streams,
> + &source_streams,
> + &source_allocated);
> + if (ret)
> + return ret;
> +
> + ret = v4l2_link_validate_get_streams(link, false, &num_sink_streams,
> + &sink_streams, &sink_allocated);
> + if (ret)
> + goto free_source;
> +
> + /*
> + * It is ok to have more source streams than sink streams as extra
> + * source streams can just be ignored (i.e. they go nowhere), but extra
Nit on the "(i.e. they go nowhere)" bit:
If it's ok to have more source streams than sinks, the extra sources
will not be connected. So I would drop or change the above phrase as
to me it sounds like sink streams go somewhere, while source stream
are rather not connected (or not active).
> + * sink streams is an error.
> + */
> + if (num_source_streams < num_sink_streams) {
> + dev_err(dev,
> + "Not enough source streams: %d < %d\n",
> + num_source_streams, num_sink_streams);
> + ret = -EINVAL;
> + goto out;
> + }
> +
> + /* Validate source and sink stream formats */
> +
> + source_idx = 0;
> +
> + for (sink_idx = 0; sink_idx < num_sink_streams; ++sink_idx) {
> + struct v4l2_subdev_format sink_fmt, source_fmt;
> + u32 stream;
> +
> + stream = sink_streams[sink_idx];
> +
> + for (; source_idx < num_source_streams; ++source_idx) {
> + if (source_streams[source_idx] == stream)
> + break;
> + }
> +
> + if (source_idx == num_source_streams) {
> + dev_err(dev, "No source stream for sink stream %u\n",
> + stream);
> + ret = -EINVAL;
> + goto out;
> + }
> +
> + dev_dbg(dev, "validating stream \"%s\":%u:%u -> \"%s\":%u:%u\n",
> + link->source->entity->name, link->source->index, stream,
> + link->sink->entity->name, link->sink->index, stream);
> +
> + ret = v4l2_subdev_link_validate_get_format(link->source, stream,
> + &source_fmt);
> + if (ret < 0) {
> + dev_dbg(dev, "Failed to get format for \"%s\":%u:%u (but that's ok)\n",
For my education: why is it ok not to have a format for the stream ?
Don't we risk to pass a non-initialized source/sink_fmt to
link_validate() in that case ?
> + link->source->entity->name, link->source->index,
> + stream);
> + ret = 0;
> + continue;
> + }
> +
> + ret = v4l2_subdev_link_validate_get_format(link->sink, stream,
> + &sink_fmt);
> + if (ret < 0) {
> + dev_dbg(dev, "Failed to get format for \"%s\":%u:%u (but that's ok)\n",
> + link->sink->entity->name, link->sink->index,
> + stream);
> + ret = 0;
> + continue;
> + }
> +
> + /* TODO: add stream number to link_validate() */
> + ret = v4l2_subdev_call(sink_subdev, pad, link_validate, link,
> + &source_fmt, &sink_fmt);
> + if (!ret)
> + continue;
> +
> + if (ret != -ENOIOCTLCMD)
> + goto out;
> +
> + ret = v4l2_subdev_link_validate_default(sink_subdev, link,
> + &source_fmt, &sink_fmt);
> +
> + if (ret)
> + goto out;
> + }
> +
> +out:
I would call it free_sinks:
Thanks
j
> + if (sink_allocated)
> + kfree(sink_streams);
> +
> +free_source:
> + if (source_allocated)
> + kfree(source_streams);
> +
> + return ret;
> +}
> +
> +int v4l2_subdev_link_validate(struct media_link *link)
> +{
> + struct v4l2_subdev *source_sd, *sink_sd;
> + struct v4l2_subdev_state *source_state, *sink_state;
> + int ret;
> +
> + sink_sd = media_entity_to_v4l2_subdev(link->sink->entity);
> + source_sd = media_entity_to_v4l2_subdev(link->source->entity);
> +
> + sink_state = v4l2_subdev_get_active_state(sink_sd);
> + source_state = v4l2_subdev_get_active_state(source_sd);
> +
> + if (sink_state)
> + v4l2_subdev_lock_state(sink_state);
> +
> + if (source_state)
> + v4l2_subdev_lock_state(source_state);
> +
> + ret = v4l2_subdev_link_validate_locked(link);
> +
> + if (sink_state)
> + v4l2_subdev_unlock_state(sink_state);
> +
> + if (source_state)
> + v4l2_subdev_unlock_state(source_state);
> +
> + return ret;
> }
> EXPORT_SYMBOL_GPL(v4l2_subdev_link_validate);
>
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 28/36] media: subdev: add "opposite" stream helper funcs
2022-03-01 16:11 ` [PATCH v11 28/36] media: subdev: add "opposite" stream helper funcs Tomi Valkeinen
@ 2022-03-16 10:50 ` Jacopo Mondi
2022-03-17 9:10 ` Tomi Valkeinen
0 siblings, 1 reply; 67+ messages in thread
From: Jacopo Mondi @ 2022-03-16 10:50 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Hi Tomi,
On Tue, Mar 01, 2022 at 06:11:48PM +0200, Tomi Valkeinen wrote:
> Add two helper functions to make dealing with streams easier:
>
> v4l2_subdev_routing_find_opposite_end - given a routing table and a pad
> + stream, return the pad + stream on the opposite side of the subdev.
>
> v4l2_subdev_state_get_opposite_stream_format - return a pointer to the
> format on the pad + stream on the opposite side from the given pad +
> stream.
>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> Reviewed-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
> ---
> drivers/media/v4l2-core/v4l2-subdev.c | 48 +++++++++++++++++++++++++++
> include/media/v4l2-subdev.h | 36 ++++++++++++++++++++
> 2 files changed, 84 insertions(+)
>
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 091b854e00d0..e65f802fe2aa 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -1459,6 +1459,54 @@ v4l2_subdev_state_get_stream_format(struct v4l2_subdev_state *state,
> }
> EXPORT_SYMBOL_GPL(v4l2_subdev_state_get_stream_format);
>
> +int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
> + u32 pad, u32 stream, u32 *other_pad,
> + u32 *other_stream)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < routing->num_routes; ++i) {
> + struct v4l2_subdev_route *route = &routing->routes[i];
> +
Is it intentional to include routes that might be active ?
If that's the case
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Thanks
j
> + if (route->source_pad == pad &&
> + route->source_stream == stream) {
> + if (other_pad)
> + *other_pad = route->sink_pad;
> + if (other_stream)
> + *other_stream = route->sink_stream;
> + return 0;
> + }
> +
> + if (route->sink_pad == pad && route->sink_stream == stream) {
> + if (other_pad)
> + *other_pad = route->source_pad;
> + if (other_stream)
> + *other_stream = route->source_stream;
> + return 0;
> + }
> + }
> +
> + return -EINVAL;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_routing_find_opposite_end);
> +
> +struct v4l2_mbus_framefmt *
> +v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
> + u32 pad, u32 stream)
> +{
> + u32 other_pad, other_stream;
> + int ret;
> +
> + ret = v4l2_subdev_routing_find_opposite_end(&state->routing, pad, stream,
> + &other_pad, &other_stream);
> + if (ret)
> + return NULL;
> +
> + return v4l2_subdev_state_get_stream_format(state, other_pad,
> + other_stream);
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_state_get_opposite_stream_format);
> +
> #endif /* CONFIG_MEDIA_CONTROLLER */
>
> void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index 2a40ad273cf8..ed3fe21637e6 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -1445,6 +1445,42 @@ struct v4l2_mbus_framefmt *
> v4l2_subdev_state_get_stream_format(struct v4l2_subdev_state *state,
> unsigned int pad, u32 stream);
>
> +/**
> + * v4l2_subdev_routing_find_opposite_end() - Find the opposite stream
> + * @routing: routing used to find the opposite side
> + * @pad: pad id
> + * @stream: stream id
> + * @other_pad: pointer used to return the opposite pad
> + * @other_stream: pointer used to return the opposite stream
> + *
> + * This function uses the routing table to find the pad + stream which is
> + * opposite the given pad + stream.
> + *
> + * @other_pad and/or @other_stream can be NULL if the caller does not need the
> + * value.
> + *
> + * Returns 0 on success, or -EINVAL if no matching route is found.
> + */
> +int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
> + u32 pad, u32 stream, u32 *other_pad,
> + u32 *other_stream);
> +
> +/**
> + * v4l2_subdev_state_get_opposite_stream_format() - Get pointer to opposite
> + * stream format
> + * @state: subdevice state
> + * @pad: pad id
> + * @stream: stream id
> + *
> + * This returns a pointer to &struct v4l2_mbus_framefmt for the pad + stream
> + * that is opposite the given pad + stream in the subdev state.
> + *
> + * If the state does not contain the given pad + stream, NULL is returned.
> + */
> +struct v4l2_mbus_framefmt *
> +v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
> + u32 pad, u32 stream);
> +
> #endif /* CONFIG_MEDIA_CONTROLLER */
>
> /**
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 29/36] media: subdev: add v4l2_subdev_get_fmt() helper function
2022-03-01 16:11 ` [PATCH v11 29/36] media: subdev: add v4l2_subdev_get_fmt() helper function Tomi Valkeinen
@ 2022-03-16 11:00 ` Jacopo Mondi
0 siblings, 0 replies; 67+ messages in thread
From: Jacopo Mondi @ 2022-03-16 11:00 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Hi Tomi
On Tue, Mar 01, 2022 at 06:11:49PM +0200, Tomi Valkeinen wrote:
> Add v4l2_subdev_get_fmt() helper function which implements
> v4l2_subdev_pad_ops.get_fmt using streams. Subdev drivers that do not
> need to do anything special in their get_fmt op can use this helper
> directly for v4l2_subdev_pad_ops.get_fmt.
>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> Reviewed-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Thanks
j
> ---
> drivers/media/v4l2-core/v4l2-subdev.c | 16 ++++++++++++++++
> include/media/v4l2-subdev.h | 17 +++++++++++++++++
> 2 files changed, 33 insertions(+)
>
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index e65f802fe2aa..c1cc9b91dba7 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -1507,6 +1507,22 @@ v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
> }
> EXPORT_SYMBOL_GPL(v4l2_subdev_state_get_opposite_stream_format);
>
> +int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
> + struct v4l2_subdev_format *format)
> +{
> + struct v4l2_mbus_framefmt *fmt;
> +
> + fmt = v4l2_subdev_state_get_stream_format(state, format->pad,
> + format->stream);
> + if (!fmt)
> + return -EINVAL;
> +
> + format->format = *fmt;
> +
> + return 0;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_get_fmt);
> +
> #endif /* CONFIG_MEDIA_CONTROLLER */
>
> void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index ed3fe21637e6..a80830801a7f 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -1481,6 +1481,23 @@ struct v4l2_mbus_framefmt *
> v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
> u32 pad, u32 stream);
>
> +/**
> + * v4l2_subdev_get_fmt() - Fill format based on state
> + * @sd: subdevice
> + * @state: subdevice state
> + * @format: pointer to &struct v4l2_subdev_format
> + *
> + * Fill @format based on the pad and stream given in the @format struct.
> + *
> + * This function can be used by the subdev drivers to implement
> + * v4l2_subdev_pad_ops.get_fmt if the subdev driver does not need to do
> + * anything special in their get_fmt op.
> + *
> + * Returns 0 on success, error value otherwise.
> + */
> +int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
> + struct v4l2_subdev_format *format);
> +
> #endif /* CONFIG_MEDIA_CONTROLLER */
>
> /**
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 30/36] media: subdev: Fallback to pad config in v4l2_subdev_get_fmt()
2022-03-01 16:11 ` [PATCH v11 30/36] media: subdev: Fallback to pad config in v4l2_subdev_get_fmt() Tomi Valkeinen
@ 2022-03-16 11:03 ` Jacopo Mondi
2022-03-17 9:18 ` Tomi Valkeinen
0 siblings, 1 reply; 67+ messages in thread
From: Jacopo Mondi @ 2022-03-16 11:03 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
On Tue, Mar 01, 2022 at 06:11:50PM +0200, Tomi Valkeinen wrote:
> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>
> If the subdev doesn't implement routing support, fallback to pad config
> as the storage for pad formats. This allows using the V4L2 subdev active
> state API and the v4l2_subdev_get_fmt() helper in subdev drivers that
> don't implement routing support.
>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
> drivers/media/v4l2-core/v4l2-subdev.c | 10 ++++++++--
> 1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index c1cc9b91dba7..7f50871054cd 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -1512,8 +1512,14 @@ int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
> {
> struct v4l2_mbus_framefmt *fmt;
>
> - fmt = v4l2_subdev_state_get_stream_format(state, format->pad,
> - format->stream);
> + if (sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED)
> + fmt = v4l2_subdev_state_get_stream_format(state, format->pad,
> + format->stream);
> + else if (format->pad < sd->entity.num_pads && format->stream == 0)
> + fmt = v4l2_subdev_get_try_format(sd, state, format->pad);
> + else
> + fmt = NULL;
You could initialize fmt = NULL and skip the else
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Thanks
j
> +
> if (!fmt)
> return -EINVAL;
>
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 31/36] media: subdev: add v4l2_subdev_set_routing_with_fmt() helper
2022-03-01 16:11 ` [PATCH v11 31/36] media: subdev: add v4l2_subdev_set_routing_with_fmt() helper Tomi Valkeinen
@ 2022-03-16 11:04 ` Jacopo Mondi
0 siblings, 0 replies; 67+ messages in thread
From: Jacopo Mondi @ 2022-03-16 11:04 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Hi Tomi
On Tue, Mar 01, 2022 at 06:11:51PM +0200, Tomi Valkeinen wrote:
> v4l2_subdev_set_routing_with_fmt() is the same as
> v4l2_subdev_set_routing(), but additionally initializes all the streams
> with the given format.
>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> Reviewed-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Thanks
j
> ---
> drivers/media/v4l2-core/v4l2-subdev.c | 22 ++++++++++++++++++++++
> include/media/v4l2-subdev.h | 16 ++++++++++++++++
> 2 files changed, 38 insertions(+)
>
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 7f50871054cd..1ceee8313246 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -1438,6 +1438,28 @@ int v4l2_subdev_set_routing(struct v4l2_subdev *sd,
> }
> EXPORT_SYMBOL_GPL(v4l2_subdev_set_routing);
>
> +int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_krouting *routing,
> + const struct v4l2_mbus_framefmt *fmt)
> +{
> + struct v4l2_subdev_stream_configs *stream_configs;
> + unsigned int i;
> + int ret;
> +
> + ret = v4l2_subdev_set_routing(sd, state, routing);
> + if (ret)
> + return ret;
> +
> + stream_configs = &state->stream_configs;
> +
> + for (i = 0; i < stream_configs->num_configs; ++i)
> + stream_configs->configs[i].fmt = *fmt;
> +
> + return 0;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_set_routing_with_fmt);
> +
> struct v4l2_mbus_framefmt *
> v4l2_subdev_state_get_stream_format(struct v4l2_subdev_state *state,
> unsigned int pad, u32 stream)
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index a80830801a7f..97db6dfc0b7a 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -1430,6 +1430,22 @@ int v4l2_subdev_set_routing(struct v4l2_subdev *sd,
> struct v4l2_subdev_state *state,
> struct v4l2_subdev_krouting *routing);
>
> +/**
> + * v4l2_subdev_set_routing_with_fmt() - Set given routing and format to subdev
> + * state
> + * @sd: The subdevice
> + * @state: The subdevice state
> + * @routing: Routing that will be copied to subdev state
> + * @fmt: Format used to initialize all the streams
> + *
> + * This is the same as v4l2_subdev_set_routing, but additionally initializes
> + * all the streams using the given format.
> + */
> +int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_krouting *routing,
> + const struct v4l2_mbus_framefmt *fmt);
> +
> /**
> * v4l2_subdev_state_get_stream_format() - Get pointer to a stream format
> * @state: subdevice state
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 32/36] media: subdev: add v4l2_subdev_routing_validate() helper
2022-03-01 16:11 ` [PATCH v11 32/36] media: subdev: add v4l2_subdev_routing_validate() helper Tomi Valkeinen
@ 2022-03-16 11:10 ` Jacopo Mondi
2022-03-17 9:15 ` Tomi Valkeinen
0 siblings, 1 reply; 67+ messages in thread
From: Jacopo Mondi @ 2022-03-16 11:10 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Hi Tomi
On Tue, Mar 01, 2022 at 06:11:52PM +0200, Tomi Valkeinen wrote:
> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>
> Add a v4l2_subdev_routing_validate() helper for verifying routing for
> common cases like only allowing non-overlapping 1-to-1 streams.
>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
> drivers/media/v4l2-core/v4l2-subdev.c | 102 ++++++++++++++++++++++++++
> include/media/v4l2-subdev.h | 39 ++++++++++
> 2 files changed, 141 insertions(+)
>
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 1ceee8313246..a19236cf11b8 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -1551,6 +1551,108 @@ int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
> }
> EXPORT_SYMBOL_GPL(v4l2_subdev_get_fmt);
>
> +int v4l2_subdev_routing_validate(struct v4l2_subdev *sd,
> + const struct v4l2_subdev_krouting *routing,
> + enum v4l2_subdev_routing_restriction disallow)
> +{
> + u32 *remote_pads = NULL;
> + unsigned int i, j;
> + int ret = -EINVAL;
> +
> + if (disallow & V4L2_SUBDEV_ROUTING_NO_STREAM_MIX) {
> + remote_pads = kcalloc(sd->entity.num_pads, sizeof(*remote_pads),
> + GFP_KERNEL);
> + if (!remote_pads)
> + return -ENOMEM;
> +
> + for (i = 0; i < sd->entity.num_pads; ++i)
> + remote_pads[i] = U32_MAX;
> + }
> +
> + for (i = 0; i < routing->num_routes; ++i) {
> + const struct v4l2_subdev_route *route = &routing->routes[i];
I guess validating non-active routes is intentional.
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Thanks
j
> +
> + /* Validate the sink and source pad numbers. */
> + if (route->sink_pad >= sd->entity.num_pads ||
> + !(sd->entity.pads[route->sink_pad].flags & MEDIA_PAD_FL_SINK)) {
> + dev_dbg(sd->dev, "route %u sink (%u) is not a sink pad\n",
> + i, route->sink_pad);
> + goto out;
> + }
> +
> + if (route->source_pad >= sd->entity.num_pads ||
> + !(sd->entity.pads[route->source_pad].flags & MEDIA_PAD_FL_SOURCE)) {
> + dev_dbg(sd->dev, "route %u source (%u) is not a source pad\n",
> + i, route->source_pad);
> + goto out;
> + }
> +
> + /*
> + * V4L2_SUBDEV_ROUTING_NO_STREAM_MIX: Streams on the same pad
> + * may not be routed to streams on different pads.
> + */
> + if (disallow & V4L2_SUBDEV_ROUTING_NO_STREAM_MIX) {
> + if (remote_pads[route->sink_pad] != U32_MAX &&
> + remote_pads[route->sink_pad] != route->source_pad) {
> + dev_dbg(sd->dev,
> + "route %u attempts to mix %s streams\n",
> + i, "sink");
> + goto out;
> + }
> +
> + if (remote_pads[route->source_pad] != U32_MAX &&
> + remote_pads[route->source_pad] != route->sink_pad) {
> + dev_dbg(sd->dev,
> + "route %u attempts to mix %s streams\n",
> + i, "source");
> + goto out;
> + }
> +
> + remote_pads[route->sink_pad] = route->source_pad;
> + remote_pads[route->source_pad] = route->sink_pad;
> + }
> +
> + for (j = i + 1; j < routing->num_routes; ++j) {
> + const struct v4l2_subdev_route *r = &routing->routes[j];
> +
> + /*
> + * V4L2_SUBDEV_ROUTING_NO_1_TO_N: No two routes can
> + * originate from the same (sink) stream.
> + */
> + if ((disallow & V4L2_SUBDEV_ROUTING_NO_1_TO_N) &&
> + route->sink_pad == r->sink_pad &&
> + route->sink_stream == r->sink_stream) {
> + dev_dbg(sd->dev,
> + "routes %u and %u originate from same sink (%u/%u)\n",
> + i, j, route->sink_pad,
> + route->sink_stream);
> + goto out;
> + }
> +
> + /*
> + * V4L2_SUBDEV_ROUTING_NO_N_TO_1: No two routes can end
> + * at the same (source) stream.
> + */
> + if ((disallow & V4L2_SUBDEV_ROUTING_NO_N_TO_1) &&
> + route->source_pad == r->source_pad &&
> + route->source_stream == r->source_stream) {
> + dev_dbg(sd->dev,
> + "routes %u and %u end at same source (%u/%u)\n",
> + i, j, route->source_pad,
> + route->source_stream);
> + goto out;
> + }
> + }
> + }
> +
> + ret = 0;
> +
> +out:
> + kfree(remote_pads);
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_routing_validate);
> +
> #endif /* CONFIG_MEDIA_CONTROLLER */
>
> void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index 97db6dfc0b7a..f75e161d646b 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -1514,6 +1514,45 @@ v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
> int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
> struct v4l2_subdev_format *format);
>
> +/**
> + * enum v4l2_subdev_routing_restriction - Subdevice internal routing restrictions
> + *
> + * @V4L2_SUBDEV_ROUTING_NO_1_TO_N:
> + * an input stream may not be routed to multiple output streams (stream
> + * duplication)
> + * @V4L2_SUBDEV_ROUTING_NO_N_TO_1:
> + * multiple input streams may not be routed to the same output stream
> + * (stream merging)
> + * @V4L2_SUBDEV_ROUTING_NO_STREAM_MIX:
> + * streams on the same pad may not be routed to streams on different pads
> + * @V4L2_SUBDEV_ROUTING_ONLY_1_TO_1:
> + * only non-overlapping 1-to-1 stream routing is allowed (a combination of
> + * @V4L2_SUBDEV_ROUTING_NO_1_TO_N and @V4L2_SUBDEV_ROUTING_NO_N_TO_1)
> + */
> +enum v4l2_subdev_routing_restriction {
> + V4L2_SUBDEV_ROUTING_NO_1_TO_N = BIT(0),
> + V4L2_SUBDEV_ROUTING_NO_N_TO_1 = BIT(1),
> + V4L2_SUBDEV_ROUTING_NO_STREAM_MIX = BIT(2),
> + V4L2_SUBDEV_ROUTING_ONLY_1_TO_1 =
> + V4L2_SUBDEV_ROUTING_NO_1_TO_N |
> + V4L2_SUBDEV_ROUTING_NO_N_TO_1,
> +};
> +
> +/**
> + * v4l2_subdev_routing_validate() - Verify that routes comply with driver
> + * constraints
> + * @sd: The subdevice
> + * @routing: Routing to verify
> + * @disallow: Restrictions on routes
> + *
> + * This verifies that the given routing complies with the @disallow constraints.
> + *
> + * Returns 0 on success, error value otherwise.
> + */
> +int v4l2_subdev_routing_validate(struct v4l2_subdev *sd,
> + const struct v4l2_subdev_krouting *routing,
> + enum v4l2_subdev_routing_restriction disallow);
> +
> #endif /* CONFIG_MEDIA_CONTROLLER */
>
> /**
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 34/36] media: v4l2-subdev: Add v4l2_subdev_state_xlate_streams() helper
2022-03-01 16:11 ` [PATCH v11 34/36] media: v4l2-subdev: Add v4l2_subdev_state_xlate_streams() helper Tomi Valkeinen
@ 2022-03-16 11:26 ` Jacopo Mondi
2022-03-17 9:27 ` Tomi Valkeinen
2022-07-07 10:27 ` Tomasz Figa
1 sibling, 1 reply; 67+ messages in thread
From: Jacopo Mondi @ 2022-03-16 11:26 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Hi Tomi,
On Tue, Mar 01, 2022 at 06:11:54PM +0200, Tomi Valkeinen wrote:
> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>
> Add a helper function to translate streams between two pads of a subdev,
> using the subdev's internal routing table.
>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
> drivers/media/v4l2-core/v4l2-subdev.c | 25 +++++++++++++++++++++++++
> include/media/v4l2-subdev.h | 23 +++++++++++++++++++++++
> 2 files changed, 48 insertions(+)
>
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index e30338a53088..6a9fc62dacbf 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -1529,6 +1529,31 @@ v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
> }
> EXPORT_SYMBOL_GPL(v4l2_subdev_state_get_opposite_stream_format);
>
> +u64 v4l2_subdev_state_xlate_streams(const struct v4l2_subdev_state *state,
> + u32 pad0, u32 pad1, u64 *streams)
> +{
> + const struct v4l2_subdev_krouting *routing = &state->routing;
> + struct v4l2_subdev_route *route;
> + u64 streams0 = 0;
> + u64 streams1 = 0;
> +
> + for_each_active_route(routing, route) {
> + if (route->sink_pad == pad0 && route->source_pad == pad1 &&
> + (*streams & BIT(route->sink_stream))) {
> + streams0 |= BIT(route->sink_stream);
> + streams1 |= BIT(route->source_stream);
> + }
> + if (route->source_pad == pad0 && route->sink_pad == pad1 &&
> + (*streams & BIT(route->source_stream))) {
> + streams0 |= BIT(route->source_stream);
> + streams1 |= BIT(route->sink_stream);
> + }
> + }
> +
> + *streams = streams0;
> + return streams1;
I'll probably learn how this is used later, but I found it hard to
immagine how the returned mask should be used.
To a sink stream mask that contains multiple streams a source maks
with multiple entries will be associated
In example, the sink stream mask might look lik
sink_stream_mask = {1, 0, 1, 0}
So we are interested in translating stream 0 and 2
Assume 0->4 and 2->1 in the routing tabe, the resulting source stream
mask will be
source_stream_mask = {1, 0, 0, 1}
How should the caller know what stream goes where ?
> +}
> +
> int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
> struct v4l2_subdev_format *format)
> {
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index 1eced0f47296..992debe116ac 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -1497,6 +1497,29 @@ struct v4l2_mbus_framefmt *
> v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
> u32 pad, u32 stream);
>
> +/**
> + * v4l2_subdev_state_xlate_streams() - Translate streams from one pad to another
Could we stress out that pads are on the same entity and this is not
meant to translate over media links ? Is this necessary ?
> + *
> + * @state: Subdevice state
> + * @pad0: The first pad
> + * @pad1: The second pad
> + * @streams: Streams bitmask on the first pad
> + *
> + * Streams on sink pads of a subdev are routed to source pads as expressed in
> + * the subdev state routing table. Stream numbers don't necessarily match on
> + * the sink and source side of a route. This function translates stream numbers
> + * on @pad0, expressed as a bitmask in @streams, to the corresponding streams
> + * on @pad1 using the routing table from the @state. It returns the stream mask
> + * on @pad1, and updates @streams with the streams that have been found in the
> + * routing table.
> + *
> + * @pad0 and @pad1 must be a sink and a source, in any order.
> + *
> + * Return: The bitmask of streams of @pad1 that are routed to @streams on @pad0.
> + */
> +u64 v4l2_subdev_state_xlate_streams(const struct v4l2_subdev_state *state,
> + u32 pad0, u32 pad1, u64 *streams);
> +
> /**
> * v4l2_subdev_get_fmt() - Fill format based on state
> * @sd: subdevice
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 35/36] media: v4l2-subdev: Add subdev .(enable|disable)_streams() operations
2022-03-01 16:11 ` [PATCH v11 35/36] media: v4l2-subdev: Add subdev .(enable|disable)_streams() operations Tomi Valkeinen
@ 2022-03-16 12:05 ` Jacopo Mondi
2022-03-17 9:43 ` Tomi Valkeinen
0 siblings, 1 reply; 67+ messages in thread
From: Jacopo Mondi @ 2022-03-16 12:05 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Hi Tomi,
On Tue, Mar 01, 2022 at 06:11:55PM +0200, Tomi Valkeinen wrote:
> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>
> Add two new subdev pad operations, .enable_streams() and
> .disable_streams(), to allow control of individual streams per pad. This
> is a superset of what the video .s_stream() operation implements.
>
> To help with handling of backward compatibility, add two wrapper
> functions around those operations, and require their usage in drivers.
>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
> drivers/media/v4l2-core/v4l2-subdev.c | 216 ++++++++++++++++++++++++++
> include/media/v4l2-subdev.h | 85 ++++++++++
> 2 files changed, 301 insertions(+)
>
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 6a9fc62dacbf..f75a1995a70b 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -1698,6 +1698,222 @@ __v4l2_subdev_next_active_route(const struct v4l2_subdev_krouting *routing,
> }
> EXPORT_SYMBOL_GPL(__v4l2_subdev_next_active_route);
>
> +static int v4l2_subdev_enable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
> + u64 streams_mask)
> +{
> + unsigned int i;
> + int ret;
> +
> + /*
> + * The subdev doesn't implement pad-based stream enable, fall back
> + * on the .s_stream() operation. This can only be done for subdevs that
> + * have a single source pad, as sd->enabled_streams is global to the
> + * subdev.
> + */
> + if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
> + return -EOPNOTSUPP;
> +
> + for (i = 0; i < sd->entity.num_pads; ++i) {
> + if (i != pad && sd->entity.pads[i].flags & MEDIA_PAD_FL_SOURCE)
> + return -EOPNOTSUPP;
> + }
> +
> + if (sd->enabled_streams & streams_mask)
> + return -EALREADY;
I wonder if a few dev_dbg on errors might save someone an headache
> +
> + /* Start streaming when the first streams are enabled. */
> + if (!sd->enabled_streams) {
> + ret = v4l2_subdev_call(sd, video, s_stream, 1);
> + if (ret)
> + return ret;
> + }
> +
> + sd->enabled_streams |= streams_mask;
> +
> + return 0;
> +}
> +
> +int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
> + u64 streams_mask)
> +{
> + struct device *dev = sd->entity.graph_obj.mdev->dev;
> + struct v4l2_subdev_state *state;
> + u64 found_streams = 0;
> + unsigned int i;
> + int ret;
> +
> + /* A few basic sanity checks first. */
> + if (pad >= sd->entity.num_pads)
> + return -EINVAL;
Should we make sure pad is a SOURCE (and remove the same check in the
_fallback version) ?
> +
> + if (!streams_mask)
> + return 0;
> +
> + /* Fallback on .s_stream() if .enable_streams() isn't available. */
> + if (!sd->ops->pad || !sd->ops->pad->enable_streams)
> + return v4l2_subdev_enable_streams_fallback(sd, pad,
> + streams_mask);
> +
> + state = v4l2_subdev_lock_and_get_active_state(sd);
> +
> + /*
> + * Verify that the requested streams exist and that they are not
> + * already enabled.
> + */
> + for (i = 0; i < state->stream_configs.num_configs; ++i) {
> + struct v4l2_subdev_stream_config *cfg =
> + &state->stream_configs.configs[i];
> +
> + if (cfg->pad != pad || !(streams_mask & BIT(cfg->stream)))
> + continue;
> +
> + found_streams |= BIT(cfg->stream);
> +
> + if (cfg->enabled) {
> + dev_dbg(dev, "stream %u already enabled on %s/%u\n",
> + cfg->stream, sd->entity.name, pad);
> + ret = -EALREADY;
> + goto done;
> + }
> + }
> +
> + if (found_streams != streams_mask) {
> + dev_dbg(dev, "streams 0x%llx not found on %s/%u\n",
nit: I would use the more usual form of entity:pad in the error
message
I like the idea :)
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Thanks
j
> + streams_mask & ~found_streams, sd->entity.name, pad);
> + ret = -EINVAL;
> + goto done;
> + }
> +
> + /* Call the .enable_streams() operation. */
> + ret = v4l2_subdev_call(sd, pad, enable_streams, state, pad,
> + streams_mask);
> + if (ret)
> + goto done;
> +
> + /* Mark the streams as enabled. */
> + for (i = 0; i < state->stream_configs.num_configs; ++i) {
> + struct v4l2_subdev_stream_config *cfg =
> + &state->stream_configs.configs[i];
> +
> + if (cfg->pad == pad && (streams_mask & BIT(cfg->stream)))
> + cfg->enabled = true;
> + }
> +
> +done:
> + v4l2_subdev_unlock_state(state);
> +
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_enable_streams);
> +
> +static int v4l2_subdev_disable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
> + u64 streams_mask)
> +{
> + unsigned int i;
> + int ret;
> +
> + /*
> + * If the subdev doesn't implement pad-based stream enable, fall back
> + * on the .s_stream() operation. This can only be done for subdevs that
> + * have a single source pad, as sd->enabled_streams is global to the
> + * subdev.
> + */
> + if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
> + return -EOPNOTSUPP;
> +
> + for (i = 0; i < sd->entity.num_pads; ++i) {
> + if (i != pad && sd->entity.pads[i].flags & MEDIA_PAD_FL_SOURCE)
> + return -EOPNOTSUPP;
> + }
> +
> + if ((sd->enabled_streams & streams_mask) != streams_mask)
> + return -EALREADY;
> +
> + /* Stop streaming when the last streams are disabled. */
> + if (!(sd->enabled_streams & ~streams_mask)) {
> + ret = v4l2_subdev_call(sd, video, s_stream, 0);
> + if (ret)
> + return ret;
> + }
> +
> + sd->enabled_streams &= ~streams_mask;
> +
> + return 0;
> +}
> +
> +int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
> + u64 streams_mask)
> +{
> + struct device *dev = sd->entity.graph_obj.mdev->dev;
> + struct v4l2_subdev_state *state;
> + u64 found_streams = 0;
> + unsigned int i;
> + int ret;
> +
> + /* A few basic sanity checks first. */
> + if (pad >= sd->entity.num_pads)
> + return -EINVAL;
> +
> + if (!streams_mask)
> + return 0;
> +
> + /* Fallback on .s_stream() if .disable_streams() isn't available. */
> + if (!sd->ops->pad || !sd->ops->pad->disable_streams)
> + return v4l2_subdev_disable_streams_fallback(sd, pad,
> + streams_mask);
> +
> + state = v4l2_subdev_lock_and_get_active_state(sd);
> +
> + /*
> + * Verify that the requested streams exist and that they are not
> + * already disabled.
> + */
> + for (i = 0; i < state->stream_configs.num_configs; ++i) {
> + struct v4l2_subdev_stream_config *cfg =
> + &state->stream_configs.configs[i];
> +
> + if (cfg->pad != pad || !(streams_mask & BIT(cfg->stream)))
> + continue;
> +
> + found_streams |= BIT(cfg->stream);
> +
> + if (!cfg->enabled) {
> + dev_dbg(dev, "stream %u already disabled on %s/%u\n",
> + cfg->stream, sd->entity.name, pad);
> + ret = -EALREADY;
> + goto done;
> + }
> + }
> +
> + if (found_streams != streams_mask) {
> + dev_dbg(dev, "streams 0x%llx not found on %s/%u\n",
> + streams_mask & ~found_streams, sd->entity.name, pad);
> + ret = -EINVAL;
> + goto done;
> + }
> +
> + /* Call the .disable_streams() operation. */
> + ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
> + streams_mask);
> + if (ret)
> + goto done;
> +
> + /* Mark the streams as disabled. */
> + for (i = 0; i < state->stream_configs.num_configs; ++i) {
> + struct v4l2_subdev_stream_config *cfg =
> + &state->stream_configs.configs[i];
> +
> + if (cfg->pad == pad && (streams_mask & BIT(cfg->stream)))
> + cfg->enabled = false;
> + }
> +
> +done:
> + v4l2_subdev_unlock_state(state);
> +
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_disable_streams);
> +
> #endif /* CONFIG_MEDIA_CONTROLLER */
>
> void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index 992debe116ac..bb1713863973 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -702,6 +702,7 @@ struct v4l2_subdev_pad_config {
> *
> * @pad: pad number
> * @stream: stream number
> + * @enabled: has the stream been enabled with v4l2_subdev_enable_stream()
> * @fmt: &struct v4l2_mbus_framefmt
> * @crop: &struct v4l2_rect to be used for crop
> * @compose: &struct v4l2_rect to be used for compose
> @@ -711,6 +712,7 @@ struct v4l2_subdev_pad_config {
> struct v4l2_subdev_stream_config {
> u32 pad;
> u32 stream;
> + bool enabled;
>
> struct v4l2_mbus_framefmt fmt;
> struct v4l2_rect crop;
> @@ -816,6 +818,18 @@ struct v4l2_subdev_state {
> *
> * @set_routing: enable or disable data connection routes described in the
> * subdevice routing table.
> + *
> + * @enable_streams: Enable the streams defined in streams_mask on the given
> + * source pad. Subdevs that implement this operation must use the active
> + * state management provided by the subdev core (enabled through a call to
> + * v4l2_subdev_init_finalize() at initialization time). Do not call
> + * directly, use v4l2_subdev_enable_streams() instead.
> + *
> + * @disable_streams: Disable the streams defined in streams_mask on the given
> + * source pad. Subdevs that implement this operation must use the active
> + * state management provided by the subdev core (enabled through a call to
> + * v4l2_subdev_init_finalize() at initialization time). Do not call
> + * directly, use v4l2_subdev_disable_streams() instead.
> */
> struct v4l2_subdev_pad_ops {
> int (*init_cfg)(struct v4l2_subdev *sd,
> @@ -862,6 +876,12 @@ struct v4l2_subdev_pad_ops {
> struct v4l2_subdev_state *state,
> enum v4l2_subdev_format_whence which,
> struct v4l2_subdev_krouting *route);
> + int (*enable_streams)(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state, u32 pad,
> + u64 streams_mask);
> + int (*disable_streams)(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state, u32 pad,
> + u64 streams_mask);
> };
>
> /**
> @@ -1007,6 +1027,10 @@ struct v4l2_subdev_platform_data {
> * @active_state: Active state for the subdev (NULL for subdevs tracking the
> * state internally). Initialized by calling
> * v4l2_subdev_init_finalize().
> + * @enabled_streams: Bitmask of enabled streams used by
> + * v4l2_subdev_enable_streams() and
> + * v4l2_subdev_disable_streams() helper functions for fallback
> + * cases.
> *
> * Each instance of a subdev driver should create this struct, either
> * stand-alone or embedded in a larger struct.
> @@ -1052,6 +1076,7 @@ struct v4l2_subdev {
> * doesn't support it.
> */
> struct v4l2_subdev_state *active_state;
> + u64 enabled_streams;
> };
>
>
> @@ -1589,6 +1614,66 @@ __v4l2_subdev_next_active_route(const struct v4l2_subdev_krouting *routing,
> for ((route) = NULL; \
> ((route) = __v4l2_subdev_next_active_route((routing), (route)));)
>
> +/**
> + * v4l2_subdev_enable_streams() - Enable streams on a pad
> + * @sd: The subdevice
> + * @pad: The pad
> + * @streams_mask: Bitmask of streams to enable
> + *
> + * This function enables streams on a source @pad of a subdevice. The pad is
> + * identified by its index, while the streams are identified by the
> + * @streams_mask bitmask. This allows enabling multiple streams on a pad at
> + * once.
> + *
> + * Enabling a stream that is already enabled isn't allowed. If @streams_mask
> + * contains an already enabled stream, this function returns -EALREADY without
> + * performing any operation.
> + *
> + * Per-stream enable is only available for subdevs that implement the
> + * .enable_streams() and .disable_streams() operations. For other subdevs, this
> + * function implements a best-effort compatibility by calling the .s_stream()
> + * operation, limited to subdevs that have a single source pad.
> + *
> + * Return:
> + * * 0: Success
> + * * -EALREADY: One of the streams in streams_mask is already enabled
> + * * -EINVAL: The pad index is invalid, or doesn't correspond to a source pad
> + * * -EOPNOTSUPP: Falling back to the legacy .s_stream() operation is
> + * impossible because the subdev has multiple source pads
> + */
> +int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
> + u64 streams_mask);
> +
> +/**
> + * v4l2_subdev_disable_streams() - Disable streams on a pad
> + * @sd: The subdevice
> + * @pad: The pad
> + * @streams_mask: Bitmask of streams to disable
> + *
> + * This function disables streams on a source @pad of a subdevice. The pad is
> + * identified by its index, while the streams are identified by the
> + * @streams_mask bitmask. This allows disabling multiple streams on a pad at
> + * once.
> + *
> + * Disabling a streams that is not enabled isn't allowed. If @streams_mask
> + * contains a disabled stream, this function returns -EALREADY without
> + * performing any operation.
> + *
> + * Per-stream disable is only available for subdevs that implement the
> + * .enable_streams() and .disable_streams() operations. For other subdevs, this
> + * function implements a best-effort compatibility by calling the .s_stream()
> + * operation, limited to subdevs that have a single source pad.
> + *
> + * Return:
> + * * 0: Success
> + * * -EALREADY: One of the streams in streams_mask is not enabled
> + * * -EINVAL: The pad index is invalid, or doesn't correspond to a source pad
> + * * -EOPNOTSUPP: Falling back to the legacy .s_stream() operation is
> + * impossible because the subdev has multiple source pads
> + */
> +int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
> + u64 streams_mask);
> +
> #endif /* CONFIG_MEDIA_CONTROLLER */
>
> /**
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 36/36] media: v4l2-subdev: Add v4l2_subdev_s_stream_helper() function
2022-03-01 16:11 ` [PATCH v11 36/36] media: v4l2-subdev: Add v4l2_subdev_s_stream_helper() function Tomi Valkeinen
@ 2022-03-16 12:10 ` Jacopo Mondi
2022-03-17 9:45 ` Tomi Valkeinen
0 siblings, 1 reply; 67+ messages in thread
From: Jacopo Mondi @ 2022-03-16 12:10 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Hi Laurent,
On Tue, Mar 01, 2022 at 06:11:56PM +0200, Tomi Valkeinen wrote:
> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>
> The v4l2_subdev_s_stream_helper() helper can be used by subdevs that
> implement the stream-aware .enable_streams() and .disable_streams()
> operations to implement .s_stream(). This is limited to subdevs that
> have a single source pad.
>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
> drivers/media/v4l2-core/v4l2-subdev.c | 40 +++++++++++++++++++++++++++
> include/media/v4l2-subdev.h | 17 ++++++++++++
> 2 files changed, 57 insertions(+)
>
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index f75a1995a70b..270445821f06 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -1914,6 +1914,46 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
> }
> EXPORT_SYMBOL_GPL(v4l2_subdev_disable_streams);
>
> +int v4l2_subdev_s_stream_helper(struct v4l2_subdev *sd, int enable)
> +{
> + struct v4l2_subdev_state *state;
> + struct v4l2_subdev_route *route;
> + struct media_pad *pad;
> + u64 source_mask = 0;
> + int pad_index = -1;
> +
> + /*
> + * Find the source pad. This helper is meant for subdevs that have a
> + * single source pad, so failures shouldn't happen, but catch them
> + * loudly nonetheless as they indicate a driver bug.
> + */
> + media_entity_for_each_pad(&sd->entity, pad) {
> + if (pad->flags & MEDIA_PAD_FL_SOURCE) {
> + pad_index = pad->index;
> + break;
> + }
> + }
> +
> + if (WARN_ON(pad_index == -1))
> + return -EINVAL;
> +
> + /*
> + * As there's a single source pad, just collect all the source streams.
> + */
> + state = v4l2_subdev_lock_and_get_active_state(sd);
> +
> + for_each_active_route(&state->routing, route)
> + source_mask |= BIT(route->source_stream);
> +
> + v4l2_subdev_unlock_state(state);
> +
> + if (enable)
> + return v4l2_subdev_enable_streams(sd, pad_index, source_mask);
> + else
> + return v4l2_subdev_disable_streams(sd, pad_index, source_mask);
Or
return enable ? ...
: ... ;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_s_stream_helper);
> +
> #endif /* CONFIG_MEDIA_CONTROLLER */
>
> void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index bb1713863973..817452ec30bb 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -1674,6 +1674,23 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
> int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
> u64 streams_mask);
>
> +/**
> + * v4l2_subdev_s_stream_helper() - Helper to implement the subdev s_stream
> + * operation using enable_streams and disable_streams
> + * @sd: The subdevice
> + * @enable: Enable to disable streaming
Enable or disable
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Thanks
j
> + *
> + * Subdevice drivers that implement the streams-aware
> + * &v4l2_subdev_pad_ops.enable_streams and &v4l2_subdev_pad_ops.disable_streams
> + * operations can use this helper to implement the legacy
> + * &v4l2_subdev_video_ops.s_stream operation.
> + *
> + * This helper can only be used by subdevs that have a single source pad.
> + *
> + * Return: 0 on success, or a negative error code otherwise.
> + */
> +int v4l2_subdev_s_stream_helper(struct v4l2_subdev *sd, int enable);
> +
> #endif /* CONFIG_MEDIA_CONTROLLER */
>
> /**
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 25/36] media: Documentation: add multiplexed streams documentation
2022-03-16 9:04 ` Jacopo Mondi
@ 2022-03-17 7:18 ` Tomi Valkeinen
2022-03-17 8:27 ` Jacopo Mondi
0 siblings, 1 reply; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-17 7:18 UTC (permalink / raw)
To: Jacopo Mondi
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
On 16/03/2022 11:04, Jacopo Mondi wrote:
> Hi Tomi
>
> On Tue, Mar 01, 2022 at 06:11:45PM +0200, Tomi Valkeinen wrote:
>> Add documentation related to multiplexed streams.
>>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> ---
>> .../driver-api/media/v4l2-subdev.rst | 8 +
>> .../userspace-api/media/v4l/dev-subdev.rst | 165 ++++++++++++++++++
>> 2 files changed, 173 insertions(+)
>>
>> diff --git a/Documentation/driver-api/media/v4l2-subdev.rst b/Documentation/driver-api/media/v4l2-subdev.rst
>> index 115211cef4d1..80654f1bcac9 100644
>> --- a/Documentation/driver-api/media/v4l2-subdev.rst
>> +++ b/Documentation/driver-api/media/v4l2-subdev.rst
>> @@ -593,6 +593,14 @@ before calling v4l2_subdev_init_finalize():
>>
>> This shares the driver's private mutex between the controls and the states.
>>
>> +Streams, multiplexed media pads and internal routing
>> +----------------------------------------------------
>> +
>> +A subdevice driver can implement support for multiplexed streams by setting
>
> Let me start by being picky with a minor thing: the rest of the
> documentation seems to use "sub-device". Here you have "sub-device",
> "subdevice" and "subdev". I think "sub-device" should be used
> everywhere
I can see lots of all those three in the docs, so I don't really know
which one to pick...
>> +the V4L2_SUBDEV_FL_MULTIPLEXED subdev flag and implementing support for
>> +centrally managed subdev active state, routing and stream based
>> +configuration.
>> +
>> V4L2 sub-device functions and data structures
>> ---------------------------------------------
>>
>> diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
>> index a67c2749089a..fd042afeddd6 100644
>> --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
>> +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
>> @@ -503,3 +503,168 @@ source pads.
>> :maxdepth: 1
>>
>> subdev-formats
>> +
>> +Streams, multiplexed media pads and internal routing
>> +----------------------------------------------------
>> +
>> +Commonly V4L2 subdevices support only separate video streams, that is, each
>> +link in the media graph and each pad in a subdevice pass through a single
>> +video stream. Thus each pad contains a format configuration for that single
>
> Isn't it the other way around ? A single video stream passes through a
> media link.
It's a bit confusing. I meant "pass through" as in "each link and pad
let a single stream pass through". How about:
Commonly V4L2 subdevices support only separate video streams, that is,
only a single stream can pass through a media link and a media pad.
>> +stream. In some cases a subdev can do stream processing and split a stream
>> +into two or compose two streams into one, but the inputs and outputs for the
>> +subdev are still a single stream per pad.
>> +
>> +Some hardware, e.g. MIPI CSI-2, support multiplexed streams, that is, a single
>> +bus carries multiple streams. Thus a camera could output two streams, a pixel
>> +stream and a metadata stream, and a bridge subdev could route the streams
>> +from multiple input pads into a single output pad.
>
> I would use sink/source and not input/output and slightly rephrase
> this, because I understand your "bridge subdev" in this example is a
> CSI-2 transmitter but "bridge" usually refers to receiver drivers on
> the SoC.
Ok. In DRM, bridges are not the display controllers in the SoC, but the
"bridges" between the SoC and the panel.
> What about
>
> Some hardware, e.g. MIPI CSI-2, support multiplexed streams, that is,
> multiple data streams are transmitted on the same bus, which is
> represented by a media link connecting a transmitter source pad with a
> sink pad on the receiver. In example, a camera sensor can produce two
> distinct streams, a pixel stream and a metadata stream, which are
> transmitted on the multiplexed data bus, represented by a media link
> which connects the single sensor's source pad with the receiver sink
> pad. The stream-aware receiver will de-multiplex the streams received
> on the its sink pad and allows to route them individually to one of
> its source pads.
This sounds fine.
> Sensor Receiver
> +----------+ +--------------+
> | Pixel | | ___________(1)
> | \_(0) --------(0)/ |
> | Meta / | | \___________(2)
> | | | |
> +----------+ +---------------
>
> (not sure if makes sense to have a drawing here, but... )
>
>> +
>> +Subdevice drivers that support multiplexed streams are compatible with
>> +non-multiplexed subdev drivers, but, of course, require a routing configuration
>> +where the link between those two types of drivers contains only a single
>> +stream.
>> +
>> +Understanding streams
>> +^^^^^^^^^^^^^^^^^^^^^
>> +
>> +A stream is a stream of content (e.g. pixel data or metadata) flowing through
>> +the media pipeline from a source (e.g. a sensor) towards the final sink(e.g. a
>
> space between "sink("
>
>> +receiver and demultiplexer in a SoC). Each media link carries all the streams
>
> the enabled streams ?
>
>> +from one end of the link to the other, and subdevices have routing tables which
>> +describe how the incoming streams from sink pads are routed to the source
>> +pads.
>
> "... from sink pad", singular..
No, the routing table describes how the streams from the sink pads are
routed to the source pads. Why would it be singular?
> I would rather say "how the incoming streams from the multiplexed sink
> pad are routed to source pads".
>
>> +
>> +A stream ID (often just "stream") is a media link-local identifier for a
>> +stream. In other words, a configuration for a particular stream ID must exist
>
> s/In other words//
Why would you remove it? The sentence explains the previous sentence in
a more verbose way.
> Should the configuration on both ends of the link also be identical ?
If you refer to the format etc. config, the same rules apply here as for
non-multiplexed case. I'm not sure if they need to be identical, but
they have to match. But I'm not really talking about configuration here.
>> +on both sides of a media link, but another stream ID can be used for the same
>> +stream at the other side of the subdevice.
>
> I would
>
> A stream ID (often just "stream") is a media link-local identifier for a
> stream. The configuration for a stream ID must exist and be identical
> on both ends of a media link connecting two multiplexed pads.
You're describing a different thing there. I'm talking about the stream
IDs. I think it becomes clearer if I remove "a configuration for". So:
> A stream ID (often just "stream") is a media link-local identifier for a stream.
> In other words, a particular stream ID must exist on both sides of a media
> link, but another stream ID can be used for the same stream at the other side
> of the subdevice.
So it's explaining the stream IDs, how a single stream can be
represented by multiple stream IDs.
> and leave the bits about routing out ?
>
>> +
>> +A stream at a specific point in the media pipeline is identified with the
>> +subdev and a (pad, stream) pair. For subdevices that do not support
>> +multiplexed streams the 'stream' is always 0.
>> +
>> +Configuring streams
>> +^^^^^^^^^^^^^^^^^^^
>> +
>> +The configuration of the streams is done individually for each subdevice and
>> +the validity of the streams between subdevices is validated when the pipeline
>> +is started.
>> +
>> +There are three steps in configuring the streams:
>> +
>> +1) Set up links. Connect the pads between subdevices using the :ref:`Media
>> +Controller API <media_controller>`
>> +
>> +2) Routing. The routing table for the subdevice must be set with
>> +:ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl. Note that
>> +setting the routing table will reset all the stream configurations.
>
> "stream configurations in a media entity".
Ok.
>> +
>> +3) Configure streams. Each route endpoint must be configured
>> +with :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>`.
>> +
>> +Multiplexed streams setup example
>> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> +
>> +A simple example of a multiplexed stream setup might be as follows:
>> +
>> +- Two identical sensors (Sensor A and Sensor B). Each sensor has a single
>> + source pad (pad 0), and outputs two streams, pixel data and metadata.
>
> a single source pad (pad 0) which carries two streams: a pixel data
> stream and a metadata stream.
Ok.
>> +
>> +- Multiplexer bridge (Bridge). The bridge has two sink pads, connected to the
>> + sensors (pads 0, 1), and one source pad (pad 2), which outputs all 4
>> + streams.
>> +
>> +- Receiver in the SoC (Receiver). The receiver has a single sink pad (pad 0),
>> + connected to the bridge, and four source pads (pads 1-4), going to the DMA
>> + engine. The receiver demultiplexes the incoming streams to the four source
>> + pads.
>> +
>> +- Four DMA Engines in the SoC (DMA Engine). Each DMA engine is connected to a
>> + single source pad in the receiver.
>> +
>> +The sensors, the bridge and the receiver are modeled as V4L2 subdevices,
>> +exposed to userspace via /dev/v4l-subdevX device nodes. The DMA engines are
>> +modeled as V4L2 devices, exposed to userspace via /dev/videoX nodes.
>> +
>> +To configure this pipeline, the userspace must take the following steps:
>> +
>> +1) Set up media links between entities: connect the sensors to the bridge,
>> +bridge to the receiver, and the receiver to the DMA engines. This step does
>> +not differ from normal non-multiplexed media controller setup.
>> +
>> +2) Configure routing.
>> +
>> +.. flat-table:: Sensor routing table (identical on both sensors)
>> + :header-rows: 1
>> +
>> + * - Sink Pad/Stream
>> + - Source Pad/Stream
>> + - Routing Flags
>> + - Comments
>> + * - 0/0 (unused)
>> + - 0/0
>> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE | V4L2_SUBDEV_ROUTE_FL_SOURCE
>> + - Pixel data stream. Source route, i.e. the sink fields are unused.
>> + * - 0/0 (unused)
>> + - 0/1
>> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE | V4L2_SUBDEV_ROUTE_FL_SOURCE
>> + - Metadata stream. Source route, i.e. the sink fields are unused.
>> +
>> +.. flat-table:: Bridge routing table
>> + :header-rows: 1
>> +
>> + * - Sink Pad/Stream
>> + - Source Pad/Stream
>> + - Routing Flags
>> + - Comments
>> + * - 0/0
>> + - 2/0
>> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE
>> + - Pixel data stream from Sensor A
>> + * - 0/1
>> + - 2/1
>> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE
>> + - Metadata stream from Sensor A
>> + * - 1/0
>> + - 2/2
>> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE
>> + - Pixel data stream from Sensor B
>> + * - 1/1
>> + - 2/3
>> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE
>> + - Metadata stream from Sensor B
>> +
>> +.. flat-table:: Receiver routing table
>> + :header-rows: 1
>> +
>> + * - Sink Pad/Stream
>> + - Source Pad/Stream
>> + - Routing Flags
>> + - Comments
>> + * - 0/0
>> + - 1/0
>> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE
>> + - Pixel data stream from Sensor A
>> + * - 0/1
>> + - 2/0
>> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE
>> + - Metadata stream from Sensor A
>> + * - 0/2
>> + - 3/0
>> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE
>> + - Pixel data stream from Sensor B
>> + * - 0/3
>> + - 4/0
>> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE
>> + - Metadata stream from Sensor B
>> +
>> +3) Configure streams
>> +
>> +After configuring the routing table, the next step is configuring the streams.
>> +This step is similar to configuring the pads in a non-multiplexed streams
>> +setup, with the difference that we need to configure each (pad, stream) pair
>> +(i.e. route endpoint) instead of just a pad.
>> +
>> +A common way to accomplish this is to start from the sensors and propagate the
>> +configurations along the stream towards the receiver, using VIDIOC_SUBDEV_S_FMT
>
> using the :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls
>
>> +to configure each stream endpoint in each subdev.
>
> All minors or just suggestions:
>
> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Thanks!
Tomi
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 26/36] media: subdev: add stream based configuration
2022-03-16 9:59 ` Jacopo Mondi
@ 2022-03-17 8:01 ` Tomi Valkeinen
2022-03-17 8:38 ` Jacopo Mondi
0 siblings, 1 reply; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-17 8:01 UTC (permalink / raw)
To: Jacopo Mondi
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
On 16/03/2022 11:59, Jacopo Mondi wrote:
> Hi Tomi
>
> On Tue, Mar 01, 2022 at 06:11:46PM +0200, Tomi Valkeinen wrote:
>> Add support to manage configurations (format, crop, compose) per stream,
>> instead of per pad. This is accomplished with data structures that hold
>> an array of all subdev's stream configurations.
>>
>> The number of streams can vary at runtime based on routing. Every time
>> the routing is changed, the stream configurations need to be
>> re-initialized.
>>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> ---
>> .../v4l/vidioc-subdev-enum-frame-interval.rst | 5 +-
>> .../v4l/vidioc-subdev-enum-frame-size.rst | 5 +-
>> .../v4l/vidioc-subdev-enum-mbus-code.rst | 5 +-
>> .../media/v4l/vidioc-subdev-g-crop.rst | 5 +-
>> .../media/v4l/vidioc-subdev-g-fmt.rst | 5 +-
>> .../v4l/vidioc-subdev-g-frame-interval.rst | 5 +-
>> .../media/v4l/vidioc-subdev-g-selection.rst | 5 +-
>> drivers/media/v4l2-core/v4l2-subdev.c | 129 ++++++++++++++++--
>> include/media/v4l2-subdev.h | 48 +++++++
>> include/uapi/linux/v4l2-subdev.h | 28 +++-
>> 10 files changed, 218 insertions(+), 22 deletions(-)
>>
>> diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-interval.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-interval.rst
>> index 3703943b412f..8def4c05d3da 100644
>> --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-interval.rst
>> +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-interval.rst
>> @@ -92,7 +92,10 @@ multiple pads of the same sub-device is not defined.
>> - Frame intervals to be enumerated, from enum
>> :ref:`v4l2_subdev_format_whence <v4l2-subdev-format-whence>`.
>> * - __u32
>> - - ``reserved``\ [8]
>> + - ``stream``
>> + - Stream identifier.
>> + * - __u32
>> + - ``reserved``\ [7]
>
> Does VIDIOC_SUBDEV_ENUM_FRAME_INTERVAL (as well as
> VIDIOC_SUBDEV_G_FRAME_INTERVAL) need to be stream-aware ?
>
> What is the semantic of the stream identifiers for IOCTLs that seem to
> control a paramter which is global to the subdev ? Isn't the stream semantic
> required to be specified in the IOCTL documentation and not just added
> to the list of fields ?
Why would it be global to the subdev? struct
v4l2_subdev_frame_interval_enum already has 'pad' field, so it operates
on that pad. With streams, each stream in a pad may have different
characteristics, (similarly to different pads in non-stream case), so it
feels logical to me to add the 'stream' field.
>> - Reserved for future extensions. Applications and drivers must set
>> the array to zero.
>>
>> diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-size.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-size.rst
>> index c25a9896df0e..3ef361c0dca7 100644
>> --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-size.rst
>> +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-size.rst
>> @@ -97,7 +97,10 @@ information about try formats.
>> - Frame sizes to be enumerated, from enum
>> :ref:`v4l2_subdev_format_whence <v4l2-subdev-format-whence>`.
>> * - __u32
>> - - ``reserved``\ [8]
>> + - ``stream``
>> + - Stream identifier.
>> + * - __u32
>> + - ``reserved``\ [7]
>> - Reserved for future extensions. Applications and drivers must set
>> the array to zero.
>>
>> diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-mbus-code.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-mbus-code.rst
>> index 417f1a19bcc4..248f6f9ee7c5 100644
>> --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-mbus-code.rst
>> +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-mbus-code.rst
>> @@ -73,7 +73,10 @@ information about the try formats.
>> - ``flags``
>> - See :ref:`v4l2-subdev-mbus-code-flags`
>> * - __u32
>> - - ``reserved``\ [7]
>> + - ``stream``
>> + - Stream identifier.
>> + * - __u32
>> + - ``reserved``\ [6]
>> - Reserved for future extensions. Applications and drivers must set
>> the array to zero.
>>
>> diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-crop.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-crop.rst
>> index bd15c0a5a66b..1d267f7e7991 100644
>> --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-crop.rst
>> +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-crop.rst
>> @@ -96,7 +96,10 @@ modified format should be as close as possible to the original request.
>> - ``rect``
>> - Crop rectangle boundaries, in pixels.
>> * - __u32
>> - - ``reserved``\ [8]
>> + - ``stream``
>> + - Stream identifier.
>> + * - __u32
>> + - ``reserved``\ [7]
>> - Reserved for future extensions. Applications and drivers must set
>> the array to zero.
>>
>> diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-fmt.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-fmt.rst
>> index 7acdbb939d89..ed253a1e44b7 100644
>> --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-fmt.rst
>> +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-fmt.rst
>> @@ -102,7 +102,10 @@ should be as close as possible to the original request.
>> - Definition of an image format, see :c:type:`v4l2_mbus_framefmt` for
>> details.
>> * - __u32
>> - - ``reserved``\ [8]
>> + - ``stream``
>> + - Stream identifier.
>> + * - __u32
>> + - ``reserved``\ [7]
>> - Reserved for future extensions. Applications and drivers must set
>> the array to zero.
>>
>> diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-frame-interval.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-frame-interval.rst
>> index d7fe7543c506..842f962d2aea 100644
>> --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-frame-interval.rst
>> +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-frame-interval.rst
>> @@ -90,7 +90,10 @@ the same sub-device is not defined.
>> - ``interval``
>> - Period, in seconds, between consecutive video frames.
>> * - __u32
>> - - ``reserved``\ [9]
>> + - ``stream``
>> + - Stream identifier.
>> + * - __u32
>> + - ``reserved``\ [8]
>> - Reserved for future extensions. Applications and drivers must set
>> the array to zero.
>>
>> diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-selection.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-selection.rst
>> index f9172a42f036..6b629c19168c 100644
>> --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-selection.rst
>> +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-selection.rst
>> @@ -94,7 +94,10 @@ Selection targets and flags are documented in
>> - ``r``
>> - Selection rectangle, in pixels.
>> * - __u32
>> - - ``reserved``\ [8]
>> + - ``stream``
>> + - Stream identifier.
>> + * - __u32
>> + - ``reserved``\ [7]
>> - Reserved for future extensions. Applications and drivers must set
>> the array to zero.
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
>> index 1c836c2de86e..339d7b15e26c 100644
>> --- a/drivers/media/v4l2-core/v4l2-subdev.c
>> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
>> @@ -149,14 +149,34 @@ static inline int check_pad(struct v4l2_subdev *sd, u32 pad)
>> return 0;
>> }
>
> To be honest, the only IOCTL for which I have a clear idea of the
> stream paramter semantic is s/g_format.
I admit I don't have much experience with some of these ioctls. But the
idea is simple: in non-stream case ioctls operate on a subdev pad, in
stream case those ioctls operate on a subdev pad + stream tuple.
If there's an ioctl that truly operates on a pad only, then we need to
drop the stream parameter. But the ioctls don't look like that to me.
>> -static int check_state_pads(u32 which, struct v4l2_subdev_state *state)
>> +static int check_state_pads(struct v4l2_subdev *sd, u32 which,
>> + struct v4l2_subdev_state *state)
>> {
>> + if (sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED)
>> + return 0;
>> +
>> if (which == V4L2_SUBDEV_FORMAT_TRY && (!state || !state->pads))
>> return -EINVAL;
>>
>> return 0;
>> }
>>
>> +static int check_state_pad_stream(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state, u32 pad,
>> + u32 stream)
>> +{
>> + struct v4l2_mbus_framefmt *fmt;
>> +
>> + if (!(sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED))
>> + return 0;
>> +
>> + fmt = v4l2_subdev_state_get_stream_format(state, pad, stream);
>> + if (!fmt)
>> + return -EINVAL;
>> +
>> + return 0;
>> +}
>> +
>
> check_state_pads() is always called in conjunction with
> check_state_pad_stream(). I would have made a check_state() that
> handles the multiplexed and non-multiplexed case. But that's an
> implementation detail, so up to you.
That's true. I think it looked a bit different in earlier versions, but
looking at it now, combining those two makes sense.
>> static inline int check_format(struct v4l2_subdev *sd,
>> struct v4l2_subdev_state *state,
>> struct v4l2_subdev_format *format)
>> @@ -165,7 +185,8 @@ static inline int check_format(struct v4l2_subdev *sd,
>> return -EINVAL;
>>
>> return check_which(format->which) ? : check_pad(sd, format->pad) ? :
>> - check_state_pads(format->which, state);
>> + check_state_pads(sd, format->which, state) ? :
>> + check_state_pad_stream(sd, state, format->pad, format->stream);
>> }
>>
>> static int call_get_fmt(struct v4l2_subdev *sd,
>> @@ -192,7 +213,8 @@ static int call_enum_mbus_code(struct v4l2_subdev *sd,
>> return -EINVAL;
>>
>> return check_which(code->which) ? : check_pad(sd, code->pad) ? :
>> - check_state_pads(code->which, state) ? :
>> + check_state_pads(sd, code->which, state) ? :
>> + check_state_pad_stream(sd, state, code->pad, code->stream) ? :
>> sd->ops->pad->enum_mbus_code(sd, state, code);
>> }
>>
>> @@ -204,7 +226,8 @@ static int call_enum_frame_size(struct v4l2_subdev *sd,
>> return -EINVAL;
>>
>> return check_which(fse->which) ? : check_pad(sd, fse->pad) ? :
>> - check_state_pads(fse->which, state) ? :
>> + check_state_pads(sd, fse->which, state) ? :
>> + check_state_pad_stream(sd, state, fse->pad, fse->stream) ? :
>> sd->ops->pad->enum_frame_size(sd, state, fse);
>> }
>>
>> @@ -239,7 +262,8 @@ static int call_enum_frame_interval(struct v4l2_subdev *sd,
>> return -EINVAL;
>>
>> return check_which(fie->which) ? : check_pad(sd, fie->pad) ? :
>> - check_state_pads(fie->which, state) ? :
>> + check_state_pads(sd, fie->which, state) ? :
>> + check_state_pad_stream(sd, state, fie->pad, fie->stream) ? :
>> sd->ops->pad->enum_frame_interval(sd, state, fie);
>
> call_g_frame_interval and call_s_frame_interval do accept a struct
> v4l2_subdev_frame_interval paramter now. Should the validity of
> streams be checked there too ?
Yes, I need to check this.
>> }
>>
>> @@ -251,7 +275,8 @@ static inline int check_selection(struct v4l2_subdev *sd,
>> return -EINVAL;
>>
>> return check_which(sel->which) ? : check_pad(sd, sel->pad) ? :
>> - check_state_pads(sel->which, state);
>> + check_state_pads(sd, sel->which, state) ? :
>> + check_state_pad_stream(sd, state, sel->pad, sel->stream);
>> }
>>
>> static int call_get_selection(struct v4l2_subdev *sd,
>> @@ -865,6 +890,71 @@ const struct v4l2_file_operations v4l2_subdev_fops = {
>>
>> #ifdef CONFIG_MEDIA_CONTROLLER
>>
>> +static int
>> +v4l2_subdev_init_stream_configs(struct v4l2_subdev_stream_configs *stream_configs,
>> + const struct v4l2_subdev_krouting *routing)
>> +{
>> + u32 num_configs = 0;
>> + unsigned int i;
>> + u32 format_idx = 0;
>> +
>> + kvfree(stream_configs->configs);
>> + stream_configs->configs = NULL;
>> + stream_configs->num_configs = 0;
>> +
>> + /* Count number of formats needed */
>> + for (i = 0; i < routing->num_routes; ++i) {
>> + struct v4l2_subdev_route *route = &routing->routes[i];
>
> This is a good candidate for for_each_active_route()
Indeed.
>> +
>> + if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
>> + continue;
>> +
>> + /*
>> + * Each route needs a format on both ends of the route, except
>> + * for source streams which only need one format.
>> + */
>> + num_configs +=
>> + (route->flags & V4L2_SUBDEV_ROUTE_FL_SOURCE) ? 1 : 2;
>> + }
>> +
>> + if (num_configs) {
>> + stream_configs->configs =
>> + kvcalloc(num_configs, sizeof(*stream_configs->configs),
>> + GFP_KERNEL);
>> +
>> + if (!stream_configs->configs)
>> + return -ENOMEM;
>> +
>> + stream_configs->num_configs = num_configs;
>> + }
>> +
>> + /*
>> + * Fill in the 'pad' and stream' value for each item in the array from
>> + * the routing table
>> + */
>> + for (i = 0; i < routing->num_routes; ++i) {
>> + struct v4l2_subdev_route *route = &routing->routes[i];
>> + u32 idx;
>> +
>> + if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
>> + continue;
>
> you iterate only active routes again. Which makes me think that you
> could return after the first loop if (!num_configs) ?
Yes, I think you're right.
>> +
>> + if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_SOURCE)) {
>> + idx = format_idx++;
>> +
>> + stream_configs->configs[idx].pad = route->sink_pad;
>> + stream_configs->configs[idx].stream = route->sink_stream;
>> + }
>> +
>> + idx = format_idx++;
>> +
>> + stream_configs->configs[idx].pad = route->source_pad;
>> + stream_configs->configs[idx].stream = route->source_stream;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> int v4l2_subdev_get_fwnode_pad_1_to_1(struct media_entity *entity,
>> struct fwnode_endpoint *endpoint)
>> {
>> @@ -1042,7 +1132,8 @@ __v4l2_subdev_state_alloc(struct v4l2_subdev *sd, const char *lock_name,
>> else
>> state->lock = &state->_lock;
>>
>> - if (sd->entity.num_pads) {
>> + /* Drivers that support streams do not need the legacy pad config */
>> + if (!(sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED) && sd->entity.num_pads) {
>> state->pads = kvmalloc_array(sd->entity.num_pads,
>> sizeof(*state->pads),
>> GFP_KERNEL | __GFP_ZERO);
>> @@ -1083,6 +1174,7 @@ void __v4l2_subdev_state_free(struct v4l2_subdev_state *state)
>> mutex_destroy(&state->_lock);
>>
>> kfree(state->routing.routes);
>> + kvfree(state->stream_configs.configs);
>> kvfree(state->pads);
>> kfree(state);
>> }
>> @@ -1133,10 +1225,31 @@ int v4l2_subdev_set_routing(struct v4l2_subdev *sd,
>> dst->num_routes = src->num_routes;
>> }
>>
>> - return 0;
>> + return v4l2_subdev_init_stream_configs(&state->stream_configs, dst);
>> }
>> EXPORT_SYMBOL_GPL(v4l2_subdev_set_routing);
>>
>> +struct v4l2_mbus_framefmt *
>> +v4l2_subdev_state_get_stream_format(struct v4l2_subdev_state *state,
>> + unsigned int pad, u32 stream)
>> +{
>> + struct v4l2_subdev_stream_configs *stream_configs;
>> + unsigned int i;
>> +
>> + lockdep_assert_held(state->lock);
>
> This function is only called by check_state_pad_stream(). Does it
It is used in multiple places later.
> locks the state ? I understand this is temporary only as other users
> introduced later will lock the state ?
The caller is supposed to lock the state. subdev_do_ioctl_lock() does it
here. Do you mean the call wrappers could be called via some other route?
Tomi
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 25/36] media: Documentation: add multiplexed streams documentation
2022-03-17 7:18 ` Tomi Valkeinen
@ 2022-03-17 8:27 ` Jacopo Mondi
0 siblings, 0 replies; 67+ messages in thread
From: Jacopo Mondi @ 2022-03-17 8:27 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Hi Tomi
On Thu, Mar 17, 2022 at 09:18:42AM +0200, Tomi Valkeinen wrote:
> On 16/03/2022 11:04, Jacopo Mondi wrote:
> > Hi Tomi
> >
> > On Tue, Mar 01, 2022 at 06:11:45PM +0200, Tomi Valkeinen wrote:
> > > Add documentation related to multiplexed streams.
> > >
> > > Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> > > ---
> > > .../driver-api/media/v4l2-subdev.rst | 8 +
> > > .../userspace-api/media/v4l/dev-subdev.rst | 165 ++++++++++++++++++
> > > 2 files changed, 173 insertions(+)
> > >
> > > diff --git a/Documentation/driver-api/media/v4l2-subdev.rst b/Documentation/driver-api/media/v4l2-subdev.rst
> > > index 115211cef4d1..80654f1bcac9 100644
> > > --- a/Documentation/driver-api/media/v4l2-subdev.rst
> > > +++ b/Documentation/driver-api/media/v4l2-subdev.rst
> > > @@ -593,6 +593,14 @@ before calling v4l2_subdev_init_finalize():
> > >
> > > This shares the driver's private mutex between the controls and the states.
> > >
> > > +Streams, multiplexed media pads and internal routing
> > > +----------------------------------------------------
> > > +
> > > +A subdevice driver can implement support for multiplexed streams by setting
> >
> > Let me start by being picky with a minor thing: the rest of the
> > documentation seems to use "sub-device". Here you have "sub-device",
> > "subdevice" and "subdev". I think "sub-device" should be used
> > everywhere
>
> I can see lots of all those three in the docs, so I don't really know which
> one to pick...
>
You're right, the number of "subdev" entries is larger that what I
first thought. Let's keep using whatever then :)
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 26/36] media: subdev: add stream based configuration
2022-03-17 8:01 ` Tomi Valkeinen
@ 2022-03-17 8:38 ` Jacopo Mondi
2022-03-17 8:48 ` Tomi Valkeinen
0 siblings, 1 reply; 67+ messages in thread
From: Jacopo Mondi @ 2022-03-17 8:38 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Hi Tomi
On Thu, Mar 17, 2022 at 10:01:47AM +0200, Tomi Valkeinen wrote:
> On 16/03/2022 11:59, Jacopo Mondi wrote:
> > Hi Tomi
> >
> > On Tue, Mar 01, 2022 at 06:11:46PM +0200, Tomi Valkeinen wrote:
> > > Add support to manage configurations (format, crop, compose) per stream,
> > > instead of per pad. This is accomplished with data structures that hold
> > > an array of all subdev's stream configurations.
> > >
> > > The number of streams can vary at runtime based on routing. Every time
> > > the routing is changed, the stream configurations need to be
> > > re-initialized.
> > >
> > > Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> > > ---
> > > .../v4l/vidioc-subdev-enum-frame-interval.rst | 5 +-
> > > .../v4l/vidioc-subdev-enum-frame-size.rst | 5 +-
> > > .../v4l/vidioc-subdev-enum-mbus-code.rst | 5 +-
> > > .../media/v4l/vidioc-subdev-g-crop.rst | 5 +-
> > > .../media/v4l/vidioc-subdev-g-fmt.rst | 5 +-
> > > .../v4l/vidioc-subdev-g-frame-interval.rst | 5 +-
> > > .../media/v4l/vidioc-subdev-g-selection.rst | 5 +-
> > > drivers/media/v4l2-core/v4l2-subdev.c | 129 ++++++++++++++++--
> > > include/media/v4l2-subdev.h | 48 +++++++
> > > include/uapi/linux/v4l2-subdev.h | 28 +++-
> > > 10 files changed, 218 insertions(+), 22 deletions(-)
> > >
> > > diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-interval.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-interval.rst
> > > index 3703943b412f..8def4c05d3da 100644
> > > --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-interval.rst
> > > +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-interval.rst
> > > @@ -92,7 +92,10 @@ multiple pads of the same sub-device is not defined.
> > > - Frame intervals to be enumerated, from enum
> > > :ref:`v4l2_subdev_format_whence <v4l2-subdev-format-whence>`.
> > > * - __u32
> > > - - ``reserved``\ [8]
> > > + - ``stream``
> > > + - Stream identifier.
> > > + * - __u32
> > > + - ``reserved``\ [7]
> >
> > Does VIDIOC_SUBDEV_ENUM_FRAME_INTERVAL (as well as
> > VIDIOC_SUBDEV_G_FRAME_INTERVAL) need to be stream-aware ?
> >
> > What is the semantic of the stream identifiers for IOCTLs that seem to
> > control a paramter which is global to the subdev ? Isn't the stream semantic
> > required to be specified in the IOCTL documentation and not just added
> > to the list of fields ?
>
> Why would it be global to the subdev? struct v4l2_subdev_frame_interval_enum
> already has 'pad' field, so it operates on that pad. With streams, each
> stream in a pad may have different characteristics, (similarly to different
> pads in non-stream case), so it feels logical to me to add the 'stream'
> field.
>
I understand a device with multiple output interfaces can decide to
clock out frames at different speeds due to some bus configuration
parameters. But I have an hard time immagine a device that can clock
out frames at different rates on the same bus.
> > > - Reserved for future extensions. Applications and drivers must set
> > > the array to zero.
> > >
> > > diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-size.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-size.rst
> > > index c25a9896df0e..3ef361c0dca7 100644
> > > --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-size.rst
> > > +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-size.rst
> > > @@ -97,7 +97,10 @@ information about try formats.
> > > - Frame sizes to be enumerated, from enum
> > > :ref:`v4l2_subdev_format_whence <v4l2-subdev-format-whence>`.
> > > * - __u32
> > > - - ``reserved``\ [8]
> > > + - ``stream``
> > > + - Stream identifier.
> > > + * - __u32
> > > + - ``reserved``\ [7]
> > > - Reserved for future extensions. Applications and drivers must set
> > > the array to zero.
> > >
> > > diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-mbus-code.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-mbus-code.rst
> > > index 417f1a19bcc4..248f6f9ee7c5 100644
> > > --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-mbus-code.rst
> > > +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-mbus-code.rst
> > > @@ -73,7 +73,10 @@ information about the try formats.
> > > - ``flags``
> > > - See :ref:`v4l2-subdev-mbus-code-flags`
> > > * - __u32
> > > - - ``reserved``\ [7]
> > > + - ``stream``
> > > + - Stream identifier.
> > > + * - __u32
> > > + - ``reserved``\ [6]
> > > - Reserved for future extensions. Applications and drivers must set
> > > the array to zero.
> > >
> > > diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-crop.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-crop.rst
> > > index bd15c0a5a66b..1d267f7e7991 100644
> > > --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-crop.rst
> > > +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-crop.rst
> > > @@ -96,7 +96,10 @@ modified format should be as close as possible to the original request.
> > > - ``rect``
> > > - Crop rectangle boundaries, in pixels.
> > > * - __u32
> > > - - ``reserved``\ [8]
> > > + - ``stream``
> > > + - Stream identifier.
> > > + * - __u32
> > > + - ``reserved``\ [7]
> > > - Reserved for future extensions. Applications and drivers must set
> > > the array to zero.
> > >
> > > diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-fmt.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-fmt.rst
> > > index 7acdbb939d89..ed253a1e44b7 100644
> > > --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-fmt.rst
> > > +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-fmt.rst
> > > @@ -102,7 +102,10 @@ should be as close as possible to the original request.
> > > - Definition of an image format, see :c:type:`v4l2_mbus_framefmt` for
> > > details.
> > > * - __u32
> > > - - ``reserved``\ [8]
> > > + - ``stream``
> > > + - Stream identifier.
> > > + * - __u32
> > > + - ``reserved``\ [7]
> > > - Reserved for future extensions. Applications and drivers must set
> > > the array to zero.
> > >
> > > diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-frame-interval.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-frame-interval.rst
> > > index d7fe7543c506..842f962d2aea 100644
> > > --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-frame-interval.rst
> > > +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-frame-interval.rst
> > > @@ -90,7 +90,10 @@ the same sub-device is not defined.
> > > - ``interval``
> > > - Period, in seconds, between consecutive video frames.
> > > * - __u32
> > > - - ``reserved``\ [9]
> > > + - ``stream``
> > > + - Stream identifier.
> > > + * - __u32
> > > + - ``reserved``\ [8]
> > > - Reserved for future extensions. Applications and drivers must set
> > > the array to zero.
> > >
> > > diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-selection.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-selection.rst
> > > index f9172a42f036..6b629c19168c 100644
> > > --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-selection.rst
> > > +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-selection.rst
> > > @@ -94,7 +94,10 @@ Selection targets and flags are documented in
> > > - ``r``
> > > - Selection rectangle, in pixels.
> > > * - __u32
> > > - - ``reserved``\ [8]
> > > + - ``stream``
> > > + - Stream identifier.
> > > + * - __u32
> > > + - ``reserved``\ [7]
> > > - Reserved for future extensions. Applications and drivers must set
> > > the array to zero.
> > >
> > > diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> > > index 1c836c2de86e..339d7b15e26c 100644
> > > --- a/drivers/media/v4l2-core/v4l2-subdev.c
> > > +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> > > @@ -149,14 +149,34 @@ static inline int check_pad(struct v4l2_subdev *sd, u32 pad)
> > > return 0;
> > > }
> >
> > To be honest, the only IOCTL for which I have a clear idea of the
> > stream paramter semantic is s/g_format.
>
> I admit I don't have much experience with some of these ioctls. But the idea
> is simple: in non-stream case ioctls operate on a subdev pad, in stream case
> those ioctls operate on a subdev pad + stream tuple.
>
> If there's an ioctl that truly operates on a pad only, then we need to drop
> the stream parameter. But the ioctls don't look like that to me.
>
As above mentioned I have an hard time reconciling streams and frame
rate handling. Thinking about image sensors, changing the rate at
which frames are clocked out for one stream modifies the frame rate
globally for all of them. Other devices might behave differently ?
> > > -static int check_state_pads(u32 which, struct v4l2_subdev_state *state)
> > > +static int check_state_pads(struct v4l2_subdev *sd, u32 which,
> > > + struct v4l2_subdev_state *state)
> > > {
> > > + if (sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED)
> > > + return 0;
> > > +
> > > if (which == V4L2_SUBDEV_FORMAT_TRY && (!state || !state->pads))
> > > return -EINVAL;
> > >
> > > return 0;
> > > }
> > >
> > > +static int check_state_pad_stream(struct v4l2_subdev *sd,
> > > + struct v4l2_subdev_state *state, u32 pad,
> > > + u32 stream)
> > > +{
> > > + struct v4l2_mbus_framefmt *fmt;
> > > +
> > > + if (!(sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED))
> > > + return 0;
> > > +
> > > + fmt = v4l2_subdev_state_get_stream_format(state, pad, stream);
> > > + if (!fmt)
> > > + return -EINVAL;
> > > +
> > > + return 0;
> > > +}
> > > +
> >
> > check_state_pads() is always called in conjunction with
> > check_state_pad_stream(). I would have made a check_state() that
> > handles the multiplexed and non-multiplexed case. But that's an
> > implementation detail, so up to you.
>
> That's true. I think it looked a bit different in earlier versions, but
> looking at it now, combining those two makes sense.
>
> > > static inline int check_format(struct v4l2_subdev *sd,
> > > struct v4l2_subdev_state *state,
> > > struct v4l2_subdev_format *format)
> > > @@ -165,7 +185,8 @@ static inline int check_format(struct v4l2_subdev *sd,
> > > return -EINVAL;
> > >
> > > return check_which(format->which) ? : check_pad(sd, format->pad) ? :
> > > - check_state_pads(format->which, state);
> > > + check_state_pads(sd, format->which, state) ? :
> > > + check_state_pad_stream(sd, state, format->pad, format->stream);
> > > }
> > >
> > > static int call_get_fmt(struct v4l2_subdev *sd,
> > > @@ -192,7 +213,8 @@ static int call_enum_mbus_code(struct v4l2_subdev *sd,
> > > return -EINVAL;
> > >
> > > return check_which(code->which) ? : check_pad(sd, code->pad) ? :
> > > - check_state_pads(code->which, state) ? :
> > > + check_state_pads(sd, code->which, state) ? :
> > > + check_state_pad_stream(sd, state, code->pad, code->stream) ? :
> > > sd->ops->pad->enum_mbus_code(sd, state, code);
> > > }
> > >
> > > @@ -204,7 +226,8 @@ static int call_enum_frame_size(struct v4l2_subdev *sd,
> > > return -EINVAL;
> > >
> > > return check_which(fse->which) ? : check_pad(sd, fse->pad) ? :
> > > - check_state_pads(fse->which, state) ? :
> > > + check_state_pads(sd, fse->which, state) ? :
> > > + check_state_pad_stream(sd, state, fse->pad, fse->stream) ? :
> > > sd->ops->pad->enum_frame_size(sd, state, fse);
> > > }
> > >
> > > @@ -239,7 +262,8 @@ static int call_enum_frame_interval(struct v4l2_subdev *sd,
> > > return -EINVAL;
> > >
> > > return check_which(fie->which) ? : check_pad(sd, fie->pad) ? :
> > > - check_state_pads(fie->which, state) ? :
> > > + check_state_pads(sd, fie->which, state) ? :
> > > + check_state_pad_stream(sd, state, fie->pad, fie->stream) ? :
> > > sd->ops->pad->enum_frame_interval(sd, state, fie);
> >
> > call_g_frame_interval and call_s_frame_interval do accept a struct
> > v4l2_subdev_frame_interval paramter now. Should the validity of
> > streams be checked there too ?
>
> Yes, I need to check this.
>
> > > }
> > >
> > > @@ -251,7 +275,8 @@ static inline int check_selection(struct v4l2_subdev *sd,
> > > return -EINVAL;
> > >
> > > return check_which(sel->which) ? : check_pad(sd, sel->pad) ? :
> > > - check_state_pads(sel->which, state);
> > > + check_state_pads(sd, sel->which, state) ? :
> > > + check_state_pad_stream(sd, state, sel->pad, sel->stream);
> > > }
> > >
> > > static int call_get_selection(struct v4l2_subdev *sd,
> > > @@ -865,6 +890,71 @@ const struct v4l2_file_operations v4l2_subdev_fops = {
> > >
> > > #ifdef CONFIG_MEDIA_CONTROLLER
> > >
> > > +static int
> > > +v4l2_subdev_init_stream_configs(struct v4l2_subdev_stream_configs *stream_configs,
> > > + const struct v4l2_subdev_krouting *routing)
> > > +{
> > > + u32 num_configs = 0;
> > > + unsigned int i;
> > > + u32 format_idx = 0;
> > > +
> > > + kvfree(stream_configs->configs);
> > > + stream_configs->configs = NULL;
> > > + stream_configs->num_configs = 0;
> > > +
> > > + /* Count number of formats needed */
> > > + for (i = 0; i < routing->num_routes; ++i) {
> > > + struct v4l2_subdev_route *route = &routing->routes[i];
> >
> > This is a good candidate for for_each_active_route()
>
> Indeed.
>
> > > +
> > > + if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
> > > + continue;
> > > +
> > > + /*
> > > + * Each route needs a format on both ends of the route, except
> > > + * for source streams which only need one format.
> > > + */
> > > + num_configs +=
> > > + (route->flags & V4L2_SUBDEV_ROUTE_FL_SOURCE) ? 1 : 2;
> > > + }
> > > +
> > > + if (num_configs) {
> > > + stream_configs->configs =
> > > + kvcalloc(num_configs, sizeof(*stream_configs->configs),
> > > + GFP_KERNEL);
> > > +
> > > + if (!stream_configs->configs)
> > > + return -ENOMEM;
> > > +
> > > + stream_configs->num_configs = num_configs;
> > > + }
> > > +
> > > + /*
> > > + * Fill in the 'pad' and stream' value for each item in the array from
> > > + * the routing table
> > > + */
> > > + for (i = 0; i < routing->num_routes; ++i) {
> > > + struct v4l2_subdev_route *route = &routing->routes[i];
> > > + u32 idx;
> > > +
> > > + if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
> > > + continue;
> >
> > you iterate only active routes again. Which makes me think that you
> > could return after the first loop if (!num_configs) ?
>
> Yes, I think you're right.
>
> > > +
> > > + if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_SOURCE)) {
> > > + idx = format_idx++;
> > > +
> > > + stream_configs->configs[idx].pad = route->sink_pad;
> > > + stream_configs->configs[idx].stream = route->sink_stream;
> > > + }
> > > +
> > > + idx = format_idx++;
> > > +
> > > + stream_configs->configs[idx].pad = route->source_pad;
> > > + stream_configs->configs[idx].stream = route->source_stream;
> > > + }
> > > +
> > > + return 0;
> > > +}
> > > +
> > > int v4l2_subdev_get_fwnode_pad_1_to_1(struct media_entity *entity,
> > > struct fwnode_endpoint *endpoint)
> > > {
> > > @@ -1042,7 +1132,8 @@ __v4l2_subdev_state_alloc(struct v4l2_subdev *sd, const char *lock_name,
> > > else
> > > state->lock = &state->_lock;
> > >
> > > - if (sd->entity.num_pads) {
> > > + /* Drivers that support streams do not need the legacy pad config */
> > > + if (!(sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED) && sd->entity.num_pads) {
> > > state->pads = kvmalloc_array(sd->entity.num_pads,
> > > sizeof(*state->pads),
> > > GFP_KERNEL | __GFP_ZERO);
> > > @@ -1083,6 +1174,7 @@ void __v4l2_subdev_state_free(struct v4l2_subdev_state *state)
> > > mutex_destroy(&state->_lock);
> > >
> > > kfree(state->routing.routes);
> > > + kvfree(state->stream_configs.configs);
> > > kvfree(state->pads);
> > > kfree(state);
> > > }
> > > @@ -1133,10 +1225,31 @@ int v4l2_subdev_set_routing(struct v4l2_subdev *sd,
> > > dst->num_routes = src->num_routes;
> > > }
> > >
> > > - return 0;
> > > + return v4l2_subdev_init_stream_configs(&state->stream_configs, dst);
> > > }
> > > EXPORT_SYMBOL_GPL(v4l2_subdev_set_routing);
> > >
> > > +struct v4l2_mbus_framefmt *
> > > +v4l2_subdev_state_get_stream_format(struct v4l2_subdev_state *state,
> > > + unsigned int pad, u32 stream)
> > > +{
> > > + struct v4l2_subdev_stream_configs *stream_configs;
> > > + unsigned int i;
> > > +
> > > + lockdep_assert_held(state->lock);
> >
> > This function is only called by check_state_pad_stream(). Does it
>
> It is used in multiple places later.
>
> > locks the state ? I understand this is temporary only as other users
> > introduced later will lock the state ?
>
> The caller is supposed to lock the state. subdev_do_ioctl_lock() does it
> here. Do you mean the call wrappers could be called via some other route?
I was concerned calling v4l2_subdev_state_get_stream_format() from
check_state_pad_stream() would trigger a lockdep warning. But I had
missed subdev_do_ioctl_lock() in the call path.
Thanks
j
>
> Tomi
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 27/36] media: subdev: use streams in v4l2_subdev_link_validate()
2022-03-16 10:47 ` Jacopo Mondi
@ 2022-03-17 8:39 ` Tomi Valkeinen
0 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-17 8:39 UTC (permalink / raw)
To: Jacopo Mondi
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
On 16/03/2022 12:47, Jacopo Mondi wrote:
> Hi Tomi
>
> On Tue, Mar 01, 2022 at 06:11:47PM +0200, Tomi Valkeinen wrote:
>> Update v4l2_subdev_link_validate() to use routing and streams for
>> validation.
>>
>> Instead of just looking at the format on the pad on both ends of the
>> link, the routing tables are used to collect all the streams going from
>> the source to the sink over the link, and the streams' formats on both
>> ends of the link are verified.
>>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> ---
>> drivers/media/v4l2-core/v4l2-subdev.c | 245 ++++++++++++++++++++++++--
>> 1 file changed, 227 insertions(+), 18 deletions(-)
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
>> index 339d7b15e26c..091b854e00d0 100644
>> --- a/drivers/media/v4l2-core/v4l2-subdev.c
>> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
>> @@ -16,6 +16,7 @@
>> #include <linux/videodev2.h>
>> #include <linux/export.h>
>> #include <linux/version.h>
>> +#include <linux/sort.h>
>
> Not sorted, but it was already like this
>>
>> #include <media/v4l2-ctrls.h>
>> #include <media/v4l2-device.h>
>> @@ -1035,6 +1036,7 @@ EXPORT_SYMBOL_GPL(v4l2_subdev_link_validate_default);
>>
>> static int
>> v4l2_subdev_link_validate_get_format(struct media_pad *pad,
>> + u32 stream,
>
> fits on the previous line
>
>> struct v4l2_subdev_format *fmt)
>> {
>> if (is_media_entity_v4l2_subdev(pad->entity)) {
>> @@ -1046,6 +1048,7 @@ v4l2_subdev_link_validate_get_format(struct media_pad *pad,
>>
>> fmt->which = V4L2_SUBDEV_FORMAT_ACTIVE;
>> fmt->pad = pad->index;
>> + fmt->stream = stream;
>> return v4l2_subdev_call(sd, pad, get_fmt, state, fmt);
>> }
>>
>> @@ -1056,31 +1059,237 @@ v4l2_subdev_link_validate_get_format(struct media_pad *pad,
>> return -EINVAL;
>> }
>>
>> -int v4l2_subdev_link_validate(struct media_link *link)
>> +static int cmp_u32(const void *a, const void *b)
>> {
>> - struct v4l2_subdev *sink;
>> - struct v4l2_subdev_format sink_fmt, source_fmt;
>> - int rval;
>> + u32 a32 = *(u32 *)a;
>> + u32 b32 = *(u32 *)b;
>>
>> - rval = v4l2_subdev_link_validate_get_format(
>> - link->source, &source_fmt);
>> - if (rval < 0)
>> - return 0;
>> + return a32 > b32 ? 1 : (a32 < b32 ? -1 : 0);
>> +}
>> +
>> +static int v4l2_link_validate_get_streams(struct media_link *link,
>> + bool is_source, u32 *out_num_streams,
>> + const u32 **out_streams,
>> + bool *allocated)
>> +{
>> + static const u32 default_streams[] = { 0 };
>> + struct v4l2_subdev_krouting *routing;
>> + struct v4l2_subdev *subdev;
>> + u32 num_streams;
>> + u32 *streams;
>> + unsigned int i;
>> + struct v4l2_subdev_state *state;
>> +
>> + if (is_source)
>> + subdev = media_entity_to_v4l2_subdev(link->source->entity);
>> + else
>> + subdev = media_entity_to_v4l2_subdev(link->sink->entity);
>>
>> - rval = v4l2_subdev_link_validate_get_format(
>> - link->sink, &sink_fmt);
>> - if (rval < 0)
>> + if (!(subdev->flags & V4L2_SUBDEV_FL_MULTIPLEXED)) {
>> + *out_num_streams = 1;
>> + *out_streams = default_streams;
>> + *allocated = false;
>> return 0;
>> + }
>>
>> - sink = media_entity_to_v4l2_subdev(link->sink->entity);
>> + state = v4l2_subdev_get_locked_active_state(subdev);
>>
>> - rval = v4l2_subdev_call(sink, pad, link_validate, link,
>> - &source_fmt, &sink_fmt);
>> - if (rval != -ENOIOCTLCMD)
>> - return rval;
>> + routing = &state->routing;
>> +
>> + streams = kmalloc_array(routing->num_routes, sizeof(u32), GFP_KERNEL);
>> + if (!streams)
>> + return -ENOMEM;
>> +
>> + num_streams = 0;
>> +
>> + for (i = 0; i < routing->num_routes; ++i) {
>> + struct v4l2_subdev_route *route = &routing->routes[i];
>> + int j;
>> + u32 route_pad;
>> + u32 route_stream;
>> + u32 link_pad;
>> +
>> + if (is_source) {
>> + route_pad = route->source_pad;
>> + route_stream = route->source_stream;
>> + link_pad = link->source->index;
>> + } else {
>> + route_pad = route->sink_pad;
>> + route_stream = route->sink_stream;
>> + link_pad = link->sink->index;
>> + }
>> +
>> + if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
>> + continue;
>
> Should this check be done before the previous if/else block ?
Yes, and I think this can use for_each_active_route.
>> +
>> + if (route_pad != link_pad)
>> + continue;
>> +
>> + /* look for duplicates... */
>> + for (j = 0; j < num_streams; ++j) {
>> + if (streams[j] == route_stream)
>> + break;
>> + }
>> +
>> + /* ...and drop the stream if we already have it */
>> + if (j != num_streams)
>> + continue;
>
> Is this an error ? can the same stream be routed to multiple ones on a
> media link ?
There must have been a reason for me to add this code, but... Indeed, we
shouldn't have duplicates. I'll change this to an error.
>> +
>> + streams[num_streams++] = route_stream;
>> + }
>> +
>> + sort(streams, num_streams, sizeof(u32), &cmp_u32, NULL);
>>
>> - return v4l2_subdev_link_validate_default(
>> - sink, link, &source_fmt, &sink_fmt);
>> + *out_num_streams = num_streams;
>> + *out_streams = streams;
>> + *allocated = true;
>> +
>> + return 0;
>> +}
>> +
>> +static int v4l2_subdev_link_validate_locked(struct media_link *link)
>> +{
>> + struct v4l2_subdev *sink_subdev =
>> + media_entity_to_v4l2_subdev(link->sink->entity);
>> + struct device *dev = sink_subdev->entity.graph_obj.mdev->dev;
>> + u32 num_source_streams;
>> + const u32 *source_streams;
>> + bool source_allocated;
>> + u32 num_sink_streams;
>> + const u32 *sink_streams;
>> + bool sink_allocated;
>> + unsigned int sink_idx;
>> + unsigned int source_idx;
>> + int ret;
>> +
>> + dev_dbg(dev, "validating link \"%s\":%u -> \"%s\":%u\n",
>> + link->source->entity->name, link->source->index,
>> + link->sink->entity->name, link->sink->index);
>> +
>> + ret = v4l2_link_validate_get_streams(link, true, &num_source_streams,
>> + &source_streams,
>> + &source_allocated);
>> + if (ret)
>> + return ret;
>> +
>> + ret = v4l2_link_validate_get_streams(link, false, &num_sink_streams,
>> + &sink_streams, &sink_allocated);
>> + if (ret)
>> + goto free_source;
>> +
>> + /*
>> + * It is ok to have more source streams than sink streams as extra
>> + * source streams can just be ignored (i.e. they go nowhere), but extra
>
> Nit on the "(i.e. they go nowhere)" bit:
>
> If it's ok to have more source streams than sinks, the extra sources
> will not be connected. So I would drop or change the above phrase as
> to me it sounds like sink streams go somewhere, while source stream
> are rather not connected (or not active).
Ok, I'll clarify.
>> + * sink streams is an error.
>> + */
>> + if (num_source_streams < num_sink_streams) {
>> + dev_err(dev,
>> + "Not enough source streams: %d < %d\n",
>> + num_source_streams, num_sink_streams);
>> + ret = -EINVAL;
>> + goto out;
>> + }
>> +
>> + /* Validate source and sink stream formats */
>> +
>> + source_idx = 0;
>> +
>> + for (sink_idx = 0; sink_idx < num_sink_streams; ++sink_idx) {
>> + struct v4l2_subdev_format sink_fmt, source_fmt;
>> + u32 stream;
>> +
>> + stream = sink_streams[sink_idx];
>> +
>> + for (; source_idx < num_source_streams; ++source_idx) {
>> + if (source_streams[source_idx] == stream)
>> + break;
>> + }
>> +
>> + if (source_idx == num_source_streams) {
>> + dev_err(dev, "No source stream for sink stream %u\n",
>> + stream);
>> + ret = -EINVAL;
>> + goto out;
>> + }
>> +
>> + dev_dbg(dev, "validating stream \"%s\":%u:%u -> \"%s\":%u:%u\n",
>> + link->source->entity->name, link->source->index, stream,
>> + link->sink->entity->name, link->sink->index, stream);
>> +
>> + ret = v4l2_subdev_link_validate_get_format(link->source, stream,
>> + &source_fmt);
>> + if (ret < 0) {
>> + dev_dbg(dev, "Failed to get format for \"%s\":%u:%u (but that's ok)\n",
>
> For my education: why is it ok not to have a format for the stream ?
I don't know, but afaics that's how it has always been. In upstream
v4l2_subdev_link_validate() returns 0 if
v4l2_subdev_link_validate_get_format() fails.
> Don't we risk to pass a non-initialized source/sink_fmt to
> link_validate() in that case ?
We don't call link_validate, we accept the stream right away.
>> + link->source->entity->name, link->source->index,
>> + stream);
>> + ret = 0;
>> + continue;
>> + }
>> +
>> + ret = v4l2_subdev_link_validate_get_format(link->sink, stream,
>> + &sink_fmt);
>> + if (ret < 0) {
>> + dev_dbg(dev, "Failed to get format for \"%s\":%u:%u (but that's ok)\n",
>> + link->sink->entity->name, link->sink->index,
>> + stream);
>> + ret = 0;
>> + continue;
>> + }
>> +
>> + /* TODO: add stream number to link_validate() */
>> + ret = v4l2_subdev_call(sink_subdev, pad, link_validate, link,
>> + &source_fmt, &sink_fmt);
>> + if (!ret)
>> + continue;
>> +
>> + if (ret != -ENOIOCTLCMD)
>> + goto out;
>> +
>> + ret = v4l2_subdev_link_validate_default(sink_subdev, link,
>> + &source_fmt, &sink_fmt);
>> +
>> + if (ret)
>> + goto out;
>> + }
>> +
>> +out:
>
> I would call it free_sinks:
Ok.
Tomi
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 26/36] media: subdev: add stream based configuration
2022-03-17 8:38 ` Jacopo Mondi
@ 2022-03-17 8:48 ` Tomi Valkeinen
0 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-17 8:48 UTC (permalink / raw)
To: Jacopo Mondi
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
On 17/03/2022 10:38, Jacopo Mondi wrote:
> Hi Tomi
>
> On Thu, Mar 17, 2022 at 10:01:47AM +0200, Tomi Valkeinen wrote:
>> On 16/03/2022 11:59, Jacopo Mondi wrote:
>>> Hi Tomi
>>>
>>> On Tue, Mar 01, 2022 at 06:11:46PM +0200, Tomi Valkeinen wrote:
>>>> Add support to manage configurations (format, crop, compose) per stream,
>>>> instead of per pad. This is accomplished with data structures that hold
>>>> an array of all subdev's stream configurations.
>>>>
>>>> The number of streams can vary at runtime based on routing. Every time
>>>> the routing is changed, the stream configurations need to be
>>>> re-initialized.
>>>>
>>>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>>>> ---
>>>> .../v4l/vidioc-subdev-enum-frame-interval.rst | 5 +-
>>>> .../v4l/vidioc-subdev-enum-frame-size.rst | 5 +-
>>>> .../v4l/vidioc-subdev-enum-mbus-code.rst | 5 +-
>>>> .../media/v4l/vidioc-subdev-g-crop.rst | 5 +-
>>>> .../media/v4l/vidioc-subdev-g-fmt.rst | 5 +-
>>>> .../v4l/vidioc-subdev-g-frame-interval.rst | 5 +-
>>>> .../media/v4l/vidioc-subdev-g-selection.rst | 5 +-
>>>> drivers/media/v4l2-core/v4l2-subdev.c | 129 ++++++++++++++++--
>>>> include/media/v4l2-subdev.h | 48 +++++++
>>>> include/uapi/linux/v4l2-subdev.h | 28 +++-
>>>> 10 files changed, 218 insertions(+), 22 deletions(-)
>>>>
>>>> diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-interval.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-interval.rst
>>>> index 3703943b412f..8def4c05d3da 100644
>>>> --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-interval.rst
>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-enum-frame-interval.rst
>>>> @@ -92,7 +92,10 @@ multiple pads of the same sub-device is not defined.
>>>> - Frame intervals to be enumerated, from enum
>>>> :ref:`v4l2_subdev_format_whence <v4l2-subdev-format-whence>`.
>>>> * - __u32
>>>> - - ``reserved``\ [8]
>>>> + - ``stream``
>>>> + - Stream identifier.
>>>> + * - __u32
>>>> + - ``reserved``\ [7]
>>>
>>> Does VIDIOC_SUBDEV_ENUM_FRAME_INTERVAL (as well as
>>> VIDIOC_SUBDEV_G_FRAME_INTERVAL) need to be stream-aware ?
>>>
>>> What is the semantic of the stream identifiers for IOCTLs that seem to
>>> control a paramter which is global to the subdev ? Isn't the stream semantic
>>> required to be specified in the IOCTL documentation and not just added
>>> to the list of fields ?
>>
>> Why would it be global to the subdev? struct v4l2_subdev_frame_interval_enum
>> already has 'pad' field, so it operates on that pad. With streams, each
>> stream in a pad may have different characteristics, (similarly to different
>> pads in non-stream case), so it feels logical to me to add the 'stream'
>> field.
>>
>
> I understand a device with multiple output interfaces can decide to
> clock out frames at different speeds due to some bus configuration
> parameters. But I have an hard time immagine a device that can clock
> out frames at different rates on the same bus.
I see your point, but doesn't the viability depend on the bus and
protocol? A sensor could send pixel frames 60 fps, but metadata only at
10 fps. Or a HD picture 10 fps, and a low res picture 60 fps. All it
needs is a bus that can interleave the streams seamlessly.
Tomi
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 28/36] media: subdev: add "opposite" stream helper funcs
2022-03-16 10:50 ` Jacopo Mondi
@ 2022-03-17 9:10 ` Tomi Valkeinen
0 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-17 9:10 UTC (permalink / raw)
To: Jacopo Mondi
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
On 16/03/2022 12:50, Jacopo Mondi wrote:
> Hi Tomi,
>
> On Tue, Mar 01, 2022 at 06:11:48PM +0200, Tomi Valkeinen wrote:
>> Add two helper functions to make dealing with streams easier:
>>
>> v4l2_subdev_routing_find_opposite_end - given a routing table and a pad
>> + stream, return the pad + stream on the opposite side of the subdev.
>>
>> v4l2_subdev_state_get_opposite_stream_format - return a pointer to the
>> format on the pad + stream on the opposite side from the given pad +
>> stream.
>>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> Reviewed-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
>> ---
>> drivers/media/v4l2-core/v4l2-subdev.c | 48 +++++++++++++++++++++++++++
>> include/media/v4l2-subdev.h | 36 ++++++++++++++++++++
>> 2 files changed, 84 insertions(+)
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
>> index 091b854e00d0..e65f802fe2aa 100644
>> --- a/drivers/media/v4l2-core/v4l2-subdev.c
>> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
>> @@ -1459,6 +1459,54 @@ v4l2_subdev_state_get_stream_format(struct v4l2_subdev_state *state,
>> }
>> EXPORT_SYMBOL_GPL(v4l2_subdev_state_get_stream_format);
>>
>> +int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
>> + u32 pad, u32 stream, u32 *other_pad,
>> + u32 *other_stream)
>> +{
>> + unsigned int i;
>> +
>> + for (i = 0; i < routing->num_routes; ++i) {
>> + struct v4l2_subdev_route *route = &routing->routes[i];
>> +
>
> Is it intentional to include routes that might be active ?
You mean not active? I don't see why this function should only consider
active routes. So yes, I guess you could call it intentional.
Tomi
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 32/36] media: subdev: add v4l2_subdev_routing_validate() helper
2022-03-16 11:10 ` Jacopo Mondi
@ 2022-03-17 9:15 ` Tomi Valkeinen
0 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-17 9:15 UTC (permalink / raw)
To: Jacopo Mondi
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
On 16/03/2022 13:10, Jacopo Mondi wrote:
> Hi Tomi
>
> On Tue, Mar 01, 2022 at 06:11:52PM +0200, Tomi Valkeinen wrote:
>> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>
>> Add a v4l2_subdev_routing_validate() helper for verifying routing for
>> common cases like only allowing non-overlapping 1-to-1 streams.
>>
>> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> ---
>> drivers/media/v4l2-core/v4l2-subdev.c | 102 ++++++++++++++++++++++++++
>> include/media/v4l2-subdev.h | 39 ++++++++++
>> 2 files changed, 141 insertions(+)
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
>> index 1ceee8313246..a19236cf11b8 100644
>> --- a/drivers/media/v4l2-core/v4l2-subdev.c
>> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
>> @@ -1551,6 +1551,108 @@ int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
>> }
>> EXPORT_SYMBOL_GPL(v4l2_subdev_get_fmt);
>>
>> +int v4l2_subdev_routing_validate(struct v4l2_subdev *sd,
>> + const struct v4l2_subdev_krouting *routing,
>> + enum v4l2_subdev_routing_restriction disallow)
>> +{
>> + u32 *remote_pads = NULL;
>> + unsigned int i, j;
>> + int ret = -EINVAL;
>> +
>> + if (disallow & V4L2_SUBDEV_ROUTING_NO_STREAM_MIX) {
>> + remote_pads = kcalloc(sd->entity.num_pads, sizeof(*remote_pads),
>> + GFP_KERNEL);
>> + if (!remote_pads)
>> + return -ENOMEM;
>> +
>> + for (i = 0; i < sd->entity.num_pads; ++i)
>> + remote_pads[i] = U32_MAX;
>> + }
>> +
>> + for (i = 0; i < routing->num_routes; ++i) {
>> + const struct v4l2_subdev_route *route = &routing->routes[i];
>
> I guess validating non-active routes is intentional.
I think it's better to ensure non-active routes are valid too.
Tomi
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 30/36] media: subdev: Fallback to pad config in v4l2_subdev_get_fmt()
2022-03-16 11:03 ` Jacopo Mondi
@ 2022-03-17 9:18 ` Tomi Valkeinen
0 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-17 9:18 UTC (permalink / raw)
To: Jacopo Mondi
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
On 16/03/2022 13:03, Jacopo Mondi wrote:
> On Tue, Mar 01, 2022 at 06:11:50PM +0200, Tomi Valkeinen wrote:
>> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>
>> If the subdev doesn't implement routing support, fallback to pad config
>> as the storage for pad formats. This allows using the V4L2 subdev active
>> state API and the v4l2_subdev_get_fmt() helper in subdev drivers that
>> don't implement routing support.
>>
>> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>> ---
>> drivers/media/v4l2-core/v4l2-subdev.c | 10 ++++++++--
>> 1 file changed, 8 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
>> index c1cc9b91dba7..7f50871054cd 100644
>> --- a/drivers/media/v4l2-core/v4l2-subdev.c
>> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
>> @@ -1512,8 +1512,14 @@ int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
>> {
>> struct v4l2_mbus_framefmt *fmt;
>>
>> - fmt = v4l2_subdev_state_get_stream_format(state, format->pad,
>> - format->stream);
>> + if (sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED)
>> + fmt = v4l2_subdev_state_get_stream_format(state, format->pad,
>> + format->stream);
>> + else if (format->pad < sd->entity.num_pads && format->stream == 0)
>> + fmt = v4l2_subdev_get_try_format(sd, state, format->pad);
>> + else
>> + fmt = NULL;
>
> You could initialize fmt = NULL and skip the else
I do like it better this way, I think it's more understandable to have
all the three cases listed there.
Tomi
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 34/36] media: v4l2-subdev: Add v4l2_subdev_state_xlate_streams() helper
2022-03-16 11:26 ` Jacopo Mondi
@ 2022-03-17 9:27 ` Tomi Valkeinen
2022-03-17 9:35 ` Tomi Valkeinen
0 siblings, 1 reply; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-17 9:27 UTC (permalink / raw)
To: Jacopo Mondi
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
On 16/03/2022 13:26, Jacopo Mondi wrote:
> Hi Tomi,
>
> On Tue, Mar 01, 2022 at 06:11:54PM +0200, Tomi Valkeinen wrote:
>> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>
>> Add a helper function to translate streams between two pads of a subdev,
>> using the subdev's internal routing table.
>>
>> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> ---
>> drivers/media/v4l2-core/v4l2-subdev.c | 25 +++++++++++++++++++++++++
>> include/media/v4l2-subdev.h | 23 +++++++++++++++++++++++
>> 2 files changed, 48 insertions(+)
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
>> index e30338a53088..6a9fc62dacbf 100644
>> --- a/drivers/media/v4l2-core/v4l2-subdev.c
>> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
>> @@ -1529,6 +1529,31 @@ v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
>> }
>> EXPORT_SYMBOL_GPL(v4l2_subdev_state_get_opposite_stream_format);
>>
>> +u64 v4l2_subdev_state_xlate_streams(const struct v4l2_subdev_state *state,
>> + u32 pad0, u32 pad1, u64 *streams)
>> +{
>> + const struct v4l2_subdev_krouting *routing = &state->routing;
>> + struct v4l2_subdev_route *route;
>> + u64 streams0 = 0;
>> + u64 streams1 = 0;
>> +
>> + for_each_active_route(routing, route) {
>> + if (route->sink_pad == pad0 && route->source_pad == pad1 &&
>> + (*streams & BIT(route->sink_stream))) {
>> + streams0 |= BIT(route->sink_stream);
>> + streams1 |= BIT(route->source_stream);
>> + }
>> + if (route->source_pad == pad0 && route->sink_pad == pad1 &&
>> + (*streams & BIT(route->source_stream))) {
>> + streams0 |= BIT(route->source_stream);
>> + streams1 |= BIT(route->sink_stream);
>> + }
>> + }
>> +
>> + *streams = streams0;
>> + return streams1;
>
> I'll probably learn how this is used later, but I found it hard to
> immagine how the returned mask should be used.
>
> To a sink stream mask that contains multiple streams a source maks
> with multiple entries will be associated
>
> In example, the sink stream mask might look lik
>
> sink_stream_mask = {1, 0, 1, 0}
>
> So we are interested in translating stream 0 and 2
> Assume 0->4 and 2->1 in the routing tabe, the resulting source stream
> mask will be
>
> source_stream_mask = {1, 0, 0, 1}
>
> How should the caller know what stream goes where ?
The use case for the function is for cases where all the streams go from
one pad to another. Probably the main use is in subdevs with a single
source and sink pad. This makes it easy to implement enable_streams()
op: you just basically call v4l2_subdev_state_xlate_streams() to get the
streams on the other side, and call v4l2_subdev_enable_streams(). Here's
a version from ub913:
> static int ub913_enable_streams(struct v4l2_subdev *sd,
> struct v4l2_subdev_state *state, u32 pad,
> u64 streams_mask)
> {
> struct ub913_data *priv = sd_to_ub913(sd);
> struct media_pad *remote_pad;
> u64 sink_streams;
> int ret;
>
> if (streams_mask & priv->enabled_source_streams)
> return -EALREADY;
>
> sink_streams = v4l2_subdev_state_xlate_streams(
> state, UB913_PAD_SOURCE, UB913_PAD_SINK, &streams_mask);
>
> remote_pad = media_entity_remote_pad(&priv->pads[UB913_PAD_SINK]);
>
> ret = v4l2_subdev_enable_streams(priv->source_sd, remote_pad->index,
> sink_streams);
> if (ret)
> return ret;
>
> priv->enabled_source_streams |= streams_mask;
>
> return 0;
> }
>> +}
>> +
>> int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
>> struct v4l2_subdev_format *format)
>> {
>> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
>> index 1eced0f47296..992debe116ac 100644
>> --- a/include/media/v4l2-subdev.h
>> +++ b/include/media/v4l2-subdev.h
>> @@ -1497,6 +1497,29 @@ struct v4l2_mbus_framefmt *
>> v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
>> u32 pad, u32 stream);
>>
>> +/**
>> + * v4l2_subdev_state_xlate_streams() - Translate streams from one pad to another
>
> Could we stress out that pads are on the same entity and this is not
> meant to translate over media links ? Is this necessary ?
The streams on both sides of a link are identical, so there can be no
translation in that case.
The doc below explains the use, I'd rather not try to squeeze in a
longer explanation in the title.
But I think we should emphasize that this only makes sense on a simple
routing case.
>> + *
>> + * @state: Subdevice state
>> + * @pad0: The first pad
>> + * @pad1: The second pad
>> + * @streams: Streams bitmask on the first pad
>> + *
>> + * Streams on sink pads of a subdev are routed to source pads as expressed in
>> + * the subdev state routing table. Stream numbers don't necessarily match on
>> + * the sink and source side of a route. This function translates stream numbers
>> + * on @pad0, expressed as a bitmask in @streams, to the corresponding streams
>> + * on @pad1 using the routing table from the @state. It returns the stream mask
>> + * on @pad1, and updates @streams with the streams that have been found in the
>> + * routing table.
>> + *
>> + * @pad0 and @pad1 must be a sink and a source, in any order.
>> + *
>> + * Return: The bitmask of streams of @pad1 that are routed to @streams on @pad0.
>> + */
>> +u64 v4l2_subdev_state_xlate_streams(const struct v4l2_subdev_state *state,
>> + u32 pad0, u32 pad1, u64 *streams);
>> +
>> /**
>> * v4l2_subdev_get_fmt() - Fill format based on state
>> * @sd: subdevice
>> --
>> 2.25.1
>>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 34/36] media: v4l2-subdev: Add v4l2_subdev_state_xlate_streams() helper
2022-03-17 9:27 ` Tomi Valkeinen
@ 2022-03-17 9:35 ` Tomi Valkeinen
0 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-17 9:35 UTC (permalink / raw)
To: Jacopo Mondi
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
On 17/03/2022 11:27, Tomi Valkeinen wrote:
> On 16/03/2022 13:26, Jacopo Mondi wrote:
>> Hi Tomi,
>>
>> On Tue, Mar 01, 2022 at 06:11:54PM +0200, Tomi Valkeinen wrote:
>>> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>>
>>> Add a helper function to translate streams between two pads of a subdev,
>>> using the subdev's internal routing table.
>>>
>>> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>>> ---
>>> drivers/media/v4l2-core/v4l2-subdev.c | 25 +++++++++++++++++++++++++
>>> include/media/v4l2-subdev.h | 23 +++++++++++++++++++++++
>>> 2 files changed, 48 insertions(+)
>>>
>>> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c
>>> b/drivers/media/v4l2-core/v4l2-subdev.c
>>> index e30338a53088..6a9fc62dacbf 100644
>>> --- a/drivers/media/v4l2-core/v4l2-subdev.c
>>> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
>>> @@ -1529,6 +1529,31 @@
>>> v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state
>>> *state,
>>> }
>>> EXPORT_SYMBOL_GPL(v4l2_subdev_state_get_opposite_stream_format);
>>>
>>> +u64 v4l2_subdev_state_xlate_streams(const struct v4l2_subdev_state
>>> *state,
>>> + u32 pad0, u32 pad1, u64 *streams)
>>> +{
>>> + const struct v4l2_subdev_krouting *routing = &state->routing;
>>> + struct v4l2_subdev_route *route;
>>> + u64 streams0 = 0;
>>> + u64 streams1 = 0;
>>> +
>>> + for_each_active_route(routing, route) {
>>> + if (route->sink_pad == pad0 && route->source_pad == pad1 &&
>>> + (*streams & BIT(route->sink_stream))) {
>>> + streams0 |= BIT(route->sink_stream);
>>> + streams1 |= BIT(route->source_stream);
>>> + }
>>> + if (route->source_pad == pad0 && route->sink_pad == pad1 &&
>>> + (*streams & BIT(route->source_stream))) {
>>> + streams0 |= BIT(route->source_stream);
>>> + streams1 |= BIT(route->sink_stream);
>>> + }
>>> + }
>>> +
>>> + *streams = streams0;
>>> + return streams1;
>>
>> I'll probably learn how this is used later, but I found it hard to
>> immagine how the returned mask should be used.
>>
>> To a sink stream mask that contains multiple streams a source maks
>> with multiple entries will be associated
>>
>> In example, the sink stream mask might look lik
>>
>> sink_stream_mask = {1, 0, 1, 0}
>>
>> So we are interested in translating stream 0 and 2
>> Assume 0->4 and 2->1 in the routing tabe, the resulting source stream
>> mask will be
>>
>> source_stream_mask = {1, 0, 0, 1}
>>
>> How should the caller know what stream goes where ?
>
> The use case for the function is for cases where all the streams go from
> one pad to another. Probably the main use is in subdevs with a single
> source and sink pad. This makes it easy to implement enable_streams()
> op: you just basically call v4l2_subdev_state_xlate_streams() to get the
> streams on the other side, and call v4l2_subdev_enable_streams(). Here's
> a version from ub913:
>
>> static int ub913_enable_streams(struct v4l2_subdev *sd,
>> struct v4l2_subdev_state *state, u32 pad,
>> u64 streams_mask)
>> {
>> struct ub913_data *priv = sd_to_ub913(sd);
>> struct media_pad *remote_pad;
>> u64 sink_streams;
>> int ret;
>>
>> if (streams_mask & priv->enabled_source_streams)
>> return -EALREADY;
>>
>> sink_streams = v4l2_subdev_state_xlate_streams(
>> state, UB913_PAD_SOURCE, UB913_PAD_SINK, &streams_mask);
>>
>> remote_pad = media_entity_remote_pad(&priv->pads[UB913_PAD_SINK]);
>>
>> ret = v4l2_subdev_enable_streams(priv->source_sd, remote_pad->index,
>> sink_streams);
>> if (ret)
>> return ret;
>>
>> priv->enabled_source_streams |= streams_mask;
>>
>> return 0;
>> }
>
>
>
>>> +}
>>> +
>>> int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct
>>> v4l2_subdev_state *state,
>>> struct v4l2_subdev_format *format)
>>> {
>>> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
>>> index 1eced0f47296..992debe116ac 100644
>>> --- a/include/media/v4l2-subdev.h
>>> +++ b/include/media/v4l2-subdev.h
>>> @@ -1497,6 +1497,29 @@ struct v4l2_mbus_framefmt *
>>> v4l2_subdev_state_get_opposite_stream_format(struct
>>> v4l2_subdev_state *state,
>>> u32 pad, u32 stream);
>>>
>>> +/**
>>> + * v4l2_subdev_state_xlate_streams() - Translate streams from one
>>> pad to another
>>
>> Could we stress out that pads are on the same entity and this is not
>> meant to translate over media links ? Is this necessary ?
>
> The streams on both sides of a link are identical, so there can be no
> translation in that case.
>
> The doc below explains the use, I'd rather not try to squeeze in a
> longer explanation in the title.
>
> But I think we should emphasize that this only makes sense on a simple
> routing case.
Ah, never mind. The function can be used in multi-pad cases too, so we
don't need to emphasize the above.
The function tells how a set of streams from pad A translate to a set of
streams on pad B. So you only know the set of streams, not how a single
stream goes. For that you need to look at the routing table.
Tomi
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 35/36] media: v4l2-subdev: Add subdev .(enable|disable)_streams() operations
2022-03-16 12:05 ` Jacopo Mondi
@ 2022-03-17 9:43 ` Tomi Valkeinen
0 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-17 9:43 UTC (permalink / raw)
To: Jacopo Mondi
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
On 16/03/2022 14:05, Jacopo Mondi wrote:
> Hi Tomi,
>
> On Tue, Mar 01, 2022 at 06:11:55PM +0200, Tomi Valkeinen wrote:
>> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>
>> Add two new subdev pad operations, .enable_streams() and
>> .disable_streams(), to allow control of individual streams per pad. This
>> is a superset of what the video .s_stream() operation implements.
>>
>> To help with handling of backward compatibility, add two wrapper
>> functions around those operations, and require their usage in drivers.
>>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>> ---
>> drivers/media/v4l2-core/v4l2-subdev.c | 216 ++++++++++++++++++++++++++
>> include/media/v4l2-subdev.h | 85 ++++++++++
>> 2 files changed, 301 insertions(+)
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
>> index 6a9fc62dacbf..f75a1995a70b 100644
>> --- a/drivers/media/v4l2-core/v4l2-subdev.c
>> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
>> @@ -1698,6 +1698,222 @@ __v4l2_subdev_next_active_route(const struct v4l2_subdev_krouting *routing,
>> }
>> EXPORT_SYMBOL_GPL(__v4l2_subdev_next_active_route);
>>
>> +static int v4l2_subdev_enable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
>> + u64 streams_mask)
>> +{
>> + unsigned int i;
>> + int ret;
>> +
>> + /*
>> + * The subdev doesn't implement pad-based stream enable, fall back
>> + * on the .s_stream() operation. This can only be done for subdevs that
>> + * have a single source pad, as sd->enabled_streams is global to the
>> + * subdev.
>> + */
>> + if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
>> + return -EOPNOTSUPP;
>> +
>> + for (i = 0; i < sd->entity.num_pads; ++i) {
>> + if (i != pad && sd->entity.pads[i].flags & MEDIA_PAD_FL_SOURCE)
>> + return -EOPNOTSUPP;
>> + }
>> +
>> + if (sd->enabled_streams & streams_mask)
>> + return -EALREADY;
>
> I wonder if a few dev_dbg on errors might save someone an headache
Yep, I'll add a few.
>> +
>> + /* Start streaming when the first streams are enabled. */
>> + if (!sd->enabled_streams) {
>> + ret = v4l2_subdev_call(sd, video, s_stream, 1);
>> + if (ret)
>> + return ret;
>> + }
>> +
>> + sd->enabled_streams |= streams_mask;
>> +
>> + return 0;
>> +}
>> +
>> +int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>> + u64 streams_mask)
>> +{
>> + struct device *dev = sd->entity.graph_obj.mdev->dev;
>> + struct v4l2_subdev_state *state;
>> + u64 found_streams = 0;
>> + unsigned int i;
>> + int ret;
>> +
>> + /* A few basic sanity checks first. */
>> + if (pad >= sd->entity.num_pads)
>> + return -EINVAL;
>
> Should we make sure pad is a SOURCE (and remove the same check in the
> _fallback version) ?
I tried that, but according to Laurent: no. Enable streams may work on
both directions, i.e. for output drivers it is called on a sink pad.
>> +
>> + if (!streams_mask)
>> + return 0;
>> +
>> + /* Fallback on .s_stream() if .enable_streams() isn't available. */
>> + if (!sd->ops->pad || !sd->ops->pad->enable_streams)
>> + return v4l2_subdev_enable_streams_fallback(sd, pad,
>> + streams_mask);
>> +
>> + state = v4l2_subdev_lock_and_get_active_state(sd);
>> +
>> + /*
>> + * Verify that the requested streams exist and that they are not
>> + * already enabled.
>> + */
>> + for (i = 0; i < state->stream_configs.num_configs; ++i) {
>> + struct v4l2_subdev_stream_config *cfg =
>> + &state->stream_configs.configs[i];
>> +
>> + if (cfg->pad != pad || !(streams_mask & BIT(cfg->stream)))
>> + continue;
>> +
>> + found_streams |= BIT(cfg->stream);
>> +
>> + if (cfg->enabled) {
>> + dev_dbg(dev, "stream %u already enabled on %s/%u\n",
>> + cfg->stream, sd->entity.name, pad);
>> + ret = -EALREADY;
>> + goto done;
>> + }
>> + }
>> +
>> + if (found_streams != streams_mask) {
>> + dev_dbg(dev, "streams 0x%llx not found on %s/%u\n",
>
> nit: I would use the more usual form of entity:pad in the error
> message
Ok.
> I like the idea :)
>
> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
>
> Thanks
> j
>
>> + streams_mask & ~found_streams, sd->entity.name, pad);
>> + ret = -EINVAL;
>> + goto done;
>> + }
>> +
>> + /* Call the .enable_streams() operation. */
>> + ret = v4l2_subdev_call(sd, pad, enable_streams, state, pad,
>> + streams_mask);
>> + if (ret)
>> + goto done;
>> +
>> + /* Mark the streams as enabled. */
>> + for (i = 0; i < state->stream_configs.num_configs; ++i) {
>> + struct v4l2_subdev_stream_config *cfg =
>> + &state->stream_configs.configs[i];
>> +
>> + if (cfg->pad == pad && (streams_mask & BIT(cfg->stream)))
>> + cfg->enabled = true;
>> + }
>> +
>> +done:
>> + v4l2_subdev_unlock_state(state);
>> +
>> + return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(v4l2_subdev_enable_streams);
>> +
>> +static int v4l2_subdev_disable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
>> + u64 streams_mask)
>> +{
>> + unsigned int i;
>> + int ret;
>> +
>> + /*
>> + * If the subdev doesn't implement pad-based stream enable, fall back
>> + * on the .s_stream() operation. This can only be done for subdevs that
>> + * have a single source pad, as sd->enabled_streams is global to the
>> + * subdev.
>> + */
>> + if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
>> + return -EOPNOTSUPP;
>> +
>> + for (i = 0; i < sd->entity.num_pads; ++i) {
>> + if (i != pad && sd->entity.pads[i].flags & MEDIA_PAD_FL_SOURCE)
>> + return -EOPNOTSUPP;
>> + }
>> +
>> + if ((sd->enabled_streams & streams_mask) != streams_mask)
>> + return -EALREADY;
>> +
>> + /* Stop streaming when the last streams are disabled. */
>> + if (!(sd->enabled_streams & ~streams_mask)) {
>> + ret = v4l2_subdev_call(sd, video, s_stream, 0);
>> + if (ret)
>> + return ret;
>> + }
>> +
>> + sd->enabled_streams &= ~streams_mask;
>> +
>> + return 0;
>> +}
>> +
>> +int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>> + u64 streams_mask)
>> +{
>> + struct device *dev = sd->entity.graph_obj.mdev->dev;
>> + struct v4l2_subdev_state *state;
>> + u64 found_streams = 0;
>> + unsigned int i;
>> + int ret;
>> +
>> + /* A few basic sanity checks first. */
>> + if (pad >= sd->entity.num_pads)
>> + return -EINVAL;
>> +
>> + if (!streams_mask)
>> + return 0;
>> +
>> + /* Fallback on .s_stream() if .disable_streams() isn't available. */
>> + if (!sd->ops->pad || !sd->ops->pad->disable_streams)
>> + return v4l2_subdev_disable_streams_fallback(sd, pad,
>> + streams_mask);
>> +
>> + state = v4l2_subdev_lock_and_get_active_state(sd);
>> +
>> + /*
>> + * Verify that the requested streams exist and that they are not
>> + * already disabled.
>> + */
>> + for (i = 0; i < state->stream_configs.num_configs; ++i) {
>> + struct v4l2_subdev_stream_config *cfg =
>> + &state->stream_configs.configs[i];
>> +
>> + if (cfg->pad != pad || !(streams_mask & BIT(cfg->stream)))
>> + continue;
>> +
>> + found_streams |= BIT(cfg->stream);
>> +
>> + if (!cfg->enabled) {
>> + dev_dbg(dev, "stream %u already disabled on %s/%u\n",
>> + cfg->stream, sd->entity.name, pad);
>> + ret = -EALREADY;
>> + goto done;
>> + }
>> + }
>> +
>> + if (found_streams != streams_mask) {
>> + dev_dbg(dev, "streams 0x%llx not found on %s/%u\n",
>> + streams_mask & ~found_streams, sd->entity.name, pad);
>> + ret = -EINVAL;
>> + goto done;
>> + }
>> +
>> + /* Call the .disable_streams() operation. */
>> + ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
>> + streams_mask);
>> + if (ret)
>> + goto done;
>> +
>> + /* Mark the streams as disabled. */
>> + for (i = 0; i < state->stream_configs.num_configs; ++i) {
>> + struct v4l2_subdev_stream_config *cfg =
>> + &state->stream_configs.configs[i];
>> +
>> + if (cfg->pad == pad && (streams_mask & BIT(cfg->stream)))
>> + cfg->enabled = false;
>> + }
>> +
>> +done:
>> + v4l2_subdev_unlock_state(state);
>> +
>> + return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(v4l2_subdev_disable_streams);
>> +
>> #endif /* CONFIG_MEDIA_CONTROLLER */
>>
>> void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
>> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
>> index 992debe116ac..bb1713863973 100644
>> --- a/include/media/v4l2-subdev.h
>> +++ b/include/media/v4l2-subdev.h
>> @@ -702,6 +702,7 @@ struct v4l2_subdev_pad_config {
>> *
>> * @pad: pad number
>> * @stream: stream number
>> + * @enabled: has the stream been enabled with v4l2_subdev_enable_stream()
>> * @fmt: &struct v4l2_mbus_framefmt
>> * @crop: &struct v4l2_rect to be used for crop
>> * @compose: &struct v4l2_rect to be used for compose
>> @@ -711,6 +712,7 @@ struct v4l2_subdev_pad_config {
>> struct v4l2_subdev_stream_config {
>> u32 pad;
>> u32 stream;
>> + bool enabled;
>>
>> struct v4l2_mbus_framefmt fmt;
>> struct v4l2_rect crop;
>> @@ -816,6 +818,18 @@ struct v4l2_subdev_state {
>> *
>> * @set_routing: enable or disable data connection routes described in the
>> * subdevice routing table.
>> + *
>> + * @enable_streams: Enable the streams defined in streams_mask on the given
>> + * source pad. Subdevs that implement this operation must use the active
>> + * state management provided by the subdev core (enabled through a call to
>> + * v4l2_subdev_init_finalize() at initialization time). Do not call
>> + * directly, use v4l2_subdev_enable_streams() instead.
>> + *
>> + * @disable_streams: Disable the streams defined in streams_mask on the given
>> + * source pad. Subdevs that implement this operation must use the active
>> + * state management provided by the subdev core (enabled through a call to
>> + * v4l2_subdev_init_finalize() at initialization time). Do not call
>> + * directly, use v4l2_subdev_disable_streams() instead.
>> */
>> struct v4l2_subdev_pad_ops {
>> int (*init_cfg)(struct v4l2_subdev *sd,
>> @@ -862,6 +876,12 @@ struct v4l2_subdev_pad_ops {
>> struct v4l2_subdev_state *state,
>> enum v4l2_subdev_format_whence which,
>> struct v4l2_subdev_krouting *route);
>> + int (*enable_streams)(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state, u32 pad,
>> + u64 streams_mask);
>> + int (*disable_streams)(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state, u32 pad,
>> + u64 streams_mask);
>> };
>>
>> /**
>> @@ -1007,6 +1027,10 @@ struct v4l2_subdev_platform_data {
>> * @active_state: Active state for the subdev (NULL for subdevs tracking the
>> * state internally). Initialized by calling
>> * v4l2_subdev_init_finalize().
>> + * @enabled_streams: Bitmask of enabled streams used by
>> + * v4l2_subdev_enable_streams() and
>> + * v4l2_subdev_disable_streams() helper functions for fallback
>> + * cases.
>> *
>> * Each instance of a subdev driver should create this struct, either
>> * stand-alone or embedded in a larger struct.
>> @@ -1052,6 +1076,7 @@ struct v4l2_subdev {
>> * doesn't support it.
>> */
>> struct v4l2_subdev_state *active_state;
>> + u64 enabled_streams;
>> };
>>
>>
>> @@ -1589,6 +1614,66 @@ __v4l2_subdev_next_active_route(const struct v4l2_subdev_krouting *routing,
>> for ((route) = NULL; \
>> ((route) = __v4l2_subdev_next_active_route((routing), (route)));)
>>
>> +/**
>> + * v4l2_subdev_enable_streams() - Enable streams on a pad
>> + * @sd: The subdevice
>> + * @pad: The pad
>> + * @streams_mask: Bitmask of streams to enable
>> + *
>> + * This function enables streams on a source @pad of a subdevice. The pad is
>> + * identified by its index, while the streams are identified by the
>> + * @streams_mask bitmask. This allows enabling multiple streams on a pad at
>> + * once.
>> + *
>> + * Enabling a stream that is already enabled isn't allowed. If @streams_mask
>> + * contains an already enabled stream, this function returns -EALREADY without
>> + * performing any operation.
>> + *
>> + * Per-stream enable is only available for subdevs that implement the
>> + * .enable_streams() and .disable_streams() operations. For other subdevs, this
>> + * function implements a best-effort compatibility by calling the .s_stream()
>> + * operation, limited to subdevs that have a single source pad.
>> + *
>> + * Return:
>> + * * 0: Success
>> + * * -EALREADY: One of the streams in streams_mask is already enabled
>> + * * -EINVAL: The pad index is invalid, or doesn't correspond to a source pad
>> + * * -EOPNOTSUPP: Falling back to the legacy .s_stream() operation is
>> + * impossible because the subdev has multiple source pads
>> + */
>> +int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>> + u64 streams_mask);
>> +
>> +/**
>> + * v4l2_subdev_disable_streams() - Disable streams on a pad
>> + * @sd: The subdevice
>> + * @pad: The pad
>> + * @streams_mask: Bitmask of streams to disable
>> + *
>> + * This function disables streams on a source @pad of a subdevice. The pad is
>> + * identified by its index, while the streams are identified by the
>> + * @streams_mask bitmask. This allows disabling multiple streams on a pad at
>> + * once.
>> + *
>> + * Disabling a streams that is not enabled isn't allowed. If @streams_mask
>> + * contains a disabled stream, this function returns -EALREADY without
>> + * performing any operation.
>> + *
>> + * Per-stream disable is only available for subdevs that implement the
>> + * .enable_streams() and .disable_streams() operations. For other subdevs, this
>> + * function implements a best-effort compatibility by calling the .s_stream()
>> + * operation, limited to subdevs that have a single source pad.
>> + *
>> + * Return:
>> + * * 0: Success
>> + * * -EALREADY: One of the streams in streams_mask is not enabled
>> + * * -EINVAL: The pad index is invalid, or doesn't correspond to a source pad
>> + * * -EOPNOTSUPP: Falling back to the legacy .s_stream() operation is
>> + * impossible because the subdev has multiple source pads
>> + */
>> +int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>> + u64 streams_mask);
>> +
>> #endif /* CONFIG_MEDIA_CONTROLLER */
>>
>> /**
>> --
>> 2.25.1
>>
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 36/36] media: v4l2-subdev: Add v4l2_subdev_s_stream_helper() function
2022-03-16 12:10 ` Jacopo Mondi
@ 2022-03-17 9:45 ` Tomi Valkeinen
0 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-03-17 9:45 UTC (permalink / raw)
To: Jacopo Mondi
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
On 16/03/2022 14:10, Jacopo Mondi wrote:
> Hi Laurent,
>
> On Tue, Mar 01, 2022 at 06:11:56PM +0200, Tomi Valkeinen wrote:
>> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>
>> The v4l2_subdev_s_stream_helper() helper can be used by subdevs that
>> implement the stream-aware .enable_streams() and .disable_streams()
>> operations to implement .s_stream(). This is limited to subdevs that
>> have a single source pad.
>>
>> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> ---
>> drivers/media/v4l2-core/v4l2-subdev.c | 40 +++++++++++++++++++++++++++
>> include/media/v4l2-subdev.h | 17 ++++++++++++
>> 2 files changed, 57 insertions(+)
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
>> index f75a1995a70b..270445821f06 100644
>> --- a/drivers/media/v4l2-core/v4l2-subdev.c
>> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
>> @@ -1914,6 +1914,46 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>> }
>> EXPORT_SYMBOL_GPL(v4l2_subdev_disable_streams);
>>
>> +int v4l2_subdev_s_stream_helper(struct v4l2_subdev *sd, int enable)
>> +{
>> + struct v4l2_subdev_state *state;
>> + struct v4l2_subdev_route *route;
>> + struct media_pad *pad;
>> + u64 source_mask = 0;
>> + int pad_index = -1;
>> +
>> + /*
>> + * Find the source pad. This helper is meant for subdevs that have a
>> + * single source pad, so failures shouldn't happen, but catch them
>> + * loudly nonetheless as they indicate a driver bug.
>> + */
>> + media_entity_for_each_pad(&sd->entity, pad) {
>> + if (pad->flags & MEDIA_PAD_FL_SOURCE) {
>> + pad_index = pad->index;
>> + break;
>> + }
>> + }
>> +
>> + if (WARN_ON(pad_index == -1))
>> + return -EINVAL;
>> +
>> + /*
>> + * As there's a single source pad, just collect all the source streams.
>> + */
>> + state = v4l2_subdev_lock_and_get_active_state(sd);
>> +
>> + for_each_active_route(&state->routing, route)
>> + source_mask |= BIT(route->source_stream);
>> +
>> + v4l2_subdev_unlock_state(state);
>> +
>> + if (enable)
>> + return v4l2_subdev_enable_streams(sd, pad_index, source_mask);
>> + else
>> + return v4l2_subdev_disable_streams(sd, pad_index, source_mask);
>
> Or
> return enable ? ...
> : ... ;
I'm not sure if that's any better...
>> +}
>> +EXPORT_SYMBOL_GPL(v4l2_subdev_s_stream_helper);
>> +
>> #endif /* CONFIG_MEDIA_CONTROLLER */
>>
>> void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
>> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
>> index bb1713863973..817452ec30bb 100644
>> --- a/include/media/v4l2-subdev.h
>> +++ b/include/media/v4l2-subdev.h
>> @@ -1674,6 +1674,23 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>> int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>> u64 streams_mask);
>>
>> +/**
>> + * v4l2_subdev_s_stream_helper() - Helper to implement the subdev s_stream
>> + * operation using enable_streams and disable_streams
>> + * @sd: The subdevice
>> + * @enable: Enable to disable streaming
>
> Enable or disable
Thanks.
Tomi
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 22/36] media: subdev: Add [GS]_ROUTING subdev ioctls and operations
2022-03-01 16:11 ` [PATCH v11 22/36] media: subdev: Add [GS]_ROUTING subdev ioctls and operations Tomi Valkeinen
@ 2022-07-07 9:15 ` Tomasz Figa
2022-07-14 6:47 ` Tomi Valkeinen
0 siblings, 1 reply; 67+ messages in thread
From: Tomasz Figa @ 2022-07-07 9:15 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav, Michal Simek
Hi Tomi, Laurent,
On Tue, Mar 01, 2022 at 06:11:42PM +0200, Tomi Valkeinen wrote:
> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>
> Add support for subdev internal routing. A route is defined as a single
> stream from a sink pad to a source pad.
>
> The userspace can configure the routing via two new ioctls,
> VIDIOC_SUBDEV_G_ROUTING and VIDIOC_SUBDEV_S_ROUTING, and subdevs can
> implement the functionality with v4l2_subdev_pad_ops.set_routing().
Thanks for the patch! Please check my comment inline.
>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Michal Simek <michal.simek@xilinx.com>
>
> - Add sink and source streams for multiplexed links
> - Copy the argument back in case of an error. This is needed to let the
> caller know the number of routes.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
>
> - Expand and refine documentation.
> - Make the 'routes' pointer a __u64 __user pointer so that a compat32
> version of the ioctl is not required.
> - Add struct v4l2_subdev_krouting to be used for subdevice operations.
>
> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
>
> - Fix typecasing warnings
> - Check sink & source pad types
> - Add 'which' field
> - Add V4L2_SUBDEV_ROUTE_FL_SOURCE
> - Routing to subdev state
> - Dropped get_routing subdev op
>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
> drivers/media/v4l2-core/v4l2-ioctl.c | 25 ++++++++-
> drivers/media/v4l2-core/v4l2-subdev.c | 73 +++++++++++++++++++++++++++
> include/media/v4l2-subdev.h | 22 ++++++++
> include/uapi/linux/v4l2-subdev.h | 57 +++++++++++++++++++++
> 4 files changed, 176 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 642cb90f457c..add3b28d446e 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -16,6 +16,7 @@
> #include <linux/kernel.h>
> #include <linux/version.h>
>
> +#include <linux/v4l2-subdev.h>
> #include <linux/videodev2.h>
>
> #include <media/v4l2-common.h>
> @@ -3093,6 +3094,21 @@ static int check_array_args(unsigned int cmd, void *parg, size_t *array_size,
> ret = 1;
> break;
> }
> +
> + case VIDIOC_SUBDEV_G_ROUTING:
> + case VIDIOC_SUBDEV_S_ROUTING: {
> + struct v4l2_subdev_routing *routing = parg;
> +
> + if (routing->num_routes > 256)
Should we define and document this constant?
> + return -EINVAL;
> +
> + *user_ptr = u64_to_user_ptr(routing->routes);
> + *kernel_ptr = (void **)&routing->routes;
> + *array_size = sizeof(struct v4l2_subdev_route)
> + * routing->num_routes;
> + ret = 1;
> + break;
> + }
> }
>
> return ret;
> @@ -3356,8 +3372,15 @@ video_usercopy(struct file *file, unsigned int orig_cmd, unsigned long arg,
> /*
> * Some ioctls can return an error, but still have valid
> * results that must be returned.
> + *
> + * FIXME: subdev IOCTLS are partially handled here and partially in
> + * v4l2-subdev.c and the 'always_copy' flag can only be set for IOCTLS
> + * defined here as part of the 'v4l2_ioctls' array. As
> + * VIDIOC_SUBDEV_G_ROUTING needs to return results to applications even
> + * in case of failure, but it is not defined here as part of the
> + * 'v4l2_ioctls' array, insert an ad-hoc check to address that.
> */
> - if (err < 0 && !always_copy)
> + if (err < 0 && !always_copy && cmd != VIDIOC_SUBDEV_G_ROUTING)
> goto out;
>
> out_array_args:
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 3ad24093abe9..89c97bde4575 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -377,6 +377,10 @@ subdev_ioctl_get_state(struct v4l2_subdev *sd, struct v4l2_subdev_fh *subdev_fh,
> case VIDIOC_SUBDEV_S_SELECTION:
> which = ((struct v4l2_subdev_selection *)arg)->which;
> break;
> + case VIDIOC_SUBDEV_G_ROUTING:
> + case VIDIOC_SUBDEV_S_ROUTING:
> + which = ((struct v4l2_subdev_routing *)arg)->which;
> + break;
> }
>
> return which == V4L2_SUBDEV_FORMAT_TRY ?
> @@ -692,6 +696,74 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
> case VIDIOC_SUBDEV_QUERYSTD:
> return v4l2_subdev_call(sd, video, querystd, arg);
>
> + case VIDIOC_SUBDEV_G_ROUTING: {
> + struct v4l2_subdev_routing *routing = arg;
> + struct v4l2_subdev_krouting *krouting;
> +
> + if (!(sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED))
> + return -ENOIOCTLCMD;
> +
> + memset(routing->reserved, 0, sizeof(routing->reserved));
> +
> + krouting = &state->routing;
> +
> + if (routing->num_routes < krouting->num_routes) {
> + routing->num_routes = krouting->num_routes;
> + return -ENOSPC;
> + }
> +
> + memcpy((struct v4l2_subdev_route *)(uintptr_t)routing->routes,
> + krouting->routes,
> + krouting->num_routes * sizeof(*krouting->routes));
> + routing->num_routes = krouting->num_routes;
> +
> + return 0;
> + }
> +
> + case VIDIOC_SUBDEV_S_ROUTING: {
> + struct v4l2_subdev_routing *routing = arg;
> + struct v4l2_subdev_route *routes =
> + (struct v4l2_subdev_route *)(uintptr_t)routing->routes;
> + struct v4l2_subdev_krouting krouting = {};
> + unsigned int i;
> +
> + if (!(sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED))
> + return -ENOIOCTLCMD;
> +
> + if (routing->which != V4L2_SUBDEV_FORMAT_TRY && ro_subdev)
> + return -EPERM;
> +
> + memset(routing->reserved, 0, sizeof(routing->reserved));
> +
> + for (i = 0; i < routing->num_routes; ++i) {
> + const struct v4l2_subdev_route *route = &routes[i];
> + const struct media_pad *pads = sd->entity.pads;
> +
> + /* Do not check sink pad for source routes */
> + if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_SOURCE)) {
> + if (route->sink_pad >= sd->entity.num_pads)
> + return -EINVAL;
> +
> + if (!(pads[route->sink_pad].flags &
> + MEDIA_PAD_FL_SINK))
> + return -EINVAL;
> + }
> +
> + if (route->source_pad >= sd->entity.num_pads)
> + return -EINVAL;
> +
> + if (!(pads[route->source_pad].flags &
> + MEDIA_PAD_FL_SOURCE))
> + return -EINVAL;
> + }
> +
> + krouting.num_routes = routing->num_routes;
> + krouting.routes = routes;
> +
> + return v4l2_subdev_call(sd, pad, set_routing, state,
> + routing->which, &krouting);
> + }
> +
> default:
> return v4l2_subdev_call(sd, core, ioctl, cmd, arg);
> }
> @@ -979,6 +1051,7 @@ void __v4l2_subdev_state_free(struct v4l2_subdev_state *state)
>
> mutex_destroy(&state->_lock);
>
> + kfree(state->routing.routes);
Do we have any guarantee that this array was allocated with kmalloc()?
Maybe kvfree() could be more appropriate here?
Best regards,
Tomasz
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 34/36] media: v4l2-subdev: Add v4l2_subdev_state_xlate_streams() helper
2022-03-01 16:11 ` [PATCH v11 34/36] media: v4l2-subdev: Add v4l2_subdev_state_xlate_streams() helper Tomi Valkeinen
2022-03-16 11:26 ` Jacopo Mondi
@ 2022-07-07 10:27 ` Tomasz Figa
2022-07-14 7:41 ` Tomi Valkeinen
1 sibling, 1 reply; 67+ messages in thread
From: Tomasz Figa @ 2022-07-07 10:27 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Hi Tomi, Laurent,
On Tue, Mar 01, 2022 at 06:11:54PM +0200, Tomi Valkeinen wrote:
> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>
> Add a helper function to translate streams between two pads of a subdev,
> using the subdev's internal routing table.
>
Thanks for the patch! Please see my comments inline.
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
> drivers/media/v4l2-core/v4l2-subdev.c | 25 +++++++++++++++++++++++++
> include/media/v4l2-subdev.h | 23 +++++++++++++++++++++++
> 2 files changed, 48 insertions(+)
>
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index e30338a53088..6a9fc62dacbf 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -1529,6 +1529,31 @@ v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
> }
> EXPORT_SYMBOL_GPL(v4l2_subdev_state_get_opposite_stream_format);
>
> +u64 v4l2_subdev_state_xlate_streams(const struct v4l2_subdev_state *state,
> + u32 pad0, u32 pad1, u64 *streams)
> +{
> + const struct v4l2_subdev_krouting *routing = &state->routing;
> + struct v4l2_subdev_route *route;
> + u64 streams0 = 0;
> + u64 streams1 = 0;
> +
> + for_each_active_route(routing, route) {
> + if (route->sink_pad == pad0 && route->source_pad == pad1 &&
> + (*streams & BIT(route->sink_stream))) {
> + streams0 |= BIT(route->sink_stream);
> + streams1 |= BIT(route->source_stream);
> + }
> + if (route->source_pad == pad0 && route->sink_pad == pad1 &&
> + (*streams & BIT(route->source_stream))) {
> + streams0 |= BIT(route->source_stream);
> + streams1 |= BIT(route->sink_stream);
> + }
> + }
> +
> + *streams = streams0;
> + return streams1;
> +}
> +
> int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
> struct v4l2_subdev_format *format)
> {
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index 1eced0f47296..992debe116ac 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -1497,6 +1497,29 @@ struct v4l2_mbus_framefmt *
> v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
> u32 pad, u32 stream);
>
> +/**
> + * v4l2_subdev_state_xlate_streams() - Translate streams from one pad to another
> + *
> + * @state: Subdevice state
> + * @pad0: The first pad
> + * @pad1: The second pad
> + * @streams: Streams bitmask on the first pad
> + *
> + * Streams on sink pads of a subdev are routed to source pads as expressed in
> + * the subdev state routing table. Stream numbers don't necessarily match on
> + * the sink and source side of a route. This function translates stream numbers
> + * on @pad0, expressed as a bitmask in @streams, to the corresponding streams
> + * on @pad1 using the routing table from the @state. It returns the stream mask
> + * on @pad1, and updates @streams with the streams that have been found in the
> + * routing table.
> + *
> + * @pad0 and @pad1 must be a sink and a source, in any order.
> + *
> + * Return: The bitmask of streams of @pad1 that are routed to @streams on @pad0.
It might be just me, but somehow I associate "translate" with turning a
name from one namespace into a corresponding name from another
namespace. In this case I initially assumed that this function is
supposed to accept stream IDs from pad0 and return corresponding
stream IDs from pad1. However, it looks like this is move like - "tell
me which streams on pad1 are connected to the given pad0 streams". Is
this correct?
If yes, maybe v4l2_subdev_state_get_routed_streams() be a better name?
Best regards,
Tomasz
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 00/36] v4l: routing and streams support
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
` (35 preceding siblings ...)
2022-03-01 16:11 ` [PATCH v11 36/36] media: v4l2-subdev: Add v4l2_subdev_s_stream_helper() function Tomi Valkeinen
@ 2022-07-07 10:38 ` Tomasz Figa
36 siblings, 0 replies; 67+ messages in thread
From: Tomasz Figa @ 2022-07-07 10:38 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
On Tue, Mar 01, 2022 at 06:11:20PM +0200, Tomi Valkeinen wrote:
> Hi,
>
> Here's v11 of the streams series (used to be "multiplexed streams").
>
> v10 can be found from:
>
> https://lore.kernel.org/all/20211130141536.891878-1-tomi.valkeinen@ideasonboard.com/
>
> This series is based on the v5 of the subdev active state:
>
> https://lore.kernel.org/all/20220301105548.305191-1-tomi.valkeinen@ideasonboard.com/
>
> My work branch with additional drivers can be found from:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/tomba/linux.git streams/work-v11
>
> And there's also the v4l-utils series to add support to v4l2-ctl and
> media-ctl:
>
> https://lore.kernel.org/all/20211130141815.892354-1-tomi.valkeinen@ideasonboard.com/
>
> The main changes compared to v10:
>
> - Rebased on the v5 of the active state series, which has different
> locking system than what we had in v10 of this series.
> - Moved code around in v4l2-subdev.[ch] so that the functions are inside
> the correct ifdefs
> - A better route validation helper v4l2_subdev_routing_validate
> - Subdev enable/disable_streams ops (see "media: v4l2-subdev: Add subdev
> .(enable|disable)_streams() operations")
>
> There are still some comments I haven't addressed from the v10 review,
> and I'd like to change the route flags a bit. We've also thought about
> adding more support for drivers that don't need multiple streams but
> would still use the new streams APIs to simplify the drivers.
>
> However, as the v10 is already quite old, and I posted the v5 for the
> active state, I thought it's better to also post the current version of
> the streams series.
Thanks Tomi and everyone for working on this series. With the few nits I
posted addressed:
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Best regards,
Tomasz
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 22/36] media: subdev: Add [GS]_ROUTING subdev ioctls and operations
2022-07-07 9:15 ` Tomasz Figa
@ 2022-07-14 6:47 ` Tomi Valkeinen
0 siblings, 0 replies; 67+ messages in thread
From: Tomi Valkeinen @ 2022-07-14 6:47 UTC (permalink / raw)
To: Tomasz Figa
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav, Michal Simek
Hi Tomasz,
On 07/07/2022 12:15, Tomasz Figa wrote:
> Hi Tomi, Laurent,
>
> On Tue, Mar 01, 2022 at 06:11:42PM +0200, Tomi Valkeinen wrote:
>> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>
>> Add support for subdev internal routing. A route is defined as a single
>> stream from a sink pad to a source pad.
>>
>> The userspace can configure the routing via two new ioctls,
>> VIDIOC_SUBDEV_G_ROUTING and VIDIOC_SUBDEV_S_ROUTING, and subdevs can
>> implement the functionality with v4l2_subdev_pad_ops.set_routing().
>
> Thanks for the patch! Please check my comment inline.
>
>>
>> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>> Signed-off-by: Michal Simek <michal.simek@xilinx.com>
>>
>> - Add sink and source streams for multiplexed links
>> - Copy the argument back in case of an error. This is needed to let the
>> caller know the number of routes.
>>
>> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
>>
>> - Expand and refine documentation.
>> - Make the 'routes' pointer a __u64 __user pointer so that a compat32
>> version of the ioctl is not required.
>> - Add struct v4l2_subdev_krouting to be used for subdevice operations.
>>
>> Signed-off-by: Jacopo Mondi <jacopo+renesas@jmondi.org>
>>
>> - Fix typecasing warnings
>> - Check sink & source pad types
>> - Add 'which' field
>> - Add V4L2_SUBDEV_ROUTE_FL_SOURCE
>> - Routing to subdev state
>> - Dropped get_routing subdev op
>>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> ---
>> drivers/media/v4l2-core/v4l2-ioctl.c | 25 ++++++++-
>> drivers/media/v4l2-core/v4l2-subdev.c | 73 +++++++++++++++++++++++++++
>> include/media/v4l2-subdev.h | 22 ++++++++
>> include/uapi/linux/v4l2-subdev.h | 57 +++++++++++++++++++++
>> 4 files changed, 176 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>> index 642cb90f457c..add3b28d446e 100644
>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>> @@ -16,6 +16,7 @@
>> #include <linux/kernel.h>
>> #include <linux/version.h>
>>
>> +#include <linux/v4l2-subdev.h>
>> #include <linux/videodev2.h>
>>
>> #include <media/v4l2-common.h>
>> @@ -3093,6 +3094,21 @@ static int check_array_args(unsigned int cmd, void *parg, size_t *array_size,
>> ret = 1;
>> break;
>> }
>> +
>> + case VIDIOC_SUBDEV_G_ROUTING:
>> + case VIDIOC_SUBDEV_S_ROUTING: {
>> + struct v4l2_subdev_routing *routing = parg;
>> +
>> + if (routing->num_routes > 256)
>
> Should we define and document this constant?
It's just an arbitrary high number as a sanity check. We don't have any
specific reason to limit the number of routes. If we publicize it to the
userspace, then the userspace might depend on it. On the other hand,
failing mystically when num-routes > 256 is also not so nice (not that
we have any drivers that go there, 8 has been the max so far).
I think we should just keep it here internally, and try to make sure to
increase it if we ever get drivers that might support more routes.
>> + return -EINVAL;
>> +
>> + *user_ptr = u64_to_user_ptr(routing->routes);
>> + *kernel_ptr = (void **)&routing->routes;
>> + *array_size = sizeof(struct v4l2_subdev_route)
>> + * routing->num_routes;
>> + ret = 1;
>> + break;
>> + }
>> }
>>
>> return ret;
>> @@ -3356,8 +3372,15 @@ video_usercopy(struct file *file, unsigned int orig_cmd, unsigned long arg,
>> /*
>> * Some ioctls can return an error, but still have valid
>> * results that must be returned.
>> + *
>> + * FIXME: subdev IOCTLS are partially handled here and partially in
>> + * v4l2-subdev.c and the 'always_copy' flag can only be set for IOCTLS
>> + * defined here as part of the 'v4l2_ioctls' array. As
>> + * VIDIOC_SUBDEV_G_ROUTING needs to return results to applications even
>> + * in case of failure, but it is not defined here as part of the
>> + * 'v4l2_ioctls' array, insert an ad-hoc check to address that.
>> */
>> - if (err < 0 && !always_copy)
>> + if (err < 0 && !always_copy && cmd != VIDIOC_SUBDEV_G_ROUTING)
>> goto out;
>>
>> out_array_args:
>> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
>> index 3ad24093abe9..89c97bde4575 100644
>> --- a/drivers/media/v4l2-core/v4l2-subdev.c
>> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
>> @@ -377,6 +377,10 @@ subdev_ioctl_get_state(struct v4l2_subdev *sd, struct v4l2_subdev_fh *subdev_fh,
>> case VIDIOC_SUBDEV_S_SELECTION:
>> which = ((struct v4l2_subdev_selection *)arg)->which;
>> break;
>> + case VIDIOC_SUBDEV_G_ROUTING:
>> + case VIDIOC_SUBDEV_S_ROUTING:
>> + which = ((struct v4l2_subdev_routing *)arg)->which;
>> + break;
>> }
>>
>> return which == V4L2_SUBDEV_FORMAT_TRY ?
>> @@ -692,6 +696,74 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
>> case VIDIOC_SUBDEV_QUERYSTD:
>> return v4l2_subdev_call(sd, video, querystd, arg);
>>
>> + case VIDIOC_SUBDEV_G_ROUTING: {
>> + struct v4l2_subdev_routing *routing = arg;
>> + struct v4l2_subdev_krouting *krouting;
>> +
>> + if (!(sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED))
>> + return -ENOIOCTLCMD;
>> +
>> + memset(routing->reserved, 0, sizeof(routing->reserved));
>> +
>> + krouting = &state->routing;
>> +
>> + if (routing->num_routes < krouting->num_routes) {
>> + routing->num_routes = krouting->num_routes;
>> + return -ENOSPC;
>> + }
>> +
>> + memcpy((struct v4l2_subdev_route *)(uintptr_t)routing->routes,
>> + krouting->routes,
>> + krouting->num_routes * sizeof(*krouting->routes));
>> + routing->num_routes = krouting->num_routes;
>> +
>> + return 0;
>> + }
>> +
>> + case VIDIOC_SUBDEV_S_ROUTING: {
>> + struct v4l2_subdev_routing *routing = arg;
>> + struct v4l2_subdev_route *routes =
>> + (struct v4l2_subdev_route *)(uintptr_t)routing->routes;
>> + struct v4l2_subdev_krouting krouting = {};
>> + unsigned int i;
>> +
>> + if (!(sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED))
>> + return -ENOIOCTLCMD;
>> +
>> + if (routing->which != V4L2_SUBDEV_FORMAT_TRY && ro_subdev)
>> + return -EPERM;
>> +
>> + memset(routing->reserved, 0, sizeof(routing->reserved));
>> +
>> + for (i = 0; i < routing->num_routes; ++i) {
>> + const struct v4l2_subdev_route *route = &routes[i];
>> + const struct media_pad *pads = sd->entity.pads;
>> +
>> + /* Do not check sink pad for source routes */
>> + if (!(route->flags & V4L2_SUBDEV_ROUTE_FL_SOURCE)) {
>> + if (route->sink_pad >= sd->entity.num_pads)
>> + return -EINVAL;
>> +
>> + if (!(pads[route->sink_pad].flags &
>> + MEDIA_PAD_FL_SINK))
>> + return -EINVAL;
>> + }
>> +
>> + if (route->source_pad >= sd->entity.num_pads)
>> + return -EINVAL;
>> +
>> + if (!(pads[route->source_pad].flags &
>> + MEDIA_PAD_FL_SOURCE))
>> + return -EINVAL;
>> + }
>> +
>> + krouting.num_routes = routing->num_routes;
>> + krouting.routes = routes;
>> +
>> + return v4l2_subdev_call(sd, pad, set_routing, state,
>> + routing->which, &krouting);
>> + }
>> +
>> default:
>> return v4l2_subdev_call(sd, core, ioctl, cmd, arg);
>> }
>> @@ -979,6 +1051,7 @@ void __v4l2_subdev_state_free(struct v4l2_subdev_state *state)
>>
>> mutex_destroy(&state->_lock);
>>
>> + kfree(state->routing.routes);
>
> Do we have any guarantee that this array was allocated with kmalloc()?
> Maybe kvfree() could be more appropriate here?
The array is allocated with kmemdup() in v4l2_subdev_set_routing().
Tomi
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 34/36] media: v4l2-subdev: Add v4l2_subdev_state_xlate_streams() helper
2022-07-07 10:27 ` Tomasz Figa
@ 2022-07-14 7:41 ` Tomi Valkeinen
2022-07-19 5:38 ` Tomasz Figa
0 siblings, 1 reply; 67+ messages in thread
From: Tomi Valkeinen @ 2022-07-14 7:41 UTC (permalink / raw)
To: Tomasz Figa
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
Hi,
On 07/07/2022 13:27, Tomasz Figa wrote:
> Hi Tomi, Laurent,
>
> On Tue, Mar 01, 2022 at 06:11:54PM +0200, Tomi Valkeinen wrote:
>> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>
>> Add a helper function to translate streams between two pads of a subdev,
>> using the subdev's internal routing table.
>>
>
> Thanks for the patch! Please see my comments inline.
>
>> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> ---
>> drivers/media/v4l2-core/v4l2-subdev.c | 25 +++++++++++++++++++++++++
>> include/media/v4l2-subdev.h | 23 +++++++++++++++++++++++
>> 2 files changed, 48 insertions(+)
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
>> index e30338a53088..6a9fc62dacbf 100644
>> --- a/drivers/media/v4l2-core/v4l2-subdev.c
>> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
>> @@ -1529,6 +1529,31 @@ v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
>> }
>> EXPORT_SYMBOL_GPL(v4l2_subdev_state_get_opposite_stream_format);
>>
>> +u64 v4l2_subdev_state_xlate_streams(const struct v4l2_subdev_state *state,
>> + u32 pad0, u32 pad1, u64 *streams)
>> +{
>> + const struct v4l2_subdev_krouting *routing = &state->routing;
>> + struct v4l2_subdev_route *route;
>> + u64 streams0 = 0;
>> + u64 streams1 = 0;
>> +
>> + for_each_active_route(routing, route) {
>> + if (route->sink_pad == pad0 && route->source_pad == pad1 &&
>> + (*streams & BIT(route->sink_stream))) {
>> + streams0 |= BIT(route->sink_stream);
>> + streams1 |= BIT(route->source_stream);
>> + }
>> + if (route->source_pad == pad0 && route->sink_pad == pad1 &&
>> + (*streams & BIT(route->source_stream))) {
>> + streams0 |= BIT(route->source_stream);
>> + streams1 |= BIT(route->sink_stream);
>> + }
>> + }
>> +
>> + *streams = streams0;
>> + return streams1;
>> +}
>> +
>> int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
>> struct v4l2_subdev_format *format)
>> {
>> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
>> index 1eced0f47296..992debe116ac 100644
>> --- a/include/media/v4l2-subdev.h
>> +++ b/include/media/v4l2-subdev.h
>> @@ -1497,6 +1497,29 @@ struct v4l2_mbus_framefmt *
>> v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
>> u32 pad, u32 stream);
>>
>> +/**
>> + * v4l2_subdev_state_xlate_streams() - Translate streams from one pad to another
>> + *
>> + * @state: Subdevice state
>> + * @pad0: The first pad
>> + * @pad1: The second pad
>> + * @streams: Streams bitmask on the first pad
>> + *
>> + * Streams on sink pads of a subdev are routed to source pads as expressed in
>> + * the subdev state routing table. Stream numbers don't necessarily match on
>> + * the sink and source side of a route. This function translates stream numbers
>> + * on @pad0, expressed as a bitmask in @streams, to the corresponding streams
>> + * on @pad1 using the routing table from the @state. It returns the stream mask
>> + * on @pad1, and updates @streams with the streams that have been found in the
>> + * routing table.
>> + *
>> + * @pad0 and @pad1 must be a sink and a source, in any order.
>> + *
>> + * Return: The bitmask of streams of @pad1 that are routed to @streams on @pad0.
>
> It might be just me, but somehow I associate "translate" with turning a
> name from one namespace into a corresponding name from another
> namespace. In this case I initially assumed that this function is
> supposed to accept stream IDs from pad0 and return corresponding
> stream IDs from pad1. However, it looks like this is move like - "tell
> me which streams on pad1 are connected to the given pad0 streams". Is
> this correct?
So is your point that as the function returns a bitmask, and not a
mapping of the stream IDs, this is not a translate function?
> If yes, maybe v4l2_subdev_state_get_routed_streams() be a better name?
I think this name would also fit quite well for a function that returns
a mapping of the streams. So... I don't have a strong opinion on this.
To me, neither the current name nor the proposed one clearly explains
alone what the function does.
Tomi
^ permalink raw reply [flat|nested] 67+ messages in thread
* Re: [PATCH v11 34/36] media: v4l2-subdev: Add v4l2_subdev_state_xlate_streams() helper
2022-07-14 7:41 ` Tomi Valkeinen
@ 2022-07-19 5:38 ` Tomasz Figa
0 siblings, 0 replies; 67+ messages in thread
From: Tomasz Figa @ 2022-07-19 5:38 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: linux-media, sakari.ailus, Jacopo Mondi, Laurent Pinchart,
niklas.soderlund+renesas, Mauro Carvalho Chehab, Hans Verkuil,
Pratyush Yadav
On Thu, Jul 14, 2022 at 4:41 PM Tomi Valkeinen
<tomi.valkeinen@ideasonboard.com> wrote:
>
> Hi,
>
> On 07/07/2022 13:27, Tomasz Figa wrote:
> > Hi Tomi, Laurent,
> >
> > On Tue, Mar 01, 2022 at 06:11:54PM +0200, Tomi Valkeinen wrote:
> >> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> >>
> >> Add a helper function to translate streams between two pads of a subdev,
> >> using the subdev's internal routing table.
> >>
> >
> > Thanks for the patch! Please see my comments inline.
> >
> >> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> >> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> >> ---
> >> drivers/media/v4l2-core/v4l2-subdev.c | 25 +++++++++++++++++++++++++
> >> include/media/v4l2-subdev.h | 23 +++++++++++++++++++++++
> >> 2 files changed, 48 insertions(+)
> >>
> >> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> >> index e30338a53088..6a9fc62dacbf 100644
> >> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> >> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> >> @@ -1529,6 +1529,31 @@ v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
> >> }
> >> EXPORT_SYMBOL_GPL(v4l2_subdev_state_get_opposite_stream_format);
> >>
> >> +u64 v4l2_subdev_state_xlate_streams(const struct v4l2_subdev_state *state,
> >> + u32 pad0, u32 pad1, u64 *streams)
> >> +{
> >> + const struct v4l2_subdev_krouting *routing = &state->routing;
> >> + struct v4l2_subdev_route *route;
> >> + u64 streams0 = 0;
> >> + u64 streams1 = 0;
> >> +
> >> + for_each_active_route(routing, route) {
> >> + if (route->sink_pad == pad0 && route->source_pad == pad1 &&
> >> + (*streams & BIT(route->sink_stream))) {
> >> + streams0 |= BIT(route->sink_stream);
> >> + streams1 |= BIT(route->source_stream);
> >> + }
> >> + if (route->source_pad == pad0 && route->sink_pad == pad1 &&
> >> + (*streams & BIT(route->source_stream))) {
> >> + streams0 |= BIT(route->source_stream);
> >> + streams1 |= BIT(route->sink_stream);
> >> + }
> >> + }
> >> +
> >> + *streams = streams0;
> >> + return streams1;
> >> +}
> >> +
> >> int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
> >> struct v4l2_subdev_format *format)
> >> {
> >> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> >> index 1eced0f47296..992debe116ac 100644
> >> --- a/include/media/v4l2-subdev.h
> >> +++ b/include/media/v4l2-subdev.h
> >> @@ -1497,6 +1497,29 @@ struct v4l2_mbus_framefmt *
> >> v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
> >> u32 pad, u32 stream);
> >>
> >> +/**
> >> + * v4l2_subdev_state_xlate_streams() - Translate streams from one pad to another
> >> + *
> >> + * @state: Subdevice state
> >> + * @pad0: The first pad
> >> + * @pad1: The second pad
> >> + * @streams: Streams bitmask on the first pad
> >> + *
> >> + * Streams on sink pads of a subdev are routed to source pads as expressed in
> >> + * the subdev state routing table. Stream numbers don't necessarily match on
> >> + * the sink and source side of a route. This function translates stream numbers
> >> + * on @pad0, expressed as a bitmask in @streams, to the corresponding streams
> >> + * on @pad1 using the routing table from the @state. It returns the stream mask
> >> + * on @pad1, and updates @streams with the streams that have been found in the
> >> + * routing table.
> >> + *
> >> + * @pad0 and @pad1 must be a sink and a source, in any order.
> >> + *
> >> + * Return: The bitmask of streams of @pad1 that are routed to @streams on @pad0.
> >
> > It might be just me, but somehow I associate "translate" with turning a
> > name from one namespace into a corresponding name from another
> > namespace. In this case I initially assumed that this function is
> > supposed to accept stream IDs from pad0 and return corresponding
> > stream IDs from pad1. However, it looks like this is move like - "tell
> > me which streams on pad1 are connected to the given pad0 streams". Is
> > this correct?
>
> So is your point that as the function returns a bitmask, and not a
> mapping of the stream IDs, this is not a translate function?
Yep.
>
> > If yes, maybe v4l2_subdev_state_get_routed_streams() be a better name?
>
> I think this name would also fit quite well for a function that returns
> a mapping of the streams. So... I don't have a strong opinion on this.
> To me, neither the current name nor the proposed one clearly explains
> alone what the function does.
Well, since it's a kernel function, it probably can be renamed later.
Anyway, I forgot to place "nit:" before the comment. It's not really a
strong opinion from me either.
Best regards,
Tomasz
^ permalink raw reply [flat|nested] 67+ messages in thread
end of thread, other threads:[~2022-07-19 5:38 UTC | newest]
Thread overview: 67+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-03-01 16:11 [PATCH v11 00/36] v4l: routing and streams support Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 01/36] media: entity: Use pad as a starting point for graph walk Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 02/36] media: entity: Use pads instead of entities in the media graph walk stack Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 03/36] media: entity: Walk the graph based on pads Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 04/36] media: mc: Start walk from a specific pad in use count calculation Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 05/36] media: entity: Add iterator helper for entity pads Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 06/36] media: entity: Move the pipeline from entity to pads Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 07/36] media: entity: Use pad as the starting point for a pipeline Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 08/36] media: entity: Add has_route entity operation Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 09/36] media: entity: Add media_entity_has_route() function Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 10/36] media: entity: Use routing information during graph traversal Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 11/36] media: entity: Skip link validation for pads to which there is no route Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 12/36] media: entity: Add an iterator helper for connected pads Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 13/36] media: entity: Add only connected pads to the pipeline Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 14/36] media: entity: Add debug information in graph walk route check Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 15/36] media: Add bus type to frame descriptors Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 16/36] media: Add CSI-2 bus configuration " Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 17/36] media: Add stream to frame descriptor Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 18/36] media: subdev: increase V4L2_FRAME_DESC_ENTRY_MAX to 8 Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 19/36] media: add V4L2_SUBDEV_FL_MULTIPLEXED Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 20/36] media: add V4L2_SUBDEV_CAP_MPLEXED Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 21/36] media: Documentation: Add GS_ROUTING documentation Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 22/36] media: subdev: Add [GS]_ROUTING subdev ioctls and operations Tomi Valkeinen
2022-07-07 9:15 ` Tomasz Figa
2022-07-14 6:47 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 23/36] media: subdev: add v4l2_subdev_has_route() Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 24/36] media: subdev: add v4l2_subdev_set_routing helper() Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 25/36] media: Documentation: add multiplexed streams documentation Tomi Valkeinen
2022-03-16 9:04 ` Jacopo Mondi
2022-03-17 7:18 ` Tomi Valkeinen
2022-03-17 8:27 ` Jacopo Mondi
2022-03-01 16:11 ` [PATCH v11 26/36] media: subdev: add stream based configuration Tomi Valkeinen
2022-03-16 9:59 ` Jacopo Mondi
2022-03-17 8:01 ` Tomi Valkeinen
2022-03-17 8:38 ` Jacopo Mondi
2022-03-17 8:48 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 27/36] media: subdev: use streams in v4l2_subdev_link_validate() Tomi Valkeinen
2022-03-16 10:47 ` Jacopo Mondi
2022-03-17 8:39 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 28/36] media: subdev: add "opposite" stream helper funcs Tomi Valkeinen
2022-03-16 10:50 ` Jacopo Mondi
2022-03-17 9:10 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 29/36] media: subdev: add v4l2_subdev_get_fmt() helper function Tomi Valkeinen
2022-03-16 11:00 ` Jacopo Mondi
2022-03-01 16:11 ` [PATCH v11 30/36] media: subdev: Fallback to pad config in v4l2_subdev_get_fmt() Tomi Valkeinen
2022-03-16 11:03 ` Jacopo Mondi
2022-03-17 9:18 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 31/36] media: subdev: add v4l2_subdev_set_routing_with_fmt() helper Tomi Valkeinen
2022-03-16 11:04 ` Jacopo Mondi
2022-03-01 16:11 ` [PATCH v11 32/36] media: subdev: add v4l2_subdev_routing_validate() helper Tomi Valkeinen
2022-03-16 11:10 ` Jacopo Mondi
2022-03-17 9:15 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 33/36] media: subdev: Add for_each_active_route() macro Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 34/36] media: v4l2-subdev: Add v4l2_subdev_state_xlate_streams() helper Tomi Valkeinen
2022-03-16 11:26 ` Jacopo Mondi
2022-03-17 9:27 ` Tomi Valkeinen
2022-03-17 9:35 ` Tomi Valkeinen
2022-07-07 10:27 ` Tomasz Figa
2022-07-14 7:41 ` Tomi Valkeinen
2022-07-19 5:38 ` Tomasz Figa
2022-03-01 16:11 ` [PATCH v11 35/36] media: v4l2-subdev: Add subdev .(enable|disable)_streams() operations Tomi Valkeinen
2022-03-16 12:05 ` Jacopo Mondi
2022-03-17 9:43 ` Tomi Valkeinen
2022-03-01 16:11 ` [PATCH v11 36/36] media: v4l2-subdev: Add v4l2_subdev_s_stream_helper() function Tomi Valkeinen
2022-03-16 12:10 ` Jacopo Mondi
2022-03-17 9:45 ` Tomi Valkeinen
2022-07-07 10:38 ` [PATCH v11 00/36] v4l: routing and streams support Tomasz Figa
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).