From: "Niklas Söderlund" <niklas.soderlund+renesas@ragnatech.se>
To: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Cc: linux-media@vger.kernel.org, sakari.ailus@linux.intel.com,
Jacopo Mondi <jacopo+renesas@jmondi.org>,
Laurent Pinchart <laurent.pinchart@ideasonboard.com>,
Mauro Carvalho Chehab <mchehab@kernel.org>,
Hans Verkuil <hverkuil-cisco@xs4all.nl>
Subject: Re: [PATCH v5 00/24] v4l: subdev internal routing
Date: Fri, 16 Apr 2021 10:38:39 +0200 [thread overview]
Message-ID: <YHlND8NhEs+lkDZB@oden.dyn.berto.se> (raw)
In-Reply-To: <20210415130450.421168-1-tomi.valkeinen@ideasonboard.com>
Hi Tomi,
I'm very happy to see this being worked on again!
Is there code somewhere that demonstrates the v5 API in use? I still
have old branches of this series and it would be nice to see how the API
have evolved for drivers.
Likewise are there some user-space code around that can be used to test
the API? For v2 and v3 I had some hack patches [1], do they still work?
More likely they have gone stale by now :-)
1. git://git.ragnatech.se/v4l-utils routing
On 2021-04-15 16:04:26 +0300, Tomi Valkeinen wrote:
> Hi,
>
> This is an RFC for subdev internal routing which is needed for
> multiplexed streams support. I believe this is essentially a v5 of the
> series, the v4 posted here:
>
> https://lore.kernel.org/linux-media/20190328200608.9463-1-jacopo+renesas@jmondi.org/
>
> Most of the patches are not changed (aside from fixing rebase issues
> etc). The major changes in this version are:
>
> 1) Added 'which' field to the routing structs. It is currently not used,
> as implementing it is not trivial. However, I think it's good to add it
> to the uAPI now, and require the field to be set to
> V4L2_SUBDEV_FORMAT_ACTIVE for now. See this RFC for an idea how this
> could be implemented:
>
> https://lore.kernel.org/linux-media/20210409133659.389544-1-tomi.valkeinen@ideasonboard.com/
>
> 2) No hardcoded maximum number of routes. Defining a maximum is not
> possible, as there can be an arbitrary amount of routes per pad, and
> there can be an arbitrary amount of pads per subdev. This series
> allocates space for the routing table dynamically, which unfortunately
> leads to not-just-a-few allocs and frees.
>
> 3) When searching for a format for a stream, the v4 looked for a
> non-multiplexed pad only as far as the "other" side of the subdev. It
> wouldn't work for a subdev which has multiplexed sink and source pads.
> This series implements a "deep" get-format (v4l2_subdev_get_format_dir)
> which follows a stream either towards the original source or the final
> sink, while looking for a non-multiplexed pad with a format.
>
> Some thoughts:
>
> 1) Link validation and v4l2_subdev_get_format_dir need to look at the
> routing, and this leads to multiple allocs to get a copy of the routing
> table. There might be a possibility here to keep a table allocated and
> re-use it in consecutive get_routing calls.
>
> Or even better, perhaps the kAPI could be changed so that allocs are not
> needed. I thought about a kAPI where the subdev just returns a pointer
> to its routing table, but then we hit the life-cycle problem: how to
> ensure the table won't be freed or changed until the caller is done.
>
> 2) The routing uAPI is a bit vague. There is no way for the userspace to
> figure out what kind of routing is allowed. Also, the existence of a
> route in the routing table already indicates that the route is active,
> but we also have V4L2_SUBDEV_ROUTE_FL_ACTIVE. I decided to keep
> V4L2_SUBDEV_ROUTE_FL_ACTIVE for now, even if it doesn't really provide
> any feature.
>
> 3) V4L2_FRAME_DESC_ENTRY_MAX is defined as 8 (I change it from 4 to 8 in
> this series). This limits the number of streams per pad to 8. Preferably
> the number of frame descs would be unlimited, but I didn't start
> tackling this. I believe 8 is quite safe number (4 pixel streams and 4
> embedded data stream).
>
> 4) Link validation ends up following the same routes multiple times, as
> each stream in each subdev is validated separately.
>
> Tomi
>
> Jacopo Mondi (2):
> media: entity: Add iterator helper for entity pads
> media: Documentation: Add GS_ROUTING documentation
>
> Laurent Pinchart (4):
> media: entity: Add has_route entity operation
> media: entity: Add media_entity_has_route() function
> media: entity: Use routing information during graph traversal
> v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations
>
> Sakari Ailus (14):
> media: entity: Use pad as a starting point for graph walk
> media: entity: Use pads instead of entities in the media graph walk
> stack
> media: entity: Walk the graph based on pads
> v4l: mc: Start walk from a specific pad in use count calculation
> media: entity: Move the pipeline from entity to pads
> media: entity: Use pad as the starting point for a pipeline
> media: entity: Skip link validation for pads to which there is no
> route to
> media: entity: Add an iterator helper for connected pads
> media: entity: Add only connected pads to the pipeline
> media: entity: Add debug information in graph walk route check
> v4l: Add bus type to frame descriptors
> v4l: Add CSI-2 bus configuration to frame descriptors
> v4l: Add stream to frame descriptor
> v4l: mc: Add an S_ROUTING helper function for power state changes
>
> Tomi Valkeinen (4):
> v4l: subdev: routing kernel helper functions
> v4l: subdev: add v4l2_subdev_get_format_dir()
> v4l: subdev: Take routing information into account in link validation
> v4l: subdev: increase V4L2_FRAME_DESC_ENTRY_MAX to 8
>
> Documentation/driver-api/media/mc-core.rst | 15 +-
> .../userspace-api/media/v4l/dev-subdev.rst | 92 +++++
> .../userspace-api/media/v4l/user-func.rst | 1 +
> .../media/v4l/vidioc-subdev-g-routing.rst | 143 +++++++
> drivers/media/mc/mc-device.c | 13 +-
> drivers/media/mc/mc-entity.c | 239 +++++++-----
> drivers/media/pci/intel/ipu3/ipu3-cio2-main.c | 6 +-
> .../media/platform/exynos4-is/fimc-capture.c | 8 +-
> .../platform/exynos4-is/fimc-isp-video.c | 8 +-
> drivers/media/platform/exynos4-is/fimc-isp.c | 2 +-
> drivers/media/platform/exynos4-is/fimc-lite.c | 10 +-
> drivers/media/platform/exynos4-is/media-dev.c | 20 +-
> drivers/media/platform/omap3isp/isp.c | 2 +-
> drivers/media/platform/omap3isp/ispvideo.c | 25 +-
> drivers/media/platform/omap3isp/ispvideo.h | 2 +-
> .../media/platform/qcom/camss/camss-video.c | 6 +-
> drivers/media/platform/rcar-vin/rcar-core.c | 16 +-
> drivers/media/platform/rcar-vin/rcar-dma.c | 8 +-
> .../platform/rockchip/rkisp1/rkisp1-capture.c | 6 +-
> .../media/platform/s3c-camif/camif-capture.c | 6 +-
> drivers/media/platform/stm32/stm32-dcmi.c | 6 +-
> .../platform/sunxi/sun4i-csi/sun4i_dma.c | 6 +-
> .../platform/sunxi/sun6i-csi/sun6i_video.c | 6 +-
> drivers/media/platform/ti-vpe/cal-video.c | 6 +-
> drivers/media/platform/vsp1/vsp1_video.c | 18 +-
> drivers/media/platform/xilinx/xilinx-dma.c | 20 +-
> drivers/media/platform/xilinx/xilinx-dma.h | 2 +-
> .../media/test-drivers/vimc/vimc-capture.c | 6 +-
> drivers/media/usb/au0828/au0828-core.c | 8 +-
> drivers/media/v4l2-core/v4l2-ioctl.c | 25 +-
> drivers/media/v4l2-core/v4l2-mc.c | 77 ++--
> drivers/media/v4l2-core/v4l2-subdev.c | 368 ++++++++++++++++++
> drivers/staging/media/imx/imx-media-utils.c | 8 +-
> drivers/staging/media/ipu3/ipu3-v4l2.c | 6 +-
> drivers/staging/media/omap4iss/iss.c | 2 +-
> drivers/staging/media/omap4iss/iss_video.c | 38 +-
> drivers/staging/media/omap4iss/iss_video.h | 2 +-
> drivers/staging/media/tegra-video/tegra210.c | 6 +-
> include/media/media-entity.h | 143 +++++--
> include/media/v4l2-mc.h | 22 ++
> include/media/v4l2-subdev.h | 93 ++++-
> include/uapi/linux/v4l2-subdev.h | 44 +++
> 42 files changed, 1241 insertions(+), 299 deletions(-)
> create mode 100644 Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
>
> --
> 2.25.1
>
--
Regards,
Niklas Söderlund
next prev parent reply other threads:[~2021-04-16 8:38 UTC|newest]
Thread overview: 72+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-15 13:04 [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 01/24] media: entity: Use pad as a starting point for graph walk Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 02/24] media: entity: Use pads instead of entities in the media graph walk stack Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 03/24] media: entity: Walk the graph based on pads Tomi Valkeinen
2021-04-18 17:47 ` Laurent Pinchart
2021-04-20 11:30 ` Sakari Ailus
2021-04-15 13:04 ` [PATCH v5 04/24] v4l: mc: Start walk from a specific pad in use count calculation Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 05/24] media: entity: Add iterator helper for entity pads Tomi Valkeinen
2021-04-18 17:52 ` Laurent Pinchart
2021-04-22 12:04 ` Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 06/24] media: entity: Move the pipeline from entity to pads Tomi Valkeinen
2021-04-18 18:00 ` Laurent Pinchart
2021-04-20 11:38 ` Sakari Ailus
2021-04-15 13:04 ` [PATCH v5 07/24] media: entity: Use pad as the starting point for a pipeline Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 08/24] media: entity: Add has_route entity operation Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 09/24] media: entity: Add media_entity_has_route() function Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 10/24] media: entity: Use routing information during graph traversal Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 11/24] media: entity: Skip link validation for pads to which there is no route to Tomi Valkeinen
2021-04-18 18:06 ` Laurent Pinchart
2021-04-20 11:41 ` Sakari Ailus
2021-04-23 12:37 ` Tomi Valkeinen
2021-04-29 12:06 ` Sakari Ailus
2021-04-29 14:10 ` Laurent Pinchart
2021-04-15 13:04 ` [PATCH v5 12/24] media: entity: Add an iterator helper for connected pads Tomi Valkeinen
2021-04-18 18:20 ` Laurent Pinchart
2021-04-20 11:48 ` Sakari Ailus
2021-04-29 1:33 ` Laurent Pinchart
2021-04-29 11:56 ` Sakari Ailus
2021-04-29 12:04 ` Tomi Valkeinen
2021-04-29 12:07 ` Sakari Ailus
2021-04-29 12:14 ` Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 13/24] media: entity: Add only connected pads to the pipeline Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 14/24] media: entity: Add debug information in graph walk route check Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 15/24] v4l: Add bus type to frame descriptors Tomi Valkeinen
2021-04-18 19:23 ` Laurent Pinchart
2021-04-20 11:50 ` Sakari Ailus
2021-04-22 12:30 ` Tomi Valkeinen
2021-04-29 11:58 ` Sakari Ailus
2021-04-29 14:09 ` Laurent Pinchart
2021-04-15 13:04 ` [PATCH v5 16/24] v4l: Add CSI-2 bus configuration " Tomi Valkeinen
2021-04-18 19:24 ` Laurent Pinchart
2021-04-20 16:32 ` Sakari Ailus
2021-04-15 13:04 ` [PATCH v5 17/24] v4l: Add stream to frame descriptor Tomi Valkeinen
2021-04-18 19:27 ` Laurent Pinchart
2021-04-22 12:47 ` Tomi Valkeinen
2021-04-22 16:18 ` Laurent Pinchart
2021-04-15 13:04 ` [PATCH v5 18/24] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations Tomi Valkeinen
2021-04-18 18:32 ` Laurent Pinchart
2021-04-22 11:16 ` Tomi Valkeinen
2021-04-22 16:20 ` Laurent Pinchart
2021-04-22 16:58 ` Tomi Valkeinen
2021-04-20 16:35 ` Sakari Ailus
2021-04-15 13:04 ` [PATCH v5 19/24] media: Documentation: Add GS_ROUTING documentation Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 20/24] v4l: mc: Add an S_ROUTING helper function for power state changes Tomi Valkeinen
2021-04-18 18:55 ` Laurent Pinchart
2021-04-20 16:41 ` Sakari Ailus
2021-04-15 13:04 ` [PATCH v5 21/24] v4l: subdev: routing kernel helper functions Tomi Valkeinen
2021-04-18 19:18 ` Laurent Pinchart
2021-04-15 13:04 ` [PATCH v5 22/24] v4l: subdev: add v4l2_subdev_get_format_dir() Tomi Valkeinen
2021-04-18 19:04 ` Laurent Pinchart
2021-04-21 13:04 ` Tomi Valkeinen
2021-04-29 1:43 ` Laurent Pinchart
2021-05-04 6:49 ` Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 23/24] v4l: subdev: Take routing information into account in link validation Tomi Valkeinen
2021-04-15 13:04 ` [PATCH v5 24/24] v4l: subdev: increase V4L2_FRAME_DESC_ENTRY_MAX to 8 Tomi Valkeinen
2021-04-18 19:06 ` Laurent Pinchart
2021-04-16 8:38 ` Niklas Söderlund [this message]
2021-04-16 8:47 ` [PATCH v5 00/24] v4l: subdev internal routing Tomi Valkeinen
2021-04-16 8:56 ` Niklas Söderlund
2021-04-18 17:32 ` Laurent Pinchart
2021-04-21 12:57 ` Tomi Valkeinen
2021-04-29 1:27 ` Laurent Pinchart
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YHlND8NhEs+lkDZB@oden.dyn.berto.se \
--to=niklas.soderlund+renesas@ragnatech.se \
--cc=hverkuil-cisco@xs4all.nl \
--cc=jacopo+renesas@jmondi.org \
--cc=laurent.pinchart@ideasonboard.com \
--cc=linux-media@vger.kernel.org \
--cc=mchehab@kernel.org \
--cc=sakari.ailus@linux.intel.com \
--cc=tomi.valkeinen@ideasonboard.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).