From: Julien Massot <julien.massot@collabora.com>
To: Sakari Ailus <sakari.ailus@linux.intel.com>
Cc: linux-media@vger.kernel.org,
Laurent Pinchart <laurent.pinchart@ideasonboard.com>,
tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com,
hongju.wang@intel.com, hverkuil@xs4all.nl,
Andrey Konovalov <andrey.konovalov@linaro.org>,
Jacopo Mondi <jacopo.mondi@ideasonboard.com>,
Dmitry Perchanov <dmitry.perchanov@intel.com>,
"Ng, Khai Wen" <khai.wen.ng@intel.com>,
Alain Volmat <alain.volmat@foss.st.com>
Subject: Re: [PATCH v8 09/38] media: Documentation: v4l: Document internal source pads
Date: Tue, 19 Mar 2024 15:38:39 +0100 [thread overview]
Message-ID: <b9ff9d89-ac54-46c5-ac9f-8b83fe604dc9@collabora.com> (raw)
In-Reply-To: <ZfmXW4tkh3FOKXU8@kekkonen.localdomain>
Hi Sakari,
On 3/19/24 14:47, Sakari Ailus wrote:
> Hi Julien,
>
> On Fri, Mar 15, 2024 at 04:32:59PM +0100, Julien Massot wrote:
>>
>>
>> On 3/13/24 08:24, Sakari Ailus wrote:
>>> Document internal source pads, pads that have both SINK and INTERNAL flags
>>> set. Use the IMX219 camera sensor as an example.
>>>
>>> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
>>> ---
>>> .../userspace-api/media/v4l/dev-subdev.rst | 145 ++++++++++++++++++
>>> 1 file changed, 145 insertions(+)
>>>
>>> diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
>>> index a387e8a15b8d..1808f40f63e3 100644
>>> --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
>>> +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
>>> @@ -553,6 +553,27 @@ A stream at a specific point in the media pipeline is identified by the
>>> sub-device and a (pad, stream) pair. For sub-devices that do not support
>>> multiplexed streams the 'stream' field is always 0.
>>> +.. _v4l2-subdev-internal-source-pads:
>>> +
>>> +Internal source pads and routing
>>> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>> +
>>> +Cases where a single sub-device source pad is traversed by multiple streams, one
>>> +or more of which originate from within the sub-device itself, are special as
>>> +there is no external sink pad for such routes. In those cases, the sources of
>>> +the internally generated streams are represented by internal source pads, which
>>> +are sink pads that have the :ref:`MEDIA_PAD_FL_INTERNAL <MEDIA-PAD-FL-INTERNAL>`
>>> +pad flag set.
>>> +
>>> +Internal pads have all the properties of an external pad, including formats and
>>> +selections. The format in this case is the source format of the stream. An
>>> +internal pad always has a single stream only (0).
>>> +
>>> +Routes from an internal source pad to an external source pad are typically not
>>> +modifiable but they can be activated and deactivated using the
>>> +:ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE <v4l2-subdev-routing-flags>` flag, depending
>>> +on driver capabilities.
>>> +
>>> Interaction between routes, streams, formats and selections
>>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>> @@ -668,3 +689,127 @@ To configure this pipeline, the userspace must take the following steps:
>>> the configurations along the stream towards the receiver, using
>>> :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls to configure each
>>> stream endpoint in each sub-device.
>>> +
>>> +Internal pads setup example
>>> +^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>> +
>>> +A simple example of a multiplexed stream setup might be as follows:
>>> +
>>> +- An IMX219 camera sensor source sub-device, with one sink pad (0), one source pad
>>> + (1), an internal sink pad (2) that represents the source of embedded
>> The pixel pad is an internal pad as well ?
>
> How about:
>
> - An IMX219 camera sensor source sub-device, with one internal sink pad
> (0) for image data, one source pad (1), an internal sink pad (2) that
> represents the source of embedded
sure looks good to me!
>
>>
>>> + data. There are two routes, one from the sink pad to the source, and another
>>> + from the internal sink pad to the source pad. Both streams are always active,
>>> + i.e. there is no need to separately enable the embedded data stream. The
>>> + sensor uses the CSI-2 bus.
>>> +
>>> +- A CSI-2 receiver in the SoC (Receiver). The receiver has a single sink pad
>>> + (pad 0), connected to the bridge, and two source pads (pads 1-2), going to the
>>> + DMA engine. The receiver demultiplexes the incoming streams to the source
>>> + pads.
>>> +
>>> +- DMA Engines in the SoC (DMA Engine), one for each stream. Each DMA engine is
>>> + connected to a single source pad in the receiver.
>>> +
>>> +The sensor, the bridge and the receiver are modeled as V4L2 sub-devices,
>>> +exposed to userspace via /dev/v4l-subdevX device nodes. The DMA engines are
>>> +modeled as V4L2 devices, exposed to userspace via /dev/videoX nodes.
>>> +
>>> +To configure this pipeline, the userspace must take the following steps:
>>> +
>>> +1) Set up media links between entities: connect the sensors to the bridge,
>>> + bridge to the receiver, and the receiver to the DMA engines. This step does
>>> + not differ from normal non-multiplexed media controller setup.
>>> +
>>> +2) Configure routing
>>> +
>>> +.. flat-table:: Camera sensor. There are no configurable routes.
>>> + :header-rows: 1
>>> +
>>> + * - Sink Pad/Stream
>>> + - Source Pad/Stream
>>> + - Routing Flags
>>> + - Comments
>>> + * - 0/0
>>> + - 1/0
>> - 1/0
>> - 0/0
>>> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE
>>> + - Pixel data stream from the sink pad
>>> + * - 2/0
>>> + - 1/1
>> - 0/1
>>> + - V4L2_SUBDEV_ROUTE_FL_ACTIVE
>>> + - Metadata stream from the internal sink pad
>>
>> In latest patch "[PATCH v6 05/15] media: i2c: imx219: Add embedded data
>> support"
>> we have:
>> - 0 -> source pad
>> - 1 -> pixel/image
>> - 2 -> EDATA
>>
>> This is also what you did for ov2740.
>
> Yes, these were leftovers from the CCS example. I've fixed them.
>
>>
>> With that fixed:
>> Reviewed-by Julien Massot <julien.massot@collabora.com>
>
> Thank you!
>
next prev parent reply other threads:[~2024-03-19 14:38 UTC|newest]
Thread overview: 149+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-13 7:24 [PATCH v8 00/38] Generic line based metadata support, internal pads Sakari Ailus
2024-03-13 7:24 ` [PATCH v8 01/38] media: mc: Add INTERNAL pad flag Sakari Ailus
2024-03-14 7:17 ` Tomi Valkeinen
2024-03-19 13:21 ` Sakari Ailus
2024-03-19 22:17 ` Laurent Pinchart
2024-03-20 7:49 ` Sakari Ailus
2024-03-21 17:20 ` Laurent Pinchart
2024-03-28 9:47 ` Sakari Ailus
2024-03-28 10:05 ` Sakari Ailus
2024-03-28 15:25 ` Laurent Pinchart
2024-04-11 7:25 ` Sakari Ailus
2024-03-13 7:24 ` [PATCH v8 02/38] media: Documentation: Add "stream" into glossary Sakari Ailus
2024-03-14 7:18 ` Tomi Valkeinen
2024-03-19 22:20 ` Laurent Pinchart
2024-03-13 7:24 ` [PATCH v8 03/38] media: uapi: Add generic serial metadata mbus formats Sakari Ailus
2024-03-14 7:30 ` Tomi Valkeinen
2024-03-19 13:27 ` Sakari Ailus
2024-03-19 14:20 ` Tomi Valkeinen
2024-03-19 22:33 ` Laurent Pinchart
2024-03-19 23:00 ` Laurent Pinchart
2024-03-20 8:48 ` Sakari Ailus
2024-03-21 17:30 ` Laurent Pinchart
2024-03-22 6:50 ` Tomi Valkeinen
2024-03-25 14:02 ` Sakari Ailus
2024-03-20 8:36 ` Sakari Ailus
2024-03-19 22:59 ` Laurent Pinchart
2024-03-20 16:23 ` Sakari Ailus
2024-03-21 17:38 ` Laurent Pinchart
2024-03-13 7:24 ` [PATCH v8 04/38] media: uapi: Document which mbus format fields are valid for metadata Sakari Ailus
2024-03-14 15:23 ` Tomi Valkeinen
2024-03-19 23:14 ` Laurent Pinchart
2024-03-20 16:49 ` Sakari Ailus
2024-03-13 7:24 ` [PATCH v8 05/38] media: uapi: Add generic 8-bit metadata format definitions Sakari Ailus
2024-03-19 23:37 ` Laurent Pinchart
2024-04-15 14:05 ` Sakari Ailus
2024-03-13 7:24 ` [PATCH v8 06/38] media: v4l: Support line-based metadata capture Sakari Ailus
2024-03-19 23:40 ` Laurent Pinchart
2024-03-13 7:24 ` [PATCH v8 07/38] media: Documentation: Additional streams generally don't harm capture Sakari Ailus
2024-03-19 23:48 ` Laurent Pinchart
2024-04-15 14:27 ` Sakari Ailus
2024-03-13 7:24 ` [PATCH v8 08/38] media: Documentation: Document embedded data guidelines for camera sensors Sakari Ailus
2024-03-15 14:49 ` Julien Massot
2024-03-20 0:03 ` Laurent Pinchart
2024-04-09 11:12 ` Sakari Ailus
2024-03-13 7:24 ` [PATCH v8 09/38] media: Documentation: v4l: Document internal source pads Sakari Ailus
2024-03-15 15:32 ` Julien Massot
2024-03-19 13:47 ` Sakari Ailus
2024-03-19 14:38 ` Julien Massot [this message]
2024-03-20 0:26 ` Laurent Pinchart
2024-04-09 12:14 ` Sakari Ailus
2024-03-13 7:24 ` [PATCH v8 10/38] media: Documentation: Document S_ROUTING behaviour Sakari Ailus
2024-03-15 15:38 ` Julien Massot
2024-03-20 0:33 ` Laurent Pinchart
2024-04-11 8:02 ` Sakari Ailus
2024-03-13 7:24 ` [PATCH v8 11/38] media: v4l: subdev: Add a function to lock two sub-device states, use it Sakari Ailus
2024-03-15 15:42 ` Julien Massot
2024-03-20 0:36 ` Laurent Pinchart
2024-03-13 7:24 ` [PATCH v8 12/38] media: v4l: subdev: Move G_ROUTING handling below S_ROUTING Sakari Ailus
2024-03-15 15:43 ` Julien Massot
2024-03-20 0:37 ` Laurent Pinchart
2024-03-13 7:24 ` [PATCH v8 13/38] media: v4l: subdev: Copy argument back to user also for S_ROUTING Sakari Ailus
2024-03-15 15:50 ` Julien Massot
2024-03-20 0:39 ` Laurent Pinchart
2024-04-11 8:06 ` Sakari Ailus
2024-03-13 7:24 ` [PATCH v8 14/38] media: v4l: subdev: Add len_routes field to struct v4l2_subdev_routing Sakari Ailus
2024-03-20 1:36 ` Laurent Pinchart
2024-04-16 7:09 ` Sakari Ailus
2024-03-13 7:24 ` [PATCH v8 15/38] media: v4l: subdev: Return routes set using S_ROUTING Sakari Ailus
2024-03-20 1:45 ` Laurent Pinchart
2024-04-16 7:12 ` Sakari Ailus
2024-03-13 7:24 ` [PATCH v8 16/38] media: v4l: subdev: Allow a larger number of routes than there's room for Sakari Ailus
2024-03-20 1:53 ` Laurent Pinchart
2024-04-16 8:08 ` Sakari Ailus
2024-03-13 7:24 ` [PATCH v8 17/38] media: v4l: subdev: Add trivial set_routing support Sakari Ailus
2024-03-15 15:51 ` Julien Massot
2024-03-20 1:55 ` Laurent Pinchart
2024-04-01 23:41 ` Laurent Pinchart
2024-04-11 8:13 ` Sakari Ailus
2024-04-12 19:14 ` Laurent Pinchart
2024-04-15 8:10 ` Sakari Ailus
2024-03-13 7:24 ` [PATCH v8 18/38] media: ccs: No need to set streaming to false in power off Sakari Ailus
2024-03-13 9:31 ` Kieran Bingham
2024-03-21 16:35 ` Laurent Pinchart
2024-03-13 7:24 ` [PATCH v8 19/38] media: ccs: Use {enable,disable}_streams operations Sakari Ailus
2024-03-21 16:21 ` Laurent Pinchart
2024-03-13 7:24 ` [PATCH v8 20/38] media: ccs: Track streaming state Sakari Ailus
2024-03-15 15:56 ` Julien Massot
2024-03-21 16:36 ` Laurent Pinchart
2024-03-13 7:24 ` [PATCH v8 21/38] media: ccs: Move ccs_validate_csi_data_format up Sakari Ailus
2024-03-15 15:57 ` Julien Massot
2024-03-21 16:37 ` Laurent Pinchart
2024-03-13 7:25 ` [PATCH v8 22/38] media: ccs: Support frame descriptors Sakari Ailus
2024-03-15 16:02 ` Julien Massot
2024-03-21 16:44 ` Laurent Pinchart
2024-04-11 8:33 ` Sakari Ailus
2024-03-13 7:25 ` [PATCH v8 23/38] media: uapi: ccs: Add media bus code for MIPI CCS embedded data Sakari Ailus
2024-03-15 16:03 ` Julien Massot
2024-03-21 16:49 ` Laurent Pinchart
2024-04-11 9:04 ` Sakari Ailus
2024-04-12 19:07 ` Laurent Pinchart
2024-04-14 10:48 ` Sakari Ailus
2024-04-20 8:07 ` Laurent Pinchart
2024-03-13 7:25 ` [PATCH v8 24/38] media: ccs: Add support for embedded data stream Sakari Ailus
2024-03-13 7:25 ` [PATCH v8 25/38] media: ccs: Remove ccs_get_crop_compose helper Sakari Ailus
2024-03-21 18:05 ` Laurent Pinchart
2024-04-16 7:30 ` Sakari Ailus
2024-03-13 7:25 ` [PATCH v8 26/38] media: ccs: Rely on sub-device state locking Sakari Ailus
2024-03-13 7:25 ` [PATCH v8 27/38] media: ccs: Compute binning configuration from sub-device state Sakari Ailus
2024-03-21 17:57 ` Laurent Pinchart
2024-04-16 8:01 ` Sakari Ailus
2024-03-13 7:25 ` [PATCH v8 28/38] media: ccs: Compute scaling " Sakari Ailus
2024-03-21 17:50 ` Laurent Pinchart
2024-04-16 7:59 ` Sakari Ailus
2024-03-13 7:25 ` [PATCH v8 29/38] media: ccs: Remove which parameter from ccs_propagate Sakari Ailus
2024-03-21 17:39 ` Laurent Pinchart
2024-03-13 7:25 ` [PATCH v8 30/38] media: Documentation: ccs: Document routing Sakari Ailus
2024-03-21 17:43 ` Laurent Pinchart
2024-04-16 7:37 ` Sakari Ailus
2024-03-13 7:25 ` [PATCH v8 31/38] media: uapi: v4l: subdev: Enable streams API Sakari Ailus
2024-03-21 16:51 ` Laurent Pinchart
2024-03-13 7:25 ` [PATCH v8 32/38] media: uapi: Add media bus code for ov2740 embedded data Sakari Ailus
2024-03-15 16:10 ` Julien Massot
2024-03-21 16:54 ` Laurent Pinchart
2024-03-13 7:25 ` [PATCH v8 33/38] media: ov2740: Switch to {enable,disable}_streams Sakari Ailus
2024-03-15 16:13 ` Julien Massot
2024-03-21 16:56 ` Laurent Pinchart
2024-03-13 7:25 ` [PATCH v8 34/38] media: ov2740: Track streaming state Sakari Ailus
2024-03-15 16:13 ` Julien Massot
2024-03-21 16:57 ` Laurent Pinchart
2024-03-13 7:25 ` [PATCH v8 35/38] media: ov2740: Add support for embedded data Sakari Ailus
2024-03-14 7:00 ` Bingbu Cao
2024-03-19 13:13 ` Sakari Ailus
2024-03-14 8:24 ` Julien Massot
2024-03-19 13:18 ` Sakari Ailus
2024-03-21 17:16 ` Laurent Pinchart
2024-04-10 13:18 ` Sakari Ailus
2024-03-13 7:25 ` [PATCH v8 36/38] media: v4l: Add V4L2_SUBDEV_ROUTE_FL_IMMUTABLE sub-device routing flag Sakari Ailus
2024-03-13 7:34 ` Tomi Valkeinen
2024-03-13 7:39 ` Sakari Ailus
2024-03-21 17:03 ` Laurent Pinchart
2024-04-09 13:21 ` Sakari Ailus
2024-04-09 15:21 ` Laurent Pinchart
2024-03-13 7:25 ` [PATCH v8 37/38] media: ccs: Add IMMUTABLE route flag Sakari Ailus
2024-03-15 16:08 ` Julien Massot
2024-03-21 16:59 ` Laurent Pinchart
2024-04-11 9:06 ` Sakari Ailus
2024-03-13 7:25 ` [PATCH v8 38/38] media: ov2740: " Sakari Ailus
2024-03-15 16:14 ` Julien Massot
2024-03-21 17:00 ` Laurent Pinchart
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=b9ff9d89-ac54-46c5-ac9f-8b83fe604dc9@collabora.com \
--to=julien.massot@collabora.com \
--cc=alain.volmat@foss.st.com \
--cc=andrey.konovalov@linaro.org \
--cc=bingbu.cao@intel.com \
--cc=dmitry.perchanov@intel.com \
--cc=hongju.wang@intel.com \
--cc=hverkuil@xs4all.nl \
--cc=jacopo.mondi@ideasonboard.com \
--cc=khai.wen.ng@intel.com \
--cc=laurent.pinchart@ideasonboard.com \
--cc=linux-media@vger.kernel.org \
--cc=sakari.ailus@linux.intel.com \
--cc=tomi.valkeinen@ideasonboard.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).