Re: [RFC 3/7] media: uapi: v4l: Document source routes

[Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>]

On 08/05/2023 19:26, Sakari Ailus wrote:
> Moi,
> 
> On Mon, May 08, 2023 at 01:33:24PM +0300, Tomi Valkeinen wrote:
>> On 06/05/2023 00:52, Sakari Ailus wrote:
>>> Document how internal pads are used on source routes.
>>>
>>> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
>>> ---
>>>    .../userspace-api/media/v4l/dev-subdev.rst     | 18 ++++++++++++++++++
>>>    .../media/v4l/vidioc-subdev-g-routing.rst      |  5 +++++
>>>    include/uapi/linux/v4l2-subdev.h               |  6 +++++-
>>>    3 files changed, 28 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
>>> index a4f1df7093e8..395e06d2f0f2 100644
>>> --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
>>> +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
>>> @@ -551,6 +551,24 @@ A stream at a specific point in the media pipeline is identified by the
>>>    sub-device and a (pad, stream) pair. For sub-devices that do not support
>>>    multiplexed streams the 'stream' field is always 0.
>>> +.. _v4l2-subdev-source-routes:
>>> +
>>> +Source routes
>>> +^^^^^^^^^^^^^
>>> +
>>> +Cases where a single sub-device pad is a source of multiple streams are special
>>> +as there is no sink pad for such a route. In those cases, an internal pad is
>>> +used as the sink pad. Such pads have the :ref:`MEDIA_PAD_FL_INTERNAL_SOURCE
>>> +<MEDIA-PAD-FL-INTERNAL-SOURCE>` flag set.
>>> +
>>> +Internal pads have all the properties of a sink pad in such case, including
>>> +formats and selections. The format in this case is the source format of the
>>> +stream. An internal pad always has a single stream only (0).
>>> +
>>> +Generally source routes are not modifiable but they can be activated and
>>> +inactivated using the :ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE
>>> +<v4l2-subdev-routing-flags>` flag, depending on driver capabilities.
>>> +
>>
>> I find the above chapter a bit difficult to read, but I think we need to
>> conclude on the internal-pad vs internal-source-pad discussion first.
>> However, one point/question:
>>
>> You write that there's only one stream in an internal pad. I wonder if
>> that's a good or a necessary limitation... Do you see that allowing multiple
>> streams brings extra complexity? It's still up to each driver to decide how
>> many streams they support (most would just support a single one), so each
>> driver can easily enforce that limit.
> 
> Good question. As we don't seem to have a tangible reason to allow it, I'd
> deny it until there's a use case.
> 
> Or did you have a use case in mind?
> 
> I thought indicating which streams will be bound together (i.e. either are
> all disabled or enabled) could be one, but I'm not sure we need that at the
> moment at least.

I don't have nothing concrete in mind. Maybe a TPG which also generates 
some kind of metadata. But that could be implemented as two internal pads.

>> I'm fine with starting with only single-stream support, but if we later need
>> to loosen that restriction, I wonder if it'll be difficult if we have
>> specified that there can be only one stream. I.e. will the drivers and the
>> userspace depend on that limit.
> 
> We can always allow what wasn't allowed previously so that's a non-issue,
> really. But it needs to bring new functionality, otherwise there's no
> reason to do that.

It's not always that easy. If the drivers and the userspace expect that 
there's a single route with ID 0, and then with a new kernel there are 
more streams or the single stream is ID 1, things could go wrong.

  Tomi