Re: [PATCH v3 26/31] adv748x: csi2: add internal routing configuration

[Jacopo Mondi <jacopo@jmondi.org>]

[-- Attachment #1: Type: text/plain, Size: 5609 bytes --]

Hi Luca,

On Fri, Mar 22, 2019 at 05:52:15PM +0100, Luca Ceresoli wrote:
> Hi,
>
> thanks for the follow-up.
>
> On 20/03/19 18:14, Jacopo Mondi wrote:
> >>>> This is probably the wrong patch to use an example, as this one is for
> >>>> a multiplexed interface, where there is no need to go through an
> >>>> s_stream() for the two CSI-2 endpoints, but as you pointed out in our
> >>>> brief offline chat, the AFE->TX routing example for this very device
> >>>> is a good one: if we change the analogue source that is internally
> >>>> routed to the CSI-2 output of the adv748x, do we need to s_stream(1)
> >>>> the now routed entity and s_stream(0) on the not not-anymore-routed
> >>>> one?
> >>>>
> >>>> My gut feeling is that this is up to userspace, as it should know
> >>>> what are the requirements of the devices in the system, but this mean
> >>>> going through an s_stream(0)/s_stream(1) sequence on the video device,
> >>>> and that would interrupt the streaming for sure.
> >>>>
> >>>> At the same time, I don't feel too much at ease with the idea of
> >>>> s_routing calling s_stream on the entity' remote subdevices, as this
> >>>> would skip the link format validation that media_pipeline_start()
> >>>> performs.
> >>>
> >>> The link validation must be done in this case as well, it may not be
> >>> simply skipped.
> >>
> >> Agreed.
> >>
> >> The routing VS pipeline validation point is a very important one. The
> >> current proposed workflow is:
> >>
> >>  1. the pipeline is validated as a whole, having knowledge of all the
> >>     entities
> >
> > let me specify this to avoid confusions:
> >      "all the entities -with an active route in the pipeline-"
> >
> >>  2. streaming is started
> >>  3. s_routing is called on an entity (not on the pipeline!)
> >>
> >> Now the s_routing function in the entity driver is not in a good
> >> position to validate the candidate future pipeline as a whole.
> >>
> >> Naively I'd say there are two possible solutions:
> >>
> >>  1. the s_routing reaches the pipeline first, then the new pipeline is
> >>     computed and verified, and if verification succeeds it is applied
> >>  2. a partial pipeline verification mechanism is added, so the entity
> >>     receiving a s_routing request to e.g. change the sink pad can invoke
> >>     a verification on the part of pipeline that is about to be
> >>     activated, and if verification succeeds it is applied
> >>
> >> Somehow I suspect neither is trivial...
> >
> > I would say it is not, but if you have such a device that does not
> > require going through a s_stream(0)/s_stream(1) cycle and all the
> > associated re-negotiation and validations, it seems to me nothing
> > prevents you from handling this in the driver implementation. Maybe it
> > won't look that great, but this seems to be quite a custom design that
> > requires all input sources to be linked to your sink pads, their
> > format validated all at the same time, power, stream activation and
> > internal mux configuration controlled by s_routing. Am I wrong or
> > nothing in this series would prevent your from doing this?
>
> You're right, nothing prevents me from doing a custom hack for my custom
> design. It's what I'm doing right now. My concern is whether the
> framework will evolve to allow modifying a running pipeline without
> custom hacks.
>
> > tl;dr: I would not make this something the framework should be
> > concerned about, as there's nothing preventing you from
> > implementing support for such a use case. But again, without a real
> > example we can only guess, and I might be overlooking the issue or
> > missing some relevant detail for sure.
>
> I'm a bit surprised in observing that my use case looks so strange,
> perhaps yours is so different that we don't quite understand each other.
> So below is an example of what I have in mind. Can you explain your use
> case too?

I'm mostly interested in this series as it allows me to add data lane
negotiation at run time. In my case, there are no stream continuity
constraints, but I get your point here.
>
>
> Here's a use case. Consider a product that takes 3 camera inputs,
> selects one of them and produces a continuous video stream with the
> camera image and an OSD on top of it. The selected camera can be changed
> at any time (e.g. upon user selection).
>
>                   OSD FB ---.
>                             |
>             .--------.      V
> Camera 0 -->|        |   .-----.
> Camera 1 -->|  MUX   |-->| OSD |--> DMA --> /dev/video0
> Camera 2 -->|        |   `-----'
>             `--------'
>
> A prerequisite is obviously that each piece of hardware and software
> involved is able to cope with a sudden stream change. Perhaps not that
> common, but no rocket science.
>
> It looks to me that each of these pieces can be modeled as an entity and
> the s_routing API is a perfect fit for the mux block. Am I wrong?
>

Personally, I would say that your muxer driver, if able to switch
source without requiring a pipeline restart, should handle it
internally. There are specific details that the mux should be able to
handle, and we're still pretty vague on the details. As a start, which
is the bus type your sources connects to the muxer with? Parallel?
CSI-2? How is the video stream multiplexed? with CSI-2 VC? Do you need
to control your source power during inactive period? I'm sure there
are more questions... I agree a partial pipeline restart might be
something to consider, but at the same time I think this is not
something strictly required to get this series merged, isn't it?

Thanks
  j


> Thanks,
> --
> Luca

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]