Re: [ANN] Media Summit at ELCE Dublin: Request for Topics

[Dave Stevenson <dave.stevenson@raspberrypi.com>]

Hi Hans and Jacopo

On Fri, 20 May 2022 at 09:22, Jacopo Mondi <jacopo@jmondi.org> wrote:
>
> Hello Hans,
>
> On Fri, May 06, 2022 at 03:20:09PM +0200, Hans Verkuil wrote:
> > Hi all,
> >
> > Since countries are opening up again and travel is (at least for now!) a lot easier,
> > I am considering a media summit during the ELCE in Dublin (Sep 13-16).
> >
> > See here for more details about the conference:
> >
> > https://events.linuxfoundation.org/open-source-summit-europe/
> >
> > Of course, this only makes sense if there is something to talk about. So please reply
> > with any suggestions for topics!
> >
> > Also please let me know if you would expect to be at such a media summit in person.
> > If only a few people would be there, then there isn't much point to this.
> >
> >
> > I have two topics:
> >
> > 1) Discussion of the media subsystem development process: any bottlenecks, any ideas
> >    for improvements?
> >
> > 2) I can give a presentation on the work I've done in the CTA-861 standard (used by
> >    HDMI) and the edid-decode utility.
> >
> > I'd like to make a decision on whether or not it is worthwhile to do this in a week
> > or two. If we wait too long it might be difficult to get a room for the summit.
>
> There are a few topics around image sensor support, especially
> relevant for RAW sensor drivers
>
> - Recently Dave posted an question about how to represent additional
>   processing stages that happens on the sensor side, mostly
>   additional subsampling/cropping that happen between the analogue cropping
>   on the full pixel array and the final image sent on the wire.
>
>   https://lore.kernel.org/linux-media/CAPY8ntA06L1Xsph79sv9t7MiDSNeSO2vADevuXZdXQdhWpSmow@mail.gmail.com/
>
>   Dave made a good introduction of the issue his email which got
>   largely unanswered.
>
>   The issue is particularly relevant for RAW sensors, where applying
>   subsampling has an impact on the sensor's sensitivity and requires
>   to adjust the gains and exposure accordingly.
>
>   The V4L2 selection API falls short on this and the only other
>   solution I am aware of is registering additional subdevices as the
>   CCS driver does.

If you want to throw in some more image sensor related issues, I can
think of a couple more areas that could do with some consensus on how
to implement:

On-sensor temperature reporting. Thread started by Benjamin at
https://lore.kernel.org/linux-media/20220415111845.27130-3-benjamin.mugnier@foss.st.com/
, but no resolution over using hwmon API or V4L2 control. If hwmon
then we need Media Controller framework to tie the sensor and thermal
device together.
It's recently been queried for IMX477 with the Pi
(https://github.com/raspberrypi/libcamera/issues/19), but it will
apply to many sensors.

Synchronising sensors for stereoscopic operation (trying to avoid the
master / slave terminonlogy). How should that be configured? Allowing
configuration from userspace would allow sensors to be operated
independently which can be useful for test purposes, or should it be
enforced from DT / ACPI? Do we set a default configuration for each
sensor from DT/ACPI and then allow userspace to override should it
wish?

Controlling sensor GPIO outputs for things such as flash triggers,
vsync, frame start/end, exposure start/end, etc.
There is a huge range of features available so do we have any hope of
standardising some of it, or do we end up hiding these away in the
drivers with custom DT bindings to configure them? If we accept that
there will be variation, can we vaguely standardise what those
bindings look like? Or should these be V4L2 controls as things like
pulse widths may want to be adjusted by userspace?

Lens drivers. Each driver will have a "useful" range which is
effectively dictated by the overall module. Should that be defined via
DT as it is a feature of the platform, or leave the driver totally
generic and expect userspace to do something sensible?
In the case of simple systems without libcamera (this is Video 4 Linux
2, not Video 4 Libcamera 2), do we set default for
V4L2_CID_FOCUS_ABSOLUTE to a sensible hyperfocal distance, and can
that again be defined in DT as it is defining the hardware?

Thanks.
  Dave

> - Probably less relevant for a media summit, but we recently got a few
>   series trying to reconcile handling of regulators, gpios and clocks
>   on OF and ACPI platforms all of them doing the usual "similar but
>   slightly different" thing:
>
>   https://lore.kernel.org/linux-media/20220425061022.1569480-1-paul.elder@ideasonboard.com/
>   https://lore.kernel.org/linux-media/20220329090133.338073-1-jacopo@jmondi.org/
>   https://lore.kernel.org/linux-media/20220509143226.531117-1-foss+kernel@0leil.net/
>   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0c2c7a1e0d69221b9d489bfd8cf53262d6f82446
>
>   ACPI and OF handles clocks slightly differently, and it is not clear
>   to me if ACPI based platform need explicit handling of
>   clocks/regulator or ACPI does "the right thing" by itself (I'm
>   afraid the answer is actually "it depends"). I'm ACPI illiterate
>   so I cannot propose anything meaningful but if anyone is interested
>   in discussing this further this might be a good time to do so ?
>
> Let me know if those points might be of any interest and I can try to
> prepare something about them.
>
> Thanks
>    j
>
> >
> > Regards,
> >
> >       Hans