From: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
To: Sakari Ailus <sakari.ailus@iki.fi>
Cc: linux-media@vger.kernel.org, devicetree@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-sunxi@lists.linux.dev, linux-kernel@vger.kernel.org,
linux-staging@lists.linux.dev,
Mauro Carvalho Chehab <mchehab@kernel.org>,
Rob Herring <robh+dt@kernel.org>,
Krzysztof Kozlowski <krzk+dt@kernel.org>,
Chen-Yu Tsai <wens@csie.org>,
Jernej Skrabec <jernej.skrabec@gmail.com>,
Samuel Holland <samuel@sholland.org>,
Sakari Ailus <sakari.ailus@linux.intel.com>,
Hans Verkuil <hans.verkuil@cisco.com>,
Laurent Pinchart <laurent.pinchart@ideasonboard.com>,
Maxime Ripard <mripard@kernel.org>,
Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Subject: Re: [PATCH v3 3/4] staging: media: Add support for the Allwinner A31 ISP
Date: Fri, 20 May 2022 16:57:35 +0200 [thread overview]
Message-ID: <YoesXywA4yzBDSwU@aptenodytes> (raw)
In-Reply-To: <YmsCJicyzf+Bz98y@valkosipuli.retiisi.eu>
[-- Attachment #1.1: Type: text/plain, Size: 5967 bytes --]
Hi Sakari,
On Fri 29 Apr 22, 00:07, Sakari Ailus wrote:
> Hi Paul,
>
> On Thu, Apr 28, 2022 at 04:30:11PM +0200, Paul Kocialkowski wrote:
> > Hi Sakari,
> >
> > On Thu 28 Apr 22, 15:14, Sakari Ailus wrote:
> > > Hi Paul,
> > >
> > > Thanks for the set.
> > >
> > > A few comments below.
> >
> > Thanks a lot for your review!
>
> You're welcome!
>
> ...
>
> > > I understand this is an online ISP. How do you schedule the video buffer
> > > queues? Say, what happens if it's time to set up buffers for a frame and
> > > there's a buffer queued in the parameter queue but not in the image data
> > > queue? Or the other way around?
> >
> > The ISP works in a quite atypical way, with a DMA buffer that is used to
> > hold upcoming parameters (including buffer addresses) and a bit in a "direct"
> > register to schedule the update of the parameters at next vsync.
> >
> > The update (setting the bit) is triggered whenever new parameters are
> > submitted via the params video device or whenever there's a capture buffer
> > available in the capture video device.
> >
> > So you don't particularly need to have one parameter buffer matching a capture
> > buffer, the two can be updated independently. Of course, a capture buffer will
> > only be returned after another buffer becomes active.
>
> This also means it's not possible to associate a capture buffer to a
> parameter buffer by other means than timing --- which is unreliable. The
> request API would allow that but it's not free of issues either.
Yes the request API seems like a good fit for this. Note that the returned
sequence number in dequeued buffers for the capture and meta video devices
should match though, so userspace still has a way to know which captured buffer
used parameters from which meta params buffer.
> Alternatively, I think in this case you could always require the capture
> buffer and grab a parameter buffer when it's available. As ISPs are
> generally requiring device specific control software, this shouldn't be a
> problem really.
I think this is pretty much what happens already.
> I wonder what Laurent thinks.
>
> >
> > I hope this answers your concern!
> >
> > [...]
> >
> > > > +static int sun6i_isp_tables_setup(struct sun6i_isp_device *isp_dev)
> > > > +{
> > > > + struct sun6i_isp_tables *tables = &isp_dev->tables;
> > > > + int ret;
> > > > +
> > > > + /* Sizes are hardcoded for now but actually depend on the platform. */
> > >
> > > Would it be cleaner to have them defined in a platform-specific way, e.g.
> > > in a struct you obtain using device_get_match_data()?
> >
> > Absolutely! I didn't do it at this stage since only one platform is supported
> > but we could just as well introduce a variant structure already for the table
> > sizes.
>
> I think that would be nice already, especially if you know these are going
> to be different. Otherwise macros could be an option.
Understood!
> ...
>
> > > > + ret = v4l2_ctrl_handler_init(&v4l2->ctrl_handler, 0);
> > >
> > > I suppose you intend to add controls later on?
> >
> > I might be wrong but I thought this was necessary to expose sensor controls
> > registered by subdevs that end up attached to this v4l2 device.
> >
> > I doubt the drivers itself will expose controls otherwise.
>
> Now that this is an MC-enabled driver, the subdev controls should be
> accessed through the subdev nodes only. Adding them to the video device's
> control handler is quite hackish and not guaranteed to even work (as e.g.
> multiple subdevs can have the same control).
Yes I was wondering what would happen in that case. I'll drop the ctrls
handling in the next iteration then.
Paul
> ...
>
> > > > +{
> > > > + struct sun6i_isp_device *isp_dev = video_drvdata(file);
> > > > + struct video_device *video_dev = &isp_dev->capture.video_dev;
> > > > + struct mutex *lock = &isp_dev->capture.lock;
> > > > + int ret;
> > > > +
> > > > + if (mutex_lock_interruptible(lock))
> > > > + return -ERESTARTSYS;
> > > > +
> > > > + ret = v4l2_pipeline_pm_get(&video_dev->entity);
> > >
> > > Do you need this?
> > >
> > > Drivers should primarily depend on runtime PM, this is only needed for
> > > compatibility reasons. Instead I'd like to see sensor drivers being moved
> > > to runtime PM.
> >
> > Yes it's still needed to support sensor drivers that don't use rpm yet.
>
> To that I suggested adding runtime PM support for the affected sensors.
> This doesn't seem to get done otherwise. E.g. ipu3-cio2 driver does not
> call s_power() on sensor subdevs.
>
> ...
>
> > > > + ret = video_register_device(video_dev, VFL_TYPE_VIDEO, -1);
> > > > + if (ret) {
> > > > + v4l2_err(v4l2_dev, "failed to register video device: %d\n",
> > > > + ret);
> > > > + goto error_media_entity;
> > > > + }
> > > > +
> > > > + v4l2_info(v4l2_dev, "device %s registered as %s\n", video_dev->name,
> > > > + video_device_node_name(video_dev));
> > >
> > > This isn't really driver specific. I'd drop it.
> >
> > I agree but I see that many drivers are doing it and the information can
> > actually be quite useful at times.
>
> You can get that information using media-ctl -e 'entity name'.
>
> I guess this could be also added to video_register_device() on debug level.
>
> > > > +struct sun6i_isp_params_config_bdnf {
> > > > + __u8 in_dis_min; // 8
> > > > + __u8 in_dis_max; // 10
> > >
> > > Are these default values or something else? Better documentation was in the
> > > TODO.txt file already.
> >
> > Yes that's the default register values, but these comments are and overlook on
> > my side and should be removed.
>
> I'm fine leaving these here. Just wondering. Up to you.
>
> --
> Kind regards,
>
> Sakari Ailus
--
Paul Kocialkowski, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
[-- Attachment #2: Type: text/plain, Size: 176 bytes --]
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2022-05-20 14:59 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-15 15:37 [PATCH v3 0/4] Allwinner A31/A83T MIPI CSI-2 and A31 ISP / ISP Driver Paul Kocialkowski
2022-04-15 15:37 ` [PATCH v3 1/4] dt-bindings: media: Add Allwinner A31 ISP bindings documentation Paul Kocialkowski
2022-04-16 1:59 ` Samuel Holland
2022-04-19 9:45 ` Paul Kocialkowski
2022-04-19 13:48 ` Rob Herring
2022-04-16 22:17 ` Rob Herring
2022-04-15 15:37 ` [PATCH v3 2/4] dt-bindings: media: sun6i-a31-csi: Add ISP output port Paul Kocialkowski
2022-04-19 13:08 ` Rob Herring
2022-04-19 13:35 ` Paul Kocialkowski
2022-04-15 15:37 ` [PATCH v3 3/4] staging: media: Add support for the Allwinner A31 ISP Paul Kocialkowski
2022-04-20 7:42 ` Dan Carpenter
2022-04-20 7:50 ` Paul Kocialkowski
2022-04-20 8:51 ` Dan Carpenter
2022-04-20 8:53 ` Dan Carpenter
2022-04-28 12:14 ` Sakari Ailus
2022-04-28 14:30 ` Paul Kocialkowski
2022-04-28 21:07 ` Sakari Ailus
2022-05-20 14:57 ` Paul Kocialkowski [this message]
2022-05-22 17:34 ` Laurent Pinchart
2022-05-23 12:51 ` Paul Kocialkowski
2022-05-23 13:27 ` Laurent Pinchart
2022-05-24 12:37 ` Paul Kocialkowski
2022-04-15 15:37 ` [PATCH v3 4/4] MAINTAINERS: Add entry for the Allwinner A31 ISP driver Paul Kocialkowski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YoesXywA4yzBDSwU@aptenodytes \
--to=paul.kocialkowski@bootlin.com \
--cc=devicetree@vger.kernel.org \
--cc=hans.verkuil@cisco.com \
--cc=jernej.skrabec@gmail.com \
--cc=krzk+dt@kernel.org \
--cc=laurent.pinchart@ideasonboard.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=linux-staging@lists.linux.dev \
--cc=linux-sunxi@lists.linux.dev \
--cc=mchehab@kernel.org \
--cc=mripard@kernel.org \
--cc=robh+dt@kernel.org \
--cc=sakari.ailus@iki.fi \
--cc=sakari.ailus@linux.intel.com \
--cc=samuel@sholland.org \
--cc=thomas.petazzoni@bootlin.com \
--cc=wens@csie.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).