From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E25B92C85 for ; Thu, 18 Nov 2021 10:50:58 +0000 (UTC) Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DD7FE3E5; Thu, 18 Nov 2021 11:50:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637232657; bh=KoiJLeigquaEguR7NAdplKzGc4pmb6yhRotmxZIQkw0=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=dL6XfuhWX76Nv7iJuu2OF1uMB9HR5K241MJd1KJW76UC8To4ocFWG79LQjeh85gIo ZLQC2cYjA8rO9tWQ+GtznwKnHN3AYVaR+L47egWpb6parVCzydmvgBhYPH5GKLM1xd em6dxmzvPhNFlFjETWfFRl8QVA5sqLKXt9EhWkUw= Date: Thu, 18 Nov 2021 12:50:34 +0200 From: Laurent Pinchart To: Umang Jain Cc: Hans de Goede , Mauro Carvalho Chehab , Sakari Ailus , andrey.i.trufanov@gmail.com, linux-staging@lists.linux.dev, Fabio Aiuto , libcamera-devel@lists.libcamera.org, Nable , Yury Luneff , Andy Shevchenko , linux-media@vger.kernel.org Subject: Re: [libcamera-devel] [RFC] media: Auto exposure/gain support for atomisp / libcamera integration ? Message-ID: References: <20211107175013.355188-1-hdegoede@redhat.com> Precedence: bulk X-Mailing-List: linux-staging@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Hi Umang, On Thu, Nov 18, 2021 at 03:43:53PM +0530, Umang Jain wrote: > On 11/7/21 11:20 PM, Hans de Goede wrote: > > Hi All, > > > > Now that we have the atomisp2 driver running on some devices like > > the Asus T101HA; and with the exposure + gain controls for the ov2680 > > sensor found on that laptop fixed: > > > > https://lore.kernel.org/linux-media/20211107171549.267583-1-hdegoede@redhat.com/ > > > > I believe we need to start thinking about various userspace API > > concerns. Thanks to Mauro's great work on various V4L2 API things > > are starting to work (somewhat) with regular V4L2 apps, but we really > > also need some processing / 3A stuff to make the cameras really > > usable. > > > > The most important thing needed is automatic exposure + gain control, > > ATM this relies on a custom ATOMISP ioctl, but I believe that we can > > just export the controls as regular v4l2 controls on the sensor subdev, > > at least for the ov2680 the exposure is already exported this way > > but it is read-only. Does anyone see any problems with just making > > these normal v4l2 controls on the sensor subdev ? > > > > We can then simulate the custom ATOMISP ioctl through the subdevs, > > or we can just remove it alltogether. > > > > Once we have the controls available this way I think we should write > > a libcamera plugin, which like the older versions of the Rasberry Pi > > plugin (if I've understood things correctly) won't use the MC framework > > for now. I believe we first need to understand the atomisp code better > > before we add MC support (*). But I still believe that an (experimental) > > plugin would be good, both to get something usable so that we can get > > more testers / more people interested in contributing. > > I am trying to understand what 'plugin' here means? Is it a wrapper > pertaining to use libcamera (fined tuned for atomisp) that apps can use? I think it means pipeline handler (and IPA module). > > Another reason is to have another use-case where apps need to support > > libcamera to work properly (like on IPU3 devices) which will hopefully > > motivate people working on apps to put more effort in supporting libcamera > > A valid and solid use case yes! > > > (preferably through the new pipewire libcamera plugin so that things > > will also work in a flatpack sandbox). > > In the longer term plan, I see this happening. I see there SPA support > for libcamera in pipewire (not sure how much usable it is as of today). > Once pipewire has a translating layer of ' request-controls' that can be > mapped to libcamera controls, it would then make good base for > applications for capturing video feeds by sending in requests with > appropriate controls. > > On the flatpak side, I think there will be more? plumbing work since  > you need to use flatpak-portals, rather than just bundling the library > in the manifest (the sandbox cannot determine system's h/w capabilites). > The current org.freedesktop.portal.Camera [1] seems to be geared to use > pipewire so that's a starting point. CV applications are yet another > use-case libcamera will be interested in, I think. Getting the > flatpak-portal support there might get tricky as of 'quirky' requests? > for the camera and pipewire API seems to be limiting to support that > use-case? Not sure how would this work out (and also a distant future), > > [1] https://flatpak.github.io/xdg-desktop-portal/portal-docs.html#gdbus-org.freedesktop.portal.Camera > > > ### > > > > On other thing which I'm wondering about is the need to call S_INPUT to > > select front / back in this list from Mauro: > > > > $ for i in $(ls /dev/video*|sort -k2 -to -n); do echo -n $i:; v4l2-ctl -D -d $i|grep Name; done > > /dev/video0: Name : ATOMISP ISP CAPTURE output > > /dev/video1: Name : ATOMISP ISP VIEWFINDER output > > /dev/video2: Name : ATOMISP ISP PREVIEW output > > /dev/video3: Name : ATOMISP ISP VIDEO output > > /dev/video4: Name : ATOMISP ISP ACC > > /dev/video5: Name : ATOMISP ISP MEMORY input > > /dev/video6: Name : ATOMISP ISP CAPTURE output > > /dev/video7: Name : ATOMISP ISP VIEWFINDER output > > /dev/video8: Name : ATOMISP ISP PREVIEW output > > /dev/video9: Name : ATOMISP ISP VIDEO output > > /dev/video10: Name : ATOMISP ISP ACC > > > > I notice that everything is listed twice, I wonder if we can use /dev/video2 > > with input 0 together with /dev/video8 for input 1, if that is possible then > > we might set a different default input on each. > > > > And maybe also make them /dev/video0 (already done I see) and /dev/video1, > > possibly combined with a module-option to hide the others for now. This > > should make things better for regular apps. OTOH if we go the libcamera > > route then this is not really necessary I guess? > > > > Regards, > > > > Hans > > > > *) I do believe that in the end MC support makes sense at least > > to tie together the -- Regards, Laurent Pinchart