Re: [PATCH v3 0/3] media: pci: intel: ivsc: Add driver of Intel Visual Sensing Controller(IVSC)

[Sakari Ailus <sakari.ailus@linux.intel.com>]

Hi Wentong,

On Mon, Mar 27, 2023 at 07:33:48AM +0000, Wu, Wentong wrote:
> 
> 
> > -----Original Message-----
> > From: Sakari Ailus <sakari.ailus@linux.intel.com>
> > Sent: Monday, March 27, 2023 3:21 PM
> > 
> > Hi Wentong,
> > 
> > On Mon, Mar 27, 2023 at 02:23:05PM +0800, Wentong Wu wrote:
> > > Intel Visual Sensing Controller (IVSC), codenamed "Clover Falls", is a
> > > companion chip designed to provide secure and low power vision
> > > capability to IA platforms. IVSC is available in existing commercial
> > > platforms from multiple OEMs.
> > >
> > > The primary use case of IVSC is to bring in context awareness. IVSC
> > > interfaces directly with the platform main camera sensor via a CSI-2
> > > link and processes the image data with the embedded AI engine. The
> > > detected events are sent over I2C to ISH (Intel Sensor Hub) for
> > > additional data fusion from multiple sensors. The fusion results are
> > > used to implement advanced use cases like:
> > >  - Face detection to unlock screen
> > >  - Detect user presence to manage backlight setting or waking up
> > > system
> > >
> > > Since the Image Processing Unit(IPU) used on the host processor needs
> > > to configure the CSI-2 link in normal camera usages, the CSI-2 link
> > > and camera sensor can only be used in mutually-exclusive ways by host
> > > IPU and IVSC. By default the IVSC owns the CSI-2 link and camera
> > > sensor. The IPU driver can take ownership of the CSI-2 link and camera
> > > sensor using interfaces exported via v4l2 sub-device.
> > >
> > > Switching ownership requires an interface with two different hardware
> > > modules inside IVSC. The software interface to these modules is via
> > > Intel MEI (The Intel Management Engine) commands. These two hardware
> > > modules have two different MEI UUIDs to enumerate. These hardware
> > modules are:
> > >  - ACE (Algorithm Context Engine): This module is for algorithm
> > > computing when IVSC owns camera sensor. Also ACE module controls
> > > camera sensor's ownership. This hardware module is used to set ownership of
> > camera sensor.
> > >  - CSI (Camera Serial Interface): This module is used to route camera
> > > sensor data either to IVSC or to host for IPU driver and application.
> > >
> > > IVSC also provides a privacy mode. When privacy mode is turned on,
> > > camera sensor can't be used. This means that both ACE and host IPU
> > > can't get image data. And when this mode is turned on, users are
> > > informed via
> > > v4l2 control API.
> > >
> > > In summary, to acquire ownership of camera by IPU driver, first ACE
> > > module needs to be informed of ownership and then to setup MIPI CSI-2
> > > link for the camera sensor and IPU.
> > >
> > > Implementation:
> > > There are two different drivers to handle ACE and CSI hardware modules
> > > inside IVSC.
> > >  - ivsc_csi: MEI client driver to send commands and receive
> > > notifications from CSI module.
> > >  - ivsc_ace: MEI client driver to send commands and get status from
> > > ACE module.
> > > Interface is exposed via v4l2 sub-devcie APIs to acquire and release
> > > camera sensor and CSI-2 link.
> > 
> > Thanks for the update.
> > 
> > Could you elaborate the decision of keeping the csi_bridge entirely separate
> > from the cio2_bridge (to be turned to ipu_bridge first)? Both are doing
> > essentially the same and using the same data structures, aren't they?
> 
> yes, they're doing same thing to bridge the software nodes needed by
> v4l2, but they have different type devices(pci and mei_client device) and
> dependency. And they have same SSDB definition in DSDT, so the structures
> are almost same.

If there are differences, what are they?

What comes to cio2_bridge, the fact that it's related to a PCI device
doesn't seem to matter after initialisation so it could as well work with
struct device.

> 
> I have no idea what the ipu bridge would be like, but IVSC csi bridge can
> be configured via kconfig to enable/disable.

Please work out the details with Bingbu.

And please do wrap your lines at 74 or so.

-- 
Kind regards,

Sakari Ailus