RE: [RFC PATCH v2 2/6] bus/cdx: add the cdx bus driver

["Gupta, Nipun" <Nipun.Gupta@amd.com>]

[AMD Official Use Only - General]



> -----Original Message-----
> From: Greg KH <gregkh@linuxfoundation.org>
> Sent: Monday, August 22, 2022 7:00 PM
> To: Gupta, Nipun <Nipun.Gupta@amd.com>
> Cc: robh+dt@kernel.org; krzysztof.kozlowski+dt@linaro.org;
> rafael@kernel.org; eric.auger@redhat.com; alex.williamson@redhat.com;
> cohuck@redhat.com; Gupta, Puneet (DCG-ENG)
> <puneet.gupta@amd.com>; song.bao.hua@hisilicon.com;
> mchehab+huawei@kernel.org; maz@kernel.org; f.fainelli@gmail.com;
> jeffrey.l.hugo@gmail.com; saravanak@google.com;
> Michael.Srba@seznam.cz; mani@kernel.org; yishaih@nvidia.com;
> jgg@ziepe.ca; linux-kernel@vger.kernel.org; devicetree@vger.kernel.org;
> kvm@vger.kernel.org; okaya@kernel.org; Anand, Harpreet
> <harpreet.anand@amd.com>; Agarwal, Nikhil <nikhil.agarwal@amd.com>;
> Simek, Michal <michal.simek@amd.com>; git (AMD-Xilinx) <git@amd.com>;
> jgg@nvidia.com; Robin Murphy <robin.murphy@arm.com>
> Subject: Re: [RFC PATCH v2 2/6] bus/cdx: add the cdx bus driver
> 
> [CAUTION: External Email]
> 
> On Mon, Aug 22, 2022 at 01:21:47PM +0000, Gupta, Nipun wrote:
> > [AMD Official Use Only - General]
> >
> >
> >
> > > -----Original Message-----
> > > From: Greg KH <gregkh@linuxfoundation.org>
> > > Sent: Wednesday, August 17, 2022 9:03 PM
> > > To: Gupta, Nipun <Nipun.Gupta@amd.com>
> > > Cc: robh+dt@kernel.org; krzysztof.kozlowski+dt@linaro.org;
> rafael@kernel.org;
> > > eric.auger@redhat.com; alex.williamson@redhat.com;
> cohuck@redhat.com;
> > > Gupta, Puneet (DCG-ENG) <puneet.gupta@amd.com>;
> > > song.bao.hua@hisilicon.com; mchehab+huawei@kernel.org;
> maz@kernel.org;
> > > f.fainelli@gmail.com; jeffrey.l.hugo@gmail.com; saravanak@google.com;
> > > Michael.Srba@seznam.cz; mani@kernel.org; yishaih@nvidia.com;
> > > jgg@ziepe.ca; linux-kernel@vger.kernel.org;
> devicetree@vger.kernel.org;
> > > kvm@vger.kernel.org; okaya@kernel.org; Anand, Harpreet
> > > <harpreet.anand@amd.com>; Agarwal, Nikhil
> <nikhil.agarwal@amd.com>;
> > > Simek, Michal <michal.simek@amd.com>; git (AMD-Xilinx)
> <git@amd.com>
> > > Subject: Re: [RFC PATCH v2 2/6] bus/cdx: add the cdx bus driver
> > >
> > > [CAUTION: External Email]
> > >
> > > On Wed, Aug 17, 2022 at 08:35:38PM +0530, Nipun Gupta wrote:
> > > > CDX bus driver manages the scanning and populating FPGA
> > > > based devices present on the CDX bus.
> > > >
> > > > The bus driver sets up the basic infrastructure and fetches
> > > > the device related information from the firmware. These
> > > > devices are registered as platform devices.
> > >
> > > Ick, why?  These aren't platform devices, they are CDX devices.  Make
> > > them real devices here, don't abuse the platform device interface for
> > > things that are not actually on the platform bus.
> >
> > CDX is a virtual bus (FW based) which discovers FPGA based platform
> > devices based on communication with FW.
> 
> virtual busses are fine to have as a real bus in the kernel, no problem
> there.
> 
> > These devices are essentially platform devices as these are memory
> mapped
> > on system bus, but having a property that they are dynamically discovered
> > via FW and are rescannable.
> 
> If they are dynamically discoverable and rescannable, then great, it's a
> bus in the kernel and NOT a platform device.
> 
> > I think your point is correct in the sense that CDX bus is not an actual bus,
> > but a FW based mechanism to discover FPGA based platform devices.
> >
> > Can you kindly suggest us if we should have the CDX platform device
> scanning
> > code as a CDX bus in "drivers/bus/" folder OR have it in "drivers/fpga/" or
> > "drivers/platform/" or which other suitable location?
> 
> drivers/cdx/ ?

I agree that the approach, which is correct should be used, just wanted
to reconfirm as adding a new bus would lead to change in other areas
like SMMU, MSI and VFIO too and we will need vfio-cdx interface for CDX
bus, similar to vfio-platform.

On another mail Robin and Jason have suggested to use OF_DYNAMIC.
Can you please also let us know in case that is a suited option where we
use OF_DYNAMIC and have our code as part of "drivers/fpga" instead of
using the bus. (something like pseries CPU hotplug is using to add new
CPU platform devices on runtime:
https://elixir.bootlin.com/linux/v5.19.3/source/arch/powerpc/platforms/pseries/hotplug-cpu.c#L534).
We can share the RFC in case you are interested in looking at code flow
using the of_dynamic approach.

The reason we were inclined towards the platform bus is due to
existing SMMU. MSI and VFIO support available for platform, though
would work on the bus if adding to the bus is correct thing to move
ahead.

Robin/Jason,
Your comments are also kindly welcomed regarding the suitable
approach.

Thanks,
Nipun

> 
> thanks,
> 
> greg k-h