RE: [RFC 01/20] ice: Initialize and register multi-function device to provide RDMA

["Ertman, David M" <david.m.ertman@intel.com>]

> -----Original Message-----
> From: gregkh@linuxfoundation.org [mailto:gregkh@linuxfoundation.org]
> Sent: Thursday, October 24, 2019 6:31 PM
> To: Ertman, David M <david.m.ertman@intel.com>
> Cc: Jason Gunthorpe <jgg@ziepe.ca>; Nguyen, Anthony L
> <anthony.l.nguyen@intel.com>; Kirsher, Jeffrey T
> <jeffrey.t.kirsher@intel.com>; netdev@vger.kernel.org; linux-
> rdma@vger.kernel.org; dledford@redhat.com; Ismail, Mustafa
> <mustafa.ismail@intel.com>; Patil, Kiran <kiran.patil@intel.com>;
> lee.jones@linaro.org
> Subject: Re: [RFC 01/20] ice: Initialize and register multi-function device to
> provide RDMA
> 
> On Thu, Oct 24, 2019 at 10:25:36PM +0000, Ertman, David M wrote:
> > The direct access of the platform bus was unacceptable, and the MFD
> > sub-system was suggested by Greg as the solution.  The MFD sub-system
> > uses the platform bus in the background as a base to perform its
> > functions, since it is a purely software construct that is handy and
> > fulfills its needs.  The question then is:  If the MFD sub- system is
> > using the platform bus for all of its background functionality, is the platform
> bus really only for platform devices?
> 
> Yes, how many times do I have to keep saying this?
> 
> The platform bus should ONLY be used for devices that are actually platform
> devices and can not be discovered any other way and are not on any other type
> of bus.
> 
> If you try to add platform devices for a PCI device, I am going to continue to
> complain.  I keep saying this and am getting tired.
> 
> Now yes, MFD does do "fun" things here, and that should probably be fixed up
> one of these days.  But I still don't see why a real bus would not work for you.
> 
> greg "platform devices are dead, long live the platform device" k-h

> -----Original Message-----
> From: gregkh@linuxfoundation.org [mailto:gregkh@linuxfoundation.org]
> Sent: Thursday, October 24, 2019 6:31 PM
> To: Ertman, David M <david.m.ertman@intel.com>
> Cc: Jason Gunthorpe <jgg@ziepe.ca>; Nguyen, Anthony L
> <anthony.l.nguyen@intel.com>; Kirsher, Jeffrey T
> <jeffrey.t.kirsher@intel.com>; netdev@vger.kernel.org; linux-
> rdma@vger.kernel.org; dledford@redhat.com; Ismail, Mustafa
> <mustafa.ismail@intel.com>; Patil, Kiran <kiran.patil@intel.com>;
> lee.jones@linaro.org
> Subject: Re: [RFC 01/20] ice: Initialize and register multi-function device to
> provide RDMA
> 
> On Thu, Oct 24, 2019 at 10:25:36PM +0000, Ertman, David M wrote:
> > The direct access of the platform bus was unacceptable, and the MFD
> > sub-system was suggested by Greg as the solution.  The MFD sub-system
> > uses the platform bus in the background as a base to perform its
> > functions, since it is a purely software construct that is handy and
> > fulfills its needs.  The question then is:  If the MFD sub- system is
> > using the platform bus for all of its background functionality, is the platform
> bus really only for platform devices?
> 
> Yes, how many times do I have to keep saying this?
> 
> The platform bus should ONLY be used for devices that are actually platform
> devices and can not be discovered any other way and are not on any other type
> of bus.
> 
> If you try to add platform devices for a PCI device, I am going to continue to
> complain.  I keep saying this and am getting tired.
> 
> Now yes, MFD does do "fun" things here, and that should probably be fixed up
> one of these days.  But I still don't see why a real bus would not work for you.
> 
> greg "platform devices are dead, long live the platform device" k-h

I'm sorry, the last thing I want to do is to annoy you! I just need to
figure out where to go from here.  Please, don't take anything I say as
argumentative.

I don't understand what you mean by "a real bus".  The irdma driver does
not have access to any physical bus.  It utilizes resources provided by
the PCI LAN drivers, but to receive those resources it needs a mechanism
to "hook up" with the PCI drivers.  The only way it has to locate them
is to register a driver function with a software based bus of some kind
and have the bus match it up to a compatible entity to achieve that hook up.

The PCI LAN driver has a function that controls the PCI hardware, and then
we want to present an entity for the RDMA driver to connect to.

To move forward, we are thinking of the following design proposal:

We could add a new module to the kernel named generic_bus.ko.  This would
create a new generic software bus and a set of APIs that would allow for
adding and removing simple generic virtual devices and drivers, not as
a MFD cell or a platform device. The power management events would also
be handled by the generic_bus infrastructure (suspend, resume, shutdown).
We would use this for matching up by having the irdma driver register
with this generic bus and hook to virtual devices that were added from
different PCI LAN drivers.

Pros:
1) This would avoid us attaching anything to the platform bus
2) Avoid having each PCI LAN driver creating its own software bus
3) Provide a common matching ground for generic devices and drivers that
eliminates problems caused by load order (all dependent on generic_bus.ko)
4) Usable by any other entity that wants a lightweight matching system
or information exchange mechanism

Cons:
1) Duplicates part of the platform bus functionality
2) Adds a new software bus to the kernel architecture

Is this path forward acceptable?

Thanks for any clarification/guidance you can provide!

-Dave E