RE: [RFC 01/20] ice: Initialize and register multi-function device to provide RDMA

From: "Ertman, David M" <david.m.ertman@intel.com>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
	"Nguyen, Anthony L" <anthony.l.nguyen@intel.com>,
	"Kirsher, Jeffrey T" <jeffrey.t.kirsher@intel.com>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	"linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
	"dledford@redhat.com" <dledford@redhat.com>,
	"Ismail, Mustafa" <mustafa.ismail@intel.com>,
	"Patil, Kiran" <kiran.patil@intel.com>
Subject: RE: [RFC 01/20] ice: Initialize and register multi-function device to provide RDMA
Date: Wed, 23 Oct 2019 17:55:38 +0000	[thread overview]
Message-ID: <2B0E3F215D1AB84DA946C8BEE234CCC97B2E0C84@ORSMSX101.amr.corp.intel.com> (raw)
In-Reply-To: <20191023174448.GP23952@ziepe.ca>

> -----Original Message-----
> From: Jason Gunthorpe [mailto:jgg@ziepe.ca]
> Sent: Wednesday, October 23, 2019 10:45 AM
> To: Ertman, David M <david.m.ertman@intel.com>
> Cc: gregkh@linuxfoundation.org; Nguyen, Anthony L
> <anthony.l.nguyen@intel.com>; Kirsher, Jeffrey T
> <jeffrey.t.kirsher@intel.com>; netdev@vger.kernel.org; linux-
> rdma@vger.kernel.org; dledford@redhat.com
> Subject: Re: [RFC 01/20] ice: Initialize and register multi-function device to
> provide RDMA
> 
> On Fri, Sep 27, 2019 at 06:03:51PM +0000, Ertman, David M wrote:
> > > From: gregkh@linuxfoundation.org [mailto:gregkh@linuxfoundation.org]
> > > Sent: Thursday, September 26, 2019 10:13 PM
> > > To: Nguyen, Anthony L <anthony.l.nguyen@intel.com>
> > > Cc: Kirsher, Jeffrey T <jeffrey.t.kirsher@intel.com>;
> > > jgg@mellanox.com; netdev@vger.kernel.org;
> > > linux-rdma@vger.kernel.org; dledford@redhat.com; Ertman, David M
> > > <david.m.ertman@intel.com>
> > > Subject: Re: [RFC 01/20] ice: Initialize and register multi-function
> > > device to provide RDMA
> > >
> > > On Thu, Sep 26, 2019 at 11:39:22PM +0000, Nguyen, Anthony L wrote:
> > > > On Thu, 2019-09-26 at 20:05 +0200, Greg KH wrote:
> > > > > On Thu, Sep 26, 2019 at 09:45:00AM -0700, Jeff Kirsher wrote:
> > > > > > From: Tony Nguyen <anthony.l.nguyen@intel.com>
> > > > > >
> > > > > > The RDMA block does not advertise on the PCI bus or any other bus.
> > > > >
> > > > > Huh?  How do you "know" where it is then?  Isn't is usually
> > > > > assigned to a PCI device?
> > > >
> > > > The RDMA block does not have its own PCI function so it must
> > > > register and interact with the ice driver.
> > >
> > > So the "ice driver" is the real thing controlling the pci device?
> > > How does it "know" about the RDMA block?
> > >
> > > thanks,
> > >
> > > greg k-h
> >
> > The ICE driver loads and registers to control the PCI device.  It then
> > creates an MFD device with the name 'ice_rdma'. The device data
> > provided to the MFD subsystem by the ICE driver is the struct
> > iidc_peer_dev which contains all of the relevant information that the
> > IRDMA peer will need to access this PF's IIDC API callbacks
> >
> > The IRDMA driver loads as a software only driver, and then registers a
> > MFD function driver that takes ownership of MFD devices named 'ice_rdma'.
> > This causes the platform bus to perform a matching between ICE's MFD
> > device and IRDMA's driver.  Then the patform bus will call the IRDMA's
> > IIDC probe function.  This probe provides the device data to IRDMA.
> 
> Did any resolution happen here? Dave, do you know what to do to get Greg's
> approval?
> 
> Jason

This was the last communication that I saw on this topic.  I was taking Greg's silence as
"Oh ok, that works" :)  I hope I was not being too optimistic!

If there is any outstanding issue I am not aware of it, but please let me know if I am 
out of the loop!

Greg, if you have any other concerns or questions I would be happy to address them! 

-Dave E