RE: [PATCH net-next v2 4/7] devlink: allow subports on devlink PCI ports

[Parav Pandit <parav@mellanox.com>]

> -----Original Message-----
> From: Parav Pandit
> Sent: Thursday, March 14, 2019 8:29 PM
> To: 'Jakub Kicinski' <jakub.kicinski@netronome.com>
> Cc: Jiri Pirko <jiri@resnulli.us>; davem@davemloft.net;
> netdev@vger.kernel.org; oss-drivers@netronome.com
> Subject: RE: [PATCH net-next v2 4/7] devlink: allow subports on devlink PCI
> ports
> 
> 
> 
> > -----Original Message-----
> > From: Jakub Kicinski <jakub.kicinski@netronome.com>
> > Sent: Thursday, March 14, 2019 6:39 PM
> > To: Parav Pandit <parav@mellanox.com>
> > Cc: Jiri Pirko <jiri@resnulli.us>; davem@davemloft.net;
> > netdev@vger.kernel.org; oss-drivers@netronome.com
> > Subject: Re: [PATCH net-next v2 4/7] devlink: allow subports on
> > devlink PCI ports
> >
> > On Thu, 14 Mar 2019 22:35:36 +0000, Parav Pandit wrote:
> > > > > Then instances of flavour pci_vf are going to appear in the same
> > > > > devlink instance. Those are the switch ports:
> > > > > pci/0000:05:00.0/10002: type eth netdev enp5s0npf0pf0s0
> > > > >                         flavour pci_vf pf 0 vf 0
> > > > >                         switch_id 00154d130d2f peer
> > > > > pci/0000:05:10.1/0
> > > > > pci/0000:05:00.0/10003: type eth netdev enp5s0npf0pf0s0
> > > > >                         flavour pci_vf pf 0 vf 0 subport 1
> > > > >                         switch_id 00154d130d2f peer
> > > > > pci/0000:05:10.1/1
> > > > >
> > > > > With that, peers are going to appear too, and those are the
> > > > > actual VF/VF
> > > > > subport:
> > > > > pci/0000:05:10.1/0: type eth netdev ??? flavour pci_vf_host
> > > > >                     peer pci/0000:05:00.0/10002
> > > > > pci/0000:05:10.1/1: type eth netdev ??? flavour pci_vf_host
> > > > >                     peer pci/0000:05:00.0/10003
> > > > >
> > > > > Later you can push this VF along with all subports to VM. So in
> > > > > VM, you are going to see the VF like this:
> > > > > $ devlink dev
> > > > > pci/0000:00:08.0
> > > > > $ devlink port
> > > > > pci/0000:00:08.0/0: type eth netdev ??? flavour pci_vf_host
> > > > > pci/0000:00:08.0/1: type eth netdev ??? flavour pci_vf_host
> > > > >
> > > > > And back to your question of how are they connected in eswitch.
> > > > > That is totally up to the original user John who did the creation.
> > > > > He is in charge of the eswitch on baremetal, he would configure
> > > > > the forwarding however he likes.
> > > >
> > > > Ack, so I think you're saying VM has to communicate to the cloud
> > > > environment to have this provisioned using some service API, not a
> > > > kernel API.  That's what I wanted to confirm.
> > > >
> > > > I don't see any benefit to having the "host ports" under devlink,
> > > > as such I think it's a matter of preference.
> > >
> > > We need 'host ports' to configure parameters of this host port which
> > > is not exposed by the rep-netdev.
> > > Such as mac address.
> >
> > Please look at the quote of what Jiri wrote above - the host port gets
> > passed to the VM, you can't use it as a handle to set the MAC.
> >
> > The way to set the MAC remains:
> >
> > # devlink port set pci/0000:05:00.0/10002 peer mac_addr
> > 00:11:22:33:44:55
> >
> Even though it can be done, I think this is wrong model to program hostport
> mac address using eswitch port.
> All devlink objects are control objects, so what is passed to VM is what is
> represented by devlink.
> VF in the VM will anyway create its devlink object.
> What is wrong in programming hostport?
> It gives a very clear view to users of topology and objects.
> 

This also make sense for rdma where there are only hostports, and it doesn't have any peer switch ports.
User needs to configure node guid and port_guid.
Programming port_guid for 'hostport' also aligns for this requirement.
We better not program hostport params using this convoluted indirect way.

> Also eswitch is flat. There is no need of pf/vf flavour for port.
> It doesn't make sense to define 'mdev' flavour which we are already working.
> At eswitch level it is just a port, it happen to be connected to vf or pf or
> other objects, it doesn't matter.
> Port should be flavoured as 'hostport' or 'switchport'.
> 
> 
> > (using the port ids from above)