RE: [PATCH net-next v2 4/7] devlink: allow subports on devlink PCI ports

From: Parav Pandit <parav@mellanox.com>
To: Jakub Kicinski <jakub.kicinski@netronome.com>,
	Jiri Pirko <jiri@resnulli.us>
Cc: "Samudrala, Sridhar" <sridhar.samudrala@intel.com>,
	"davem@davemloft.net" <davem@davemloft.net>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	"oss-drivers@netronome.com" <oss-drivers@netronome.com>
Subject: RE: [PATCH net-next v2 4/7] devlink: allow subports on devlink PCI ports
Date: Fri, 15 Mar 2019 22:12:13 +0000	[thread overview]
Message-ID: <VI1PR0501MB22710A6E7DDDDC4E51350EECD1440@VI1PR0501MB2271.eurprd05.prod.outlook.com> (raw)
In-Reply-To: <20190315134454.581f47ca@cakuba.netronome.com>

> -----Original Message-----
> From: Jakub Kicinski <jakub.kicinski@netronome.com>
> Sent: Friday, March 15, 2019 3:45 PM
> To: Jiri Pirko <jiri@resnulli.us>
> Cc: Parav Pandit <parav@mellanox.com>; Samudrala, Sridhar
> <sridhar.samudrala@intel.com>; davem@davemloft.net;
> netdev@vger.kernel.org; oss-drivers@netronome.com
> Subject: Re: [PATCH net-next v2 4/7] devlink: allow subports on devlink PCI
> ports
> 
> On Fri, 15 Mar 2019 21:08:14 +0100, Jiri Pirko wrote:
> > >> IIUC, Jiri/Jakub are proposing creation of 2 devlink objects for
> > >> each port - host facing ports and switch facing ports. This is in
> > >> addition to the netdevs that are created today.
> 
> To be clear I'm not in favour of the dual-object proposal.
> 
> > >I am not proposing any different.
> > >I am proposing only two changes.
> > >1. control hostport params via referring hostport (not via indirect
> > >peer)
> >
> > Not really possible. If you passthrough VF into VM, the hostport goes
> > along with it.
> >
> > >2. flavour should not be vf/pf, flavour should be hostport, switchport.
> > >Because switch is flat and agnostic of pf/vf/mdev.
> >
> > Not sure. It's good to have this kind of visibility.
> 
> Yes, this subthread honestly makes me go from 60% sure to 95% sure we
> shouldn't do the dual object thing :(  Seems like Parav is already confused by
> it and suggests host port can exist without switch port :(
>
I am almost sure that I am not confused.
I am clear that hostports should be configured by devlink
instance which has the capability to program it.
When hostport is in VF, that VF usually won't have privilege
to program it and won't have visibility to eswitch either.

Why would you like to start with restrictive model of peer view only?
Hostports exist for infiniband HCA without switchport.
We should be able to manage hostport objects without creating fake eswitch sw object.

Jakub,
Can you please point to some example other than veth-pair where you configure host param (such as mac address) through a switch?
An existing example will help me to map it to devlink eswitch proposal.
If we go peer programming route, what are your thoughts on
how should we program infiniband hostports which doesn't have peer ports?

> > >> Are you suggesting that all the devlink objects should be visible
> > >> only at the hypervisor layer?
> > >>
> > >Of course not.
> > >
> > >Ports and params controlled by hypervisor should be exposed at
> hypervisor/eswitch wherever its parent devlink instance exist.
> > >Ports which should be visible inside a VM should be exposed inside a VM.
> > >So for a given VF,
> > >
> > >If eswitch is at hypervisor level,
> > >$ devlink port show
> > >pci/0000:05:00.0/10002 eth netdev flavour switchport switch_id
> > >00154d130d2f peer pci/0000:05:10.1/0
> > >pci/0000:05:10.1/0 eth netdev flavour hostport switch_id 00154d130d2f
> > >peer pci/0000:05:00.0/10002
> > >
> > >where VF is enumerated,
> > >$ devlink port show
> > >pci/0000:05:10.1/0 eth netdev flavour hostport
> >
> > So this is how it looks like in VM, right?
> >
> > >This is because unprivileged VF doesn't have visibility to eswitch and its
> links.