Re: [PATCH/RFC net-next 1/2] devlink: expose port function commands to assign VFs to multiple netdevs

From: Simon Horman <simon.horman@corigine.com>
To: Leon Romanovsky <leon@kernel.org>
Cc: Jakub Kicinski <kuba@kernel.org>,
	David Miller <davem@davemloft.net>,
	Paolo Abeni <pabeni@redhat.com>,
	Michael Chan <michael.chan@broadcom.com>,
	Andy Gospodarek <andy@greyhouse.net>,
	Gal Pressman <gal@nvidia.com>, Saeed Mahameed <saeed@kernel.org>,
	Jesse Brandeburg <jesse.brandeburg@intel.com>,
	Tony Nguyen <anthony.l.nguyen@intel.com>,
	Edward Cree <ecree.xilinx@gmail.com>,
	Vladimir Oltean <vladimir.oltean@nxp.com>,
	Andrew Lunn <andrew@lunn.ch>, Fei Qin <fei.qin@corigine.com>,
	netdev@vger.kernel.org, oss-drivers@corigine.com
Subject: Re: [PATCH/RFC net-next 1/2] devlink: expose port function commands to assign VFs to multiple netdevs
Date: Wed, 8 Feb 2023 13:05:02 +0100	[thread overview]
Message-ID: <Y+OP7rIQ+iB5NgUw@corigine.com> (raw)
In-Reply-To: <Y+ONTC6q0pqZl3/I@unreal>

On Wed, Feb 08, 2023 at 01:53:48PM +0200, Leon Romanovsky wrote:
> On Wed, Feb 08, 2023 at 12:36:53PM +0100, Simon Horman wrote:
> > On Wed, Feb 08, 2023 at 01:21:22PM +0200, Leon Romanovsky wrote:
> > > On Mon, Feb 06, 2023 at 06:42:27PM -0800, Jakub Kicinski wrote:
> > > > On Mon,  6 Feb 2023 16:36:02 +0100 Simon Horman wrote:
> > > > > +VF assignment setup
> > > > > +---------------------------
> > > > > +In some cases, NICs could have multiple physical ports per PF. Users can assign VFs to
> > > > > +different ports.
> > > > 
> > > > Please make sure you run make htmldocs when changing docs,
> > > > this will warn.
> > > > 
> > > > > +- Get count of VFs assigned to physical port::
> > > > > +
> > > > > +    $ devlink port show pci/0000:82:00.0/0
> > > > > +    pci/0000:82:00.0/0: type eth netdev enp130s0np0 flavour physical port 0 splittable true lanes 4
> > > > 
> > > > Physical port has VFs? My knee jerk reaction is that allocating
> > > > resources via devlink is fine but this seems to lean a bit into
> > > > forwarding. How do other vendors do it? What's the mapping of VFs
> > > > to ports?
> > > 
> > > I don't understand the meaning of VFs here. If we are talking about PCI
> > > VFs, other vendors follow PCI spec "9.3.3.3.1 VF Enable" section, which
> > > talks about having one bit to enable all VFs at once. All these VFs will
> > > have separate netdevs.
> > 
> > Yes, that is the case here too (before and after).
> > 
> > What we are talking about is the association of VFs to physical ports
> > (in the case where a NIC has more than one physical port).
> 
> We have devices with multiple ports too, but don't have such issues.
> So it will help if you can provide more context here.
> 
> I'm failing to see connection between physical ports and physical VFs.
> 
> Are you saying that physical ports are actual PCI VFs, which spans L2 VFs,
> which you want to assign to another port (PF)?

No, a physical port is not a VF (nor a PF, FWIIW).

The topic here is about two modes of behaviour.

One, where VFs are associated with physical ports - conceptually one might
think of this as some VFs and one phys port being plugged into one VEB,
while other VFs and another phys port are plugged into another VEB.

I believe this is the mode on most multi-port NICs.

And another mode where all VFs are associated with one physical port,
even if more than one is present. The NFP currently implements this model.

This patch is about allowing NICs, in particular the NFP based NICs,
to switch modes.