RE: [net-next 06/11] devlink: Extend SF port attributes to have external attribute

[Parav Pandit <parav@nvidia.com>]

> From: Jakub Kicinski <kuba@kernel.org>
> Sent: Thursday, April 22, 2021 10:07 PM
> 
> On Thu, 22 Apr 2021 03:55:50 +0000 Parav Pandit wrote:
> > > On Wed, 21 Apr 2021 10:47:18 -0700 Saeed Mahameed wrote:
> > > > From: Parav Pandit <parav@nvidia.com>
> > > >
> > > > Extended SF port attributes to have optional external flag similar
> > > > to PCI PF and VF port attributes.
> > > >
> > > > External atttibute is required to generate unique phys_port_name
> > > > when PF number and SF number are overlapping between two
> > > > controllers similar to SR-IOV VFs.
> > > >
> > > > When a SF is for external controller an example view of external
> > > > SF port and config sequence.
> > > >
> > > > On eswitch system:
> > > > $ devlink dev eswitch set pci/0033:01:00.0 mode switchdev
> > > >
> > > > $ devlink port show
> > > > pci/0033:01:00.0/196607: type eth netdev enP51p1s0f0np0 flavour
> > > > physical port 0 splittable false
> > > > pci/0033:01:00.0/131072: type eth netdev eth0 flavour pcipf
> > > > controller 1
> > > pfnum 0 external true splittable false
> > > >   function:
> > > >     hw_addr 00:00:00:00:00:00
> > > >
> > > > $ devlink port add pci/0033:01:00.0 flavour pcisf pfnum 0 sfnum 77
> > > > controller 1
> > > > pci/0033:01:00.0/163840: type eth netdev eth1 flavour pcisf
> > > > controller 1
> > > pfnum 0 sfnum 77 splittable false
> > > >   function:
> > > >     hw_addr 00:00:00:00:00:00 state inactive opstate detached
> > > >
> > > > phys_port_name construction:
> > > > $ cat /sys/class/net/eth1/phys_port_name
> > > > c1pf0sf77
> > > >
> > > > Signed-off-by: Parav Pandit <parav@nvidia.com>
> > > > Reviewed-by: Jiri Pirko <jiri@nvidia.com>
> > > > Reviewed-by: Vu Pham <vuhuong@nvidia.com>
> > > > Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
> > >
> > > I have a feeling I nacked this in the past, but can't find the thread.
> > > Was something similar previously posted?
> > Your memory is correct.
> > In past external flag was present but it was always set to false.
> > So you asked to move out until we set it to true, which we did.
> > This series uses it as true similar to existing PF and VF eswitch ports of an
> external controller.
> > Hence, it was removed from past series and done in this series that actually
> uses it.
> 
> Right. I still think it's a weird model to instantiate an SF from the controller
> side, but if your HW is too limited to support nested switching that's fine.

I can't locate the old email thread, but we discussed the use cases.
Nested switch may be solution to some use case but not for the current one.
In the use case of interest, multiple tenant applications are running in a bare-metal host.
Such host should not have access to switching rate, policy, filter rules, encryption keys.
Each such tenant is assigned one VF or SF running on the host system.

Also, this model doesn't prevent nested switch implementation for mlx5 and other vendors.
Each such nested switch in that case will do its own programming at its own level.
Such model is already described by Jiri in the RFCv3 [1].

[1] https://lore.kernel.org/netdev/20200519092258.GF4655@nanopsycho/#r