RE: [PATCH net-next 00/19] Mellanox, mlx5 sub function support

[Parav Pandit <parav@mellanox.com>]

Hi All,

> From: Jiri Pirko <jiri@resnulli.us>
> Sent: Monday, November 11, 2019 9:06 AM
> 
> Mon, Nov 11, 2019 at 03:58:18PM CET, parav@mellanox.com wrote:
> >
> >
> >> -----Original Message-----
> >> From: Jiri Pirko <jiri@resnulli.us>
> >> Sent: Monday, November 11, 2019 8:18 AM Sun, Nov 10, 2019 at
> >> 08:48:31PM CET, parav@mellanox.com wrote:
> >> >
> >> >> From: Jason Gunthorpe <jgg@ziepe.ca>
> >> >> Sent: Friday, November 8, 2019 6:57 PM
> >> >> > We should be creating 3 different buses, instead of mdev bus
> >> >> > being
> >> >> > de-
> >> >> multiplexer of that?
> >> >> >
> >> >> > Hence, depending the device flavour specified, create such
> >> >> > device on right
> >> >> bus?
> >> >> >
> >> >> > For example,
> >> >> > $ devlink create subdev pci/0000:05:00.0 flavour virtio name foo
> >> >> > subdev_id 1 $ devlink create subdev pci/0000:05:00.0 flavour
> >> >> > mdev <uuid> subdev_id 2 $ devlink create subdev pci/0000:05:00.0
> >> >> > flavour
> >> >> > mlx5 id 1 subdev_id 3
> >> >>
> >> >> I like the idea of specifying what kind of interface you want at
> >> >> sub device creation time. It fits the driver model pretty well and
> >> >> doesn't require abusing the vfio mdev for binding to a netdev driver.
> >> >>
> >> >> > $ devlink subdev pci/0000:05:00.0/<subdev_id> config <params> $
> >> >> > echo <respective_device_id> <sysfs_path>/bind
> >> >>
> >> >> Is explicit binding really needed?
> >> >No.
> >> >
> >> >> If you specify a vfio flavour why shouldn't the vfio driver
> >> >> autoload and bind to it right away? That is kind of the point of
> >> >> the driver model...
> >> >>
> >> >It some configuration is needed that cannot be passed at device
> >> >creation
> >> time, explicit bind later can be used.
> >> >
> >> >> (kind of related, but I don't get while all that GUID and
> >> >> lifecycle stuff in mdev should apply for something like a SF)
> >> >>
> >> >GUID is just the name of the device.
> >> >But lets park this aside for a moment.
> >> >
> >> >> > Implement power management callbacks also on all above 3 buses?
> >> >> > Abstract out mlx5_bus into more generic virtual bus (vdev bus?)
> >> >> > so that multiple vendors can reuse?
> >> >>
> >> >> In this specific case, why does the SF in mlx5 mode even need a bus?
> >> >> Is it only because of devlink? That would be unfortunate
> >> >>
> >> >Devlink is one part due to identifying using bus/dev.
> >> >How do we refer to its devlink instance of SF without bus/device?
> >>
> >> Question is, why to have devlink instance for SF itself. Same as VF,
> >> you don't
> >mlx5_core has devlink instance for PF and VF for long time now.
> >Health report, txq/rxq dumps etc all anchored to this devlink instance even for
> VF. (similar to PF).
> >And so, SF same framework should work for SF.
> 
> Right, for health it makes sense.
> 
> >
> >> need devlink instance. You only need devlink_port (or
> >> devlink_subdev) instance on the PF devlink parent for it.
> >>
> >Devlink_port or devlink_subdev are still on eswitch or mgmt side.
> >They are not present on the side where devlink instance exist on side where
> txq/rxq/eq etc exist.
> >
> 
> Got it.

I am working on the revised v1 version of these series to address below concerns to achieve all the requirements captured in this cover letter.

1. Avoid mdev bus abuse (Jason's and Greg's input)
2. Avoid dma_ops overload (Christoph's comment)
3. Update cover letter for devlink examples (Jakub's comment)
4. Update cover letter to describe rdma persistent naming scheme (Leon's comment)
5. Jiri's few comments on code restructure