From: Parav Pandit <parav@mellanox.com> To: Jason Gunthorpe <jgg@ziepe.ca> Cc: Alex Williamson <alex.williamson@redhat.com>, Jakub Kicinski <jakub.kicinski@netronome.com>, Jiri Pirko <jiri@resnulli.us>, David M <david.m.ertman@intel.com>, "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>, "davem@davemloft.net" <davem@davemloft.net>, "kvm@vger.kernel.org" <kvm@vger.kernel.org>, "netdev@vger.kernel.org" <netdev@vger.kernel.org>, Saeed Mahameed <saeedm@mellanox.com>, "kwankhede@nvidia.com" <kwankhede@nvidia.com>, "leon@kernel.org" <leon@kernel.org>, "cohuck@redhat.com" <cohuck@redhat.com>, Jiri Pirko <jiri@mellanox.com>, "linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>, Or Gerlitz <gerlitz.or@gmail.com>, "Jason Wang (jasowang@redhat.com)" <jasowang@redhat.com> Subject: RE: [PATCH net-next 00/19] Mellanox, mlx5 sub function support Date: Sun, 10 Nov 2019 19:48:31 +0000 Message-ID: <AM0PR05MB48660E49342EC2CD6AB825F8D1750@AM0PR05MB4866.eurprd05.prod.outlook.com> (raw) In-Reply-To: <20191109005708.GC31761@ziepe.ca> > From: Jason Gunthorpe <jgg@ziepe.ca> > Sent: Friday, November 8, 2019 6:57 PM > > We should be creating 3 different buses, instead of mdev bus being de- > multiplexer of that? > > > > Hence, depending the device flavour specified, create such device on right > bus? > > > > For example, > > $ devlink create subdev pci/0000:05:00.0 flavour virtio name foo > > subdev_id 1 $ devlink create subdev pci/0000:05:00.0 flavour mdev > > <uuid> subdev_id 2 $ devlink create subdev pci/0000:05:00.0 flavour > > mlx5 id 1 subdev_id 3 > > I like the idea of specifying what kind of interface you want at sub device > creation time. It fits the driver model pretty well and doesn't require abusing > the vfio mdev for binding to a netdev driver. > > > $ devlink subdev pci/0000:05:00.0/<subdev_id> config <params> $ echo > > <respective_device_id> <sysfs_path>/bind > > Is explicit binding really needed? No. > If you specify a vfio flavour why shouldn't > the vfio driver autoload and bind to it right away? That is kind of the point > of the driver model... > It some configuration is needed that cannot be passed at device creation time, explicit bind later can be used. > (kind of related, but I don't get while all that GUID and lifecycle stuff in mdev > should apply for something like a SF) > GUID is just the name of the device. But lets park this aside for a moment. > > Implement power management callbacks also on all above 3 buses? > > Abstract out mlx5_bus into more generic virtual bus (vdev bus?) so > > that multiple vendors can reuse? > > In this specific case, why does the SF in mlx5 mode even need a bus? > Is it only because of devlink? That would be unfortunate > Devlink is one part due to identifying using bus/dev. How do we refer to its devlink instance of SF without bus/device? Can we extend devlink_register() to accept optionally have sf_id? If we don't have a bus, creating sub function (a device), without a 'struct device' which will have BAR, resources, etc is odd. Now if we cannot see 'struct device' in sysfs, how do we persistently name them? Are we ok to add /sys/class/net/sf_netdev/subdev_id And /sys/class/infiniband/<rdma_dev>/subdev_id So that systemd/udev can rename them as en<X?><subdev_id> and roce<X><subdev_id> If so, what will be X without a bus type? This route without a bus is certainly helpful to overcome the IOMMU limitation where IOMMU only listens to pci bus type for DMAR setup, dmar_register_bus_notifier(), and in intel_iommu_init()-> bus_set_iommu(&pci_bus_type, &intel_iommu_ops); and other IOMMU doing similar PCI/AMBA binding. This is currently overcome using WA dma_ops.
next prev parent reply index Thread overview: 132+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-11-07 16:04 Parav Pandit 2019-11-07 16:08 ` [PATCH net-next 01/19] net/mlx5: E-switch, Move devlink port close to eswitch port Parav Pandit 2019-11-07 16:08 ` [PATCH net-next 02/19] net/mlx5: E-Switch, Add SF vport, vport-rep support Parav Pandit 2019-11-07 16:08 ` [PATCH net-next 03/19] net/mlx5: Introduce SF table framework Parav Pandit 2019-11-07 16:08 ` [PATCH net-next 04/19] net/mlx5: Introduce SF life cycle APIs to allocate/free Parav Pandit 2019-11-07 16:08 ` [PATCH net-next 05/19] net/mlx5: E-Switch, Enable/disable SF's vport during SF life cycle Parav Pandit 2019-11-07 16:08 ` [PATCH net-next 06/19] net/mlx5: Add support for mediated devices in switchdev mode Parav Pandit 2019-11-08 10:32 ` Jiri Pirko 2019-11-08 16:03 ` Parav Pandit 2019-11-08 16:22 ` Jiri Pirko 2019-11-08 16:29 ` Parav Pandit 2019-11-08 18:01 ` Jiri Pirko 2019-11-08 18:04 ` Jiri Pirko 2019-11-08 18:21 ` Parav Pandit 2019-11-07 16:08 ` [PATCH net-next 07/19] vfio/mdev: Introduce sha1 based mdev alias Parav Pandit 2019-11-08 11:04 ` Jiri Pirko 2019-11-08 15:59 ` Parav Pandit 2019-11-08 16:28 ` Jiri Pirko 2019-11-08 11:10 ` Cornelia Huck 2019-11-08 16:03 ` Parav Pandit 2019-11-07 16:08 ` [PATCH net-next 08/19] vfio/mdev: Make mdev alias unique among all mdevs Parav Pandit 2019-11-08 10:49 ` Jiri Pirko 2019-11-08 15:13 ` Parav Pandit 2019-11-07 16:08 ` [PATCH net-next 09/19] vfio/mdev: Expose mdev alias in sysfs tree Parav Pandit 2019-11-08 13:22 ` Jiri Pirko 2019-11-08 18:03 ` Alex Williamson 2019-11-08 18:16 ` Jiri Pirko 2019-11-07 16:08 ` [PATCH net-next 10/19] vfio/mdev: Introduce an API mdev_alias Parav Pandit 2019-11-07 16:08 ` [PATCH net-next 11/19] vfio/mdev: Improvise mdev life cycle and parent removal scheme Parav Pandit 2019-11-08 13:01 ` Cornelia Huck 2019-11-08 16:12 ` Parav Pandit 2019-11-07 16:08 ` [PATCH net-next 12/19] devlink: Introduce mdev port flavour Parav Pandit 2019-11-07 20:38 ` Jakub Kicinski 2019-11-07 21:03 ` Parav Pandit 2019-11-08 1:17 ` Jakub Kicinski 2019-11-08 1:44 ` Parav Pandit 2019-11-08 2:20 ` Jakub Kicinski 2019-11-08 2:31 ` Parav Pandit 2019-11-08 9:46 ` Jiri Pirko 2019-11-08 15:45 ` Parav Pandit 2019-11-08 16:31 ` Jiri Pirko 2019-11-08 16:43 ` Parav Pandit 2019-11-08 18:11 ` Jiri Pirko 2019-11-08 18:23 ` Parav Pandit 2019-11-08 18:34 ` Jiri Pirko 2019-11-08 18:56 ` Parav Pandit 2019-11-08 9:30 ` Jiri Pirko 2019-11-08 15:41 ` Parav Pandit 2019-11-07 16:08 ` [PATCH net-next 13/19] net/mlx5: Register SF devlink port Parav Pandit 2019-11-07 16:08 ` [PATCH net-next 14/19] net/mlx5: Share irqs between SFs and parent PCI device Parav Pandit 2019-11-07 16:08 ` [PATCH net-next 15/19] net/mlx5: Add load/unload routines for SF driver binding Parav Pandit 2019-11-08 9:48 ` Jiri Pirko 2019-11-08 11:13 ` Jiri Pirko 2019-11-07 16:08 ` [PATCH net-next 16/19] net/mlx5: Implement dma ops and params for mediated device Parav Pandit 2019-11-07 20:42 ` Jakub Kicinski 2019-11-07 21:30 ` Parav Pandit 2019-11-08 1:16 ` Jakub Kicinski 2019-11-08 6:37 ` Christoph Hellwig 2019-11-08 15:29 ` Parav Pandit 2019-11-07 16:08 ` [PATCH net-next 17/19] net/mlx5: Add mdev driver to bind to mdev devices Parav Pandit 2019-11-07 16:08 ` [PATCH net-next 18/19] Documentation: net: mlx5: Add mdev usage documentation Parav Pandit 2019-11-07 16:08 ` [PATCH net-next 19/19] mtty: Optionally support mtty alias Parav Pandit 2019-11-08 6:26 ` Leon Romanovsky 2019-11-08 10:45 ` Jiri Pirko 2019-11-08 15:08 ` Parav Pandit 2019-11-08 15:15 ` Jiri Pirko 2019-11-08 13:46 ` Cornelia Huck 2019-11-08 15:10 ` Parav Pandit 2019-11-08 15:28 ` Cornelia Huck 2019-11-08 15:30 ` Parav Pandit 2019-11-08 17:54 ` Alex Williamson 2019-11-08 9:51 ` [PATCH net-next 01/19] net/mlx5: E-switch, Move devlink port close to eswitch port Jiri Pirko 2019-11-08 15:50 ` Parav Pandit 2019-11-07 17:03 ` [PATCH net-next 00/19] Mellanox, mlx5 sub function support Leon Romanovsky 2019-11-07 20:10 ` Parav Pandit 2019-11-08 6:20 ` Leon Romanovsky 2019-11-08 15:01 ` Parav Pandit 2019-11-07 20:32 ` Jakub Kicinski 2019-11-07 20:52 ` Parav Pandit 2019-11-08 1:16 ` Jakub Kicinski 2019-11-08 1:49 ` Parav Pandit 2019-11-08 2:12 ` Jakub Kicinski 2019-11-08 12:12 ` Jiri Pirko 2019-11-08 14:40 ` Jason Gunthorpe 2019-11-08 15:40 ` Parav Pandit 2019-11-08 19:12 ` Jakub Kicinski 2019-11-08 20:12 ` Jason Gunthorpe 2019-11-08 20:20 ` Parav Pandit 2019-11-08 20:32 ` Jason Gunthorpe 2019-11-08 20:52 ` gregkh 2019-11-08 20:34 ` Alex Williamson 2019-11-08 21:05 ` Jason Gunthorpe 2019-11-08 21:19 ` gregkh 2019-11-08 21:52 ` Alex Williamson 2019-11-08 22:48 ` Parav Pandit 2019-11-09 0:57 ` Jason Gunthorpe 2019-11-09 17:41 ` Jakub Kicinski 2019-11-10 19:04 ` Jason Gunthorpe 2019-11-10 19:48 ` Parav Pandit [this message] 2019-11-11 14:17 ` Jiri Pirko 2019-11-11 14:58 ` Parav Pandit 2019-11-11 15:06 ` Jiri Pirko 2019-11-19 4:51 ` Parav Pandit 2019-11-09 0:12 ` Jason Gunthorpe 2019-11-09 0:45 ` Parav Pandit 2019-11-11 2:19 ` Jason Wang 2019-11-08 21:45 ` Jakub Kicinski 2019-11-09 0:44 ` Jason Gunthorpe 2019-11-09 8:46 ` gregkh 2019-11-09 11:18 ` Jiri Pirko 2019-11-09 17:28 ` Jakub Kicinski 2019-11-10 9:16 ` gregkh 2019-11-09 17:27 ` Jakub Kicinski 2019-11-10 9:18 ` gregkh 2019-11-11 3:46 ` Jakub Kicinski 2019-11-11 5:18 ` Parav Pandit 2019-11-11 13:30 ` Jiri Pirko 2019-11-11 14:14 ` gregkh 2019-11-11 14:37 ` Jiri Pirko 2019-11-10 19:37 ` Jason Gunthorpe 2019-11-11 3:57 ` Jakub Kicinski 2019-11-08 16:06 ` Parav Pandit 2019-11-08 19:06 ` Jakub Kicinski 2019-11-08 19:34 ` Parav Pandit 2019-11-08 19:48 ` Jakub Kicinski 2019-11-08 19:41 ` Jiri Pirko 2019-11-08 20:40 ` Parav Pandit 2019-11-08 21:21 ` Jakub Kicinski 2019-11-08 21:39 ` Jiri Pirko 2019-11-08 21:51 ` Jakub Kicinski 2019-11-08 22:21 ` Jiri Pirko 2019-11-07 23:57 ` David Miller
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=AM0PR05MB48660E49342EC2CD6AB825F8D1750@AM0PR05MB4866.eurprd05.prod.outlook.com \ --to=parav@mellanox.com \ --cc=alex.williamson@redhat.com \ --cc=cohuck@redhat.com \ --cc=davem@davemloft.net \ --cc=david.m.ertman@intel.com \ --cc=gerlitz.or@gmail.com \ --cc=gregkh@linuxfoundation.org \ --cc=jakub.kicinski@netronome.com \ --cc=jasowang@redhat.com \ --cc=jgg@ziepe.ca \ --cc=jiri@mellanox.com \ --cc=jiri@resnulli.us \ --cc=kvm@vger.kernel.org \ --cc=kwankhede@nvidia.com \ --cc=leon@kernel.org \ --cc=linux-rdma@vger.kernel.org \ --cc=netdev@vger.kernel.org \ --cc=saeedm@mellanox.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
KVM Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/kvm/0 kvm/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 kvm kvm/ https://lore.kernel.org/kvm \ kvm@vger.kernel.org public-inbox-index kvm Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.kvm AGPL code for this site: git clone https://public-inbox.org/public-inbox.git