Netdev Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH 0/7] Introduce vdpa management tool
@ 2020-11-12  6:39 Parav Pandit
  2020-11-12  6:39 ` [PATCH 1/7] vdpa: Add missing comment for virtqueue count Parav Pandit
                   ` (12 more replies)
  0 siblings, 13 replies; 79+ messages in thread
From: Parav Pandit @ 2020-11-12  6:39 UTC (permalink / raw)
  To: virtualization; +Cc: mst, jasowang, parav, elic, netdev

This patchset covers user requirements for managing existing vdpa devices,
using a tool and its internal design notes for kernel drivers.

Background and user requirements:
----------------------------------
(1) Currently VDPA device is created by driver when driver is loaded.
However, user should have a choice when to create or not create a vdpa device
for the underlying parent device.

For example, mlx5 PCI VF and subfunction device supports multiple classes of
device such netdev, vdpa, rdma. Howevever it is not required to always created
vdpa device for such device.

(2) In another use case, a device may support creating one or multiple vdpa
device of same or different class such as net and block.
Creating vdpa devices at driver load time further limits this use case.

(3) A user should be able to monitor and query vdpa queue level or device level
statistics for a given vdpa device.

(4) A user should be able to query what class of vdpa devices are supported
by its parent device.

(5) A user should be able to view supported features and negotiated features
of the vdpa device.

(6) A user should be able to create a vdpa device in vendor agnostic manner
using single tool.

Hence, it is required to have a tool through which user can create one or more
vdpa devices from a parent device which addresses above user requirements.

Example devices:
----------------
 +-----------+ +-----------+ +---------+ +--------+ +-----------+ 
 |vdpa dev 0 | |vdpa dev 1 | |rdma dev | |netdev  | |vdpa dev 3 |
 |type=net   | |type=block | |mlx5_0   | |ens3f0  | |type=net   |
 +----+------+ +-----+-----+ +----+----+ +-----+--+ +----+------+
      |              |            |            |         |
      |              |            |            |         |
 +----+-----+        |       +----+----+       |    +----+----+
 |  mlx5    +--------+       |mlx5     +-------+    |mlx5     |
 |pci vf 2  |                |pci vf 4 |            |pci sf 8 |
 |03:00:2   |                |03:00.4  |            |mlx5_sf.8|
 +----+-----+                +----+----+            +----+----+
      |                           |                      |
      |                      +----+-----+                |
      +----------------------+mlx5      +----------------+
                             |pci pf 0  |
                             |03:00.0   |
                             +----------+

vdpa tool:
----------
vdpa tool is a tool to create, delete vdpa devices from a parent device. It is a
tool that enables user to query statistics, features and may be more attributes
in future.

vdpa tool command draft:
------------------------
(a) List parent devices which supports creating vdpa devices.
It also shows which class types supported by this parent device.
In below command example two parent devices support vdpa device creation.
First is PCI VF whose bdf is 03.00:2.
Second is PCI VF whose name is 03:00.4.
Third is PCI SF whose name is mlx5_core.sf.8

$ vdpa parentdev list
vdpasim
  supported_classes
    net
pci/0000:03.00:3
  supported_classes
    net block
pci/0000:03.00:4
  supported_classes
    net block
auxiliary/mlx5_core.sf.8
  supported_classes
    net

(b) Now add a vdpa device of networking class and show the device.
$ vdpa dev add parentdev pci/0000:03.00:2 type net name foo0 $ vdpa dev show foo0
foo0: parentdev pci/0000:03.00:2 type network parentdev vdpasim vendor_id 0 max_vqs 2 max_vq_size 256

(c) Show features of a vdpa device
$ vdpa dev features show foo0
supported
  iommu platform
  version 1

(d) Dump vdpa device statistics
$ vdpa dev stats show foo0
kickdoorbells 10
wqes 100

(e) Now delete a vdpa device previously created.
$ vdpa dev del foo0

vdpa tool support in this patchset:
-----------------------------------
vdpa tool is created to create, delete and query vdpa devices.
examples:
Show vdpa parent device that supports creating, deleting vdpa devices.

$ vdpa parentdev show
vdpasim:
  supported_classes
    net

$ vdpa parentdev show -jp
{
    "show": {
       "vdpasim": {
          "supported_classes": {
             "net"
        }
    }
}

Create a vdpa device of type networking named as "foo2" from the parent device vdpasim:

$ vdpa dev add parentdev vdpasim type net name foo2

Show the newly created vdpa device by its name:
$ vdpa dev show foo2
foo2: type network parentdev vdpasim vendor_id 0 max_vqs 2 max_vq_size 256

$ vdpa dev show foo2 -jp
{
    "dev": {
        "foo2": {
            "type": "network",
            "parentdev": "vdpasim",
            "vendor_id": 0,
            "max_vqs": 2,
            "max_vq_size": 256
        }
    }
}

Delete the vdpa device after its use:
$ vdpa dev del foo2

vdpa tool support by kernel:
----------------------------
vdpa tool user interface will be supported by existing vdpa kernel framework,
i.e. drivers/vdpa/vdpa.c It services user command through a netlink interface.

Each parent device registers supported callback operations with vdpa subsystem
through which vdpa device(s) can be managed.

FAQs:
-----
1. Where does userspace vdpa tool reside which users can use?
Ans: vdpa tool can possibly reside in iproute2 [1] as it enables user to
create vdpa net devices.

2. Why not create and delete vdpa device using sysfs/configfs?
Ans:
(a) A device creation may involve passing one or more attributes.
Passing multiple attributes and returning error code and more verbose
information for invalid attributes cannot be handled by sysfs/configfs.

(b) netlink framework is rich that enables user space and kernel driver to
provide nested attributes.

(c) Exposing device specific file under sysfs without net namespace
awareness exposes details to multiple containers. Instead exposing
attributes via a netlink socket secures the communication channel with kernel.

(d) netlink socket interface enables to run syscaller kernel tests.

3. Why not use ioctl() interface?
Ans: ioctl() interface replicates the necessary plumbing which already
exists through netlink socket.

4. What happens when one or more user created vdpa devices exist for a
parent PCI VF or SF and such parent device is removed?
Ans: All user created vdpa devices are removed that belong to a parent.

[1] git://git.kernel.org/pub/scm/network/iproute2/iproute2-next.git

Next steps:
-----------
(a) Post this patchset and iproute2/vdpa inclusion, remaining two drivers
will be coverted to support vdpa tool instead of creating unmanaged default
device on driver load.
(b) More net specific parameters such as mac, mtu will be added.
(c) Features bits get and set interface will be added.

Parav Pandit (7):
  vdpa: Add missing comment for virtqueue count
  vdpa: Use simpler version of ida allocation
  vdpa: Extend routine to accept vdpa device name
  vdpa: Define vdpa parent device, ops and a netlink interface
  vdpa: Enable a user to add and delete a vdpa device
  vdpa: Enable user to query vdpa device info
  vdpa/vdpa_sim: Enable user to create vdpasim net devices

 drivers/vdpa/Kconfig              |   1 +
 drivers/vdpa/ifcvf/ifcvf_main.c   |   2 +-
 drivers/vdpa/mlx5/net/mlx5_vnet.c |   2 +-
 drivers/vdpa/vdpa.c               | 511 +++++++++++++++++++++++++++++-
 drivers/vdpa/vdpa_sim/vdpa_sim.c  |  81 ++++-
 include/linux/vdpa.h              |  46 ++-
 include/uapi/linux/vdpa.h         |  41 +++
 7 files changed, 660 insertions(+), 24 deletions(-)
 create mode 100644 include/uapi/linux/vdpa.h

-- 
2.26.2


^ permalink raw reply	[flat|nested] 79+ messages in thread

end of thread, back to index

Thread overview: 79+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-11-12  6:39 [PATCH 0/7] Introduce vdpa management tool Parav Pandit
2020-11-12  6:39 ` [PATCH 1/7] vdpa: Add missing comment for virtqueue count Parav Pandit
2020-11-12  6:40 ` [PATCH 2/7] vdpa: Use simpler version of ida allocation Parav Pandit
2020-11-12  6:40 ` [PATCH 3/7] vdpa: Extend routine to accept vdpa device name Parav Pandit
2020-11-12  6:40 ` [PATCH 4/7] vdpa: Define vdpa parent device, ops and a netlink interface Parav Pandit
2020-11-12  6:40 ` [PATCH 5/7] vdpa: Enable a user to add and delete a vdpa device Parav Pandit
2020-11-12  6:40 ` [PATCH 6/7] vdpa: Enable user to query vdpa device info Parav Pandit
2020-11-12  6:40 ` [PATCH 7/7] vdpa/vdpa_sim: Enable user to create vdpasim net devices Parav Pandit
2020-11-16  9:41 ` [PATCH 0/7] Introduce vdpa management tool Stefan Hajnoczi
2020-11-17 19:41   ` Parav Pandit
2020-11-16 22:23 ` Jakub Kicinski
2020-11-17 19:51   ` Parav Pandit
2020-12-16  9:13     ` Michael S. Tsirkin
     [not found]       ` <20201216080610.08541f44@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
2020-12-16 16:54         ` Parav Pandit
2020-12-16 19:57           ` Michael S. Tsirkin
2020-12-17 12:13             ` Parav Pandit
2020-11-27  3:53 ` Jason Wang
     [not found]   ` <CACycT3sYScObb9nN3g7L3cesjE7sCZWxZ5_5R1usGU9ePZEeqA@mail.gmail.com>
2020-11-30  3:36     ` [External] " Jason Wang
2020-11-30  7:07       ` Yongji Xie
2020-12-01  6:25         ` Jason Wang
2020-12-01  9:55           ` Yongji Xie
2020-12-01 11:32             ` Parav Pandit
2020-12-01 14:18               ` Yongji Xie
2020-12-01 15:58                 ` Parav Pandit
2020-12-02  3:29                   ` Yongji Xie
2020-12-02  4:53                     ` Parav Pandit
2020-12-02  5:51                       ` Jason Wang
2020-12-02  6:24                         ` Parav Pandit
2020-12-02  7:55                           ` Jason Wang
2020-12-02  9:27                         ` Yongji Xie
2020-12-02  9:21                       ` Yongji Xie
2020-12-02 11:13                         ` Parav Pandit
2020-12-02 13:18                           ` Yongji Xie
2020-12-02  5:48             ` Jason Wang
2020-12-08 22:47   ` David Ahern
2021-01-19  4:21     ` Parav Pandit
2020-12-16  9:16 ` Michael S. Tsirkin
2021-01-04  3:31 ` [PATCH linux-next v2 " Parav Pandit
2021-01-04  3:31   ` [PATCH linux-next v2 1/7] vdpa_sim_net: Make mac address array static Parav Pandit
2021-01-04  7:00     ` Jason Wang
2021-01-04  3:31   ` [PATCH linux-next v2 2/7] vdpa_sim_net: Add module param to disable default vdpa net device Parav Pandit
2021-01-04  3:31   ` [PATCH linux-next v2 3/7] vdpa: Extend routine to accept vdpa device name Parav Pandit
2021-01-04  3:31   ` [PATCH linux-next v2 4/7] vdpa: Define vdpa mgmt device, ops and a netlink interface Parav Pandit
2021-01-04  7:03     ` Jason Wang
2021-01-04  7:24       ` Parav Pandit
2021-01-05  4:10         ` Jason Wang
2021-01-05  6:33           ` Parav Pandit
2021-01-05  8:36             ` Jason Wang
2021-01-04  3:31   ` [PATCH linux-next v2 5/7] vdpa: Enable a user to add and delete a vdpa device Parav Pandit
2021-01-04  3:31   ` [PATCH linux-next v2 6/7] vdpa: Enable user to query vdpa device info Parav Pandit
2021-01-04  3:31   ` [PATCH linux-next v2 7/7] vdpa_sim_net: Add support for user supported devices Parav Pandit
2021-01-04  7:05     ` Jason Wang
2021-01-04  7:21       ` Parav Pandit
2021-01-05  4:06         ` Jason Wang
2021-01-05  6:22           ` Parav Pandit
2021-01-05 10:31 ` [PATCH linux-next v3 0/6] Introduce vdpa management tool Parav Pandit
2021-01-05 10:31   ` [PATCH linux-next v3 1/6] vdpa_sim_net: Make mac address array static Parav Pandit
2021-01-07 13:45     ` Stefano Garzarella
2021-01-05 10:31   ` [PATCH linux-next v3 2/6] vdpa: Extend routine to accept vdpa device name Parav Pandit
2021-01-05 10:32   ` [PATCH linux-next v3 3/6] vdpa: Define vdpa mgmt device, ops and a netlink interface Parav Pandit
2021-01-05 10:32   ` [PATCH linux-next v3 4/6] vdpa: Enable a user to add and delete a vdpa device Parav Pandit
2021-01-05 10:32   ` [PATCH linux-next v3 5/6] vdpa: Enable user to query vdpa device info Parav Pandit
2021-01-05 10:32   ` [PATCH linux-next v3 6/6] vdpa_sim_net: Add support for user supported devices Parav Pandit
2021-01-05 11:48     ` Michael S. Tsirkin
2021-01-05 12:02       ` Parav Pandit
2021-01-05 12:14         ` Michael S. Tsirkin
2021-01-05 12:30           ` Parav Pandit
2021-01-05 13:23             ` Michael S. Tsirkin
2021-01-07  3:48               ` Parav Pandit
2021-01-12  4:14                 ` Parav Pandit
2021-01-14  4:17                 ` Jason Wang
2021-01-14  7:58                   ` Parav Pandit
2021-01-15  5:38                     ` Jason Wang
2021-01-15  6:27                       ` Parav Pandit
2021-01-19 11:09                         ` Jason Wang
2021-01-20  3:21                           ` Parav Pandit
2021-01-20  3:46                             ` Parav Pandit
2021-01-18 18:03                       ` Parav Pandit
2021-01-20  7:53                         ` Michael S. Tsirkin

Netdev Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/netdev/0 netdev/git/0.git
	git clone --mirror https://lore.kernel.org/netdev/1 netdev/git/1.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 netdev netdev/ https://lore.kernel.org/netdev \
		netdev@vger.kernel.org
	public-inbox-index netdev

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.netdev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git