From: Jason Wang <jasowang@redhat.com> To: mst@redhat.com, jasowang@redhat.com Cc: eperezma@redhat.com, kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, lulu@redhat.com, eli@mellanox.com, lingshan.zhu@intel.com, rob.miller@broadcom.com, stefanha@redhat.com, sgarzare@redhat.com Subject: [PATCH 00/21] Control VQ support in vDPA Date: Wed, 16 Dec 2020 14:47:57 +0800 [thread overview] Message-ID: <20201216064818.48239-1-jasowang@redhat.com> (raw) Hi All: This series tries to add the support for control virtqueue in vDPA. Control virtqueue is used by networking device for accepting various commands from the driver. It's a must to support multiqueue and other configurations. When used by vhost-vDPA bus driver for VM, the control virtqueue should be shadowed via userspace VMM (Qemu) instead of being assigned directly to Guest. This is because Qemu needs to know the device state in order to start and stop device correctly (e.g for Live Migration). This requies to isolate the memory mapping for control virtqueue presented by vhost-vDPA to prevent guest from accesing it directly. To achieve this, vDPA introduce two new abstractions: - address space: identified through address space id (ASID) and a set of memory mapping in maintained - virtqueue group: the minimal set of virtqueues that must share an address space Device needs to advertise the following attributes to vDPA: - the number of address spaces supported in the device - the number of virtqueue groups supported in the device - the mappings from a specific virtqueue to its virtqueue groups The mappings from virtqueue to virtqueue groups is fixed and defined by vDPA device driver. E.g: - For the device that has hardware ASID support, it can simply advertise a per virtqueue virtqueue group. - For the device that does not have hardware ASID support, it can simply advertise a single virtqueue group that contains all virtqueues. Or if it wants a software emulated control virtqueue, it can advertise two virtqueue groups, one is for cvq, another is for the rest virtqueues. vDPA also allow to change the association between virtqueue group and address space. So in the case of control virtqueue, userspace VMM(Qemu) may use a dedicated address space for the control virtqueue group to isolate the memory mapping. The vhost/vhost-vDPA is also extend for the userspace to: - query the number of virtqueue groups and address spaces supported by the device - query the virtqueue group for a specific virtqueue - assocaite a virtqueue group with an address space - send ASID based IOTLB commands This will help userspace VMM(Qemu) to detect whether the control vq could be supported and isolate memory mappings of control virtqueue from the others. To demonstrate the usage, vDPA simulator is extended to support setting MAC address via a emulated control virtqueue. Please review. Changes since RFC: - tweak vhost uAPI documentation - switch to use device specific IOTLB really in patch 4 - tweak the commit log - fix that ASID in vhost is claimed to be 32 actually but 16bit actually - fix use after free when using ASID with IOTLB batching requests - switch to use Stefano's patch for having separated iov - remove unused "used_as" variable - fix the iotlb/asid checking in vhost_vdpa_unmap() Thanks Jason Wang (20): vhost: move the backend feature bits to vhost_types.h virtio-vdpa: don't set callback if virtio doesn't need it vhost-vdpa: passing iotlb to IOMMU mapping helpers vhost-vdpa: switch to use vhost-vdpa specific IOTLB vdpa: add the missing comment for nvqs in struct vdpa_device vdpa: introduce virtqueue groups vdpa: multiple address spaces support vdpa: introduce config operations for associating ASID to a virtqueue group vhost_iotlb: split out IOTLB initialization vhost: support ASID in IOTLB API vhost-vdpa: introduce asid based IOTLB vhost-vdpa: introduce uAPI to get the number of virtqueue groups vhost-vdpa: introduce uAPI to get the number of address spaces vhost-vdpa: uAPI to get virtqueue group id vhost-vdpa: introduce uAPI to set group ASID vhost-vdpa: support ASID based IOTLB API vdpa_sim: advertise VIRTIO_NET_F_MTU vdpa_sim: factor out buffer completion logic vdpa_sim: filter destination mac address vdpasim: control virtqueue support Stefano Garzarella (1): vdpa_sim: split vdpasim_virtqueue's iov field in out_iov and in_iov drivers/vdpa/ifcvf/ifcvf_main.c | 9 +- drivers/vdpa/mlx5/net/mlx5_vnet.c | 11 +- drivers/vdpa/vdpa.c | 8 +- drivers/vdpa/vdpa_sim/vdpa_sim.c | 292 ++++++++++++++++++++++++------ drivers/vhost/iotlb.c | 23 ++- drivers/vhost/vdpa.c | 246 ++++++++++++++++++++----- drivers/vhost/vhost.c | 23 ++- drivers/vhost/vhost.h | 4 +- drivers/virtio/virtio_vdpa.c | 2 +- include/linux/vdpa.h | 42 ++++- include/linux/vhost_iotlb.h | 2 + include/uapi/linux/vhost.h | 25 ++- include/uapi/linux/vhost_types.h | 10 +- 13 files changed, 561 insertions(+), 136 deletions(-) -- 2.25.1
WARNING: multiple messages have this Message-ID (diff)
From: Jason Wang <jasowang@redhat.com> To: mst@redhat.com, jasowang@redhat.com Cc: kvm@vger.kernel.org, lulu@redhat.com, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, eperezma@redhat.com, stefanha@redhat.com, eli@mellanox.com, lingshan.zhu@intel.com, rob.miller@broadcom.com Subject: [PATCH 00/21] Control VQ support in vDPA Date: Wed, 16 Dec 2020 14:47:57 +0800 [thread overview] Message-ID: <20201216064818.48239-1-jasowang@redhat.com> (raw) Hi All: This series tries to add the support for control virtqueue in vDPA. Control virtqueue is used by networking device for accepting various commands from the driver. It's a must to support multiqueue and other configurations. When used by vhost-vDPA bus driver for VM, the control virtqueue should be shadowed via userspace VMM (Qemu) instead of being assigned directly to Guest. This is because Qemu needs to know the device state in order to start and stop device correctly (e.g for Live Migration). This requies to isolate the memory mapping for control virtqueue presented by vhost-vDPA to prevent guest from accesing it directly. To achieve this, vDPA introduce two new abstractions: - address space: identified through address space id (ASID) and a set of memory mapping in maintained - virtqueue group: the minimal set of virtqueues that must share an address space Device needs to advertise the following attributes to vDPA: - the number of address spaces supported in the device - the number of virtqueue groups supported in the device - the mappings from a specific virtqueue to its virtqueue groups The mappings from virtqueue to virtqueue groups is fixed and defined by vDPA device driver. E.g: - For the device that has hardware ASID support, it can simply advertise a per virtqueue virtqueue group. - For the device that does not have hardware ASID support, it can simply advertise a single virtqueue group that contains all virtqueues. Or if it wants a software emulated control virtqueue, it can advertise two virtqueue groups, one is for cvq, another is for the rest virtqueues. vDPA also allow to change the association between virtqueue group and address space. So in the case of control virtqueue, userspace VMM(Qemu) may use a dedicated address space for the control virtqueue group to isolate the memory mapping. The vhost/vhost-vDPA is also extend for the userspace to: - query the number of virtqueue groups and address spaces supported by the device - query the virtqueue group for a specific virtqueue - assocaite a virtqueue group with an address space - send ASID based IOTLB commands This will help userspace VMM(Qemu) to detect whether the control vq could be supported and isolate memory mappings of control virtqueue from the others. To demonstrate the usage, vDPA simulator is extended to support setting MAC address via a emulated control virtqueue. Please review. Changes since RFC: - tweak vhost uAPI documentation - switch to use device specific IOTLB really in patch 4 - tweak the commit log - fix that ASID in vhost is claimed to be 32 actually but 16bit actually - fix use after free when using ASID with IOTLB batching requests - switch to use Stefano's patch for having separated iov - remove unused "used_as" variable - fix the iotlb/asid checking in vhost_vdpa_unmap() Thanks Jason Wang (20): vhost: move the backend feature bits to vhost_types.h virtio-vdpa: don't set callback if virtio doesn't need it vhost-vdpa: passing iotlb to IOMMU mapping helpers vhost-vdpa: switch to use vhost-vdpa specific IOTLB vdpa: add the missing comment for nvqs in struct vdpa_device vdpa: introduce virtqueue groups vdpa: multiple address spaces support vdpa: introduce config operations for associating ASID to a virtqueue group vhost_iotlb: split out IOTLB initialization vhost: support ASID in IOTLB API vhost-vdpa: introduce asid based IOTLB vhost-vdpa: introduce uAPI to get the number of virtqueue groups vhost-vdpa: introduce uAPI to get the number of address spaces vhost-vdpa: uAPI to get virtqueue group id vhost-vdpa: introduce uAPI to set group ASID vhost-vdpa: support ASID based IOTLB API vdpa_sim: advertise VIRTIO_NET_F_MTU vdpa_sim: factor out buffer completion logic vdpa_sim: filter destination mac address vdpasim: control virtqueue support Stefano Garzarella (1): vdpa_sim: split vdpasim_virtqueue's iov field in out_iov and in_iov drivers/vdpa/ifcvf/ifcvf_main.c | 9 +- drivers/vdpa/mlx5/net/mlx5_vnet.c | 11 +- drivers/vdpa/vdpa.c | 8 +- drivers/vdpa/vdpa_sim/vdpa_sim.c | 292 ++++++++++++++++++++++++------ drivers/vhost/iotlb.c | 23 ++- drivers/vhost/vdpa.c | 246 ++++++++++++++++++++----- drivers/vhost/vhost.c | 23 ++- drivers/vhost/vhost.h | 4 +- drivers/virtio/virtio_vdpa.c | 2 +- include/linux/vdpa.h | 42 ++++- include/linux/vhost_iotlb.h | 2 + include/uapi/linux/vhost.h | 25 ++- include/uapi/linux/vhost_types.h | 10 +- 13 files changed, 561 insertions(+), 136 deletions(-) -- 2.25.1 _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
next reply other threads:[~2020-12-16 6:50 UTC|newest] Thread overview: 136+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-12-16 6:47 Jason Wang [this message] 2020-12-16 6:47 ` [PATCH 00/21] Control VQ support in vDPA Jason Wang 2020-12-16 6:47 ` [PATCH 01/21] vhost: move the backend feature bits to vhost_types.h Jason Wang 2020-12-16 6:47 ` Jason Wang 2020-12-16 6:47 ` [PATCH 02/21] virtio-vdpa: don't set callback if virtio doesn't need it Jason Wang 2020-12-16 6:47 ` Jason Wang 2020-12-16 6:48 ` [PATCH 03/21] vhost-vdpa: passing iotlb to IOMMU mapping helpers Jason Wang 2020-12-16 6:48 ` Jason Wang 2020-12-16 6:48 ` [PATCH 04/21] vhost-vdpa: switch to use vhost-vdpa specific IOTLB Jason Wang 2020-12-16 6:48 ` Jason Wang 2020-12-16 6:48 ` [PATCH 05/21] vdpa: add the missing comment for nvqs in struct vdpa_device Jason Wang 2020-12-16 6:48 ` Jason Wang 2020-12-16 6:48 ` [PATCH 06/21] vdpa: introduce virtqueue groups Jason Wang 2020-12-16 6:48 ` Jason Wang 2021-01-04 10:04 ` Stefan Hajnoczi 2021-01-04 10:04 ` Stefan Hajnoczi 2021-01-05 4:13 ` Jason Wang 2021-01-05 4:13 ` Jason Wang 2020-12-16 6:48 ` [PATCH 07/21] vdpa: multiple address spaces support Jason Wang 2020-12-16 6:48 ` Jason Wang 2020-12-29 7:28 ` Eli Cohen 2020-12-30 4:00 ` Jason Wang 2020-12-30 4:00 ` Jason Wang 2020-12-30 4:04 ` Jason Wang 2020-12-30 4:04 ` Jason Wang 2020-12-30 9:44 ` Eli Cohen 2020-12-16 6:48 ` [PATCH 08/21] vdpa: introduce config operations for associating ASID to a virtqueue group Jason Wang 2020-12-16 6:48 ` Jason Wang 2020-12-16 6:48 ` [PATCH 09/21] vhost_iotlb: split out IOTLB initialization Jason Wang 2020-12-16 6:48 ` Jason Wang 2020-12-16 6:48 ` [PATCH 10/21] vhost: support ASID in IOTLB API Jason Wang 2020-12-16 6:48 ` Jason Wang 2020-12-29 10:20 ` Eli Cohen 2020-12-30 4:27 ` Jason Wang 2020-12-30 4:27 ` Jason Wang 2020-12-16 6:48 ` [PATCH 11/21] vhost-vdpa: introduce asid based IOTLB Jason Wang 2020-12-16 6:48 ` Jason Wang 2020-12-29 11:41 ` Eli Cohen 2020-12-30 6:23 ` Jason Wang 2020-12-30 6:23 ` Jason Wang 2020-12-29 11:53 ` Eli Cohen 2020-12-30 6:34 ` Jason Wang 2020-12-30 6:34 ` Jason Wang 2020-12-29 12:05 ` Eli Cohen 2020-12-30 6:33 ` Jason Wang 2020-12-30 6:33 ` Jason Wang 2020-12-16 6:48 ` [PATCH 12/21] vhost-vdpa: introduce uAPI to get the number of virtqueue groups Jason Wang 2020-12-16 6:48 ` Jason Wang 2020-12-29 12:24 ` Eli Cohen 2020-12-30 6:49 ` Jason Wang 2020-12-30 6:49 ` Jason Wang 2020-12-30 10:05 ` Eli Cohen 2020-12-31 2:36 ` Jason Wang 2020-12-31 2:36 ` Jason Wang 2020-12-16 6:48 ` [PATCH 13/21] vhost-vdpa: introduce uAPI to get the number of address spaces Jason Wang 2020-12-16 6:48 ` Jason Wang 2020-12-16 6:48 ` [PATCH 14/21] vhost-vdpa: uAPI to get virtqueue group id Jason Wang 2020-12-16 6:48 ` Jason Wang 2020-12-16 6:48 ` [PATCH 15/21] vhost-vdpa: introduce uAPI to set group ASID Jason Wang 2020-12-16 6:48 ` Jason Wang 2020-12-16 6:48 ` [PATCH 16/21] vhost-vdpa: support ASID based IOTLB API Jason Wang 2020-12-16 6:48 ` Jason Wang 2020-12-16 6:48 ` [PATCH 17/21] vdpa_sim: split vdpasim_virtqueue's iov field in out_iov and in_iov Jason Wang 2020-12-16 6:48 ` Jason Wang 2020-12-16 6:48 ` [PATCH 18/21] vdpa_sim: advertise VIRTIO_NET_F_MTU Jason Wang 2020-12-16 6:48 ` Jason Wang 2020-12-16 6:48 ` [PATCH 19/21] vdpa_sim: factor out buffer completion logic Jason Wang 2020-12-16 6:48 ` Jason Wang 2020-12-16 6:48 ` [PATCH 20/21] vdpa_sim: filter destination mac address Jason Wang 2020-12-16 6:48 ` Jason Wang 2020-12-16 6:48 ` [PATCH 21/21] vdpasim: control virtqueue support Jason Wang 2020-12-16 6:48 ` Jason Wang 2020-12-17 20:19 ` kernel test robot 2020-12-17 20:19 ` kernel test robot 2020-12-17 20:19 ` kernel test robot 2021-01-11 12:26 ` Eli Cohen 2021-01-12 3:11 ` Jason Wang 2021-01-12 3:11 ` Jason Wang 2021-01-22 19:43 ` Eugenio Perez Martin 2021-01-25 3:16 ` Jason Wang 2021-01-25 3:16 ` Jason Wang 2020-12-16 9:47 ` [PATCH 00/21] Control VQ support in vDPA Michael S. Tsirkin 2020-12-16 9:47 ` Michael S. Tsirkin 2020-12-17 3:30 ` Jason Wang 2020-12-17 3:30 ` Jason Wang 2020-12-17 7:58 ` Michael S. Tsirkin 2020-12-17 7:58 ` Michael S. Tsirkin 2020-12-17 9:02 ` Jason Wang 2020-12-17 9:02 ` Jason Wang 2020-12-17 22:28 ` Michael S. Tsirkin 2020-12-17 22:28 ` Michael S. Tsirkin 2020-12-18 2:56 ` Jason Wang 2020-12-18 2:56 ` Jason Wang 2020-12-17 7:26 ` Eli Cohen 2022-02-24 21:22 ` [RFC PATCH v2 00/19] " Gautam Dawar 2022-02-24 21:22 ` [RFC PATCH v2 01/19] vhost: move the backend feature bits to vhost_types.h Gautam Dawar 2022-02-24 21:22 ` [RFC PATCH v2 02/19] virtio-vdpa: don't set callback if virtio doesn't need it Gautam Dawar 2022-02-24 21:22 ` [RFC PATCH v2 03/19] vhost-vdpa: passing iotlb to IOMMU mapping helpers Gautam Dawar 2022-02-24 21:22 ` [RFC PATCH v2 04/19] vhost-vdpa: switch to use vhost-vdpa specific IOTLB Gautam Dawar 2022-02-24 21:22 ` [RFC PATCH v2 05/19] vdpa: introduce virtqueue groups Gautam Dawar 2022-02-28 8:07 ` Jason Wang 2022-02-28 8:07 ` Jason Wang 2022-02-28 10:57 ` Gautam Dawar 2022-02-24 21:22 ` [RFC PATCH v2 06/19] vdpa: multiple address spaces support Gautam Dawar 2022-03-03 19:39 ` Eugenio Perez Martin 2022-03-04 6:30 ` Gautam Dawar 2022-03-04 17:45 ` Eugenio Perez Martin 2022-02-24 21:22 ` [RFC PATCH v2 07/19] vdpa: introduce config operations for associating ASID to a virtqueue group Gautam Dawar 2022-03-04 9:54 ` Eugenio Perez Martin 2022-03-04 17:48 ` Gautam Dawar 2022-02-24 21:22 ` [RFC PATCH v2 08/19] vhost_iotlb: split out IOTLB initialization Gautam Dawar 2022-02-24 21:22 ` [RFC PATCH v2 09/19] vhost: support ASID in IOTLB API Gautam Dawar 2022-03-04 10:25 ` Eugenio Perez Martin 2022-03-04 17:52 ` Gautam Dawar 2022-02-24 21:22 ` [RFC PATCH v2 10/19] vhost-vdpa: introduce asid based IOTLB Gautam Dawar 2022-03-04 17:56 ` Eugenio Perez Martin 2022-03-07 10:07 ` Gautam Dawar 2022-02-24 21:22 ` [RFC PATCH v2 11/19] vhost-vdpa: introduce uAPI to get the number of virtqueue groups Gautam Dawar 2022-02-24 21:22 ` [RFC PATCH v2 12/19] vhost-vdpa: introduce uAPI to get the number of address spaces Gautam Dawar 2022-02-24 21:22 ` [RFC PATCH v2 13/19] vhost-vdpa: uAPI to get virtqueue group id Gautam Dawar 2022-02-24 21:22 ` [RFC PATCH v2 14/19] vhost-vdpa: introduce uAPI to set group ASID Gautam Dawar 2022-02-24 21:22 ` [RFC PATCH v2 15/19] vhost-vdpa: support ASID based IOTLB API Gautam Dawar 2022-03-04 18:04 ` Eugenio Perez Martin 2022-03-07 10:23 ` Gautam Dawar 2022-02-24 21:22 ` [RFC PATCH v2 16/19] vdpa_sim: advertise VIRTIO_NET_F_MTU Gautam Dawar 2022-03-10 17:55 ` Eugenio Perez Martin 2022-02-24 21:22 ` [RFC PATCH v2 17/19] vdpa_sim: factor out buffer completion logic Gautam Dawar 2022-02-24 21:22 ` [RFC PATCH v2 18/19] vdpa_sim: filter destination mac address Gautam Dawar 2022-03-10 18:22 ` Eugenio Perez Martin 2022-02-24 21:22 ` [RFC PATCH v2 19/19] vdpasim: control virtqueue support Gautam Dawar 2022-03-10 18:20 ` Eugenio Perez Martin 2022-03-18 7:35 ` Eugenio Perez Martin 2022-03-22 8:46 ` Gautam Dawar 2022-02-28 8:17 ` [RFC PATCH v2 00/19] Control VQ support in vDPA Jason Wang 2022-02-28 8:17 ` Jason Wang 2022-02-28 10:56 ` Gautam Dawar
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20201216064818.48239-1-jasowang@redhat.com \ --to=jasowang@redhat.com \ --cc=eli@mellanox.com \ --cc=eperezma@redhat.com \ --cc=kvm@vger.kernel.org \ --cc=lingshan.zhu@intel.com \ --cc=linux-kernel@vger.kernel.org \ --cc=lulu@redhat.com \ --cc=mst@redhat.com \ --cc=netdev@vger.kernel.org \ --cc=rob.miller@broadcom.com \ --cc=sgarzare@redhat.com \ --cc=stefanha@redhat.com \ --cc=virtualization@lists.linux-foundation.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.