From: Gautam Dawar <gdawar@xilinx.com>
To: Jason Wang <jasowang@redhat.com>
Cc: Martin Petrus Hubertus Habets <martinh@xilinx.com>,
Harpreet Singh Anand <hanand@xilinx.com>,
Tanuj Murlidhar Kamde <tanujk@xilinx.com>,
"eperezma@redhat.com" <eperezma@redhat.com>,
"Michael S. Tsirkin" <mst@redhat.com>,
Zhu Lingshan <lingshan.zhu@intel.com>,
Stefano Garzarella <sgarzare@redhat.com>,
Xie Yongji <xieyongji@bytedance.com>, Eli Cohen <elic@nvidia.com>,
Si-Wei Liu <si-wei.liu@oracle.com>,
Parav Pandit <parav@nvidia.com>, Longpeng <longpeng2@huawei.com>,
"virtualization@lists.linux-foundation.org"
<virtualization@lists.linux-foundation.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
"netdev@vger.kernel.org" <netdev@vger.kernel.org>
Subject: RE: [RFC PATCH v2 00/19] Control VQ support in vDPA
Date: Mon, 28 Feb 2022 10:56:15 +0000 [thread overview]
Message-ID: <BY5PR02MB6980DC90EE52FE1900F7C414B1019@BY5PR02MB6980.namprd02.prod.outlook.com> (raw)
In-Reply-To: <19843d2b-dfe9-5e2d-4d3d-ca55456947dc@redhat.com>
在 2022/2/25 上午5:22, Gautam Dawar 写道:
> Hi All:
>
> This series tries to add the support for control virtqueue in vDPA.
>
> Control virtqueue is used by networking device for accepting various
> commands from the driver. It's a must to support multiqueue and other
> configurations.
>
> When used by vhost-vDPA bus driver for VM, the control virtqueue
> should be shadowed via userspace VMM (Qemu) instead of being assigned
> directly to Guest. This is because Qemu needs to know the device state
> in order to start and stop device correctly (e.g for Live Migration).
>
> This requies to isolate the memory mapping for control virtqueue
> presented by vhost-vDPA to prevent guest from accessing it directly.
>
> To achieve this, vDPA introduce two new abstractions:
>
> - address space: identified through address space id (ASID) and a set
> of memory mapping in maintained
> - virtqueue group: the minimal set of virtqueues that must share an
> address space
>
> Device needs to advertise the following attributes to vDPA:
>
> - the number of address spaces supported in the device
> - the number of virtqueue groups supported in the device
> - the mappings from a specific virtqueue to its virtqueue groups
>
> The mappings from virtqueue to virtqueue groups is fixed and defined
> by vDPA device driver. E.g:
>
> - For the device that has hardware ASID support, it can simply
> advertise a per virtqueue virtqueue group.
> - For the device that does not have hardware ASID support, it can
> simply advertise a single virtqueue group that contains all
> virtqueues. Or if it wants a software emulated control virtqueue, it
> can advertise two virtqueue groups, one is for cvq, another is for
> the rest virtqueues.
>
> vDPA also allow to change the association between virtqueue group and
> address space. So in the case of control virtqueue, userspace
> VMM(Qemu) may use a dedicated address space for the control virtqueue
> group to isolate the memory mapping.
>
> The vhost/vhost-vDPA is also extend for the userspace to:
>
> - query the number of virtqueue groups and address spaces supported by
> the device
> - query the virtqueue group for a specific virtqueue
> - assocaite a virtqueue group with an address space
> - send ASID based IOTLB commands
>
> This will help userspace VMM(Qemu) to detect whether the control vq
> could be supported and isolate memory mappings of control virtqueue
> from the others.
>
> To demonstrate the usage, vDPA simulator is extended to support
> setting MAC address via a emulated control virtqueue.
>
> Please review.
>
> Changes since v1:
>
> - Rebased the v1 patch series on vhost branch of MST vhost git repo
> git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git/log/?h=vhost
> - Updates to accommodate vdpa_sim changes from monolithic module in
> kernel used v1 patch series to current modularized class (net, block)
> based approach.
> - Added new attributes (ngroups and nas) to "vdpasim_dev_attr" and
> propagated them from vdpa_sim_net to vdpa_sim
> - Widened the data-type for "asid" member of vhost_msg_v2 to __u32
> to accommodate PASID
This is great. Then the semantic matches exactly the PASID proposal here[1].
> - Fixed the buildbot warnings
> - Resolved all checkpatch.pl errors and warnings
> - Tested both control and datapath with Xilinx Smartnic SN1000 series
> device using QEMU implementing the Shadow virtqueue and support for
> VQ groups and ASID available at:
> github.com/eugpermar/qemu/releases/tag/vdpa_sw_live_migration.d%2F
> asid_groups-v1.d%2F00
On top, we may extend the netlink protocol to report the mapping between virtqueue to its groups.
[GD>>] Yes, I've already discussed this with Eugenio. For testing purpose, I added the mapping in Xilinx netdriver "sfc".
Thanks
[1]
https://www.mail-archive.com/virtio-dev@lists.oasis-open.org/msg08077.html
>
> Changes since RFC:
>
> - tweak vhost uAPI documentation
> - switch to use device specific IOTLB really in patch 4
> - tweak the commit log
> - fix that ASID in vhost is claimed to be 32 actually but 16bit
> actually
> - fix use after free when using ASID with IOTLB batching requests
> - switch to use Stefano's patch for having separated iov
> - remove unused "used_as" variable
> - fix the iotlb/asid checking in vhost_vdpa_unmap()
>
> Thanks
>
> Gautam Dawar (19):
> vhost: move the backend feature bits to vhost_types.h
> virtio-vdpa: don't set callback if virtio doesn't need it
> vhost-vdpa: passing iotlb to IOMMU mapping helpers
> vhost-vdpa: switch to use vhost-vdpa specific IOTLB
> vdpa: introduce virtqueue groups
> vdpa: multiple address spaces support
> vdpa: introduce config operations for associating ASID to a virtqueue
> group
> vhost_iotlb: split out IOTLB initialization
> vhost: support ASID in IOTLB API
> vhost-vdpa: introduce asid based IOTLB
> vhost-vdpa: introduce uAPI to get the number of virtqueue groups
> vhost-vdpa: introduce uAPI to get the number of address spaces
> vhost-vdpa: uAPI to get virtqueue group id
> vhost-vdpa: introduce uAPI to set group ASID
> vhost-vdpa: support ASID based IOTLB API
> vdpa_sim: advertise VIRTIO_NET_F_MTU
> vdpa_sim: factor out buffer completion logic
> vdpa_sim: filter destination mac address
> vdpasim: control virtqueue support
>
> drivers/vdpa/ifcvf/ifcvf_main.c | 8 +-
> drivers/vdpa/mlx5/net/mlx5_vnet.c | 11 +-
> drivers/vdpa/vdpa.c | 5 +
> drivers/vdpa/vdpa_sim/vdpa_sim.c | 100 ++++++++--
> drivers/vdpa/vdpa_sim/vdpa_sim.h | 3 +
> drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 169 +++++++++++++----
> drivers/vhost/iotlb.c | 23 ++-
> drivers/vhost/vdpa.c | 272 +++++++++++++++++++++------
> drivers/vhost/vhost.c | 23 ++-
> drivers/vhost/vhost.h | 4 +-
> drivers/virtio/virtio_vdpa.c | 2 +-
> include/linux/vdpa.h | 46 ++++-
> include/linux/vhost_iotlb.h | 2 +
> include/uapi/linux/vhost.h | 25 ++-
> include/uapi/linux/vhost_types.h | 11 +-
> 15 files changed, 566 insertions(+), 138 deletions(-)
>
prev parent reply other threads:[~2022-02-28 10:58 UTC|newest]
Thread overview: 92+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-12-16 6:47 [PATCH 00/21] Control VQ support in vDPA Jason Wang
2020-12-16 6:47 ` [PATCH 01/21] vhost: move the backend feature bits to vhost_types.h Jason Wang
2020-12-16 6:47 ` [PATCH 02/21] virtio-vdpa: don't set callback if virtio doesn't need it Jason Wang
2020-12-16 6:48 ` [PATCH 03/21] vhost-vdpa: passing iotlb to IOMMU mapping helpers Jason Wang
2020-12-16 6:48 ` [PATCH 04/21] vhost-vdpa: switch to use vhost-vdpa specific IOTLB Jason Wang
2020-12-16 6:48 ` [PATCH 05/21] vdpa: add the missing comment for nvqs in struct vdpa_device Jason Wang
2020-12-16 6:48 ` [PATCH 06/21] vdpa: introduce virtqueue groups Jason Wang
2021-01-04 10:04 ` Stefan Hajnoczi
2021-01-05 4:13 ` Jason Wang
2020-12-16 6:48 ` [PATCH 07/21] vdpa: multiple address spaces support Jason Wang
2020-12-29 7:28 ` Eli Cohen
2020-12-30 4:00 ` Jason Wang
2020-12-30 4:04 ` Jason Wang
2020-12-30 9:44 ` Eli Cohen
2020-12-16 6:48 ` [PATCH 08/21] vdpa: introduce config operations for associating ASID to a virtqueue group Jason Wang
2020-12-16 6:48 ` [PATCH 09/21] vhost_iotlb: split out IOTLB initialization Jason Wang
2020-12-16 6:48 ` [PATCH 10/21] vhost: support ASID in IOTLB API Jason Wang
2020-12-29 10:20 ` Eli Cohen
2020-12-30 4:27 ` Jason Wang
2020-12-16 6:48 ` [PATCH 11/21] vhost-vdpa: introduce asid based IOTLB Jason Wang
2020-12-29 11:41 ` Eli Cohen
2020-12-30 6:23 ` Jason Wang
2020-12-29 11:53 ` Eli Cohen
2020-12-30 6:34 ` Jason Wang
2020-12-29 12:05 ` Eli Cohen
2020-12-30 6:33 ` Jason Wang
2020-12-16 6:48 ` [PATCH 12/21] vhost-vdpa: introduce uAPI to get the number of virtqueue groups Jason Wang
2020-12-29 12:24 ` Eli Cohen
2020-12-30 6:49 ` Jason Wang
2020-12-30 10:05 ` Eli Cohen
2020-12-31 2:36 ` Jason Wang
2020-12-16 6:48 ` [PATCH 13/21] vhost-vdpa: introduce uAPI to get the number of address spaces Jason Wang
2020-12-16 6:48 ` [PATCH 14/21] vhost-vdpa: uAPI to get virtqueue group id Jason Wang
2020-12-16 6:48 ` [PATCH 15/21] vhost-vdpa: introduce uAPI to set group ASID Jason Wang
2020-12-16 6:48 ` [PATCH 16/21] vhost-vdpa: support ASID based IOTLB API Jason Wang
2020-12-16 6:48 ` [PATCH 17/21] vdpa_sim: split vdpasim_virtqueue's iov field in out_iov and in_iov Jason Wang
2020-12-16 6:48 ` [PATCH 18/21] vdpa_sim: advertise VIRTIO_NET_F_MTU Jason Wang
2020-12-16 6:48 ` [PATCH 19/21] vdpa_sim: factor out buffer completion logic Jason Wang
2020-12-16 6:48 ` [PATCH 20/21] vdpa_sim: filter destination mac address Jason Wang
2020-12-16 6:48 ` [PATCH 21/21] vdpasim: control virtqueue support Jason Wang
2020-12-17 20:19 ` kernel test robot
2021-01-11 12:26 ` Eli Cohen
2021-01-12 3:11 ` Jason Wang
2021-01-22 19:43 ` Eugenio Perez Martin
2021-01-25 3:16 ` Jason Wang
2020-12-16 9:47 ` [PATCH 00/21] Control VQ support in vDPA Michael S. Tsirkin
2020-12-17 3:30 ` Jason Wang
2020-12-17 7:58 ` Michael S. Tsirkin
2020-12-17 9:02 ` Jason Wang
2020-12-17 22:28 ` Michael S. Tsirkin
2020-12-18 2:56 ` Jason Wang
2020-12-17 7:26 ` Eli Cohen
2022-02-24 21:22 ` [RFC PATCH v2 00/19] " Gautam Dawar
2022-02-24 21:22 ` [RFC PATCH v2 01/19] vhost: move the backend feature bits to vhost_types.h Gautam Dawar
2022-02-24 21:22 ` [RFC PATCH v2 02/19] virtio-vdpa: don't set callback if virtio doesn't need it Gautam Dawar
2022-02-24 21:22 ` [RFC PATCH v2 03/19] vhost-vdpa: passing iotlb to IOMMU mapping helpers Gautam Dawar
2022-02-24 21:22 ` [RFC PATCH v2 04/19] vhost-vdpa: switch to use vhost-vdpa specific IOTLB Gautam Dawar
2022-02-24 21:22 ` [RFC PATCH v2 05/19] vdpa: introduce virtqueue groups Gautam Dawar
2022-02-28 8:07 ` Jason Wang
2022-02-28 10:57 ` Gautam Dawar
2022-02-24 21:22 ` [RFC PATCH v2 06/19] vdpa: multiple address spaces support Gautam Dawar
2022-03-03 19:39 ` Eugenio Perez Martin
2022-03-04 6:30 ` Gautam Dawar
2022-03-04 17:45 ` Eugenio Perez Martin
2022-02-24 21:22 ` [RFC PATCH v2 07/19] vdpa: introduce config operations for associating ASID to a virtqueue group Gautam Dawar
2022-03-04 9:54 ` Eugenio Perez Martin
2022-03-04 17:48 ` Gautam Dawar
2022-02-24 21:22 ` [RFC PATCH v2 08/19] vhost_iotlb: split out IOTLB initialization Gautam Dawar
2022-02-24 21:22 ` [RFC PATCH v2 09/19] vhost: support ASID in IOTLB API Gautam Dawar
2022-03-04 10:25 ` Eugenio Perez Martin
2022-03-04 17:52 ` Gautam Dawar
2022-02-24 21:22 ` [RFC PATCH v2 10/19] vhost-vdpa: introduce asid based IOTLB Gautam Dawar
2022-03-04 17:56 ` Eugenio Perez Martin
2022-03-07 10:07 ` Gautam Dawar
2022-02-24 21:22 ` [RFC PATCH v2 11/19] vhost-vdpa: introduce uAPI to get the number of virtqueue groups Gautam Dawar
2022-02-24 21:22 ` [RFC PATCH v2 12/19] vhost-vdpa: introduce uAPI to get the number of address spaces Gautam Dawar
2022-02-24 21:22 ` [RFC PATCH v2 13/19] vhost-vdpa: uAPI to get virtqueue group id Gautam Dawar
2022-02-24 21:22 ` [RFC PATCH v2 14/19] vhost-vdpa: introduce uAPI to set group ASID Gautam Dawar
2022-02-24 21:22 ` [RFC PATCH v2 15/19] vhost-vdpa: support ASID based IOTLB API Gautam Dawar
2022-03-04 18:04 ` Eugenio Perez Martin
2022-03-07 10:23 ` Gautam Dawar
2022-02-24 21:22 ` [RFC PATCH v2 16/19] vdpa_sim: advertise VIRTIO_NET_F_MTU Gautam Dawar
2022-03-10 17:55 ` Eugenio Perez Martin
2022-02-24 21:22 ` [RFC PATCH v2 17/19] vdpa_sim: factor out buffer completion logic Gautam Dawar
2022-02-24 21:22 ` [RFC PATCH v2 18/19] vdpa_sim: filter destination mac address Gautam Dawar
2022-03-10 18:22 ` Eugenio Perez Martin
2022-02-24 21:22 ` [RFC PATCH v2 19/19] vdpasim: control virtqueue support Gautam Dawar
2022-03-10 18:20 ` Eugenio Perez Martin
2022-03-18 7:35 ` Eugenio Perez Martin
2022-03-22 8:46 ` Gautam Dawar
2022-02-28 8:17 ` [RFC PATCH v2 00/19] Control VQ support in vDPA Jason Wang
2022-02-28 10:56 ` Gautam Dawar [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=BY5PR02MB6980DC90EE52FE1900F7C414B1019@BY5PR02MB6980.namprd02.prod.outlook.com \
--to=gdawar@xilinx.com \
--cc=elic@nvidia.com \
--cc=eperezma@redhat.com \
--cc=hanand@xilinx.com \
--cc=jasowang@redhat.com \
--cc=kvm@vger.kernel.org \
--cc=lingshan.zhu@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=longpeng2@huawei.com \
--cc=martinh@xilinx.com \
--cc=mst@redhat.com \
--cc=netdev@vger.kernel.org \
--cc=parav@nvidia.com \
--cc=sgarzare@redhat.com \
--cc=si-wei.liu@oracle.com \
--cc=tanujk@xilinx.com \
--cc=virtualization@lists.linux-foundation.org \
--cc=xieyongji@bytedance.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).