From: Jason Wang <jasowang@redhat.com>
To: mst@redhat.com, jasowang@redhat.com
Cc: lulu@redhat.com, kvm@vger.kernel.org,
virtualization@lists.linux-foundation.org,
netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
rob.miller@broadcom.com, lingshan.zhu@intel.com,
eperezma@redhat.com, hanand@xilinx.com, mhabets@solarflare.com,
eli@mellanox.com, amorenoz@redhat.com,
maxime.coquelin@redhat.com, stefanha@redhat.com,
sgarzare@redhat.com
Subject: [RFC PATCH 00/24] Control VQ support in vDPA
Date: Thu, 24 Sep 2020 11:21:01 +0800 [thread overview]
Message-ID: <20200924032125.18619-1-jasowang@redhat.com> (raw)
Hi All:
This series tries to add the support for control virtqueue in vDPA.
Control virtqueue is used by networking device for accepting various
commands from the driver. It's a must to support multiqueue and other
configurations.
When used by vhost-vDPA bus driver for VM, the control virtqueue
should be shadowed via userspace VMM (Qemu) instead of being assigned
directly to Guest. This is because Qemu needs to know the device state
in order to start and stop device correctly (e.g for Live Migration).
This requies to isolate the memory mapping for control virtqueue
presented by vhost-vDPA to prevent guest from accesing it directly.
To achieve this, vDPA introduce two new abstractions:
- address space: identified through address space id (ASID) and a set
of memory mapping in maintained
- virtqueue group: the minimal set of virtqueues that must share an
address space
Device needs to advertise the following attributes to vDPA:
- the number of address spaces supported in the device
- the number of virtqueue groups supported in the device
- the mappings from a specific virtqueue to its virtqueue groups
The mappings from virtqueue to virtqueue groups is fixed and defined
by vDPA device driver. E.g:
- For the device that has hardware ASID support, it can simply
advertise a per virtqueue virtqueue group.
- For the device that does not have hardware ASID support, it can
simply advertise a single virtqueue group that contains all
virtqueues. Or if it wants a software emulated control virtqueue, it
can advertise two virtqueue groups, one is for cvq, another is for
the rest virtqueues.
vDPA also allow to change the association between virtqueue group and
address space. So in the case of control virtqueue, userspace
VMM(Qemu) may use a dedicated address space for the control virtqueue
group to isolate the memory mapping.
The vhost/vhost-vDPA is also extend for the userspace to:
- query the number of virtqueue groups and address spaces supported by
the device
- query the virtqueue group for a specific virtqueue
- assocaite a virtqueue group with an address space
- send ASID based IOTLB commands
This will help userspace VMM(Qemu) to detect whether the control vq
could be supported and isolate memory mappings of control virtqueue
from the others.
To demonstrate the usage, vDPA simulator is extended to support
setting MAC address via a emulated control virtqueue. Please refer
patch 24 for more implementation details.
Please review.
Note that patch 1 and a equivalent of patch 2 have been posted in the
list. Those two are requirement for this series to work, so I add them
here.
Thank
Jason Wang (24):
vhost-vdpa: fix backend feature ioctls
vhost-vdpa: fix vqs leak in vhost_vdpa_open()
vhost: move the backend feature bits to vhost_types.h
virtio-vdpa: don't set callback if virtio doesn't need it
vhost-vdpa: passing iotlb to IOMMU mapping helpers
vhost-vdpa: switch to use vhost-vdpa specific IOTLB
vdpa: add the missing comment for nvqs in struct vdpa_device
vdpa: introduce virtqueue groups
vdpa: multiple address spaces support
vdpa: introduce config operations for associating ASID to a virtqueue
group
vhost_iotlb: split out IOTLB initialization
vhost: support ASID in IOTLB API
vhost-vdpa: introduce ASID based IOTLB
vhost-vdpa: introduce uAPI to get the number of virtqueue groups
vhost-vdpa: introduce uAPI to get the number of address spaces
vhost-vdpa: uAPI to get virtqueue group id
vhost-vdpa: introduce uAPI to set group ASID
vhost-vdpa: support ASID based IOTLB API
vdpa_sim: use separated iov for reading and writing
vdpa_sim: advertise VIRTIO_NET_F_MTU
vdpa_sim: advertise VIRTIO_NET_F_MAC
vdpa_sim: factor out buffer completion logic
vdpa_sim: filter destination mac address
vdpasim: control virtqueue support
drivers/vdpa/ifcvf/ifcvf_main.c | 9 +-
drivers/vdpa/mlx5/net/mlx5_vnet.c | 11 +-
drivers/vdpa/vdpa.c | 8 +-
drivers/vdpa/vdpa_sim/vdpa_sim.c | 293 ++++++++++++++++++++++++------
drivers/vhost/iotlb.c | 23 ++-
drivers/vhost/vdpa.c | 259 ++++++++++++++++++++------
drivers/vhost/vhost.c | 23 ++-
drivers/vhost/vhost.h | 4 +-
drivers/virtio/virtio_vdpa.c | 2 +-
include/linux/vdpa.h | 42 ++++-
include/linux/vhost_iotlb.h | 2 +
include/uapi/linux/vhost.h | 19 +-
include/uapi/linux/vhost_types.h | 10 +-
13 files changed, 556 insertions(+), 149 deletions(-)
--
2.20.1
next reply other threads:[~2020-09-24 3:21 UTC|newest]
Thread overview: 55+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-24 3:21 Jason Wang [this message]
2020-09-24 3:21 ` [RFC PATCH 01/24] vhost-vdpa: fix backend feature ioctls Jason Wang
2020-09-24 7:16 ` Eli Cohen
2020-09-24 7:26 ` Jason Wang
2020-09-24 7:38 ` Eli Cohen
2020-09-24 7:50 ` Michael S. Tsirkin
2020-09-24 8:28 ` Jason Wang
2020-09-24 3:21 ` [RFC PATCH 02/24] vhost-vdpa: fix vqs leak in vhost_vdpa_open() Jason Wang
2020-09-24 7:48 ` Eli Cohen
2020-09-25 11:41 ` Jason Wang
2020-09-24 9:31 ` Michael S. Tsirkin
2020-09-25 11:27 ` Jason Wang
2020-09-24 3:21 ` [RFC PATCH 03/24] vhost: move the backend feature bits to vhost_types.h Jason Wang
2020-09-24 3:21 ` [RFC PATCH 04/24] virtio-vdpa: don't set callback if virtio doesn't need it Jason Wang
2020-09-24 3:21 ` [RFC PATCH 05/24] vhost-vdpa: passing iotlb to IOMMU mapping helpers Jason Wang
2020-09-30 11:26 ` Eli Cohen
2020-10-09 2:01 ` Jason Wang
2020-09-24 3:21 ` [RFC PATCH 06/24] vhost-vdpa: switch to use vhost-vdpa specific IOTLB Jason Wang
2020-09-30 12:02 ` Eli Cohen
2020-10-09 3:46 ` Jason Wang
2020-09-24 3:21 ` [RFC PATCH 07/24] vdpa: add the missing comment for nvqs in struct vdpa_device Jason Wang
2020-09-24 3:21 ` [RFC PATCH 08/24] vdpa: introduce virtqueue groups Jason Wang
2020-09-28 15:44 ` Eugenio Perez Martin
2020-10-09 3:49 ` Jason Wang
2020-09-24 3:21 ` [RFC PATCH 09/24] vdpa: multiple address spaces support Jason Wang
2020-10-01 13:21 ` Eli Cohen
2020-10-09 3:51 ` Jason Wang
2020-10-01 13:23 ` Eli Cohen
2020-10-09 3:52 ` Jason Wang
2020-09-24 3:21 ` [RFC PATCH 10/24] vdpa: introduce config operations for associating ASID to a virtqueue group Jason Wang
2020-10-01 13:29 ` Eli Cohen
2020-10-09 3:56 ` Jason Wang
2020-10-12 6:59 ` Eli Cohen
2020-10-12 7:45 ` Jason Wang
2020-10-12 8:17 ` Eli Cohen
2020-10-13 5:40 ` Jason Wang
2020-09-24 3:21 ` [RFC PATCH 11/24] vhost_iotlb: split out IOTLB initialization Jason Wang
2020-09-24 3:21 ` [RFC PATCH 12/24] vhost: support ASID in IOTLB API Jason Wang
2020-09-24 3:21 ` [RFC PATCH 13/24] vhost-vdpa: introduce ASID based IOTLB Jason Wang
2020-09-29 14:40 ` Eugenio Perez Martin
2020-09-24 3:21 ` [RFC PATCH 14/24] vhost-vdpa: introduce uAPI to get the number of virtqueue groups Jason Wang
2020-09-24 3:21 ` [RFC PATCH 15/24] vhost-vdpa: introduce uAPI to get the number of address spaces Jason Wang
2020-09-24 3:21 ` [RFC PATCH 16/24] vhost-vdpa: uAPI to get virtqueue group id Jason Wang
2020-09-24 3:21 ` [RFC PATCH 17/24] vhost-vdpa: introduce uAPI to set group ASID Jason Wang
2020-09-24 3:21 ` [RFC PATCH 18/24] vhost-vdpa: support ASID based IOTLB API Jason Wang
2020-09-28 15:44 ` Eugenio Perez Martin
2020-10-09 3:59 ` Jason Wang
2020-09-24 3:21 ` [RFC PATCH 19/24] vdpa_sim: use separated iov for reading and writing Jason Wang
2020-09-24 3:21 ` [RFC PATCH 20/24] vdpa_sim: advertise VIRTIO_NET_F_MTU Jason Wang
2020-09-24 3:21 ` [RFC PATCH 21/24] vdpa_sim: advertise VIRTIO_NET_F_MAC Jason Wang
2020-09-24 3:21 ` [RFC PATCH 22/24] vdpa_sim: factor out buffer completion logic Jason Wang
2020-09-24 3:21 ` [RFC PATCH 23/24] vdpa_sim: filter destination mac address Jason Wang
2020-09-24 3:21 ` [RFC PATCH 24/24] vdpasim: control virtqueue support Jason Wang
2020-09-24 10:17 ` [RFC PATCH 00/24] Control VQ support in vDPA Stefan Hajnoczi
2020-09-25 11:36 ` Jason Wang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200924032125.18619-1-jasowang@redhat.com \
--to=jasowang@redhat.com \
--cc=amorenoz@redhat.com \
--cc=eli@mellanox.com \
--cc=eperezma@redhat.com \
--cc=hanand@xilinx.com \
--cc=kvm@vger.kernel.org \
--cc=lingshan.zhu@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lulu@redhat.com \
--cc=maxime.coquelin@redhat.com \
--cc=mhabets@solarflare.com \
--cc=mst@redhat.com \
--cc=netdev@vger.kernel.org \
--cc=rob.miller@broadcom.com \
--cc=sgarzare@redhat.com \
--cc=stefanha@redhat.com \
--cc=virtualization@lists.linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).