* [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern
@ 2022-08-16 1:06 Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 01/24] virtio: sync relevant definitions with linux Kangjie Xu
` (27 more replies)
0 siblings, 28 replies; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
The virtio queue reset function has already been defined in the virtio spec 1.2.
The relevant virtio spec information is here:
https://github.com/oasis-tcs/virtio-spec/issues/124
https://github.com/oasis-tcs/virtio-spec/issues/139
This patch set is to support this function in QEMU. It consists of several parts:
1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
3. Patches 13-19 support vq stop and vq restart for vhost-user.
4. Patches 20-22 support vq reset and re-enable for virtio-net.
5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.
The process of virtqueue reset can be concluded as:
1. The virtqueue is disabled when VIRTIO_PCI_COMMON_Q_RESET is written.
2. Then the virtqueue can be optionally restarted(re-enabled).
Since this patch set involves multiple modules and seems a bit messy, we briefly describe the
calling process for different modes below.
virtio-net:
1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
-> virtio_queue_reset() [virtio]
-> virtio_net_queue_reset() [virtio-net]
-> __virtio_queue_reset()
2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
-> set enabled, reset status of vq.
vhost-kernel:
1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
-> virtio_queue_reset() [virtio]
-> virtio_net_queue_reset() [virtio-net]
-> vhost_net_virtqueue_stop() [vhost-net]
-> vhost_net_set_backend() [vhost]
-> vhost_dev_virtqueue_stop()
-> vhost_virtqueue_unmap()
-> __virtio_queue_reset()
2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
-> virtio_queue_enable() [virtio]
-> virtio_net_queue_enable() [virtio-net]
-> vhost_net_virtqueue_restart() [vhost-net]
-> vhost_dev_virtqueue_restart() [vhost]
-> vhost_virtqueue_start()
-> vhost_net_set_backend()
-> set enabled, reset status of vq.
vhost-user:
1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
-> virtio_queue_reset() [virtio]
-> virtio_net_queue_reset() [virtio-net]
-> vhost_net_virtqueue_stop() [vhost-net]
-> vhost_dev_virtqueue_stop() [vhost]
-> vhost_user_reset_vring() [vhost-user]
-> send VHOST_USER_RESET_VRING to the device
-> vhost_virtqueue_unmap()
-> __virtio_queue_reset()
2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
-> virtio_queue_enable() [virtio]
-> virtio_net_queue_enable() [virtio-net]
-> vhost_net_virtqueue_restart() [vhost-net]
-> vhost_dev_virtqueue_restart() [vhost]
-> vhost_virtqueue_start()
-> vhost_user_set_single_vring_enable [vhost-user]
-> send VHOST_USER_SET_VRING_ENABLE to the device
-> set enabled, reset status of vq.
Test environment:
Host: 5.19.0-rc3 (With vq reset support)
Qemu: QEMU emulator version 7.0.50
Guest: 5.19.0-rc3 (With vq reset support)
DPDK: 22.07-rc1 (With vq reset support)
Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;
The drvier can resize the virtio queue, then virtio queue reset function should
be triggered.
The default is split mode, modify Qemu virtio-net to add PACKED feature to
test packed mode.
Guest Kernel Patch:
https://lore.kernel.org/bpf/20220801063902.129329-1-xuanzhuo@linux.alibaba.com/
DPDK Patch:
https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7
Host Kernel Patch:
https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3
Looking forward to your review and comments. Thanks.
Kangjie Xu (19):
virtio: introduce virtio_queue_enable()
virtio: core: vq reset feature negotation support
virtio-pci: support queue enable
vhost: extract the logic of unmapping the vrings and desc
vhost: introduce vhost_dev_virtqueue_stop()
vhost: introduce vhost_dev_virtqueue_restart()
vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
docs: vhost-user: add VHOST_USER_RESET_VRING message
vhost-user: introduce vhost_reset_vring() interface
vhost-user: add op to enable or disable a single vring
vhost: vhost-user: update vhost_dev_virtqueue_stop()
vhost: vhost-user: update vhost_dev_virtqueue_restart()
vhost-net: vhost-user: update vhost_net_virtqueue_stop()
vhost-net: vhost-user: update vhost_net_virtqueue_restart()
virtio-net: introduce flush_or_purge_queued_packets()
virtio-net: support queue_enable
vhost: vhost-kernel: enable vq reset feature
vhost: vhost-user: enable vq reset feature
Xuan Zhuo (5):
virtio: sync relevant definitions with linux
virtio: introduce __virtio_queue_reset()
virtio: introduce virtio_queue_reset()
virtio-pci: support queue reset
virtio-net: support queue reset
docs/interop/vhost-user.rst | 10 +++
hw/core/machine.c | 1 +
hw/net/vhost_net.c | 79 +++++++++++++++++++
hw/net/virtio-net.c | 58 ++++++++++++--
hw/virtio/vhost-user.c | 67 ++++++++++++++--
hw/virtio/vhost.c | 79 +++++++++++++++++--
hw/virtio/virtio-pci.c | 20 +++++
hw/virtio/virtio.c | 62 +++++++++++----
include/hw/virtio/vhost-backend.h | 6 ++
include/hw/virtio/vhost.h | 5 ++
include/hw/virtio/virtio-pci.h | 1 +
include/hw/virtio/virtio.h | 8 +-
include/net/vhost_net.h | 4 +
.../standard-headers/linux/virtio_config.h | 5 ++
include/standard-headers/linux/virtio_pci.h | 2 +
15 files changed, 371 insertions(+), 36 deletions(-)
--
2.32.0
^ permalink raw reply [flat|nested] 77+ messages in thread
* [PATCH v2 01/24] virtio: sync relevant definitions with linux
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 02/24] virtio: introduce __virtio_queue_reset() Kangjie Xu
` (26 subsequent siblings)
27 siblings, 0 replies; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
This is updated using scripts/update-linux-headers.sh.
Added VIRTIO_F_RING_RESET, VIRTIO_PCI_COMMON_Q_RESET. It came from here:
https://github.com/oasis-tcs/virtio-spec/issues/124
https://github.com/oasis-tcs/virtio-spec/issues/139
Add VIRTIO_PCI_COMMON_Q_NDATA, which comes from here:
https://github.com/oasis-tcs/virtio-spec/issues/89
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
---
include/standard-headers/linux/virtio_config.h | 5 +++++
include/standard-headers/linux/virtio_pci.h | 2 ++
2 files changed, 7 insertions(+)
diff --git a/include/standard-headers/linux/virtio_config.h b/include/standard-headers/linux/virtio_config.h
index 7acd8d4abc..47a7eef5e4 100644
--- a/include/standard-headers/linux/virtio_config.h
+++ b/include/standard-headers/linux/virtio_config.h
@@ -96,4 +96,9 @@
* Does the device support Single Root I/O Virtualization?
*/
#define VIRTIO_F_SR_IOV 37
+
+/*
+ * This feature indicates that the driver can reset a queue individually.
+ */
+#define VIRTIO_F_RING_RESET 40
#endif /* _LINUX_VIRTIO_CONFIG_H */
diff --git a/include/standard-headers/linux/virtio_pci.h b/include/standard-headers/linux/virtio_pci.h
index db7a8e2fcb..be912cfc95 100644
--- a/include/standard-headers/linux/virtio_pci.h
+++ b/include/standard-headers/linux/virtio_pci.h
@@ -202,6 +202,8 @@ struct virtio_pci_cfg_cap {
#define VIRTIO_PCI_COMMON_Q_AVAILHI 44
#define VIRTIO_PCI_COMMON_Q_USEDLO 48
#define VIRTIO_PCI_COMMON_Q_USEDHI 52
+#define VIRTIO_PCI_COMMON_Q_NDATA 56
+#define VIRTIO_PCI_COMMON_Q_RESET 58
#endif /* VIRTIO_PCI_NO_MODERN */
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 02/24] virtio: introduce __virtio_queue_reset()
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 01/24] virtio: sync relevant definitions with linux Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-23 7:31 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 03/24] virtio: introduce virtio_queue_reset() Kangjie Xu
` (25 subsequent siblings)
27 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Separate the logic of vq reset. This logic will be called directly
later.
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
hw/virtio/virtio.c | 37 +++++++++++++++++++++----------------
1 file changed, 21 insertions(+), 16 deletions(-)
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 5d607aeaa0..67d54832a9 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -2019,6 +2019,26 @@ static enum virtio_device_endian virtio_current_cpu_endian(void)
}
}
+static void __virtio_queue_reset(VirtIODevice *vdev, uint32_t i)
+{
+ vdev->vq[i].vring.desc = 0;
+ vdev->vq[i].vring.avail = 0;
+ vdev->vq[i].vring.used = 0;
+ vdev->vq[i].last_avail_idx = 0;
+ vdev->vq[i].shadow_avail_idx = 0;
+ vdev->vq[i].used_idx = 0;
+ vdev->vq[i].last_avail_wrap_counter = true;
+ vdev->vq[i].shadow_avail_wrap_counter = true;
+ vdev->vq[i].used_wrap_counter = true;
+ virtio_queue_set_vector(vdev, i, VIRTIO_NO_VECTOR);
+ vdev->vq[i].signalled_used = 0;
+ vdev->vq[i].signalled_used_valid = false;
+ vdev->vq[i].notification = true;
+ vdev->vq[i].vring.num = vdev->vq[i].vring.num_default;
+ vdev->vq[i].inuse = 0;
+ virtio_virtqueue_reset_region_cache(&vdev->vq[i]);
+}
+
void virtio_reset(void *opaque)
{
VirtIODevice *vdev = opaque;
@@ -2050,22 +2070,7 @@ void virtio_reset(void *opaque)
virtio_notify_vector(vdev, vdev->config_vector);
for(i = 0; i < VIRTIO_QUEUE_MAX; i++) {
- vdev->vq[i].vring.desc = 0;
- vdev->vq[i].vring.avail = 0;
- vdev->vq[i].vring.used = 0;
- vdev->vq[i].last_avail_idx = 0;
- vdev->vq[i].shadow_avail_idx = 0;
- vdev->vq[i].used_idx = 0;
- vdev->vq[i].last_avail_wrap_counter = true;
- vdev->vq[i].shadow_avail_wrap_counter = true;
- vdev->vq[i].used_wrap_counter = true;
- virtio_queue_set_vector(vdev, i, VIRTIO_NO_VECTOR);
- vdev->vq[i].signalled_used = 0;
- vdev->vq[i].signalled_used_valid = false;
- vdev->vq[i].notification = true;
- vdev->vq[i].vring.num = vdev->vq[i].vring.num_default;
- vdev->vq[i].inuse = 0;
- virtio_virtqueue_reset_region_cache(&vdev->vq[i]);
+ __virtio_queue_reset(vdev, i);
}
}
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 03/24] virtio: introduce virtio_queue_reset()
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 01/24] virtio: sync relevant definitions with linux Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 02/24] virtio: introduce __virtio_queue_reset() Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-23 7:32 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 04/24] virtio: introduce virtio_queue_enable() Kangjie Xu
` (24 subsequent siblings)
27 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Introduce a new interface function virtio_queue_reset() to implement
reset for vq.
Add a new callback to VirtioDeviceClass for queue reset operation for
each child device.
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
hw/virtio/virtio.c | 11 +++++++++++
include/hw/virtio/virtio.h | 2 ++
2 files changed, 13 insertions(+)
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 67d54832a9..0e9d41366f 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -2039,6 +2039,17 @@ static void __virtio_queue_reset(VirtIODevice *vdev, uint32_t i)
virtio_virtqueue_reset_region_cache(&vdev->vq[i]);
}
+void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
+{
+ VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
+
+ if (k->queue_reset) {
+ k->queue_reset(vdev, queue_index);
+ }
+
+ __virtio_queue_reset(vdev, queue_index);
+}
+
void virtio_reset(void *opaque)
{
VirtIODevice *vdev = opaque;
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index db1c0ddf6b..879394299b 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -130,6 +130,7 @@ struct VirtioDeviceClass {
void (*set_config)(VirtIODevice *vdev, const uint8_t *config);
void (*reset)(VirtIODevice *vdev);
void (*set_status)(VirtIODevice *vdev, uint8_t val);
+ void (*queue_reset)(VirtIODevice *vdev, uint32_t queue_index);
/* For transitional devices, this is a bitmap of features
* that are only exposed on the legacy interface but not
* the modern one.
@@ -268,6 +269,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,
MemoryRegion *mr, bool assign);
int virtio_set_status(VirtIODevice *vdev, uint8_t val);
void virtio_reset(void *opaque);
+void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index);
void virtio_update_irq(VirtIODevice *vdev);
int virtio_set_features(VirtIODevice *vdev, uint64_t val);
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 04/24] virtio: introduce virtio_queue_enable()
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (2 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 03/24] virtio: introduce virtio_queue_reset() Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-23 7:37 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 05/24] virtio: core: vq reset feature negotation support Kangjie Xu
` (23 subsequent siblings)
27 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
Introduce the interface queue_enable() in VirtioDeviceClass and the
fucntion virtio_queue_enable() in virtio, it can be called when
VIRTIO_PCI_COMMON_Q_ENABLE is written and related virtqueue can be
started. It only supports the devices of virtio 1 or later. The
not-supported devices can only start the virtqueue when DRIVER_OK.
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
hw/virtio/virtio.c | 14 ++++++++++++++
include/hw/virtio/virtio.h | 2 ++
2 files changed, 16 insertions(+)
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 0e9d41366f..141f18c633 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -2050,6 +2050,20 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
__virtio_queue_reset(vdev, queue_index);
}
+void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index)
+{
+ VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
+
+ if (!virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
+ error_report("queue_enable is only suppported in devices of virtio "
+ "1.0 or later.");
+ }
+
+ if (k->queue_enable) {
+ k->queue_enable(vdev, queue_index);
+ }
+}
+
void virtio_reset(void *opaque)
{
VirtIODevice *vdev = opaque;
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index 879394299b..085997d8f3 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -131,6 +131,7 @@ struct VirtioDeviceClass {
void (*reset)(VirtIODevice *vdev);
void (*set_status)(VirtIODevice *vdev, uint8_t val);
void (*queue_reset)(VirtIODevice *vdev, uint32_t queue_index);
+ void (*queue_enable)(VirtIODevice *vdev, uint32_t queue_index);
/* For transitional devices, this is a bitmap of features
* that are only exposed on the legacy interface but not
* the modern one.
@@ -270,6 +271,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,
int virtio_set_status(VirtIODevice *vdev, uint8_t val);
void virtio_reset(void *opaque);
void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index);
+void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index);
void virtio_update_irq(VirtIODevice *vdev);
int virtio_set_features(VirtIODevice *vdev, uint64_t val);
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 05/24] virtio: core: vq reset feature negotation support
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (3 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 04/24] virtio: introduce virtio_queue_enable() Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-23 7:34 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 06/24] virtio-pci: support queue reset Kangjie Xu
` (22 subsequent siblings)
27 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
A a new command line parameter "queue_reset" is added.
Meanwhile, the vq reset feature is disabled for pre-7.1 machines.
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
hw/core/machine.c | 1 +
include/hw/virtio/virtio.h | 4 +++-
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/hw/core/machine.c b/hw/core/machine.c
index a673302cce..8b22b4647f 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -43,6 +43,7 @@
GlobalProperty hw_compat_7_0[] = {
{ "arm-gicv3-common", "force-8-bit-prio", "on" },
{ "nvme-ns", "eui64-default", "on"},
+ { "virtio-device", "queue_reset", "false" },
};
const size_t hw_compat_7_0_len = G_N_ELEMENTS(hw_compat_7_0);
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index 085997d8f3..ed3ecbef80 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -295,7 +295,9 @@ typedef struct VirtIORNGConf VirtIORNGConf;
DEFINE_PROP_BIT64("iommu_platform", _state, _field, \
VIRTIO_F_IOMMU_PLATFORM, false), \
DEFINE_PROP_BIT64("packed", _state, _field, \
- VIRTIO_F_RING_PACKED, false)
+ VIRTIO_F_RING_PACKED, false), \
+ DEFINE_PROP_BIT64("queue_reset", _state, _field, \
+ VIRTIO_F_RING_RESET, true)
hwaddr virtio_queue_get_desc_addr(VirtIODevice *vdev, int n);
bool virtio_queue_enabled_legacy(VirtIODevice *vdev, int n);
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 06/24] virtio-pci: support queue reset
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (4 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 05/24] virtio: core: vq reset feature negotation support Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-23 7:40 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 07/24] virtio-pci: support queue enable Kangjie Xu
` (21 subsequent siblings)
27 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
PCI devices support vq reset.
Based on this function, the driver can adjust the size of the ring, and
quickly recycle the buffer in the ring.
The migration of the virtio devices will not happen during a reset
operation. This is becuase the global iothread lock is held. Migration
thread also needs the lock. As a result, we do not need to migrate the
reset state of VirtIOPCIQueue.
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
---
hw/virtio/virtio-pci.c | 19 +++++++++++++++++++
include/hw/virtio/virtio-pci.h | 1 +
2 files changed, 20 insertions(+)
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index 45327f0b31..ec8e92052f 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -1246,6 +1246,9 @@ static uint64_t virtio_pci_common_read(void *opaque, hwaddr addr,
case VIRTIO_PCI_COMMON_Q_USEDHI:
val = proxy->vqs[vdev->queue_sel].used[1];
break;
+ case VIRTIO_PCI_COMMON_Q_RESET:
+ val = proxy->vqs[vdev->queue_sel].reset;
+ break;
default:
val = 0;
}
@@ -1333,6 +1336,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,
((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << 32 |
proxy->vqs[vdev->queue_sel].used[0]);
proxy->vqs[vdev->queue_sel].enabled = 1;
+ proxy->vqs[vdev->queue_sel].reset = 0;
} else {
virtio_error(vdev, "wrong value for queue_enable %"PRIx64, val);
}
@@ -1355,6 +1359,20 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,
case VIRTIO_PCI_COMMON_Q_USEDHI:
proxy->vqs[vdev->queue_sel].used[1] = val;
break;
+ case VIRTIO_PCI_COMMON_Q_RESET:
+ if (val == 1) {
+ /*
+ * With the global iothread lock taken, the migration will not
+ * happen until the virtqueue reset is done.
+ */
+ proxy->vqs[vdev->queue_sel].reset = 1;
+
+ virtio_queue_reset(vdev, vdev->queue_sel);
+
+ proxy->vqs[vdev->queue_sel].reset = 0;
+ proxy->vqs[vdev->queue_sel].enabled = 0;
+ }
+ break;
default:
break;
}
@@ -1950,6 +1968,7 @@ static void virtio_pci_reset(DeviceState *qdev)
for (i = 0; i < VIRTIO_QUEUE_MAX; i++) {
proxy->vqs[i].enabled = 0;
+ proxy->vqs[i].reset = 0;
proxy->vqs[i].num = 0;
proxy->vqs[i].desc[0] = proxy->vqs[i].desc[1] = 0;
proxy->vqs[i].avail[0] = proxy->vqs[i].avail[1] = 0;
diff --git a/include/hw/virtio/virtio-pci.h b/include/hw/virtio/virtio-pci.h
index 2446dcd9ae..e9290e2b94 100644
--- a/include/hw/virtio/virtio-pci.h
+++ b/include/hw/virtio/virtio-pci.h
@@ -117,6 +117,7 @@ typedef struct VirtIOPCIRegion {
typedef struct VirtIOPCIQueue {
uint16_t num;
bool enabled;
+ bool reset;
uint32_t desc[2];
uint32_t avail[2];
uint32_t used[2];
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 07/24] virtio-pci: support queue enable
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (5 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 06/24] virtio-pci: support queue reset Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-23 7:44 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 08/24] vhost: extract the logic of unmapping the vrings and desc Kangjie Xu
` (20 subsequent siblings)
27 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
PCI devices support vq enable.
Based on this function, the driver can re-enable the virtqueue after the
virtqueue is reset.
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
hw/virtio/virtio-pci.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index ec8e92052f..3d560e45ad 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -1335,6 +1335,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,
proxy->vqs[vdev->queue_sel].avail[0],
((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << 32 |
proxy->vqs[vdev->queue_sel].used[0]);
+ virtio_queue_enable(vdev, vdev->queue_sel);
proxy->vqs[vdev->queue_sel].enabled = 1;
proxy->vqs[vdev->queue_sel].reset = 0;
} else {
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 08/24] vhost: extract the logic of unmapping the vrings and desc
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (6 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 07/24] virtio-pci: support queue enable Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-23 7:45 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 09/24] vhost: introduce vhost_dev_virtqueue_stop() Kangjie Xu
` (19 subsequent siblings)
27 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
Introduce vhost_virtqueue_unmap() to ummap the vrings and desc
of a virtqueue.
The function will be used later.
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
hw/virtio/vhost.c | 20 ++++++++++++++------
1 file changed, 14 insertions(+), 6 deletions(-)
diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index 0827d631c0..e467dfc7bc 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -1197,6 +1197,19 @@ fail_alloc_desc:
return r;
}
+static void vhost_virtqueue_unmap(struct vhost_dev *dev,
+ struct VirtIODevice *vdev,
+ struct vhost_virtqueue *vq,
+ unsigned idx)
+{
+ vhost_memory_unmap(dev, vq->used, virtio_queue_get_used_size(vdev, idx),
+ 1, virtio_queue_get_used_size(vdev, idx));
+ vhost_memory_unmap(dev, vq->avail, virtio_queue_get_avail_size(vdev, idx),
+ 0, virtio_queue_get_avail_size(vdev, idx));
+ vhost_memory_unmap(dev, vq->desc, virtio_queue_get_desc_size(vdev, idx),
+ 0, virtio_queue_get_desc_size(vdev, idx));
+}
+
static void vhost_virtqueue_stop(struct vhost_dev *dev,
struct VirtIODevice *vdev,
struct vhost_virtqueue *vq,
@@ -1235,12 +1248,7 @@ static void vhost_virtqueue_stop(struct vhost_dev *dev,
vhost_vq_index);
}
- vhost_memory_unmap(dev, vq->used, virtio_queue_get_used_size(vdev, idx),
- 1, virtio_queue_get_used_size(vdev, idx));
- vhost_memory_unmap(dev, vq->avail, virtio_queue_get_avail_size(vdev, idx),
- 0, virtio_queue_get_avail_size(vdev, idx));
- vhost_memory_unmap(dev, vq->desc, virtio_queue_get_desc_size(vdev, idx),
- 0, virtio_queue_get_desc_size(vdev, idx));
+ vhost_virtqueue_unmap(dev, vdev, vq, idx);
}
static void vhost_eventfd_add(MemoryListener *listener,
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 09/24] vhost: introduce vhost_dev_virtqueue_stop()
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (7 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 08/24] vhost: extract the logic of unmapping the vrings and desc Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-23 7:52 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 10/24] vhost: introduce vhost_dev_virtqueue_restart() Kangjie Xu
` (18 subsequent siblings)
27 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
Introduce vhost_dev_virtqueue_stop(), which can ummap the
vrings and the desc of it.
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
hw/virtio/vhost.c | 9 +++++++++
include/hw/virtio/vhost.h | 3 +++
2 files changed, 12 insertions(+)
diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index e467dfc7bc..1bca9ff48d 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -1904,3 +1904,12 @@ int vhost_net_set_backend(struct vhost_dev *hdev,
return -ENOSYS;
}
+
+void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice *vdev,
+ int idx)
+{
+ vhost_virtqueue_unmap(hdev,
+ vdev,
+ hdev->vqs + idx,
+ idx);
+}
diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
index a346f23d13..574888440c 100644
--- a/include/hw/virtio/vhost.h
+++ b/include/hw/virtio/vhost.h
@@ -288,4 +288,7 @@ int vhost_dev_set_inflight(struct vhost_dev *dev,
struct vhost_inflight *inflight);
int vhost_dev_get_inflight(struct vhost_dev *dev, uint16_t queue_size,
struct vhost_inflight *inflight);
+
+void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice *vdev,
+ int idx);
#endif
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 10/24] vhost: introduce vhost_dev_virtqueue_restart()
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (8 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 09/24] vhost: introduce vhost_dev_virtqueue_stop() Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-24 2:37 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 11/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop() Kangjie Xu
` (17 subsequent siblings)
27 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
Introduce vhost_dev_virtqueue_restart(), which can restart the
virtqueue when the vhost has already started running.
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
hw/virtio/vhost.c | 13 +++++++++++++
include/hw/virtio/vhost.h | 2 ++
2 files changed, 15 insertions(+)
diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index 1bca9ff48d..fc3f550c76 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -1913,3 +1913,16 @@ void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice *vdev,
hdev->vqs + idx,
idx);
}
+
+int vhost_dev_virtqueue_restart(struct vhost_dev *hdev, VirtIODevice *vdev,
+ int idx)
+{
+ const VhostOps *vhost_ops = hdev->vhost_ops;
+
+ assert(vhost_ops);
+
+ return vhost_virtqueue_start(hdev,
+ vdev,
+ hdev->vqs + idx,
+ hdev->vq_index + idx);
+}
diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
index 574888440c..b3394b6348 100644
--- a/include/hw/virtio/vhost.h
+++ b/include/hw/virtio/vhost.h
@@ -291,4 +291,6 @@ int vhost_dev_get_inflight(struct vhost_dev *dev, uint16_t queue_size,
void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice *vdev,
int idx);
+int vhost_dev_virtqueue_restart(struct vhost_dev *hdev, VirtIODevice *vdev,
+ int idx);
#endif
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 11/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (9 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 10/24] vhost: introduce vhost_dev_virtqueue_restart() Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-24 2:40 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 12/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart() Kangjie Xu
` (16 subsequent siblings)
27 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
Introduce vhost_virtqueue_stop(), which can reset the virtqueue
in the device. Then it will unmap vrings and the desc of the
virtqueue.
This patch only considers the case for vhost-kernel, when
NetClientDriver is NET_CLIENT_DRIVER_TAP.
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
hw/net/vhost_net.c | 21 +++++++++++++++++++++
include/net/vhost_net.h | 2 ++
2 files changed, 23 insertions(+)
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index ccac5b7a64..aa60dd901c 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -514,3 +514,24 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu)
return vhost_ops->vhost_net_set_mtu(&net->dev, mtu);
}
+
+void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc,
+ int vq_index)
+{
+ VHostNetState *net = get_vhost_net(nc->peer);
+ const VhostOps *vhost_ops = net->dev.vhost_ops;
+ struct vhost_vring_file file = { .fd = -1 };
+ int idx;
+
+ assert(vhost_ops);
+
+ idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index);
+
+ if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) {
+ file.index = idx;
+ int r = vhost_net_set_backend(&net->dev, &file);
+ assert(r >= 0);
+ }
+
+ vhost_dev_virtqueue_stop(&net->dev, vdev, idx);
+}
diff --git a/include/net/vhost_net.h b/include/net/vhost_net.h
index 387e913e4e..9b3aaf3814 100644
--- a/include/net/vhost_net.h
+++ b/include/net/vhost_net.h
@@ -48,4 +48,6 @@ uint64_t vhost_net_get_acked_features(VHostNetState *net);
int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu);
+void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc,
+ int vq_index);
#endif
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 12/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (10 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 11/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop() Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-24 2:44 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 13/24] docs: vhost-user: add VHOST_USER_RESET_VRING message Kangjie Xu
` (15 subsequent siblings)
27 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
Introduce vhost_net_virtqueue_restart(), which can restart the
virtqueue when the vhost net started running before. If it fails
to restart the virtqueue, the device will be stopped.
This patch only considers the case for vhost-kernel, when
NetClientDriver is NET_CLIENT_DRIVER_TAP.
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
hw/net/vhost_net.c | 48 +++++++++++++++++++++++++++++++++++++++++
include/net/vhost_net.h | 2 ++
2 files changed, 50 insertions(+)
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index aa60dd901c..2ab67e875e 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -535,3 +535,51 @@ void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc,
vhost_dev_virtqueue_stop(&net->dev, vdev, idx);
}
+
+int vhost_net_virtqueue_restart(VirtIODevice *vdev, NetClientState *nc,
+ int vq_index)
+{
+ VHostNetState *net = get_vhost_net(nc->peer);
+ const VhostOps *vhost_ops = net->dev.vhost_ops;
+ struct vhost_vring_file file = { };
+ int idx, r;
+
+ if (!net->dev.started) {
+ return 0;
+ }
+
+ assert(vhost_ops);
+
+ idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index);
+
+ r = vhost_dev_virtqueue_restart(&net->dev, vdev, idx);
+ if (r < 0) {
+ goto err_start;
+ }
+
+ if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) {
+ file.index = idx;
+ file.fd = net->backend;
+ r = vhost_net_set_backend(&net->dev, &file);
+ if (r < 0) {
+ r = -errno;
+ goto err_start;
+ }
+ }
+
+ return 0;
+
+err_start:
+ error_report("Error when restarting the queue.");
+
+ if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) {
+ file.fd = -1;
+ file.index = idx;
+ int r = vhost_net_set_backend(&net->dev, &file);
+ assert(r >= 0);
+ }
+
+ vhost_dev_stop(&net->dev, vdev);
+
+ return r;
+}
diff --git a/include/net/vhost_net.h b/include/net/vhost_net.h
index 9b3aaf3814..e11a297380 100644
--- a/include/net/vhost_net.h
+++ b/include/net/vhost_net.h
@@ -50,4 +50,6 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu);
void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc,
int vq_index);
+int vhost_net_virtqueue_restart(VirtIODevice *vdev, NetClientState *nc,
+ int vq_index);
#endif
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 13/24] docs: vhost-user: add VHOST_USER_RESET_VRING message
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (11 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 12/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart() Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-24 2:46 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 14/24] vhost-user: introduce vhost_reset_vring() interface Kangjie Xu
` (14 subsequent siblings)
27 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
To support the reset operation for an individual virtqueue,
we introduce a new message VHOST_USER_RESET_VRING. This
message is submitted by the front-end to reset an individual
virtqueue to initial states in the back-end. The reply is
needed to ensure that the reset operation is complete.
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
docs/interop/vhost-user.rst | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst
index 3f18ab424e..ce7991b9d3 100644
--- a/docs/interop/vhost-user.rst
+++ b/docs/interop/vhost-user.rst
@@ -1422,6 +1422,16 @@ Front-end message types
query the back-end for its device status as defined in the Virtio
specification.
+``VHOST_USER_RESET_VRING``
+ :id: 41
+ :equivalent ioctl: N/A
+ :request payload: vring state description
+ :reply payload: ``u64``
+
+ When the feature ``VIRTIO_F_RING_RESET`` feature has been successfully
+ negotiated, this message is submitted by the front-end to reset an
+ individual virtqueue to initial states in the back-end. It will ask
+ for a reply to ensure the virtqueue is successfully reset in the back-end.
Back-end message types
----------------------
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 14/24] vhost-user: introduce vhost_reset_vring() interface
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (12 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 13/24] docs: vhost-user: add VHOST_USER_RESET_VRING message Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-24 2:50 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 15/24] vhost-user: add op to enable or disable a single vring Kangjie Xu
` (13 subsequent siblings)
27 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
Introduce the interface vhost_reset_vring(). The interface is a wrapper
to send a VHOST_USER_RESET_VRING message to the back-end. It will reset
an individual vring in the back-end. Meanwhile, it will wait for a reply
to ensure the reset has been completed.
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
hw/virtio/vhost-user.c | 41 +++++++++++++++++++++++++++++++
include/hw/virtio/vhost-backend.h | 3 +++
2 files changed, 44 insertions(+)
diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index 75b8df21a4..56033f7a92 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -126,6 +126,7 @@ typedef enum VhostUserRequest {
VHOST_USER_GET_MAX_MEM_SLOTS = 36,
VHOST_USER_ADD_MEM_REG = 37,
VHOST_USER_REM_MEM_REG = 38,
+ VHOST_USER_RESET_VRING = 41,
VHOST_USER_MAX
} VhostUserRequest;
@@ -1498,6 +1499,45 @@ static int vhost_user_get_max_memslots(struct vhost_dev *dev,
return 0;
}
+static int vhost_user_reset_vring(struct vhost_dev *dev,
+ struct vhost_vring_state *ring)
+{
+ int ret;
+ VhostUserMsg msg = {
+ .hdr.request = VHOST_USER_RESET_VRING,
+ .hdr.flags = VHOST_USER_VERSION,
+ .payload.state = *ring,
+ .hdr.size = sizeof(msg.payload.state),
+ };
+
+ if (!virtio_has_feature(dev->acked_features, VIRTIO_F_RING_RESET)) {
+ return -ENOTSUP;
+ }
+
+ ret = vhost_user_write(dev, &msg, NULL, 0);
+ if (ret < 0) {
+ return ret;
+ }
+
+ ret = vhost_user_read(dev, &msg);
+ if (ret < 0) {
+ return ret;
+ }
+
+ if (msg.hdr.request != VHOST_USER_RESET_VRING) {
+ error_report("Received unexpected msg type. Expected %d received %d",
+ VHOST_USER_RESET_VRING, msg.hdr.request);
+ return -EPROTO;
+ }
+
+ if (msg.hdr.size != sizeof(msg.payload.state)) {
+ error_report("Received bad msg size.");
+ return -EPROTO;
+ }
+
+ return 0;
+}
+
static int vhost_user_reset_device(struct vhost_dev *dev)
{
VhostUserMsg msg = {
@@ -2625,6 +2665,7 @@ const VhostOps user_ops = {
.vhost_set_features = vhost_user_set_features,
.vhost_get_features = vhost_user_get_features,
.vhost_set_owner = vhost_user_set_owner,
+ .vhost_reset_vring = vhost_user_reset_vring,
.vhost_reset_device = vhost_user_reset_device,
.vhost_get_vq_index = vhost_user_get_vq_index,
.vhost_set_vring_enable = vhost_user_set_vring_enable,
diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
index eab46d7f0b..f23bf71a8d 100644
--- a/include/hw/virtio/vhost-backend.h
+++ b/include/hw/virtio/vhost-backend.h
@@ -79,6 +79,8 @@ typedef int (*vhost_get_features_op)(struct vhost_dev *dev,
uint64_t *features);
typedef int (*vhost_set_backend_cap_op)(struct vhost_dev *dev);
typedef int (*vhost_set_owner_op)(struct vhost_dev *dev);
+typedef int (*vhost_reset_vring_op)(struct vhost_dev *dev,
+ struct vhost_vring_state *ring);
typedef int (*vhost_reset_device_op)(struct vhost_dev *dev);
typedef int (*vhost_get_vq_index_op)(struct vhost_dev *dev, int idx);
typedef int (*vhost_set_vring_enable_op)(struct vhost_dev *dev,
@@ -154,6 +156,7 @@ typedef struct VhostOps {
vhost_set_backend_cap_op vhost_set_backend_cap;
vhost_set_owner_op vhost_set_owner;
vhost_reset_device_op vhost_reset_device;
+ vhost_reset_vring_op vhost_reset_vring;
vhost_get_vq_index_op vhost_get_vq_index;
vhost_set_vring_enable_op vhost_set_vring_enable;
vhost_requires_shm_log_op vhost_requires_shm_log;
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 15/24] vhost-user: add op to enable or disable a single vring
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (13 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 14/24] vhost-user: introduce vhost_reset_vring() interface Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-24 2:53 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 16/24] vhost: vhost-user: update vhost_dev_virtqueue_stop() Kangjie Xu
` (12 subsequent siblings)
27 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
The interface to set enable status for a single vring is lacked in
VhostOps, since the vhost_set_vring_enable_op will manipulate all
virtqueues in a device.
Resetting a single vq will rely on this interface.
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
hw/virtio/vhost-user.c | 26 +++++++++++++++++++-------
include/hw/virtio/vhost-backend.h | 3 +++
2 files changed, 22 insertions(+), 7 deletions(-)
diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index 56033f7a92..8307976cda 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -1199,6 +1199,22 @@ static int vhost_user_set_vring_base(struct vhost_dev *dev,
return vhost_set_vring(dev, VHOST_USER_SET_VRING_BASE, ring);
}
+static int vhost_user_set_single_vring_enable(struct vhost_dev *dev,
+ int index,
+ int enable)
+{
+ if (index < dev->vq_index || index >= dev->vq_index + dev->nvqs) {
+ return -EINVAL;
+ }
+
+ struct vhost_vring_state state = {
+ .index = index,
+ .num = enable,
+ };
+
+ return vhost_set_vring(dev, VHOST_USER_SET_VRING_ENABLE, &state);
+}
+
static int vhost_user_set_vring_enable(struct vhost_dev *dev, int enable)
{
int i;
@@ -1208,13 +1224,8 @@ static int vhost_user_set_vring_enable(struct vhost_dev *dev, int enable)
}
for (i = 0; i < dev->nvqs; ++i) {
- int ret;
- struct vhost_vring_state state = {
- .index = dev->vq_index + i,
- .num = enable,
- };
-
- ret = vhost_set_vring(dev, VHOST_USER_SET_VRING_ENABLE, &state);
+ int ret = vhost_user_set_single_vring_enable(dev, dev->vq_index + i,
+ enable);
if (ret < 0) {
/*
* Restoring the previous state is likely infeasible, as well as
@@ -2668,6 +2679,7 @@ const VhostOps user_ops = {
.vhost_reset_vring = vhost_user_reset_vring,
.vhost_reset_device = vhost_user_reset_device,
.vhost_get_vq_index = vhost_user_get_vq_index,
+ .vhost_set_single_vring_enable = vhost_user_set_single_vring_enable,
.vhost_set_vring_enable = vhost_user_set_vring_enable,
.vhost_requires_shm_log = vhost_user_requires_shm_log,
.vhost_migration_done = vhost_user_migration_done,
diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
index f23bf71a8d..38f6b752ff 100644
--- a/include/hw/virtio/vhost-backend.h
+++ b/include/hw/virtio/vhost-backend.h
@@ -83,6 +83,8 @@ typedef int (*vhost_reset_vring_op)(struct vhost_dev *dev,
struct vhost_vring_state *ring);
typedef int (*vhost_reset_device_op)(struct vhost_dev *dev);
typedef int (*vhost_get_vq_index_op)(struct vhost_dev *dev, int idx);
+typedef int (*vhost_set_single_vring_enable_op)(struct vhost_dev *dev,
+ int index, int enable);
typedef int (*vhost_set_vring_enable_op)(struct vhost_dev *dev,
int enable);
typedef bool (*vhost_requires_shm_log_op)(struct vhost_dev *dev);
@@ -158,6 +160,7 @@ typedef struct VhostOps {
vhost_reset_device_op vhost_reset_device;
vhost_reset_vring_op vhost_reset_vring;
vhost_get_vq_index_op vhost_get_vq_index;
+ vhost_set_single_vring_enable_op vhost_set_single_vring_enable;
vhost_set_vring_enable_op vhost_set_vring_enable;
vhost_requires_shm_log_op vhost_requires_shm_log;
vhost_migration_done_op vhost_migration_done;
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 16/24] vhost: vhost-user: update vhost_dev_virtqueue_stop()
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (14 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 15/24] vhost-user: add op to enable or disable a single vring Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-24 3:56 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 17/24] vhost: vhost-user: update vhost_dev_virtqueue_restart() Kangjie Xu
` (11 subsequent siblings)
27 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
Update vhost_dev_virtqueue_stop() for vhost-user scenario.
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
hw/virtio/vhost.c | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index fc3f550c76..a0d6824353 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -1908,10 +1908,29 @@ int vhost_net_set_backend(struct vhost_dev *hdev,
void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice *vdev,
int idx)
{
+ const VhostOps *vhost_ops = hdev->vhost_ops;
+ struct vhost_vring_state state;
+ int r;
+
+ assert(vhost_ops);
+
+ if (vhost_ops->vhost_reset_vring) {
+ state.index = hdev->vq_index + idx;
+ r = vhost_ops->vhost_reset_vring(hdev, &state);
+ if (r < 0) {
+ goto err_queue_reset;
+ }
+ }
+
vhost_virtqueue_unmap(hdev,
vdev,
hdev->vqs + idx,
idx);
+
+ return;
+
+err_queue_reset:
+ error_report("Error when stopping the qeuue.");
}
int vhost_dev_virtqueue_restart(struct vhost_dev *hdev, VirtIODevice *vdev,
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 17/24] vhost: vhost-user: update vhost_dev_virtqueue_restart()
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (15 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 16/24] vhost: vhost-user: update vhost_dev_virtqueue_stop() Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-24 4:03 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 18/24] vhost-net: vhost-user: update vhost_net_virtqueue_stop() Kangjie Xu
` (10 subsequent siblings)
27 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
Update vhost_dev_virtqueue_restart() for vhost-user scenario.
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
hw/virtio/vhost.c | 26 ++++++++++++++++++++++----
1 file changed, 22 insertions(+), 4 deletions(-)
diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index a0d6824353..bd90cfe13a 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -1937,11 +1937,29 @@ int vhost_dev_virtqueue_restart(struct vhost_dev *hdev, VirtIODevice *vdev,
int idx)
{
const VhostOps *vhost_ops = hdev->vhost_ops;
+ int r;
assert(vhost_ops);
- return vhost_virtqueue_start(hdev,
- vdev,
- hdev->vqs + idx,
- hdev->vq_index + idx);
+ r = vhost_virtqueue_start(hdev,
+ vdev,
+ hdev->vqs + idx,
+ hdev->vq_index + idx);
+ if (r < 0) {
+ goto err_start;
+ }
+
+ if (vhost_ops->vhost_set_single_vring_enable) {
+ r = vhost_ops->vhost_set_single_vring_enable(hdev,
+ hdev->vq_index + idx,
+ 1);
+ if (r < 0) {
+ goto err_start;
+ }
+ }
+
+ return 0;
+
+err_start:
+ return r;
}
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 18/24] vhost-net: vhost-user: update vhost_net_virtqueue_stop()
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (16 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 17/24] vhost: vhost-user: update vhost_dev_virtqueue_restart() Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-24 4:05 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 19/24] vhost-net: vhost-user: update vhost_net_virtqueue_restart() Kangjie Xu
` (9 subsequent siblings)
27 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
Update vhost_net_virtqueue_stop() for vhost-user scenario.
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
hw/net/vhost_net.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 2ab67e875e..c0d408f3b4 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -533,6 +533,10 @@ void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc,
assert(r >= 0);
}
+ if (net->nc->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
+ idx = idx - net->dev.vq_index;
+ }
+
vhost_dev_virtqueue_stop(&net->dev, vdev, idx);
}
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 19/24] vhost-net: vhost-user: update vhost_net_virtqueue_restart()
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (17 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 18/24] vhost-net: vhost-user: update vhost_net_virtqueue_stop() Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-24 4:06 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 20/24] virtio-net: introduce flush_or_purge_queued_packets() Kangjie Xu
` (8 subsequent siblings)
27 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
Update vhost_net_virtqueue_restart() for vhost-user scenario.
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
hw/net/vhost_net.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index c0d408f3b4..778081e54a 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -556,6 +556,10 @@ int vhost_net_virtqueue_restart(VirtIODevice *vdev, NetClientState *nc,
idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index);
+ if (net->nc->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
+ idx = idx - net->dev.vq_index;
+ }
+
r = vhost_dev_virtqueue_restart(&net->dev, vdev, idx);
if (r < 0) {
goto err_start;
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 20/24] virtio-net: introduce flush_or_purge_queued_packets()
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (18 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 19/24] vhost-net: vhost-user: update vhost_net_virtqueue_restart() Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 21/24] virtio-net: support queue reset Kangjie Xu
` (7 subsequent siblings)
27 siblings, 0 replies; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
Introduce the fucntion flush_or_purge_queued_packets(), it will be
used in device reset and virtqueue reset. Therefore, we extract the
common logic as a new function.
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
hw/net/virtio-net.c | 17 +++++++++++------
1 file changed, 11 insertions(+), 6 deletions(-)
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index dd0d056fde..27b59c0ad6 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -118,6 +118,16 @@ static int vq2q(int queue_index)
return queue_index / 2;
}
+static void flush_or_purge_queued_packets(NetClientState *nc)
+{
+ if (!nc->peer) {
+ return;
+ }
+
+ qemu_flush_or_purge_queued_packets(nc->peer, true);
+ assert(!virtio_net_get_subqueue(nc)->async_tx.elem);
+}
+
/* TODO
* - we could suppress RX interrupt if we were so inclined.
*/
@@ -560,12 +570,7 @@ static void virtio_net_reset(VirtIODevice *vdev)
/* Flush any async TX */
for (i = 0; i < n->max_queue_pairs; i++) {
- NetClientState *nc = qemu_get_subqueue(n->nic, i);
-
- if (nc->peer) {
- qemu_flush_or_purge_queued_packets(nc->peer, true);
- assert(!virtio_net_get_subqueue(nc)->async_tx.elem);
- }
+ flush_or_purge_queued_packets(qemu_get_subqueue(n->nic, i));
}
}
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 21/24] virtio-net: support queue reset
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (19 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 20/24] virtio-net: introduce flush_or_purge_queued_packets() Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 22/24] virtio-net: support queue_enable Kangjie Xu
` (6 subsequent siblings)
27 siblings, 0 replies; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
virtio-net, vhost-kernel, vhost-user implement queue reset.
Queued packets in the corresponding queue pair are flushed
or purged.
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
---
hw/net/virtio-net.c | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 27b59c0ad6..e2989487ec 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -540,6 +540,24 @@ static RxFilterInfo *virtio_net_query_rxfilter(NetClientState *nc)
return info;
}
+static void virtio_net_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
+{
+ VirtIONet *n = VIRTIO_NET(vdev);
+ NetClientState *nc = qemu_get_subqueue(n->nic, vq2q(queue_index));
+
+ if (!nc->peer) {
+ return;
+ }
+
+ if (get_vhost_net(nc->peer) &&
+ ((nc->peer->info->type == NET_CLIENT_DRIVER_TAP) ||
+ (nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_USER))) {
+ vhost_net_virtqueue_stop(vdev, nc, queue_index);
+ }
+
+ flush_or_purge_queued_packets(nc);
+}
+
static void virtio_net_reset(VirtIODevice *vdev)
{
VirtIONet *n = VIRTIO_NET(vdev);
@@ -3784,6 +3802,7 @@ static void virtio_net_class_init(ObjectClass *klass, void *data)
vdc->set_features = virtio_net_set_features;
vdc->bad_features = virtio_net_bad_features;
vdc->reset = virtio_net_reset;
+ vdc->queue_reset = virtio_net_queue_reset;
vdc->set_status = virtio_net_set_status;
vdc->guest_notifier_mask = virtio_net_guest_notifier_mask;
vdc->guest_notifier_pending = virtio_net_guest_notifier_pending;
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 22/24] virtio-net: support queue_enable
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (20 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 21/24] virtio-net: support queue reset Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 23/24] vhost: vhost-kernel: enable vq reset feature Kangjie Xu
` (5 subsequent siblings)
27 siblings, 0 replies; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
virtio-net: support queue_enable for vhost
Support queue_enable in vhost scenario. It can be called when
a vq reset operation is performed and the vq will be restared.
It should be noted that we can restart the vq when the vhost has
already started. When launching a new vhost-user device, the vhost
is not started and all vqs are not initalized until
VIRTIO_PCI_COMMON_STATUS is written. Thus, we should use vhost_started
to differentiate the two cases: vq reset and device start.
Currently it only supports vhost-user and vhost-kernel.
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
hw/net/virtio-net.c | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index e2989487ec..ba216f7b84 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -558,6 +558,27 @@ static void virtio_net_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
flush_or_purge_queued_packets(nc);
}
+static void virtio_net_queue_enable(VirtIODevice *vdev, uint32_t queue_index)
+{
+ VirtIONet *n = VIRTIO_NET(vdev);
+ NetClientState *nc = qemu_get_subqueue(n->nic, vq2q(queue_index));
+ int r;
+
+ if (!nc->peer || !vdev->vhost_started) {
+ return;
+ }
+
+ if (get_vhost_net(nc->peer) &&
+ ((nc->peer->info->type == NET_CLIENT_DRIVER_TAP) ||
+ (nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_USER))) {
+ r = vhost_net_virtqueue_restart(vdev, nc, queue_index);
+ if (r < 0) {
+ error_report("unable to restart vhost net virtqueue: %d, "
+ "when resetting the queue", queue_index);
+ }
+ }
+}
+
static void virtio_net_reset(VirtIODevice *vdev)
{
VirtIONet *n = VIRTIO_NET(vdev);
@@ -3803,6 +3824,7 @@ static void virtio_net_class_init(ObjectClass *klass, void *data)
vdc->bad_features = virtio_net_bad_features;
vdc->reset = virtio_net_reset;
vdc->queue_reset = virtio_net_queue_reset;
+ vdc->queue_enable = virtio_net_queue_enable;
vdc->set_status = virtio_net_set_status;
vdc->guest_notifier_mask = virtio_net_guest_notifier_mask;
vdc->guest_notifier_pending = virtio_net_guest_notifier_pending;
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 23/24] vhost: vhost-kernel: enable vq reset feature
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (21 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 22/24] virtio-net: support queue_enable Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 24/24] vhost: vhost-user: " Kangjie Xu
` (4 subsequent siblings)
27 siblings, 0 replies; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
Add virtqueue reset feature for vhost-kernel.
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
hw/net/vhost_net.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 778081e54a..9ea160e4e9 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -46,6 +46,7 @@ static const int kernel_feature_bits[] = {
VIRTIO_NET_F_MTU,
VIRTIO_F_IOMMU_PLATFORM,
VIRTIO_F_RING_PACKED,
+ VIRTIO_F_RING_RESET,
VIRTIO_NET_F_HASH_REPORT,
VHOST_INVALID_FEATURE_BIT
};
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v2 24/24] vhost: vhost-user: enable vq reset feature
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (22 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 23/24] vhost: vhost-kernel: enable vq reset feature Kangjie Xu
@ 2022-08-16 1:06 ` Kangjie Xu
2022-08-16 6:14 ` [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Michael S. Tsirkin
` (3 subsequent siblings)
27 siblings, 0 replies; 77+ messages in thread
From: Kangjie Xu @ 2022-08-16 1:06 UTC (permalink / raw)
To: qemu-devel
Cc: mst, jasowang, eduardo, marcel.apfelbaum, f4bug, wangyanan55,
hengqi, xuanzhuo
Add virtqueue reset feature for vhost-user.
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
hw/net/vhost_net.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 9ea160e4e9..5bc89f2842 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -74,6 +74,7 @@ static const int user_feature_bits[] = {
VIRTIO_NET_F_MTU,
VIRTIO_F_IOMMU_PLATFORM,
VIRTIO_F_RING_PACKED,
+ VIRTIO_F_RING_RESET,
VIRTIO_NET_F_RSS,
VIRTIO_NET_F_HASH_REPORT,
--
2.32.0
^ permalink raw reply related [flat|nested] 77+ messages in thread
* Re: [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (23 preceding siblings ...)
2022-08-16 1:06 ` [PATCH v2 24/24] vhost: vhost-user: " Kangjie Xu
@ 2022-08-16 6:14 ` Michael S. Tsirkin
2022-08-16 6:15 ` Xuan Zhuo
2022-08-23 1:49 ` Kangjie Xu
` (2 subsequent siblings)
27 siblings, 1 reply; 77+ messages in thread
From: Michael S. Tsirkin @ 2022-08-16 6:14 UTC (permalink / raw)
To: Kangjie Xu
Cc: qemu-devel, jasowang, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, xuanzhuo
On Tue, Aug 16, 2022 at 09:06:12AM +0800, Kangjie Xu wrote:
> The virtio queue reset function has already been defined in the virtio spec 1.2.
> The relevant virtio spec information is here:
>
> https://github.com/oasis-tcs/virtio-spec/issues/124
> https://github.com/oasis-tcs/virtio-spec/issues/139
>
> This patch set is to support this function in QEMU. It consists of several parts:
> 1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
> 2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
> 3. Patches 13-19 support vq stop and vq restart for vhost-user.
> 4. Patches 20-22 support vq reset and re-enable for virtio-net.
> 5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.
>
> The process of virtqueue reset can be concluded as:
> 1. The virtqueue is disabled when VIRTIO_PCI_COMMON_Q_RESET is written.
> 2. Then the virtqueue can be optionally restarted(re-enabled).
>
> Since this patch set involves multiple modules and seems a bit messy, we briefly describe the
> calling process for different modes below.
> virtio-net:
> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> -> virtio_queue_reset() [virtio]
> -> virtio_net_queue_reset() [virtio-net]
> -> __virtio_queue_reset()
> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> -> set enabled, reset status of vq.
>
> vhost-kernel:
> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> -> virtio_queue_reset() [virtio]
> -> virtio_net_queue_reset() [virtio-net]
> -> vhost_net_virtqueue_stop() [vhost-net]
> -> vhost_net_set_backend() [vhost]
> -> vhost_dev_virtqueue_stop()
> -> vhost_virtqueue_unmap()
> -> __virtio_queue_reset()
> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> -> virtio_queue_enable() [virtio]
> -> virtio_net_queue_enable() [virtio-net]
> -> vhost_net_virtqueue_restart() [vhost-net]
> -> vhost_dev_virtqueue_restart() [vhost]
> -> vhost_virtqueue_start()
> -> vhost_net_set_backend()
> -> set enabled, reset status of vq.
>
> vhost-user:
> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> -> virtio_queue_reset() [virtio]
> -> virtio_net_queue_reset() [virtio-net]
> -> vhost_net_virtqueue_stop() [vhost-net]
> -> vhost_dev_virtqueue_stop() [vhost]
> -> vhost_user_reset_vring() [vhost-user]
> -> send VHOST_USER_RESET_VRING to the device
> -> vhost_virtqueue_unmap()
> -> __virtio_queue_reset()
> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> -> virtio_queue_enable() [virtio]
> -> virtio_net_queue_enable() [virtio-net]
> -> vhost_net_virtqueue_restart() [vhost-net]
> -> vhost_dev_virtqueue_restart() [vhost]
> -> vhost_virtqueue_start()
> -> vhost_user_set_single_vring_enable [vhost-user]
> -> send VHOST_USER_SET_VRING_ENABLE to the device
> -> set enabled, reset status of vq.
>
>
> Test environment:
> Host: 5.19.0-rc3 (With vq reset support)
> Qemu: QEMU emulator version 7.0.50
> Guest: 5.19.0-rc3 (With vq reset support)
> DPDK: 22.07-rc1 (With vq reset support)
> Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;
>
> The drvier can resize the virtio queue, then virtio queue reset function should
> be triggered.
>
> The default is split mode, modify Qemu virtio-net to add PACKED feature to
> test packed mode.
legacy mode testing?
> Guest Kernel Patch:
> https://lore.kernel.org/bpf/20220801063902.129329-1-xuanzhuo@linux.alibaba.com/
>
> DPDK Patch:
> https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7
>
> Host Kernel Patch:
> https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3
>
> Looking forward to your review and comments. Thanks.
>
> Kangjie Xu (19):
> virtio: introduce virtio_queue_enable()
> virtio: core: vq reset feature negotation support
> virtio-pci: support queue enable
> vhost: extract the logic of unmapping the vrings and desc
> vhost: introduce vhost_dev_virtqueue_stop()
> vhost: introduce vhost_dev_virtqueue_restart()
> vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
> vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
> docs: vhost-user: add VHOST_USER_RESET_VRING message
> vhost-user: introduce vhost_reset_vring() interface
> vhost-user: add op to enable or disable a single vring
> vhost: vhost-user: update vhost_dev_virtqueue_stop()
> vhost: vhost-user: update vhost_dev_virtqueue_restart()
> vhost-net: vhost-user: update vhost_net_virtqueue_stop()
> vhost-net: vhost-user: update vhost_net_virtqueue_restart()
> virtio-net: introduce flush_or_purge_queued_packets()
> virtio-net: support queue_enable
> vhost: vhost-kernel: enable vq reset feature
> vhost: vhost-user: enable vq reset feature
>
> Xuan Zhuo (5):
> virtio: sync relevant definitions with linux
> virtio: introduce __virtio_queue_reset()
> virtio: introduce virtio_queue_reset()
> virtio-pci: support queue reset
> virtio-net: support queue reset
>
> docs/interop/vhost-user.rst | 10 +++
> hw/core/machine.c | 1 +
> hw/net/vhost_net.c | 79 +++++++++++++++++++
> hw/net/virtio-net.c | 58 ++++++++++++--
> hw/virtio/vhost-user.c | 67 ++++++++++++++--
> hw/virtio/vhost.c | 79 +++++++++++++++++--
> hw/virtio/virtio-pci.c | 20 +++++
> hw/virtio/virtio.c | 62 +++++++++++----
> include/hw/virtio/vhost-backend.h | 6 ++
> include/hw/virtio/vhost.h | 5 ++
> include/hw/virtio/virtio-pci.h | 1 +
> include/hw/virtio/virtio.h | 8 +-
> include/net/vhost_net.h | 4 +
> .../standard-headers/linux/virtio_config.h | 5 ++
> include/standard-headers/linux/virtio_pci.h | 2 +
> 15 files changed, 371 insertions(+), 36 deletions(-)
>
> --
> 2.32.0
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern
2022-08-16 6:14 ` [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Michael S. Tsirkin
@ 2022-08-16 6:15 ` Xuan Zhuo
2022-08-16 6:22 ` Michael S. Tsirkin
0 siblings, 1 reply; 77+ messages in thread
From: Xuan Zhuo @ 2022-08-16 6:15 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: qemu-devel, jasowang, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Kangjie Xu
On Tue, 16 Aug 2022 02:14:10 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Tue, Aug 16, 2022 at 09:06:12AM +0800, Kangjie Xu wrote:
> > The virtio queue reset function has already been defined in the virtio spec 1.2.
> > The relevant virtio spec information is here:
> >
> > https://github.com/oasis-tcs/virtio-spec/issues/124
> > https://github.com/oasis-tcs/virtio-spec/issues/139
> >
> > This patch set is to support this function in QEMU. It consists of several parts:
> > 1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
> > 2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
> > 3. Patches 13-19 support vq stop and vq restart for vhost-user.
> > 4. Patches 20-22 support vq reset and re-enable for virtio-net.
> > 5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.
> >
> > The process of virtqueue reset can be concluded as:
> > 1. The virtqueue is disabled when VIRTIO_PCI_COMMON_Q_RESET is written.
> > 2. Then the virtqueue can be optionally restarted(re-enabled).
> >
> > Since this patch set involves multiple modules and seems a bit messy, we briefly describe the
> > calling process for different modes below.
> > virtio-net:
> > 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> > -> virtio_queue_reset() [virtio]
> > -> virtio_net_queue_reset() [virtio-net]
> > -> __virtio_queue_reset()
> > 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> > -> set enabled, reset status of vq.
> >
> > vhost-kernel:
> > 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> > -> virtio_queue_reset() [virtio]
> > -> virtio_net_queue_reset() [virtio-net]
> > -> vhost_net_virtqueue_stop() [vhost-net]
> > -> vhost_net_set_backend() [vhost]
> > -> vhost_dev_virtqueue_stop()
> > -> vhost_virtqueue_unmap()
> > -> __virtio_queue_reset()
> > 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> > -> virtio_queue_enable() [virtio]
> > -> virtio_net_queue_enable() [virtio-net]
> > -> vhost_net_virtqueue_restart() [vhost-net]
> > -> vhost_dev_virtqueue_restart() [vhost]
> > -> vhost_virtqueue_start()
> > -> vhost_net_set_backend()
> > -> set enabled, reset status of vq.
> >
> > vhost-user:
> > 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> > -> virtio_queue_reset() [virtio]
> > -> virtio_net_queue_reset() [virtio-net]
> > -> vhost_net_virtqueue_stop() [vhost-net]
> > -> vhost_dev_virtqueue_stop() [vhost]
> > -> vhost_user_reset_vring() [vhost-user]
> > -> send VHOST_USER_RESET_VRING to the device
> > -> vhost_virtqueue_unmap()
> > -> __virtio_queue_reset()
> > 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> > -> virtio_queue_enable() [virtio]
> > -> virtio_net_queue_enable() [virtio-net]
> > -> vhost_net_virtqueue_restart() [vhost-net]
> > -> vhost_dev_virtqueue_restart() [vhost]
> > -> vhost_virtqueue_start()
> > -> vhost_user_set_single_vring_enable [vhost-user]
> > -> send VHOST_USER_SET_VRING_ENABLE to the device
> > -> set enabled, reset status of vq.
> >
> >
> > Test environment:
> > Host: 5.19.0-rc3 (With vq reset support)
> > Qemu: QEMU emulator version 7.0.50
> > Guest: 5.19.0-rc3 (With vq reset support)
> > DPDK: 22.07-rc1 (With vq reset support)
> > Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;
> >
> > The drvier can resize the virtio queue, then virtio queue reset function should
> > be triggered.
> >
> > The default is split mode, modify Qemu virtio-net to add PACKED feature to
> > test packed mode.
>
> legacy mode testing?
legacy does not support vq reset.
Thanks.
>
> > Guest Kernel Patch:
> > https://lore.kernel.org/bpf/20220801063902.129329-1-xuanzhuo@linux.alibaba.com/
> >
> > DPDK Patch:
> > https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7
> >
> > Host Kernel Patch:
> > https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3
> >
> > Looking forward to your review and comments. Thanks.
> >
> > Kangjie Xu (19):
> > virtio: introduce virtio_queue_enable()
> > virtio: core: vq reset feature negotation support
> > virtio-pci: support queue enable
> > vhost: extract the logic of unmapping the vrings and desc
> > vhost: introduce vhost_dev_virtqueue_stop()
> > vhost: introduce vhost_dev_virtqueue_restart()
> > vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
> > vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
> > docs: vhost-user: add VHOST_USER_RESET_VRING message
> > vhost-user: introduce vhost_reset_vring() interface
> > vhost-user: add op to enable or disable a single vring
> > vhost: vhost-user: update vhost_dev_virtqueue_stop()
> > vhost: vhost-user: update vhost_dev_virtqueue_restart()
> > vhost-net: vhost-user: update vhost_net_virtqueue_stop()
> > vhost-net: vhost-user: update vhost_net_virtqueue_restart()
> > virtio-net: introduce flush_or_purge_queued_packets()
> > virtio-net: support queue_enable
> > vhost: vhost-kernel: enable vq reset feature
> > vhost: vhost-user: enable vq reset feature
> >
> > Xuan Zhuo (5):
> > virtio: sync relevant definitions with linux
> > virtio: introduce __virtio_queue_reset()
> > virtio: introduce virtio_queue_reset()
> > virtio-pci: support queue reset
> > virtio-net: support queue reset
> >
> > docs/interop/vhost-user.rst | 10 +++
> > hw/core/machine.c | 1 +
> > hw/net/vhost_net.c | 79 +++++++++++++++++++
> > hw/net/virtio-net.c | 58 ++++++++++++--
> > hw/virtio/vhost-user.c | 67 ++++++++++++++--
> > hw/virtio/vhost.c | 79 +++++++++++++++++--
> > hw/virtio/virtio-pci.c | 20 +++++
> > hw/virtio/virtio.c | 62 +++++++++++----
> > include/hw/virtio/vhost-backend.h | 6 ++
> > include/hw/virtio/vhost.h | 5 ++
> > include/hw/virtio/virtio-pci.h | 1 +
> > include/hw/virtio/virtio.h | 8 +-
> > include/net/vhost_net.h | 4 +
> > .../standard-headers/linux/virtio_config.h | 5 ++
> > include/standard-headers/linux/virtio_pci.h | 2 +
> > 15 files changed, 371 insertions(+), 36 deletions(-)
> >
> > --
> > 2.32.0
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern
2022-08-16 6:15 ` Xuan Zhuo
@ 2022-08-16 6:22 ` Michael S. Tsirkin
2022-08-16 6:40 ` Xuan Zhuo
2022-08-17 6:46 ` Kangjie Xu
0 siblings, 2 replies; 77+ messages in thread
From: Michael S. Tsirkin @ 2022-08-16 6:22 UTC (permalink / raw)
To: Xuan Zhuo
Cc: qemu-devel, jasowang, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Kangjie Xu
On Tue, Aug 16, 2022 at 02:15:57PM +0800, Xuan Zhuo wrote:
> On Tue, 16 Aug 2022 02:14:10 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > On Tue, Aug 16, 2022 at 09:06:12AM +0800, Kangjie Xu wrote:
> > > The virtio queue reset function has already been defined in the virtio spec 1.2.
> > > The relevant virtio spec information is here:
> > >
> > > https://github.com/oasis-tcs/virtio-spec/issues/124
> > > https://github.com/oasis-tcs/virtio-spec/issues/139
> > >
> > > This patch set is to support this function in QEMU. It consists of several parts:
> > > 1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
> > > 2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
> > > 3. Patches 13-19 support vq stop and vq restart for vhost-user.
> > > 4. Patches 20-22 support vq reset and re-enable for virtio-net.
> > > 5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.
> > >
> > > The process of virtqueue reset can be concluded as:
> > > 1. The virtqueue is disabled when VIRTIO_PCI_COMMON_Q_RESET is written.
> > > 2. Then the virtqueue can be optionally restarted(re-enabled).
> > >
> > > Since this patch set involves multiple modules and seems a bit messy, we briefly describe the
> > > calling process for different modes below.
> > > virtio-net:
> > > 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> > > -> virtio_queue_reset() [virtio]
> > > -> virtio_net_queue_reset() [virtio-net]
> > > -> __virtio_queue_reset()
> > > 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> > > -> set enabled, reset status of vq.
> > >
> > > vhost-kernel:
> > > 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> > > -> virtio_queue_reset() [virtio]
> > > -> virtio_net_queue_reset() [virtio-net]
> > > -> vhost_net_virtqueue_stop() [vhost-net]
> > > -> vhost_net_set_backend() [vhost]
> > > -> vhost_dev_virtqueue_stop()
> > > -> vhost_virtqueue_unmap()
> > > -> __virtio_queue_reset()
> > > 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> > > -> virtio_queue_enable() [virtio]
> > > -> virtio_net_queue_enable() [virtio-net]
> > > -> vhost_net_virtqueue_restart() [vhost-net]
> > > -> vhost_dev_virtqueue_restart() [vhost]
> > > -> vhost_virtqueue_start()
> > > -> vhost_net_set_backend()
> > > -> set enabled, reset status of vq.
> > >
> > > vhost-user:
> > > 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> > > -> virtio_queue_reset() [virtio]
> > > -> virtio_net_queue_reset() [virtio-net]
> > > -> vhost_net_virtqueue_stop() [vhost-net]
> > > -> vhost_dev_virtqueue_stop() [vhost]
> > > -> vhost_user_reset_vring() [vhost-user]
> > > -> send VHOST_USER_RESET_VRING to the device
> > > -> vhost_virtqueue_unmap()
> > > -> __virtio_queue_reset()
> > > 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> > > -> virtio_queue_enable() [virtio]
> > > -> virtio_net_queue_enable() [virtio-net]
> > > -> vhost_net_virtqueue_restart() [vhost-net]
> > > -> vhost_dev_virtqueue_restart() [vhost]
> > > -> vhost_virtqueue_start()
> > > -> vhost_user_set_single_vring_enable [vhost-user]
> > > -> send VHOST_USER_SET_VRING_ENABLE to the device
> > > -> set enabled, reset status of vq.
> > >
> > >
> > > Test environment:
> > > Host: 5.19.0-rc3 (With vq reset support)
> > > Qemu: QEMU emulator version 7.0.50
> > > Guest: 5.19.0-rc3 (With vq reset support)
> > > DPDK: 22.07-rc1 (With vq reset support)
> > > Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;
> > >
> > > The drvier can resize the virtio queue, then virtio queue reset function should
> > > be triggered.
> > >
> > > The default is split mode, modify Qemu virtio-net to add PACKED feature to
> > > test packed mode.
> >
> > legacy mode testing?
>
>
> legacy does not support vq reset.
>
> Thanks.
yes but did it break with all these code changes?
> >
> > > Guest Kernel Patch:
> > > https://lore.kernel.org/bpf/20220801063902.129329-1-xuanzhuo@linux.alibaba.com/
> > >
> > > DPDK Patch:
> > > https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7
> > >
> > > Host Kernel Patch:
> > > https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3
> > >
> > > Looking forward to your review and comments. Thanks.
> > >
> > > Kangjie Xu (19):
> > > virtio: introduce virtio_queue_enable()
> > > virtio: core: vq reset feature negotation support
> > > virtio-pci: support queue enable
> > > vhost: extract the logic of unmapping the vrings and desc
> > > vhost: introduce vhost_dev_virtqueue_stop()
> > > vhost: introduce vhost_dev_virtqueue_restart()
> > > vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
> > > vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
> > > docs: vhost-user: add VHOST_USER_RESET_VRING message
> > > vhost-user: introduce vhost_reset_vring() interface
> > > vhost-user: add op to enable or disable a single vring
> > > vhost: vhost-user: update vhost_dev_virtqueue_stop()
> > > vhost: vhost-user: update vhost_dev_virtqueue_restart()
> > > vhost-net: vhost-user: update vhost_net_virtqueue_stop()
> > > vhost-net: vhost-user: update vhost_net_virtqueue_restart()
> > > virtio-net: introduce flush_or_purge_queued_packets()
> > > virtio-net: support queue_enable
> > > vhost: vhost-kernel: enable vq reset feature
> > > vhost: vhost-user: enable vq reset feature
> > >
> > > Xuan Zhuo (5):
> > > virtio: sync relevant definitions with linux
> > > virtio: introduce __virtio_queue_reset()
> > > virtio: introduce virtio_queue_reset()
> > > virtio-pci: support queue reset
> > > virtio-net: support queue reset
> > >
> > > docs/interop/vhost-user.rst | 10 +++
> > > hw/core/machine.c | 1 +
> > > hw/net/vhost_net.c | 79 +++++++++++++++++++
> > > hw/net/virtio-net.c | 58 ++++++++++++--
> > > hw/virtio/vhost-user.c | 67 ++++++++++++++--
> > > hw/virtio/vhost.c | 79 +++++++++++++++++--
> > > hw/virtio/virtio-pci.c | 20 +++++
> > > hw/virtio/virtio.c | 62 +++++++++++----
> > > include/hw/virtio/vhost-backend.h | 6 ++
> > > include/hw/virtio/vhost.h | 5 ++
> > > include/hw/virtio/virtio-pci.h | 1 +
> > > include/hw/virtio/virtio.h | 8 +-
> > > include/net/vhost_net.h | 4 +
> > > .../standard-headers/linux/virtio_config.h | 5 ++
> > > include/standard-headers/linux/virtio_pci.h | 2 +
> > > 15 files changed, 371 insertions(+), 36 deletions(-)
> > >
> > > --
> > > 2.32.0
> >
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern
2022-08-16 6:22 ` Michael S. Tsirkin
@ 2022-08-16 6:40 ` Xuan Zhuo
2022-08-17 6:46 ` Kangjie Xu
1 sibling, 0 replies; 77+ messages in thread
From: Xuan Zhuo @ 2022-08-16 6:40 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: qemu-devel, jasowang, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Kangjie Xu
On Tue, 16 Aug 2022 02:22:16 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Tue, Aug 16, 2022 at 02:15:57PM +0800, Xuan Zhuo wrote:
> > On Tue, 16 Aug 2022 02:14:10 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > On Tue, Aug 16, 2022 at 09:06:12AM +0800, Kangjie Xu wrote:
> > > > The virtio queue reset function has already been defined in the virtio spec 1.2.
> > > > The relevant virtio spec information is here:
> > > >
> > > > https://github.com/oasis-tcs/virtio-spec/issues/124
> > > > https://github.com/oasis-tcs/virtio-spec/issues/139
> > > >
> > > > This patch set is to support this function in QEMU. It consists of several parts:
> > > > 1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
> > > > 2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
> > > > 3. Patches 13-19 support vq stop and vq restart for vhost-user.
> > > > 4. Patches 20-22 support vq reset and re-enable for virtio-net.
> > > > 5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.
> > > >
> > > > The process of virtqueue reset can be concluded as:
> > > > 1. The virtqueue is disabled when VIRTIO_PCI_COMMON_Q_RESET is written.
> > > > 2. Then the virtqueue can be optionally restarted(re-enabled).
> > > >
> > > > Since this patch set involves multiple modules and seems a bit messy, we briefly describe the
> > > > calling process for different modes below.
> > > > virtio-net:
> > > > 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> > > > -> virtio_queue_reset() [virtio]
> > > > -> virtio_net_queue_reset() [virtio-net]
> > > > -> __virtio_queue_reset()
> > > > 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> > > > -> set enabled, reset status of vq.
> > > >
> > > > vhost-kernel:
> > > > 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> > > > -> virtio_queue_reset() [virtio]
> > > > -> virtio_net_queue_reset() [virtio-net]
> > > > -> vhost_net_virtqueue_stop() [vhost-net]
> > > > -> vhost_net_set_backend() [vhost]
> > > > -> vhost_dev_virtqueue_stop()
> > > > -> vhost_virtqueue_unmap()
> > > > -> __virtio_queue_reset()
> > > > 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> > > > -> virtio_queue_enable() [virtio]
> > > > -> virtio_net_queue_enable() [virtio-net]
> > > > -> vhost_net_virtqueue_restart() [vhost-net]
> > > > -> vhost_dev_virtqueue_restart() [vhost]
> > > > -> vhost_virtqueue_start()
> > > > -> vhost_net_set_backend()
> > > > -> set enabled, reset status of vq.
> > > >
> > > > vhost-user:
> > > > 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> > > > -> virtio_queue_reset() [virtio]
> > > > -> virtio_net_queue_reset() [virtio-net]
> > > > -> vhost_net_virtqueue_stop() [vhost-net]
> > > > -> vhost_dev_virtqueue_stop() [vhost]
> > > > -> vhost_user_reset_vring() [vhost-user]
> > > > -> send VHOST_USER_RESET_VRING to the device
> > > > -> vhost_virtqueue_unmap()
> > > > -> __virtio_queue_reset()
> > > > 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> > > > -> virtio_queue_enable() [virtio]
> > > > -> virtio_net_queue_enable() [virtio-net]
> > > > -> vhost_net_virtqueue_restart() [vhost-net]
> > > > -> vhost_dev_virtqueue_restart() [vhost]
> > > > -> vhost_virtqueue_start()
> > > > -> vhost_user_set_single_vring_enable [vhost-user]
> > > > -> send VHOST_USER_SET_VRING_ENABLE to the device
> > > > -> set enabled, reset status of vq.
> > > >
> > > >
> > > > Test environment:
> > > > Host: 5.19.0-rc3 (With vq reset support)
> > > > Qemu: QEMU emulator version 7.0.50
> > > > Guest: 5.19.0-rc3 (With vq reset support)
> > > > DPDK: 22.07-rc1 (With vq reset support)
> > > > Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;
> > > >
> > > > The drvier can resize the virtio queue, then virtio queue reset function should
> > > > be triggered.
> > > >
> > > > The default is split mode, modify Qemu virtio-net to add PACKED feature to
> > > > test packed mode.
> > >
> > > legacy mode testing?
> >
> >
> > legacy does not support vq reset.
> >
> > Thanks.
>
> yes but did it break with all these code changes?
OK, I see, we'll test this.
Thanks.
>
> > >
> > > > Guest Kernel Patch:
> > > > https://lore.kernel.org/bpf/20220801063902.129329-1-xuanzhuo@linux.alibaba.com/
> > > >
> > > > DPDK Patch:
> > > > https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7
> > > >
> > > > Host Kernel Patch:
> > > > https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3
> > > >
> > > > Looking forward to your review and comments. Thanks.
> > > >
> > > > Kangjie Xu (19):
> > > > virtio: introduce virtio_queue_enable()
> > > > virtio: core: vq reset feature negotation support
> > > > virtio-pci: support queue enable
> > > > vhost: extract the logic of unmapping the vrings and desc
> > > > vhost: introduce vhost_dev_virtqueue_stop()
> > > > vhost: introduce vhost_dev_virtqueue_restart()
> > > > vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
> > > > vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
> > > > docs: vhost-user: add VHOST_USER_RESET_VRING message
> > > > vhost-user: introduce vhost_reset_vring() interface
> > > > vhost-user: add op to enable or disable a single vring
> > > > vhost: vhost-user: update vhost_dev_virtqueue_stop()
> > > > vhost: vhost-user: update vhost_dev_virtqueue_restart()
> > > > vhost-net: vhost-user: update vhost_net_virtqueue_stop()
> > > > vhost-net: vhost-user: update vhost_net_virtqueue_restart()
> > > > virtio-net: introduce flush_or_purge_queued_packets()
> > > > virtio-net: support queue_enable
> > > > vhost: vhost-kernel: enable vq reset feature
> > > > vhost: vhost-user: enable vq reset feature
> > > >
> > > > Xuan Zhuo (5):
> > > > virtio: sync relevant definitions with linux
> > > > virtio: introduce __virtio_queue_reset()
> > > > virtio: introduce virtio_queue_reset()
> > > > virtio-pci: support queue reset
> > > > virtio-net: support queue reset
> > > >
> > > > docs/interop/vhost-user.rst | 10 +++
> > > > hw/core/machine.c | 1 +
> > > > hw/net/vhost_net.c | 79 +++++++++++++++++++
> > > > hw/net/virtio-net.c | 58 ++++++++++++--
> > > > hw/virtio/vhost-user.c | 67 ++++++++++++++--
> > > > hw/virtio/vhost.c | 79 +++++++++++++++++--
> > > > hw/virtio/virtio-pci.c | 20 +++++
> > > > hw/virtio/virtio.c | 62 +++++++++++----
> > > > include/hw/virtio/vhost-backend.h | 6 ++
> > > > include/hw/virtio/vhost.h | 5 ++
> > > > include/hw/virtio/virtio-pci.h | 1 +
> > > > include/hw/virtio/virtio.h | 8 +-
> > > > include/net/vhost_net.h | 4 +
> > > > .../standard-headers/linux/virtio_config.h | 5 ++
> > > > include/standard-headers/linux/virtio_pci.h | 2 +
> > > > 15 files changed, 371 insertions(+), 36 deletions(-)
> > > >
> > > > --
> > > > 2.32.0
> > >
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern
2022-08-16 6:22 ` Michael S. Tsirkin
2022-08-16 6:40 ` Xuan Zhuo
@ 2022-08-17 6:46 ` Kangjie Xu
1 sibling, 0 replies; 77+ messages in thread
From: Kangjie Xu @ 2022-08-17 6:46 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: qemu-devel, Xuan Zhuo, Jason Wang, eduardo, marcel.apfelbaum,
f4bug, wangyanan55, hengqi
在 2022/8/16 14:22, Michael S. Tsirkin 写道:
> On Tue, Aug 16, 2022 at 02:15:57PM +0800, Xuan Zhuo wrote:
>> On Tue, 16 Aug 2022 02:14:10 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
>>> On Tue, Aug 16, 2022 at 09:06:12AM +0800, Kangjie Xu wrote:
>>>> The virtio queue reset function has already been defined in the virtio spec 1.2.
>>>> The relevant virtio spec information is here:
>>>>
>>>> https://github.com/oasis-tcs/virtio-spec/issues/124
>>>> https://github.com/oasis-tcs/virtio-spec/issues/139
>>>>
>>>> This patch set is to support this function in QEMU. It consists of several parts:
>>>> 1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
>>>> 2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
>>>> 3. Patches 13-19 support vq stop and vq restart for vhost-user.
>>>> 4. Patches 20-22 support vq reset and re-enable for virtio-net.
>>>> 5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.
>>>>
>>>> The process of virtqueue reset can be concluded as:
>>>> 1. The virtqueue is disabled when VIRTIO_PCI_COMMON_Q_RESET is written.
>>>> 2. Then the virtqueue can be optionally restarted(re-enabled).
>>>>
>>>> Since this patch set involves multiple modules and seems a bit messy, we briefly describe the
>>>> calling process for different modes below.
>>>> virtio-net:
>>>> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
>>>> -> virtio_queue_reset() [virtio]
>>>> -> virtio_net_queue_reset() [virtio-net]
>>>> -> __virtio_queue_reset()
>>>> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
>>>> -> set enabled, reset status of vq.
>>>>
>>>> vhost-kernel:
>>>> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
>>>> -> virtio_queue_reset() [virtio]
>>>> -> virtio_net_queue_reset() [virtio-net]
>>>> -> vhost_net_virtqueue_stop() [vhost-net]
>>>> -> vhost_net_set_backend() [vhost]
>>>> -> vhost_dev_virtqueue_stop()
>>>> -> vhost_virtqueue_unmap()
>>>> -> __virtio_queue_reset()
>>>> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
>>>> -> virtio_queue_enable() [virtio]
>>>> -> virtio_net_queue_enable() [virtio-net]
>>>> -> vhost_net_virtqueue_restart() [vhost-net]
>>>> -> vhost_dev_virtqueue_restart() [vhost]
>>>> -> vhost_virtqueue_start()
>>>> -> vhost_net_set_backend()
>>>> -> set enabled, reset status of vq.
>>>>
>>>> vhost-user:
>>>> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
>>>> -> virtio_queue_reset() [virtio]
>>>> -> virtio_net_queue_reset() [virtio-net]
>>>> -> vhost_net_virtqueue_stop() [vhost-net]
>>>> -> vhost_dev_virtqueue_stop() [vhost]
>>>> -> vhost_user_reset_vring() [vhost-user]
>>>> -> send VHOST_USER_RESET_VRING to the device
>>>> -> vhost_virtqueue_unmap()
>>>> -> __virtio_queue_reset()
>>>> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
>>>> -> virtio_queue_enable() [virtio]
>>>> -> virtio_net_queue_enable() [virtio-net]
>>>> -> vhost_net_virtqueue_restart() [vhost-net]
>>>> -> vhost_dev_virtqueue_restart() [vhost]
>>>> -> vhost_virtqueue_start()
>>>> -> vhost_user_set_single_vring_enable [vhost-user]
>>>> -> send VHOST_USER_SET_VRING_ENABLE to the device
>>>> -> set enabled, reset status of vq.
>>>>
>>>>
>>>> Test environment:
>>>> Host: 5.19.0-rc3 (With vq reset support)
>>>> Qemu: QEMU emulator version 7.0.50
>>>> Guest: 5.19.0-rc3 (With vq reset support)
>>>> DPDK: 22.07-rc1 (With vq reset support)
>>>> Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;
>>>>
>>>> The drvier can resize the virtio queue, then virtio queue reset function should
>>>> be triggered.
>>>>
>>>> The default is split mode, modify Qemu virtio-net to add PACKED feature to
>>>> test packed mode.
>>> legacy mode testing?
>>
>> legacy does not support vq reset.
>>
>> Thanks.
> yes but did it break with all these code changes?
For pci legacy mode, we tested resizing the vq using 'ethtool -G', it
reported an error code -ENOENT because the transport did not support the
vq reset feature.
We also tested the connectivity and performance in pci legacy mode using
iperf 3.9. We placed a client in the host and a server in the guest, or
swapped the them.
In all the cases, the connectivity did not break down. The performance
was about the same with all our code changes.
Thanks.
>>>> Guest Kernel Patch:
>>>> https://lore.kernel.org/bpf/20220801063902.129329-1-xuanzhuo@linux.alibaba.com/
>>>>
>>>> DPDK Patch:
>>>> https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7
>>>>
>>>> Host Kernel Patch:
>>>> https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3
>>>>
>>>> Looking forward to your review and comments. Thanks.
>>>>
>>>> Kangjie Xu (19):
>>>> virtio: introduce virtio_queue_enable()
>>>> virtio: core: vq reset feature negotation support
>>>> virtio-pci: support queue enable
>>>> vhost: extract the logic of unmapping the vrings and desc
>>>> vhost: introduce vhost_dev_virtqueue_stop()
>>>> vhost: introduce vhost_dev_virtqueue_restart()
>>>> vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
>>>> vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
>>>> docs: vhost-user: add VHOST_USER_RESET_VRING message
>>>> vhost-user: introduce vhost_reset_vring() interface
>>>> vhost-user: add op to enable or disable a single vring
>>>> vhost: vhost-user: update vhost_dev_virtqueue_stop()
>>>> vhost: vhost-user: update vhost_dev_virtqueue_restart()
>>>> vhost-net: vhost-user: update vhost_net_virtqueue_stop()
>>>> vhost-net: vhost-user: update vhost_net_virtqueue_restart()
>>>> virtio-net: introduce flush_or_purge_queued_packets()
>>>> virtio-net: support queue_enable
>>>> vhost: vhost-kernel: enable vq reset feature
>>>> vhost: vhost-user: enable vq reset feature
>>>>
>>>> Xuan Zhuo (5):
>>>> virtio: sync relevant definitions with linux
>>>> virtio: introduce __virtio_queue_reset()
>>>> virtio: introduce virtio_queue_reset()
>>>> virtio-pci: support queue reset
>>>> virtio-net: support queue reset
>>>>
>>>> docs/interop/vhost-user.rst | 10 +++
>>>> hw/core/machine.c | 1 +
>>>> hw/net/vhost_net.c | 79 +++++++++++++++++++
>>>> hw/net/virtio-net.c | 58 ++++++++++++--
>>>> hw/virtio/vhost-user.c | 67 ++++++++++++++--
>>>> hw/virtio/vhost.c | 79 +++++++++++++++++--
>>>> hw/virtio/virtio-pci.c | 20 +++++
>>>> hw/virtio/virtio.c | 62 +++++++++++----
>>>> include/hw/virtio/vhost-backend.h | 6 ++
>>>> include/hw/virtio/vhost.h | 5 ++
>>>> include/hw/virtio/virtio-pci.h | 1 +
>>>> include/hw/virtio/virtio.h | 8 +-
>>>> include/net/vhost_net.h | 4 +
>>>> .../standard-headers/linux/virtio_config.h | 5 ++
>>>> include/standard-headers/linux/virtio_pci.h | 2 +
>>>> 15 files changed, 371 insertions(+), 36 deletions(-)
>>>>
>>>> --
>>>> 2.32.0
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (24 preceding siblings ...)
2022-08-16 6:14 ` [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Michael S. Tsirkin
@ 2022-08-23 1:49 ` Kangjie Xu
2022-08-24 4:10 ` Jason Wang
2022-08-24 4:11 ` Jason Wang
27 siblings, 0 replies; 77+ messages in thread
From: Kangjie Xu @ 2022-08-23 1:49 UTC (permalink / raw)
To: qemu-devel
Cc: Michael S. Tsirkin, Jason Wang, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
Forgot to append changelog for v2 in the cover letter, so I add it in
this email.
changelog:
1. Add support for vhost-net scenario
2. Add a new vhost-user message VHOST_USER_RESET_VRING
3. Add migration compatibility for virtqueue reset
Looking forward to your review and comments to this patch set, thanks. :)
Best regards,
Kangjie
在 2022/8/16 09:06, Kangjie Xu 写道:
> The virtio queue reset function has already been defined in the virtio spec 1.2.
> The relevant virtio spec information is here:
>
> https://github.com/oasis-tcs/virtio-spec/issues/124
> https://github.com/oasis-tcs/virtio-spec/issues/139
>
> This patch set is to support this function in QEMU. It consists of several parts:
> 1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
> 2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
> 3. Patches 13-19 support vq stop and vq restart for vhost-user.
> 4. Patches 20-22 support vq reset and re-enable for virtio-net.
> 5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.
>
> The process of virtqueue reset can be concluded as:
> 1. The virtqueue is disabled when VIRTIO_PCI_COMMON_Q_RESET is written.
> 2. Then the virtqueue can be optionally restarted(re-enabled).
>
> Since this patch set involves multiple modules and seems a bit messy, we briefly describe the
> calling process for different modes below.
> virtio-net:
> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> -> virtio_queue_reset() [virtio]
> -> virtio_net_queue_reset() [virtio-net]
> -> __virtio_queue_reset()
> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> -> set enabled, reset status of vq.
>
> vhost-kernel:
> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> -> virtio_queue_reset() [virtio]
> -> virtio_net_queue_reset() [virtio-net]
> -> vhost_net_virtqueue_stop() [vhost-net]
> -> vhost_net_set_backend() [vhost]
> -> vhost_dev_virtqueue_stop()
> -> vhost_virtqueue_unmap()
> -> __virtio_queue_reset()
> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> -> virtio_queue_enable() [virtio]
> -> virtio_net_queue_enable() [virtio-net]
> -> vhost_net_virtqueue_restart() [vhost-net]
> -> vhost_dev_virtqueue_restart() [vhost]
> -> vhost_virtqueue_start()
> -> vhost_net_set_backend()
> -> set enabled, reset status of vq.
>
> vhost-user:
> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> -> virtio_queue_reset() [virtio]
> -> virtio_net_queue_reset() [virtio-net]
> -> vhost_net_virtqueue_stop() [vhost-net]
> -> vhost_dev_virtqueue_stop() [vhost]
> -> vhost_user_reset_vring() [vhost-user]
> -> send VHOST_USER_RESET_VRING to the device
> -> vhost_virtqueue_unmap()
> -> __virtio_queue_reset()
> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> -> virtio_queue_enable() [virtio]
> -> virtio_net_queue_enable() [virtio-net]
> -> vhost_net_virtqueue_restart() [vhost-net]
> -> vhost_dev_virtqueue_restart() [vhost]
> -> vhost_virtqueue_start()
> -> vhost_user_set_single_vring_enable [vhost-user]
> -> send VHOST_USER_SET_VRING_ENABLE to the device
> -> set enabled, reset status of vq.
>
>
> Test environment:
> Host: 5.19.0-rc3 (With vq reset support)
> Qemu: QEMU emulator version 7.0.50
> Guest: 5.19.0-rc3 (With vq reset support)
> DPDK: 22.07-rc1 (With vq reset support)
> Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;
>
> The drvier can resize the virtio queue, then virtio queue reset function should
> be triggered.
>
> The default is split mode, modify Qemu virtio-net to add PACKED feature to
> test packed mode.
>
> Guest Kernel Patch:
> https://lore.kernel.org/bpf/20220801063902.129329-1-xuanzhuo@linux.alibaba.com/
>
> DPDK Patch:
> https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7
>
> Host Kernel Patch:
> https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3
>
> Looking forward to your review and comments. Thanks.
>
> Kangjie Xu (19):
> virtio: introduce virtio_queue_enable()
> virtio: core: vq reset feature negotation support
> virtio-pci: support queue enable
> vhost: extract the logic of unmapping the vrings and desc
> vhost: introduce vhost_dev_virtqueue_stop()
> vhost: introduce vhost_dev_virtqueue_restart()
> vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
> vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
> docs: vhost-user: add VHOST_USER_RESET_VRING message
> vhost-user: introduce vhost_reset_vring() interface
> vhost-user: add op to enable or disable a single vring
> vhost: vhost-user: update vhost_dev_virtqueue_stop()
> vhost: vhost-user: update vhost_dev_virtqueue_restart()
> vhost-net: vhost-user: update vhost_net_virtqueue_stop()
> vhost-net: vhost-user: update vhost_net_virtqueue_restart()
> virtio-net: introduce flush_or_purge_queued_packets()
> virtio-net: support queue_enable
> vhost: vhost-kernel: enable vq reset feature
> vhost: vhost-user: enable vq reset feature
>
> Xuan Zhuo (5):
> virtio: sync relevant definitions with linux
> virtio: introduce __virtio_queue_reset()
> virtio: introduce virtio_queue_reset()
> virtio-pci: support queue reset
> virtio-net: support queue reset
>
> docs/interop/vhost-user.rst | 10 +++
> hw/core/machine.c | 1 +
> hw/net/vhost_net.c | 79 +++++++++++++++++++
> hw/net/virtio-net.c | 58 ++++++++++++--
> hw/virtio/vhost-user.c | 67 ++++++++++++++--
> hw/virtio/vhost.c | 79 +++++++++++++++++--
> hw/virtio/virtio-pci.c | 20 +++++
> hw/virtio/virtio.c | 62 +++++++++++----
> include/hw/virtio/vhost-backend.h | 6 ++
> include/hw/virtio/vhost.h | 5 ++
> include/hw/virtio/virtio-pci.h | 1 +
> include/hw/virtio/virtio.h | 8 +-
> include/net/vhost_net.h | 4 +
> .../standard-headers/linux/virtio_config.h | 5 ++
> include/standard-headers/linux/virtio_pci.h | 2 +
> 15 files changed, 371 insertions(+), 36 deletions(-)
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 02/24] virtio: introduce __virtio_queue_reset()
2022-08-16 1:06 ` [PATCH v2 02/24] virtio: introduce __virtio_queue_reset() Kangjie Xu
@ 2022-08-23 7:31 ` Jason Wang
0 siblings, 0 replies; 77+ messages in thread
From: Jason Wang @ 2022-08-23 7:31 UTC (permalink / raw)
To: Kangjie Xu, qemu-devel
Cc: mst, eduardo, marcel.apfelbaum, f4bug, wangyanan55, hengqi, xuanzhuo
在 2022/8/16 09:06, Kangjie Xu 写道:
> From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>
> Separate the logic of vq reset. This logic will be called directly
> later.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
> ---
> hw/virtio/virtio.c | 37 +++++++++++++++++++++----------------
> 1 file changed, 21 insertions(+), 16 deletions(-)
>
> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> index 5d607aeaa0..67d54832a9 100644
> --- a/hw/virtio/virtio.c
> +++ b/hw/virtio/virtio.c
> @@ -2019,6 +2019,26 @@ static enum virtio_device_endian virtio_current_cpu_endian(void)
> }
> }
>
> +static void __virtio_queue_reset(VirtIODevice *vdev, uint32_t i)
> +{
> + vdev->vq[i].vring.desc = 0;
> + vdev->vq[i].vring.avail = 0;
> + vdev->vq[i].vring.used = 0;
> + vdev->vq[i].last_avail_idx = 0;
> + vdev->vq[i].shadow_avail_idx = 0;
> + vdev->vq[i].used_idx = 0;
> + vdev->vq[i].last_avail_wrap_counter = true;
> + vdev->vq[i].shadow_avail_wrap_counter = true;
> + vdev->vq[i].used_wrap_counter = true;
> + virtio_queue_set_vector(vdev, i, VIRTIO_NO_VECTOR);
> + vdev->vq[i].signalled_used = 0;
> + vdev->vq[i].signalled_used_valid = false;
> + vdev->vq[i].notification = true;
> + vdev->vq[i].vring.num = vdev->vq[i].vring.num_default;
> + vdev->vq[i].inuse = 0;
> + virtio_virtqueue_reset_region_cache(&vdev->vq[i]);
> +}
> +
> void virtio_reset(void *opaque)
> {
> VirtIODevice *vdev = opaque;
> @@ -2050,22 +2070,7 @@ void virtio_reset(void *opaque)
> virtio_notify_vector(vdev, vdev->config_vector);
>
> for(i = 0; i < VIRTIO_QUEUE_MAX; i++) {
> - vdev->vq[i].vring.desc = 0;
> - vdev->vq[i].vring.avail = 0;
> - vdev->vq[i].vring.used = 0;
> - vdev->vq[i].last_avail_idx = 0;
> - vdev->vq[i].shadow_avail_idx = 0;
> - vdev->vq[i].used_idx = 0;
> - vdev->vq[i].last_avail_wrap_counter = true;
> - vdev->vq[i].shadow_avail_wrap_counter = true;
> - vdev->vq[i].used_wrap_counter = true;
> - virtio_queue_set_vector(vdev, i, VIRTIO_NO_VECTOR);
> - vdev->vq[i].signalled_used = 0;
> - vdev->vq[i].signalled_used_valid = false;
> - vdev->vq[i].notification = true;
> - vdev->vq[i].vring.num = vdev->vq[i].vring.num_default;
> - vdev->vq[i].inuse = 0;
> - virtio_virtqueue_reset_region_cache(&vdev->vq[i]);
> + __virtio_queue_reset(vdev, i);
> }
> }
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 03/24] virtio: introduce virtio_queue_reset()
2022-08-16 1:06 ` [PATCH v2 03/24] virtio: introduce virtio_queue_reset() Kangjie Xu
@ 2022-08-23 7:32 ` Jason Wang
0 siblings, 0 replies; 77+ messages in thread
From: Jason Wang @ 2022-08-23 7:32 UTC (permalink / raw)
To: Kangjie Xu, qemu-devel
Cc: mst, eduardo, marcel.apfelbaum, f4bug, wangyanan55, hengqi, xuanzhuo
在 2022/8/16 09:06, Kangjie Xu 写道:
> From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>
> Introduce a new interface function virtio_queue_reset() to implement
> reset for vq.
>
> Add a new callback to VirtioDeviceClass for queue reset operation for
> each child device.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
> ---
> hw/virtio/virtio.c | 11 +++++++++++
> include/hw/virtio/virtio.h | 2 ++
> 2 files changed, 13 insertions(+)
>
> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> index 67d54832a9..0e9d41366f 100644
> --- a/hw/virtio/virtio.c
> +++ b/hw/virtio/virtio.c
> @@ -2039,6 +2039,17 @@ static void __virtio_queue_reset(VirtIODevice *vdev, uint32_t i)
> virtio_virtqueue_reset_region_cache(&vdev->vq[i]);
> }
>
> +void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
> +{
> + VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
> +
> + if (k->queue_reset) {
> + k->queue_reset(vdev, queue_index);
> + }
> +
> + __virtio_queue_reset(vdev, queue_index);
> +}
> +
> void virtio_reset(void *opaque)
> {
> VirtIODevice *vdev = opaque;
> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> index db1c0ddf6b..879394299b 100644
> --- a/include/hw/virtio/virtio.h
> +++ b/include/hw/virtio/virtio.h
> @@ -130,6 +130,7 @@ struct VirtioDeviceClass {
> void (*set_config)(VirtIODevice *vdev, const uint8_t *config);
> void (*reset)(VirtIODevice *vdev);
> void (*set_status)(VirtIODevice *vdev, uint8_t val);
> + void (*queue_reset)(VirtIODevice *vdev, uint32_t queue_index);
> /* For transitional devices, this is a bitmap of features
> * that are only exposed on the legacy interface but not
> * the modern one.
> @@ -268,6 +269,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,
> MemoryRegion *mr, bool assign);
> int virtio_set_status(VirtIODevice *vdev, uint8_t val);
> void virtio_reset(void *opaque);
> +void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index);
> void virtio_update_irq(VirtIODevice *vdev);
> int virtio_set_features(VirtIODevice *vdev, uint64_t val);
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 05/24] virtio: core: vq reset feature negotation support
2022-08-16 1:06 ` [PATCH v2 05/24] virtio: core: vq reset feature negotation support Kangjie Xu
@ 2022-08-23 7:34 ` Jason Wang
2022-08-23 7:42 ` Kangjie Xu
0 siblings, 1 reply; 77+ messages in thread
From: Jason Wang @ 2022-08-23 7:34 UTC (permalink / raw)
To: Kangjie Xu, qemu-devel
Cc: mst, eduardo, marcel.apfelbaum, f4bug, wangyanan55, hengqi, xuanzhuo
在 2022/8/16 09:06, Kangjie Xu 写道:
> A a new command line parameter "queue_reset" is added.
>
> Meanwhile, the vq reset feature is disabled for pre-7.1 machines.
>
> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> hw/core/machine.c | 1 +
> include/hw/virtio/virtio.h | 4 +++-
> 2 files changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index a673302cce..8b22b4647f 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -43,6 +43,7 @@
> GlobalProperty hw_compat_7_0[] = {
> { "arm-gicv3-common", "force-8-bit-prio", "on" },
> { "nvme-ns", "eui64-default", "on"},
> + { "virtio-device", "queue_reset", "false" },
7.1 is about to release so we need to do it for pre-7.2.
Thanks
> };
> const size_t hw_compat_7_0_len = G_N_ELEMENTS(hw_compat_7_0);
>
> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> index 085997d8f3..ed3ecbef80 100644
> --- a/include/hw/virtio/virtio.h
> +++ b/include/hw/virtio/virtio.h
> @@ -295,7 +295,9 @@ typedef struct VirtIORNGConf VirtIORNGConf;
> DEFINE_PROP_BIT64("iommu_platform", _state, _field, \
> VIRTIO_F_IOMMU_PLATFORM, false), \
> DEFINE_PROP_BIT64("packed", _state, _field, \
> - VIRTIO_F_RING_PACKED, false)
> + VIRTIO_F_RING_PACKED, false), \
> + DEFINE_PROP_BIT64("queue_reset", _state, _field, \
> + VIRTIO_F_RING_RESET, true)
>
> hwaddr virtio_queue_get_desc_addr(VirtIODevice *vdev, int n);
> bool virtio_queue_enabled_legacy(VirtIODevice *vdev, int n);
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 04/24] virtio: introduce virtio_queue_enable()
2022-08-16 1:06 ` [PATCH v2 04/24] virtio: introduce virtio_queue_enable() Kangjie Xu
@ 2022-08-23 7:37 ` Jason Wang
0 siblings, 0 replies; 77+ messages in thread
From: Jason Wang @ 2022-08-23 7:37 UTC (permalink / raw)
To: Kangjie Xu, qemu-devel
Cc: mst, eduardo, marcel.apfelbaum, f4bug, wangyanan55, hengqi, xuanzhuo
在 2022/8/16 09:06, Kangjie Xu 写道:
> Introduce the interface queue_enable() in VirtioDeviceClass and the
> fucntion virtio_queue_enable() in virtio, it can be called when
> VIRTIO_PCI_COMMON_Q_ENABLE is written and related virtqueue can be
> started. It only supports the devices of virtio 1 or later. The
> not-supported devices can only start the virtqueue when DRIVER_OK.
>
> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
Acked-by: Jason Wang <jasowang@redhat.com>
> hw/virtio/virtio.c | 14 ++++++++++++++
> include/hw/virtio/virtio.h | 2 ++
> 2 files changed, 16 insertions(+)
>
> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> index 0e9d41366f..141f18c633 100644
> --- a/hw/virtio/virtio.c
> +++ b/hw/virtio/virtio.c
> @@ -2050,6 +2050,20 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
> __virtio_queue_reset(vdev, queue_index);
> }
>
> +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index)
> +{
> + VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
> +
> + if (!virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
> + error_report("queue_enable is only suppported in devices of virtio "
> + "1.0 or later.");
> + }
> +
> + if (k->queue_enable) {
> + k->queue_enable(vdev, queue_index);
> + }
> +}
> +
> void virtio_reset(void *opaque)
> {
> VirtIODevice *vdev = opaque;
> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> index 879394299b..085997d8f3 100644
> --- a/include/hw/virtio/virtio.h
> +++ b/include/hw/virtio/virtio.h
> @@ -131,6 +131,7 @@ struct VirtioDeviceClass {
> void (*reset)(VirtIODevice *vdev);
> void (*set_status)(VirtIODevice *vdev, uint8_t val);
> void (*queue_reset)(VirtIODevice *vdev, uint32_t queue_index);
> + void (*queue_enable)(VirtIODevice *vdev, uint32_t queue_index);
> /* For transitional devices, this is a bitmap of features
> * that are only exposed on the legacy interface but not
> * the modern one.
> @@ -270,6 +271,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,
> int virtio_set_status(VirtIODevice *vdev, uint8_t val);
> void virtio_reset(void *opaque);
> void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index);
> +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index);
> void virtio_update_irq(VirtIODevice *vdev);
> int virtio_set_features(VirtIODevice *vdev, uint64_t val);
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 06/24] virtio-pci: support queue reset
2022-08-16 1:06 ` [PATCH v2 06/24] virtio-pci: support queue reset Kangjie Xu
@ 2022-08-23 7:40 ` Jason Wang
2022-08-23 7:52 ` Kangjie Xu
0 siblings, 1 reply; 77+ messages in thread
From: Jason Wang @ 2022-08-23 7:40 UTC (permalink / raw)
To: Kangjie Xu, qemu-devel
Cc: mst, eduardo, marcel.apfelbaum, f4bug, wangyanan55, hengqi, xuanzhuo
在 2022/8/16 09:06, Kangjie Xu 写道:
> From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>
> PCI devices support vq reset.
>
> Based on this function, the driver can adjust the size of the ring, and
> quickly recycle the buffer in the ring.
>
> The migration of the virtio devices will not happen during a reset
> operation. This is becuase the global iothread lock is held. Migration
> thread also needs the lock. As a result, we do not need to migrate the
> reset state of VirtIOPCIQueue.
>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> ---
> hw/virtio/virtio-pci.c | 19 +++++++++++++++++++
> include/hw/virtio/virtio-pci.h | 1 +
> 2 files changed, 20 insertions(+)
>
> diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
> index 45327f0b31..ec8e92052f 100644
> --- a/hw/virtio/virtio-pci.c
> +++ b/hw/virtio/virtio-pci.c
> @@ -1246,6 +1246,9 @@ static uint64_t virtio_pci_common_read(void *opaque, hwaddr addr,
> case VIRTIO_PCI_COMMON_Q_USEDHI:
> val = proxy->vqs[vdev->queue_sel].used[1];
> break;
> + case VIRTIO_PCI_COMMON_Q_RESET:
> + val = proxy->vqs[vdev->queue_sel].reset;
> + break;
> default:
> val = 0;
> }
> @@ -1333,6 +1336,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,
> ((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << 32 |
> proxy->vqs[vdev->queue_sel].used[0]);
> proxy->vqs[vdev->queue_sel].enabled = 1;
> + proxy->vqs[vdev->queue_sel].reset = 0;
> } else {
> virtio_error(vdev, "wrong value for queue_enable %"PRIx64, val);
> }
> @@ -1355,6 +1359,20 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,
> case VIRTIO_PCI_COMMON_Q_USEDHI:
> proxy->vqs[vdev->queue_sel].used[1] = val;
> break;
> + case VIRTIO_PCI_COMMON_Q_RESET:
> + if (val == 1) {
> + /*
> + * With the global iothread lock taken, the migration will not
> + * happen until the virtqueue reset is done.
> + */
This comment applies to all other common cfg operation as well, So it
looks not necessary?
> + proxy->vqs[vdev->queue_sel].reset = 1;
> +
> + virtio_queue_reset(vdev, vdev->queue_sel);
> +
> + proxy->vqs[vdev->queue_sel].reset = 0;
> + proxy->vqs[vdev->queue_sel].enabled = 0;
> + }
> + break;
> default:
> break;
> }
> @@ -1950,6 +1968,7 @@ static void virtio_pci_reset(DeviceState *qdev)
>
> for (i = 0; i < VIRTIO_QUEUE_MAX; i++) {
> proxy->vqs[i].enabled = 0;
> + proxy->vqs[i].reset = 0;
> proxy->vqs[i].num = 0;
> proxy->vqs[i].desc[0] = proxy->vqs[i].desc[1] = 0;
> proxy->vqs[i].avail[0] = proxy->vqs[i].avail[1] = 0;
> diff --git a/include/hw/virtio/virtio-pci.h b/include/hw/virtio/virtio-pci.h
> index 2446dcd9ae..e9290e2b94 100644
> --- a/include/hw/virtio/virtio-pci.h
> +++ b/include/hw/virtio/virtio-pci.h
> @@ -117,6 +117,7 @@ typedef struct VirtIOPCIRegion {
> typedef struct VirtIOPCIQueue {
> uint16_t num;
> bool enabled;
> + bool reset;
Do we need to migrate this?
Thanks
> uint32_t desc[2];
> uint32_t avail[2];
> uint32_t used[2];
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 05/24] virtio: core: vq reset feature negotation support
2022-08-23 7:34 ` Jason Wang
@ 2022-08-23 7:42 ` Kangjie Xu
0 siblings, 0 replies; 77+ messages in thread
From: Kangjie Xu @ 2022-08-23 7:42 UTC (permalink / raw)
To: Jason Wang
Cc: Michael S. Tsirkin, qemu-devel, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/23 15:34, Jason Wang 写道:
>
> 在 2022/8/16 09:06, Kangjie Xu 写道:
>> A a new command line parameter "queue_reset" is added.
>>
>> Meanwhile, the vq reset feature is disabled for pre-7.1 machines.
>>
>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>> ---
>> hw/core/machine.c | 1 +
>> include/hw/virtio/virtio.h | 4 +++-
>> 2 files changed, 4 insertions(+), 1 deletion(-)
>>
>> diff --git a/hw/core/machine.c b/hw/core/machine.c
>> index a673302cce..8b22b4647f 100644
>> --- a/hw/core/machine.c
>> +++ b/hw/core/machine.c
>> @@ -43,6 +43,7 @@
>> GlobalProperty hw_compat_7_0[] = {
>> { "arm-gicv3-common", "force-8-bit-prio", "on" },
>> { "nvme-ns", "eui64-default", "on"},
>> + { "virtio-device", "queue_reset", "false" },
>
>
> 7.1 is about to release so we need to do it for pre-7.2.
>
> Thanks
>
The current version is 7.0.92, should I wait until 7.1 to be released
and submit my patch after that? or just add hw_compat_7_1[] and related
support.
Thanks
>
>> };
>> const size_t hw_compat_7_0_len = G_N_ELEMENTS(hw_compat_7_0);
>> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
>> index 085997d8f3..ed3ecbef80 100644
>> --- a/include/hw/virtio/virtio.h
>> +++ b/include/hw/virtio/virtio.h
>> @@ -295,7 +295,9 @@ typedef struct VirtIORNGConf VirtIORNGConf;
>> DEFINE_PROP_BIT64("iommu_platform", _state, _field, \
>> VIRTIO_F_IOMMU_PLATFORM, false), \
>> DEFINE_PROP_BIT64("packed", _state, _field, \
>> - VIRTIO_F_RING_PACKED, false)
>> + VIRTIO_F_RING_PACKED, false), \
>> + DEFINE_PROP_BIT64("queue_reset", _state, _field, \
>> + VIRTIO_F_RING_RESET, true)
>> hwaddr virtio_queue_get_desc_addr(VirtIODevice *vdev, int n);
>> bool virtio_queue_enabled_legacy(VirtIODevice *vdev, int n);
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 07/24] virtio-pci: support queue enable
2022-08-16 1:06 ` [PATCH v2 07/24] virtio-pci: support queue enable Kangjie Xu
@ 2022-08-23 7:44 ` Jason Wang
2022-08-23 8:20 ` Kangjie Xu
0 siblings, 1 reply; 77+ messages in thread
From: Jason Wang @ 2022-08-23 7:44 UTC (permalink / raw)
To: Kangjie Xu, qemu-devel
Cc: mst, eduardo, marcel.apfelbaum, f4bug, wangyanan55, hengqi, xuanzhuo
在 2022/8/16 09:06, Kangjie Xu 写道:
> PCI devices support vq enable.
Nit: it might be "support device specific vq enable"
>
> Based on this function, the driver can re-enable the virtqueue after the
> virtqueue is reset.
>
> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> hw/virtio/virtio-pci.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
> index ec8e92052f..3d560e45ad 100644
> --- a/hw/virtio/virtio-pci.c
> +++ b/hw/virtio/virtio-pci.c
> @@ -1335,6 +1335,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,
> proxy->vqs[vdev->queue_sel].avail[0],
> ((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << 32 |
> proxy->vqs[vdev->queue_sel].used[0]);
> + virtio_queue_enable(vdev, vdev->queue_sel);
> proxy->vqs[vdev->queue_sel].enabled = 1;
> proxy->vqs[vdev->queue_sel].reset = 0;
Any reason we do it before the assignment of 1? It probably means the
device specific method can't depend on virtio_queue_enabled()?
Thanks
> } else {
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 08/24] vhost: extract the logic of unmapping the vrings and desc
2022-08-16 1:06 ` [PATCH v2 08/24] vhost: extract the logic of unmapping the vrings and desc Kangjie Xu
@ 2022-08-23 7:45 ` Jason Wang
0 siblings, 0 replies; 77+ messages in thread
From: Jason Wang @ 2022-08-23 7:45 UTC (permalink / raw)
To: Kangjie Xu, qemu-devel
Cc: mst, eduardo, marcel.apfelbaum, f4bug, wangyanan55, hengqi, xuanzhuo
在 2022/8/16 09:06, Kangjie Xu 写道:
> Introduce vhost_virtqueue_unmap() to ummap the vrings and desc
> of a virtqueue.
>
> The function will be used later.
>
> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
> ---
> hw/virtio/vhost.c | 20 ++++++++++++++------
> 1 file changed, 14 insertions(+), 6 deletions(-)
>
> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index 0827d631c0..e467dfc7bc 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -1197,6 +1197,19 @@ fail_alloc_desc:
> return r;
> }
>
> +static void vhost_virtqueue_unmap(struct vhost_dev *dev,
> + struct VirtIODevice *vdev,
> + struct vhost_virtqueue *vq,
> + unsigned idx)
> +{
> + vhost_memory_unmap(dev, vq->used, virtio_queue_get_used_size(vdev, idx),
> + 1, virtio_queue_get_used_size(vdev, idx));
> + vhost_memory_unmap(dev, vq->avail, virtio_queue_get_avail_size(vdev, idx),
> + 0, virtio_queue_get_avail_size(vdev, idx));
> + vhost_memory_unmap(dev, vq->desc, virtio_queue_get_desc_size(vdev, idx),
> + 0, virtio_queue_get_desc_size(vdev, idx));
> +}
> +
> static void vhost_virtqueue_stop(struct vhost_dev *dev,
> struct VirtIODevice *vdev,
> struct vhost_virtqueue *vq,
> @@ -1235,12 +1248,7 @@ static void vhost_virtqueue_stop(struct vhost_dev *dev,
> vhost_vq_index);
> }
>
> - vhost_memory_unmap(dev, vq->used, virtio_queue_get_used_size(vdev, idx),
> - 1, virtio_queue_get_used_size(vdev, idx));
> - vhost_memory_unmap(dev, vq->avail, virtio_queue_get_avail_size(vdev, idx),
> - 0, virtio_queue_get_avail_size(vdev, idx));
> - vhost_memory_unmap(dev, vq->desc, virtio_queue_get_desc_size(vdev, idx),
> - 0, virtio_queue_get_desc_size(vdev, idx));
> + vhost_virtqueue_unmap(dev, vdev, vq, idx);
> }
>
> static void vhost_eventfd_add(MemoryListener *listener,
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 06/24] virtio-pci: support queue reset
2022-08-23 7:40 ` Jason Wang
@ 2022-08-23 7:52 ` Kangjie Xu
2022-08-24 8:56 ` Jason Wang
0 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-23 7:52 UTC (permalink / raw)
To: Jason Wang
Cc: Michael S. Tsirkin, qemu-devel, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/23 15:40, Jason Wang 写道:
>
> 在 2022/8/16 09:06, Kangjie Xu 写道:
>> From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>
>> PCI devices support vq reset.
>>
>> Based on this function, the driver can adjust the size of the ring, and
>> quickly recycle the buffer in the ring.
>>
>> The migration of the virtio devices will not happen during a reset
>> operation. This is becuase the global iothread lock is held. Migration
>> thread also needs the lock. As a result, we do not need to migrate the
>> reset state of VirtIOPCIQueue.
>>
>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>> ---
>> hw/virtio/virtio-pci.c | 19 +++++++++++++++++++
>> include/hw/virtio/virtio-pci.h | 1 +
>> 2 files changed, 20 insertions(+)
>>
>> diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
>> index 45327f0b31..ec8e92052f 100644
>> --- a/hw/virtio/virtio-pci.c
>> +++ b/hw/virtio/virtio-pci.c
>> @@ -1246,6 +1246,9 @@ static uint64_t virtio_pci_common_read(void
>> *opaque, hwaddr addr,
>> case VIRTIO_PCI_COMMON_Q_USEDHI:
>> val = proxy->vqs[vdev->queue_sel].used[1];
>> break;
>> + case VIRTIO_PCI_COMMON_Q_RESET:
>> + val = proxy->vqs[vdev->queue_sel].reset;
>> + break;
>> default:
>> val = 0;
>> }
>> @@ -1333,6 +1336,7 @@ static void virtio_pci_common_write(void
>> *opaque, hwaddr addr,
>> ((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << 32 |
>> proxy->vqs[vdev->queue_sel].used[0]);
>> proxy->vqs[vdev->queue_sel].enabled = 1;
>> + proxy->vqs[vdev->queue_sel].reset = 0;
>> } else {
>> virtio_error(vdev, "wrong value for queue_enable
>> %"PRIx64, val);
>> }
>> @@ -1355,6 +1359,20 @@ static void virtio_pci_common_write(void
>> *opaque, hwaddr addr,
>> case VIRTIO_PCI_COMMON_Q_USEDHI:
>> proxy->vqs[vdev->queue_sel].used[1] = val;
>> break;
>> + case VIRTIO_PCI_COMMON_Q_RESET:
>> + if (val == 1) {
>> + /*
>> + * With the global iothread lock taken, the migration
>> will not
>> + * happen until the virtqueue reset is done.
>> + */
>
>
> This comment applies to all other common cfg operation as well, So it
> looks not necessary?
>
Get it.
>
>> + proxy->vqs[vdev->queue_sel].reset = 1;
>> +
>> + virtio_queue_reset(vdev, vdev->queue_sel);
>> +
>> + proxy->vqs[vdev->queue_sel].reset = 0;
>> + proxy->vqs[vdev->queue_sel].enabled = 0;
>> + }
>> + break;
>> default:
>> break;
>> }
>> @@ -1950,6 +1968,7 @@ static void virtio_pci_reset(DeviceState *qdev)
>> for (i = 0; i < VIRTIO_QUEUE_MAX; i++) {
>> proxy->vqs[i].enabled = 0;
>> + proxy->vqs[i].reset = 0;
>> proxy->vqs[i].num = 0;
>> proxy->vqs[i].desc[0] = proxy->vqs[i].desc[1] = 0;
>> proxy->vqs[i].avail[0] = proxy->vqs[i].avail[1] = 0;
>> diff --git a/include/hw/virtio/virtio-pci.h
>> b/include/hw/virtio/virtio-pci.h
>> index 2446dcd9ae..e9290e2b94 100644
>> --- a/include/hw/virtio/virtio-pci.h
>> +++ b/include/hw/virtio/virtio-pci.h
>> @@ -117,6 +117,7 @@ typedef struct VirtIOPCIRegion {
>> typedef struct VirtIOPCIQueue {
>> uint16_t num;
>> bool enabled;
>> + bool reset;
>
>
> Do we need to migrate this?
>
> Thanks
>
I think we do not need to migrate this because we hold the global
iothread lock when virtqueue reset is triggered. The migration of these
device states also needs this lock.
On the other hand, the 'reset' state of virtqueue is same(is 0) before
and after the process of resetting a virtqueue.
Thus, the migration will not happen when we are resetting a virtqueue
and we do not to migrate it.
Thanks
>
>> uint32_t desc[2];
>> uint32_t avail[2];
>> uint32_t used[2];
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 09/24] vhost: introduce vhost_dev_virtqueue_stop()
2022-08-16 1:06 ` [PATCH v2 09/24] vhost: introduce vhost_dev_virtqueue_stop() Kangjie Xu
@ 2022-08-23 7:52 ` Jason Wang
2022-08-23 8:03 ` Kangjie Xu
0 siblings, 1 reply; 77+ messages in thread
From: Jason Wang @ 2022-08-23 7:52 UTC (permalink / raw)
To: Kangjie Xu, qemu-devel
Cc: mst, eduardo, marcel.apfelbaum, f4bug, wangyanan55, hengqi, xuanzhuo
在 2022/8/16 09:06, Kangjie Xu 写道:
> Introduce vhost_dev_virtqueue_stop(), which can ummap the
> vrings and the desc of it.
>
> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> hw/virtio/vhost.c | 9 +++++++++
> include/hw/virtio/vhost.h | 3 +++
> 2 files changed, 12 insertions(+)
>
> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index e467dfc7bc..1bca9ff48d 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -1904,3 +1904,12 @@ int vhost_net_set_backend(struct vhost_dev *hdev,
>
> return -ENOSYS;
> }
> +
> +void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice *vdev,
> + int idx)
> +{
> + vhost_virtqueue_unmap(hdev,
> + vdev,
> + hdev->vqs + idx,
> + idx);
So I think the unmap is not sufficient, we need backend specific
support. E.g for vhost kernel, need a SET_BACKEND here?
Thanks
> +}
> diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
> index a346f23d13..574888440c 100644
> --- a/include/hw/virtio/vhost.h
> +++ b/include/hw/virtio/vhost.h
> @@ -288,4 +288,7 @@ int vhost_dev_set_inflight(struct vhost_dev *dev,
> struct vhost_inflight *inflight);
> int vhost_dev_get_inflight(struct vhost_dev *dev, uint16_t queue_size,
> struct vhost_inflight *inflight);
> +
> +void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice *vdev,
> + int idx);
> #endif
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 09/24] vhost: introduce vhost_dev_virtqueue_stop()
2022-08-23 7:52 ` Jason Wang
@ 2022-08-23 8:03 ` Kangjie Xu
2022-08-24 2:40 ` Jason Wang
0 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-23 8:03 UTC (permalink / raw)
To: Jason Wang
Cc: Michael S. Tsirkin, qemu-devel, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/23 15:52, Jason Wang 写道:
>
> 在 2022/8/16 09:06, Kangjie Xu 写道:
>> Introduce vhost_dev_virtqueue_stop(), which can ummap the
>> vrings and the desc of it.
>>
>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>> ---
>> hw/virtio/vhost.c | 9 +++++++++
>> include/hw/virtio/vhost.h | 3 +++
>> 2 files changed, 12 insertions(+)
>>
>> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
>> index e467dfc7bc..1bca9ff48d 100644
>> --- a/hw/virtio/vhost.c
>> +++ b/hw/virtio/vhost.c
>> @@ -1904,3 +1904,12 @@ int vhost_net_set_backend(struct vhost_dev *hdev,
>> return -ENOSYS;
>> }
>> +
>> +void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice
>> *vdev,
>> + int idx)
>> +{
>> + vhost_virtqueue_unmap(hdev,
>> + vdev,
>> + hdev->vqs + idx,
>> + idx);
>
>
> So I think the unmap is not sufficient, we need backend specific
> support. E.g for vhost kernel, need a SET_BACKEND here?
>
> Thanks
>
But SET_BACKEND of vhost-net needs a parameter fd in vhost_vring_file:
that is net->backend of VHostNetState.
If we add the fd parameter or struct vhost_vring_file to
vhost_dev_virtqueue_stop/restart, it exposes some implementation details
in the parameter list.
And that seems not good? So I put SET_BACKEND in the vhost-net module.
The workflow is similar to vhost_net_start_one().
Thanks
>
>> +}
>> diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
>> index a346f23d13..574888440c 100644
>> --- a/include/hw/virtio/vhost.h
>> +++ b/include/hw/virtio/vhost.h
>> @@ -288,4 +288,7 @@ int vhost_dev_set_inflight(struct vhost_dev *dev,
>> struct vhost_inflight *inflight);
>> int vhost_dev_get_inflight(struct vhost_dev *dev, uint16_t queue_size,
>> struct vhost_inflight *inflight);
>> +
>> +void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice
>> *vdev,
>> + int idx);
>> #endif
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 07/24] virtio-pci: support queue enable
2022-08-23 7:44 ` Jason Wang
@ 2022-08-23 8:20 ` Kangjie Xu
2022-08-24 8:59 ` Jason Wang
0 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-23 8:20 UTC (permalink / raw)
To: Jason Wang
Cc: Michael S. Tsirkin, qemu-devel, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/23 15:44, Jason Wang 写道:
>
> 在 2022/8/16 09:06, Kangjie Xu 写道:
>> PCI devices support vq enable.
>
>
> Nit: it might be "support device specific vq enable"
>
>
Get it.
>>
>> Based on this function, the driver can re-enable the virtqueue after the
>> virtqueue is reset.
>>
>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>> ---
>> hw/virtio/virtio-pci.c | 1 +
>> 1 file changed, 1 insertion(+)
>>
>> diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
>> index ec8e92052f..3d560e45ad 100644
>> --- a/hw/virtio/virtio-pci.c
>> +++ b/hw/virtio/virtio-pci.c
>> @@ -1335,6 +1335,7 @@ static void virtio_pci_common_write(void
>> *opaque, hwaddr addr,
>> proxy->vqs[vdev->queue_sel].avail[0],
>> ((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << 32 |
>> proxy->vqs[vdev->queue_sel].used[0]);
>> + virtio_queue_enable(vdev, vdev->queue_sel);
>> proxy->vqs[vdev->queue_sel].enabled = 1;
>> proxy->vqs[vdev->queue_sel].reset = 0;
>
>
> Any reason we do it before the assignment of 1? It probably means the
> device specific method can't depend on virtio_queue_enabled()?
>
> Thanks
>
Sorry, I don't get why device specific method can't depend on
virtio_queue_enabled().
Before virtio_queue_enable() is done, virtqueue should always be not
ready and disabled.
Otherwise, If we put it after the assignment of enabled to 1, the
virtqueue may be accessed illegally and may cause panic, because the
virtqueue is still being intialized and being configured.
Thanks
>
>> } else {
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 10/24] vhost: introduce vhost_dev_virtqueue_restart()
2022-08-16 1:06 ` [PATCH v2 10/24] vhost: introduce vhost_dev_virtqueue_restart() Kangjie Xu
@ 2022-08-24 2:37 ` Jason Wang
2022-08-24 2:44 ` Kangjie Xu
0 siblings, 1 reply; 77+ messages in thread
From: Jason Wang @ 2022-08-24 2:37 UTC (permalink / raw)
To: Kangjie Xu, qemu-devel
Cc: mst, eduardo, marcel.apfelbaum, f4bug, wangyanan55, hengqi, xuanzhuo
在 2022/8/16 09:06, Kangjie Xu 写道:
> Introduce vhost_dev_virtqueue_restart(), which can restart the
> virtqueue when the vhost has already started running.
>
> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> hw/virtio/vhost.c | 13 +++++++++++++
> include/hw/virtio/vhost.h | 2 ++
> 2 files changed, 15 insertions(+)
>
> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index 1bca9ff48d..fc3f550c76 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -1913,3 +1913,16 @@ void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice *vdev,
> hdev->vqs + idx,
> idx);
> }
> +
> +int vhost_dev_virtqueue_restart(struct vhost_dev *hdev, VirtIODevice *vdev,
> + int idx)
> +{
> + const VhostOps *vhost_ops = hdev->vhost_ops;
> +
> + assert(vhost_ops);
So we had the comment like:
/* should only be called after backend is connected */
in vhost_virtqueue_mask().
If this assert has the same reason, let's add a comment here.
> +
> + return vhost_virtqueue_start(hdev,
> + vdev,
> + hdev->vqs + idx,
> + hdev->vq_index + idx);
So it just a wrapper of vhost_virtqueue_start(), any value to have a
re-start wrapper?
Thanks
> +}
> diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
> index 574888440c..b3394b6348 100644
> --- a/include/hw/virtio/vhost.h
> +++ b/include/hw/virtio/vhost.h
> @@ -291,4 +291,6 @@ int vhost_dev_get_inflight(struct vhost_dev *dev, uint16_t queue_size,
>
> void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice *vdev,
> int idx);
> +int vhost_dev_virtqueue_restart(struct vhost_dev *hdev, VirtIODevice *vdev,
> + int idx);
> #endif
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 11/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
2022-08-16 1:06 ` [PATCH v2 11/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop() Kangjie Xu
@ 2022-08-24 2:40 ` Jason Wang
2022-08-24 2:46 ` Kangjie Xu
2022-08-24 3:33 ` Kangjie Xu
0 siblings, 2 replies; 77+ messages in thread
From: Jason Wang @ 2022-08-24 2:40 UTC (permalink / raw)
To: Kangjie Xu, qemu-devel
Cc: mst, eduardo, marcel.apfelbaum, f4bug, wangyanan55, hengqi, xuanzhuo
在 2022/8/16 09:06, Kangjie Xu 写道:
> Introduce vhost_virtqueue_stop(), which can reset the virtqueue
> in the device. Then it will unmap vrings and the desc of the
> virtqueue.
>
> This patch only considers the case for vhost-kernel, when
> NetClientDriver is NET_CLIENT_DRIVER_TAP.
>
> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> hw/net/vhost_net.c | 21 +++++++++++++++++++++
> include/net/vhost_net.h | 2 ++
> 2 files changed, 23 insertions(+)
>
> diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
> index ccac5b7a64..aa60dd901c 100644
> --- a/hw/net/vhost_net.c
> +++ b/hw/net/vhost_net.c
> @@ -514,3 +514,24 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu)
>
> return vhost_ops->vhost_net_set_mtu(&net->dev, mtu);
> }
> +
> +void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc,
> + int vq_index)
> +{
> + VHostNetState *net = get_vhost_net(nc->peer);
> + const VhostOps *vhost_ops = net->dev.vhost_ops;
> + struct vhost_vring_file file = { .fd = -1 };
> + int idx;
> +
> + assert(vhost_ops);
> +
> + idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index);
> +
> + if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) {
> + file.index = idx;
> + int r = vhost_net_set_backend(&net->dev, &file);
> + assert(r >= 0);
> + }
Let's have a vhost_ops here instead of open code it.
Thanks
> +
> + vhost_dev_virtqueue_stop(&net->dev, vdev, idx);
> +}
> diff --git a/include/net/vhost_net.h b/include/net/vhost_net.h
> index 387e913e4e..9b3aaf3814 100644
> --- a/include/net/vhost_net.h
> +++ b/include/net/vhost_net.h
> @@ -48,4 +48,6 @@ uint64_t vhost_net_get_acked_features(VHostNetState *net);
>
> int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu);
>
> +void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc,
> + int vq_index);
> #endif
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 09/24] vhost: introduce vhost_dev_virtqueue_stop()
2022-08-23 8:03 ` Kangjie Xu
@ 2022-08-24 2:40 ` Jason Wang
0 siblings, 0 replies; 77+ messages in thread
From: Jason Wang @ 2022-08-24 2:40 UTC (permalink / raw)
To: Kangjie Xu
Cc: Michael S. Tsirkin, qemu-devel, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/23 16:03, Kangjie Xu 写道:
>
> 在 2022/8/23 15:52, Jason Wang 写道:
>>
>> 在 2022/8/16 09:06, Kangjie Xu 写道:
>>> Introduce vhost_dev_virtqueue_stop(), which can ummap the
>>> vrings and the desc of it.
>>>
>>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>> ---
>>> hw/virtio/vhost.c | 9 +++++++++
>>> include/hw/virtio/vhost.h | 3 +++
>>> 2 files changed, 12 insertions(+)
>>>
>>> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
>>> index e467dfc7bc..1bca9ff48d 100644
>>> --- a/hw/virtio/vhost.c
>>> +++ b/hw/virtio/vhost.c
>>> @@ -1904,3 +1904,12 @@ int vhost_net_set_backend(struct vhost_dev
>>> *hdev,
>>> return -ENOSYS;
>>> }
>>> +
>>> +void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice
>>> *vdev,
>>> + int idx)
>>> +{
>>> + vhost_virtqueue_unmap(hdev,
>>> + vdev,
>>> + hdev->vqs + idx,
>>> + idx);
>>
>>
>> So I think the unmap is not sufficient, we need backend specific
>> support. E.g for vhost kernel, need a SET_BACKEND here?
>>
>> Thanks
>>
> But SET_BACKEND of vhost-net needs a parameter fd in vhost_vring_file:
> that is net->backend of VHostNetState.
>
> If we add the fd parameter or struct vhost_vring_file to
> vhost_dev_virtqueue_stop/restart, it exposes some implementation
> details in the parameter list.
>
> And that seems not good? So I put SET_BACKEND in the vhost-net module.
> The workflow is similar to vhost_net_start_one().
That looks fine.
Thanks
>
> Thanks
>
>>
>>> +}
>>> diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
>>> index a346f23d13..574888440c 100644
>>> --- a/include/hw/virtio/vhost.h
>>> +++ b/include/hw/virtio/vhost.h
>>> @@ -288,4 +288,7 @@ int vhost_dev_set_inflight(struct vhost_dev *dev,
>>> struct vhost_inflight *inflight);
>>> int vhost_dev_get_inflight(struct vhost_dev *dev, uint16_t
>>> queue_size,
>>> struct vhost_inflight *inflight);
>>> +
>>> +void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice
>>> *vdev,
>>> + int idx);
>>> #endif
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 10/24] vhost: introduce vhost_dev_virtqueue_restart()
2022-08-24 2:37 ` Jason Wang
@ 2022-08-24 2:44 ` Kangjie Xu
0 siblings, 0 replies; 77+ messages in thread
From: Kangjie Xu @ 2022-08-24 2:44 UTC (permalink / raw)
To: Jason Wang
Cc: Michael S. Tsirkin, qemu-devel, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/24 10:37, Jason Wang 写道:
>
> 在 2022/8/16 09:06, Kangjie Xu 写道:
>> Introduce vhost_dev_virtqueue_restart(), which can restart the
>> virtqueue when the vhost has already started running.
>>
>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>> ---
>> hw/virtio/vhost.c | 13 +++++++++++++
>> include/hw/virtio/vhost.h | 2 ++
>> 2 files changed, 15 insertions(+)
>>
>> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
>> index 1bca9ff48d..fc3f550c76 100644
>> --- a/hw/virtio/vhost.c
>> +++ b/hw/virtio/vhost.c
>> @@ -1913,3 +1913,16 @@ void vhost_dev_virtqueue_stop(struct vhost_dev
>> *hdev, VirtIODevice *vdev,
>> hdev->vqs + idx,
>> idx);
>> }
>> +
>> +int vhost_dev_virtqueue_restart(struct vhost_dev *hdev, VirtIODevice
>> *vdev,
>> + int idx)
>> +{
>> + const VhostOps *vhost_ops = hdev->vhost_ops;
>> +
>> + assert(vhost_ops);
>
>
> So we had the comment like:
>
> /* should only be called after backend is connected */
>
> in vhost_virtqueue_mask().
>
> If this assert has the same reason, let's add a comment here.
>
>
Get it.
>> +
>> + return vhost_virtqueue_start(hdev,
>> + vdev,
>> + hdev->vqs + idx,
>> + hdev->vq_index + idx);
>
>
> So it just a wrapper of vhost_virtqueue_start(), any value to have a
> re-start wrapper?
>
> Thanks
>
Because in subsequent patches, to support vhost-user, we will add
vhost_ops->vhost_set_single_vring_enable() in this function.
Thanks.
>
>> +}
>> diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
>> index 574888440c..b3394b6348 100644
>> --- a/include/hw/virtio/vhost.h
>> +++ b/include/hw/virtio/vhost.h
>> @@ -291,4 +291,6 @@ int vhost_dev_get_inflight(struct vhost_dev *dev,
>> uint16_t queue_size,
>> void vhost_dev_virtqueue_stop(struct vhost_dev *hdev,
>> VirtIODevice *vdev,
>> int idx);
>> +int vhost_dev_virtqueue_restart(struct vhost_dev *hdev, VirtIODevice
>> *vdev,
>> + int idx);
>> #endif
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 12/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
2022-08-16 1:06 ` [PATCH v2 12/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart() Kangjie Xu
@ 2022-08-24 2:44 ` Jason Wang
2022-08-24 2:53 ` Kangjie Xu
0 siblings, 1 reply; 77+ messages in thread
From: Jason Wang @ 2022-08-24 2:44 UTC (permalink / raw)
To: Kangjie Xu, qemu-devel
Cc: mst, eduardo, marcel.apfelbaum, f4bug, wangyanan55, hengqi, xuanzhuo
在 2022/8/16 09:06, Kangjie Xu 写道:
> Introduce vhost_net_virtqueue_restart(), which can restart the
> virtqueue when the vhost net started running before. If it fails
> to restart the virtqueue, the device will be stopped.
>
> This patch only considers the case for vhost-kernel, when
> NetClientDriver is NET_CLIENT_DRIVER_TAP.
>
> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
I would explain why current vhost_net_start_one()/vhost_net_stop_one()
can't work. Is it because it works at queue pair level? If yes can we
restructure the code and try to reuse ?
Thanks
> ---
> hw/net/vhost_net.c | 48 +++++++++++++++++++++++++++++++++++++++++
> include/net/vhost_net.h | 2 ++
> 2 files changed, 50 insertions(+)
>
> diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
> index aa60dd901c..2ab67e875e 100644
> --- a/hw/net/vhost_net.c
> +++ b/hw/net/vhost_net.c
> @@ -535,3 +535,51 @@ void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc,
>
> vhost_dev_virtqueue_stop(&net->dev, vdev, idx);
> }
> +
> +int vhost_net_virtqueue_restart(VirtIODevice *vdev, NetClientState *nc,
> + int vq_index)
> +{
> + VHostNetState *net = get_vhost_net(nc->peer);
> + const VhostOps *vhost_ops = net->dev.vhost_ops;
> + struct vhost_vring_file file = { };
> + int idx, r;
> +
> + if (!net->dev.started) {
> + return 0;
> + }
> +
> + assert(vhost_ops);
> +
> + idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index);
> +
> + r = vhost_dev_virtqueue_restart(&net->dev, vdev, idx);
> + if (r < 0) {
> + goto err_start;
> + }
> +
> + if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) {
> + file.index = idx;
> + file.fd = net->backend;
> + r = vhost_net_set_backend(&net->dev, &file);
> + if (r < 0) {
> + r = -errno;
> + goto err_start;
> + }
> + }
> +
> + return 0;
> +
> +err_start:
> + error_report("Error when restarting the queue.");
> +
> + if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) {
> + file.fd = -1;
> + file.index = idx;
> + int r = vhost_net_set_backend(&net->dev, &file);
> + assert(r >= 0);
> + }
> +
> + vhost_dev_stop(&net->dev, vdev);
> +
> + return r;
> +}
> diff --git a/include/net/vhost_net.h b/include/net/vhost_net.h
> index 9b3aaf3814..e11a297380 100644
> --- a/include/net/vhost_net.h
> +++ b/include/net/vhost_net.h
> @@ -50,4 +50,6 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu);
>
> void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc,
> int vq_index);
> +int vhost_net_virtqueue_restart(VirtIODevice *vdev, NetClientState *nc,
> + int vq_index);
> #endif
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 13/24] docs: vhost-user: add VHOST_USER_RESET_VRING message
2022-08-16 1:06 ` [PATCH v2 13/24] docs: vhost-user: add VHOST_USER_RESET_VRING message Kangjie Xu
@ 2022-08-24 2:46 ` Jason Wang
0 siblings, 0 replies; 77+ messages in thread
From: Jason Wang @ 2022-08-24 2:46 UTC (permalink / raw)
To: Kangjie Xu, qemu-devel
Cc: mst, eduardo, marcel.apfelbaum, f4bug, wangyanan55, hengqi,
xuanzhuo, Maxime Coquelin
在 2022/8/16 09:06, Kangjie Xu 写道:
> To support the reset operation for an individual virtqueue,
> we introduce a new message VHOST_USER_RESET_VRING. This
> message is submitted by the front-end to reset an individual
> virtqueue to initial states in the back-end. The reply is
> needed to ensure that the reset operation is complete.
>
> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
> ---
> docs/interop/vhost-user.rst | 10 ++++++++++
> 1 file changed, 10 insertions(+)
>
> diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst
> index 3f18ab424e..ce7991b9d3 100644
> --- a/docs/interop/vhost-user.rst
> +++ b/docs/interop/vhost-user.rst
> @@ -1422,6 +1422,16 @@ Front-end message types
> query the back-end for its device status as defined in the Virtio
> specification.
>
> +``VHOST_USER_RESET_VRING``
> + :id: 41
> + :equivalent ioctl: N/A
> + :request payload: vring state description
> + :reply payload: ``u64``
> +
> + When the feature ``VIRTIO_F_RING_RESET`` feature has been successfully
> + negotiated, this message is submitted by the front-end to reset an
> + individual virtqueue to initial states in the back-end. It will ask
> + for a reply to ensure the virtqueue is successfully reset in the back-end.
>
> Back-end message types
> ----------------------
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 11/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
2022-08-24 2:40 ` Jason Wang
@ 2022-08-24 2:46 ` Kangjie Xu
2022-08-24 3:33 ` Kangjie Xu
1 sibling, 0 replies; 77+ messages in thread
From: Kangjie Xu @ 2022-08-24 2:46 UTC (permalink / raw)
To: Jason Wang
Cc: Michael S. Tsirkin, qemu-devel, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
[-- Attachment #1: Type: text/plain, Size: 2273 bytes --]
在 2022/8/24 10:40, Jason Wang 写道:
>
> 在 2022/8/16 09:06, Kangjie Xu 写道:
>> Introduce vhost_virtqueue_stop(), which can reset the virtqueue
>> in the device. Then it will unmap vrings and the desc of the
>> virtqueue.
>>
>> This patch only considers the case for vhost-kernel, when
>> NetClientDriver is NET_CLIENT_DRIVER_TAP.
>>
>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>> ---
>> hw/net/vhost_net.c | 21 +++++++++++++++++++++
>> include/net/vhost_net.h | 2 ++
>> 2 files changed, 23 insertions(+)
>>
>> diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
>> index ccac5b7a64..aa60dd901c 100644
>> --- a/hw/net/vhost_net.c
>> +++ b/hw/net/vhost_net.c
>> @@ -514,3 +514,24 @@ int vhost_net_set_mtu(struct vhost_net *net,
>> uint16_t mtu)
>> return vhost_ops->vhost_net_set_mtu(&net->dev, mtu);
>> }
>> +
>> +void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc,
>> + int vq_index)
>> +{
>> + VHostNetState *net = get_vhost_net(nc->peer);
>> + const VhostOps *vhost_ops = net->dev.vhost_ops;
>> + struct vhost_vring_file file = { .fd = -1 };
>> + int idx;
>> +
>> + assert(vhost_ops);
>> +
>> + idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index);
>> +
>> + if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) {
>> + file.index = idx;
>> + int r = vhost_net_set_backend(&net->dev, &file);
>> + assert(r >= 0);
>> + }
>
>
> Let's have a vhost_ops here instead of open code it.
>
> Thanks
>
I share your view.
Thanks
>
>> +
>> + vhost_dev_virtqueue_stop(&net->dev, vdev, idx);
>> +}
>> diff --git a/include/net/vhost_net.h b/include/net/vhost_net.h
>> index 387e913e4e..9b3aaf3814 100644
>> --- a/include/net/vhost_net.h
>> +++ b/include/net/vhost_net.h
>> @@ -48,4 +48,6 @@ uint64_t vhost_net_get_acked_features(VHostNetState
>> *net);
>> int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu);
>> +void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc,
>> + int vq_index);
>> #endif
[-- Attachment #2: Type: text/html, Size: 4721 bytes --]
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 14/24] vhost-user: introduce vhost_reset_vring() interface
2022-08-16 1:06 ` [PATCH v2 14/24] vhost-user: introduce vhost_reset_vring() interface Kangjie Xu
@ 2022-08-24 2:50 ` Jason Wang
2022-08-24 3:03 ` Kangjie Xu
0 siblings, 1 reply; 77+ messages in thread
From: Jason Wang @ 2022-08-24 2:50 UTC (permalink / raw)
To: Kangjie Xu, qemu-devel
Cc: mst, eduardo, marcel.apfelbaum, f4bug, wangyanan55, hengqi, xuanzhuo
在 2022/8/16 09:06, Kangjie Xu 写道:
> Introduce the interface vhost_reset_vring(). The interface is a wrapper
> to send a VHOST_USER_RESET_VRING message to the back-end. It will reset
> an individual vring in the back-end. Meanwhile, it will wait for a reply
> to ensure the reset has been completed.
>
> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> hw/virtio/vhost-user.c | 41 +++++++++++++++++++++++++++++++
> include/hw/virtio/vhost-backend.h | 3 +++
> 2 files changed, 44 insertions(+)
>
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index 75b8df21a4..56033f7a92 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -126,6 +126,7 @@ typedef enum VhostUserRequest {
> VHOST_USER_GET_MAX_MEM_SLOTS = 36,
> VHOST_USER_ADD_MEM_REG = 37,
> VHOST_USER_REM_MEM_REG = 38,
> + VHOST_USER_RESET_VRING = 41,
> VHOST_USER_MAX
> } VhostUserRequest;
>
> @@ -1498,6 +1499,45 @@ static int vhost_user_get_max_memslots(struct vhost_dev *dev,
> return 0;
> }
>
> +static int vhost_user_reset_vring(struct vhost_dev *dev,
> + struct vhost_vring_state *ring)
> +{
> + int ret;
> + VhostUserMsg msg = {
> + .hdr.request = VHOST_USER_RESET_VRING,
> + .hdr.flags = VHOST_USER_VERSION,
Do we need VHOST_USER_NEED_REPLY_MASK here?
Other looks good.
Thanks
> + .payload.state = *ring,
> + .hdr.size = sizeof(msg.payload.state),
> + };
> +
> + if (!virtio_has_feature(dev->acked_features, VIRTIO_F_RING_RESET)) {
> + return -ENOTSUP;
> + }
> +
> + ret = vhost_user_write(dev, &msg, NULL, 0);
> + if (ret < 0) {
> + return ret;
> + }
> +
> + ret = vhost_user_read(dev, &msg);
> + if (ret < 0) {
> + return ret;
> + }
> +
> + if (msg.hdr.request != VHOST_USER_RESET_VRING) {
> + error_report("Received unexpected msg type. Expected %d received %d",
> + VHOST_USER_RESET_VRING, msg.hdr.request);
> + return -EPROTO;
> + }
> +
> + if (msg.hdr.size != sizeof(msg.payload.state)) {
> + error_report("Received bad msg size.");
> + return -EPROTO;
> + }
> +
> + return 0;
> +}
> +
> static int vhost_user_reset_device(struct vhost_dev *dev)
> {
> VhostUserMsg msg = {
> @@ -2625,6 +2665,7 @@ const VhostOps user_ops = {
> .vhost_set_features = vhost_user_set_features,
> .vhost_get_features = vhost_user_get_features,
> .vhost_set_owner = vhost_user_set_owner,
> + .vhost_reset_vring = vhost_user_reset_vring,
> .vhost_reset_device = vhost_user_reset_device,
> .vhost_get_vq_index = vhost_user_get_vq_index,
> .vhost_set_vring_enable = vhost_user_set_vring_enable,
> diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
> index eab46d7f0b..f23bf71a8d 100644
> --- a/include/hw/virtio/vhost-backend.h
> +++ b/include/hw/virtio/vhost-backend.h
> @@ -79,6 +79,8 @@ typedef int (*vhost_get_features_op)(struct vhost_dev *dev,
> uint64_t *features);
> typedef int (*vhost_set_backend_cap_op)(struct vhost_dev *dev);
> typedef int (*vhost_set_owner_op)(struct vhost_dev *dev);
> +typedef int (*vhost_reset_vring_op)(struct vhost_dev *dev,
> + struct vhost_vring_state *ring);
> typedef int (*vhost_reset_device_op)(struct vhost_dev *dev);
> typedef int (*vhost_get_vq_index_op)(struct vhost_dev *dev, int idx);
> typedef int (*vhost_set_vring_enable_op)(struct vhost_dev *dev,
> @@ -154,6 +156,7 @@ typedef struct VhostOps {
> vhost_set_backend_cap_op vhost_set_backend_cap;
> vhost_set_owner_op vhost_set_owner;
> vhost_reset_device_op vhost_reset_device;
> + vhost_reset_vring_op vhost_reset_vring;
> vhost_get_vq_index_op vhost_get_vq_index;
> vhost_set_vring_enable_op vhost_set_vring_enable;
> vhost_requires_shm_log_op vhost_requires_shm_log;
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 12/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
2022-08-24 2:44 ` Jason Wang
@ 2022-08-24 2:53 ` Kangjie Xu
2022-08-24 9:01 ` Jason Wang
0 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-24 2:53 UTC (permalink / raw)
To: Jason Wang
Cc: Michael S. Tsirkin, qemu-devel, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/24 10:44, Jason Wang 写道:
>
> 在 2022/8/16 09:06, Kangjie Xu 写道:
>> Introduce vhost_net_virtqueue_restart(), which can restart the
>> virtqueue when the vhost net started running before. If it fails
>> to restart the virtqueue, the device will be stopped.
>>
>> This patch only considers the case for vhost-kernel, when
>> NetClientDriver is NET_CLIENT_DRIVER_TAP.
>>
>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>
>
> I would explain why current vhost_net_start_one()/vhost_net_stop_one()
> can't work. Is it because it works at queue pair level? If yes can we
> restructure the code and try to reuse ?
>
> Thanks
>
Because vhost_net_start_one()/vhost_net_stop_one() works at device level.
The queue pair level start/stop are vhost_virtqueue_start() and
vhost_virtqueue_stop().
What we can reuse is the vhost_virtqueue_start(). vhost_virtqueue_stop()
cannot be reused because it will destroy device.
I think we do not need to restructure because we've already had an
abstraction vhost_virtqueue_start().
Thanks.
>
>> ---
>> hw/net/vhost_net.c | 48 +++++++++++++++++++++++++++++++++++++++++
>> include/net/vhost_net.h | 2 ++
>> 2 files changed, 50 insertions(+)
>>
>> diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
>> index aa60dd901c..2ab67e875e 100644
>> --- a/hw/net/vhost_net.c
>> +++ b/hw/net/vhost_net.c
>> @@ -535,3 +535,51 @@ void vhost_net_virtqueue_stop(VirtIODevice
>> *vdev, NetClientState *nc,
>> vhost_dev_virtqueue_stop(&net->dev, vdev, idx);
>> }
>> +
>> +int vhost_net_virtqueue_restart(VirtIODevice *vdev, NetClientState *nc,
>> + int vq_index)
>> +{
>> + VHostNetState *net = get_vhost_net(nc->peer);
>> + const VhostOps *vhost_ops = net->dev.vhost_ops;
>> + struct vhost_vring_file file = { };
>> + int idx, r;
>> +
>> + if (!net->dev.started) {
>> + return 0;
>> + }
>> +
>> + assert(vhost_ops);
>> +
>> + idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index);
>> +
>> + r = vhost_dev_virtqueue_restart(&net->dev, vdev, idx);
>> + if (r < 0) {
>> + goto err_start;
>> + }
>> +
>> + if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) {
>> + file.index = idx;
>> + file.fd = net->backend;
>> + r = vhost_net_set_backend(&net->dev, &file);
>> + if (r < 0) {
>> + r = -errno;
>> + goto err_start;
>> + }
>> + }
>> +
>> + return 0;
>> +
>> +err_start:
>> + error_report("Error when restarting the queue.");
>> +
>> + if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) {
>> + file.fd = -1;
>> + file.index = idx;
>> + int r = vhost_net_set_backend(&net->dev, &file);
>> + assert(r >= 0);
>> + }
>> +
>> + vhost_dev_stop(&net->dev, vdev);
>> +
>> + return r;
>> +}
>> diff --git a/include/net/vhost_net.h b/include/net/vhost_net.h
>> index 9b3aaf3814..e11a297380 100644
>> --- a/include/net/vhost_net.h
>> +++ b/include/net/vhost_net.h
>> @@ -50,4 +50,6 @@ int vhost_net_set_mtu(struct vhost_net *net,
>> uint16_t mtu);
>> void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState
>> *nc,
>> int vq_index);
>> +int vhost_net_virtqueue_restart(VirtIODevice *vdev, NetClientState *nc,
>> + int vq_index);
>> #endif
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 15/24] vhost-user: add op to enable or disable a single vring
2022-08-16 1:06 ` [PATCH v2 15/24] vhost-user: add op to enable or disable a single vring Kangjie Xu
@ 2022-08-24 2:53 ` Jason Wang
2022-08-24 3:09 ` Kangjie Xu
0 siblings, 1 reply; 77+ messages in thread
From: Jason Wang @ 2022-08-24 2:53 UTC (permalink / raw)
To: Kangjie Xu, qemu-devel
Cc: mst, eduardo, marcel.apfelbaum, f4bug, wangyanan55, hengqi, xuanzhuo
在 2022/8/16 09:06, Kangjie Xu 写道:
> The interface to set enable status for a single vring is lacked in
> VhostOps, since the vhost_set_vring_enable_op will manipulate all
> virtqueues in a device.
>
> Resetting a single vq will rely on this interface.
>
> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> hw/virtio/vhost-user.c | 26 +++++++++++++++++++-------
> include/hw/virtio/vhost-backend.h | 3 +++
> 2 files changed, 22 insertions(+), 7 deletions(-)
>
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index 56033f7a92..8307976cda 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -1199,6 +1199,22 @@ static int vhost_user_set_vring_base(struct vhost_dev *dev,
> return vhost_set_vring(dev, VHOST_USER_SET_VRING_BASE, ring);
> }
>
> +static int vhost_user_set_single_vring_enable(struct vhost_dev *dev,
> + int index,
> + int enable)
> +{
> + if (index < dev->vq_index || index >= dev->vq_index + dev->nvqs) {
> + return -EINVAL;
> + }
> +
> + struct vhost_vring_state state = {
> + .index = index,
> + .num = enable,
> + };
> +
> + return vhost_set_vring(dev, VHOST_USER_SET_VRING_ENABLE, &state);
> +}
> +
> static int vhost_user_set_vring_enable(struct vhost_dev *dev, int enable)
> {
> int i;
> @@ -1208,13 +1224,8 @@ static int vhost_user_set_vring_enable(struct vhost_dev *dev, int enable)
> }
>
> for (i = 0; i < dev->nvqs; ++i) {
> - int ret;
> - struct vhost_vring_state state = {
> - .index = dev->vq_index + i,
> - .num = enable,
> - };
> -
> - ret = vhost_set_vring(dev, VHOST_USER_SET_VRING_ENABLE, &state);
Then I'd squash this into previous patch or re-roder to let this patch
(vhost_user_set_single_vring_enable()) to be first.
Thanks
> + int ret = vhost_user_set_single_vring_enable(dev, dev->vq_index + i,
> + enable);
> if (ret < 0) {
> /*
> * Restoring the previous state is likely infeasible, as well as
> @@ -2668,6 +2679,7 @@ const VhostOps user_ops = {
> .vhost_reset_vring = vhost_user_reset_vring,
> .vhost_reset_device = vhost_user_reset_device,
> .vhost_get_vq_index = vhost_user_get_vq_index,
> + .vhost_set_single_vring_enable = vhost_user_set_single_vring_enable,
> .vhost_set_vring_enable = vhost_user_set_vring_enable,
> .vhost_requires_shm_log = vhost_user_requires_shm_log,
> .vhost_migration_done = vhost_user_migration_done,
> diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
> index f23bf71a8d..38f6b752ff 100644
> --- a/include/hw/virtio/vhost-backend.h
> +++ b/include/hw/virtio/vhost-backend.h
> @@ -83,6 +83,8 @@ typedef int (*vhost_reset_vring_op)(struct vhost_dev *dev,
> struct vhost_vring_state *ring);
> typedef int (*vhost_reset_device_op)(struct vhost_dev *dev);
> typedef int (*vhost_get_vq_index_op)(struct vhost_dev *dev, int idx);
> +typedef int (*vhost_set_single_vring_enable_op)(struct vhost_dev *dev,
> + int index, int enable);
> typedef int (*vhost_set_vring_enable_op)(struct vhost_dev *dev,
> int enable);
> typedef bool (*vhost_requires_shm_log_op)(struct vhost_dev *dev);
> @@ -158,6 +160,7 @@ typedef struct VhostOps {
> vhost_reset_device_op vhost_reset_device;
> vhost_reset_vring_op vhost_reset_vring;
> vhost_get_vq_index_op vhost_get_vq_index;
> + vhost_set_single_vring_enable_op vhost_set_single_vring_enable;
> vhost_set_vring_enable_op vhost_set_vring_enable;
> vhost_requires_shm_log_op vhost_requires_shm_log;
> vhost_migration_done_op vhost_migration_done;
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 14/24] vhost-user: introduce vhost_reset_vring() interface
2022-08-24 2:50 ` Jason Wang
@ 2022-08-24 3:03 ` Kangjie Xu
0 siblings, 0 replies; 77+ messages in thread
From: Kangjie Xu @ 2022-08-24 3:03 UTC (permalink / raw)
To: Jason Wang
Cc: Michael S. Tsirkin, qemu-devel, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/24 10:50, Jason Wang 写道:
>
> 在 2022/8/16 09:06, Kangjie Xu 写道:
>> Introduce the interface vhost_reset_vring(). The interface is a wrapper
>> to send a VHOST_USER_RESET_VRING message to the back-end. It will reset
>> an individual vring in the back-end. Meanwhile, it will wait for a reply
>> to ensure the reset has been completed.
>>
>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>> ---
>> hw/virtio/vhost-user.c | 41 +++++++++++++++++++++++++++++++
>> include/hw/virtio/vhost-backend.h | 3 +++
>> 2 files changed, 44 insertions(+)
>>
>> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
>> index 75b8df21a4..56033f7a92 100644
>> --- a/hw/virtio/vhost-user.c
>> +++ b/hw/virtio/vhost-user.c
>> @@ -126,6 +126,7 @@ typedef enum VhostUserRequest {
>> VHOST_USER_GET_MAX_MEM_SLOTS = 36,
>> VHOST_USER_ADD_MEM_REG = 37,
>> VHOST_USER_REM_MEM_REG = 38,
>> + VHOST_USER_RESET_VRING = 41,
>> VHOST_USER_MAX
>> } VhostUserRequest;
>> @@ -1498,6 +1499,45 @@ static int
>> vhost_user_get_max_memslots(struct vhost_dev *dev,
>> return 0;
>> }
>> +static int vhost_user_reset_vring(struct vhost_dev *dev,
>> + struct vhost_vring_state *ring)
>> +{
>> + int ret;
>> + VhostUserMsg msg = {
>> + .hdr.request = VHOST_USER_RESET_VRING,
>> + .hdr.flags = VHOST_USER_VERSION,
>
>
> Do we need VHOST_USER_NEED_REPLY_MASK here?
>
> Other looks good.
>
> Thanks
>
No, it works the same as vhost_user_get_features() or
vhost_user_get_vring_base().
Thanks
>
>> + .payload.state = *ring,
>> + .hdr.size = sizeof(msg.payload.state),
>> + };
>> +
>> + if (!virtio_has_feature(dev->acked_features,
>> VIRTIO_F_RING_RESET)) {
>> + return -ENOTSUP;
>> + }
>> +
>> + ret = vhost_user_write(dev, &msg, NULL, 0);
>> + if (ret < 0) {
>> + return ret;
>> + }
>> +
>> + ret = vhost_user_read(dev, &msg);
>> + if (ret < 0) {
>> + return ret;
>> + }
>> +
>> + if (msg.hdr.request != VHOST_USER_RESET_VRING) {
>> + error_report("Received unexpected msg type. Expected %d
>> received %d",
>> + VHOST_USER_RESET_VRING, msg.hdr.request);
>> + return -EPROTO;
>> + }
>> +
>> + if (msg.hdr.size != sizeof(msg.payload.state)) {
>> + error_report("Received bad msg size.");
>> + return -EPROTO;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> static int vhost_user_reset_device(struct vhost_dev *dev)
>> {
>> VhostUserMsg msg = {
>> @@ -2625,6 +2665,7 @@ const VhostOps user_ops = {
>> .vhost_set_features = vhost_user_set_features,
>> .vhost_get_features = vhost_user_get_features,
>> .vhost_set_owner = vhost_user_set_owner,
>> + .vhost_reset_vring = vhost_user_reset_vring,
>> .vhost_reset_device = vhost_user_reset_device,
>> .vhost_get_vq_index = vhost_user_get_vq_index,
>> .vhost_set_vring_enable = vhost_user_set_vring_enable,
>> diff --git a/include/hw/virtio/vhost-backend.h
>> b/include/hw/virtio/vhost-backend.h
>> index eab46d7f0b..f23bf71a8d 100644
>> --- a/include/hw/virtio/vhost-backend.h
>> +++ b/include/hw/virtio/vhost-backend.h
>> @@ -79,6 +79,8 @@ typedef int (*vhost_get_features_op)(struct
>> vhost_dev *dev,
>> uint64_t *features);
>> typedef int (*vhost_set_backend_cap_op)(struct vhost_dev *dev);
>> typedef int (*vhost_set_owner_op)(struct vhost_dev *dev);
>> +typedef int (*vhost_reset_vring_op)(struct vhost_dev *dev,
>> + struct vhost_vring_state *ring);
>> typedef int (*vhost_reset_device_op)(struct vhost_dev *dev);
>> typedef int (*vhost_get_vq_index_op)(struct vhost_dev *dev, int idx);
>> typedef int (*vhost_set_vring_enable_op)(struct vhost_dev *dev,
>> @@ -154,6 +156,7 @@ typedef struct VhostOps {
>> vhost_set_backend_cap_op vhost_set_backend_cap;
>> vhost_set_owner_op vhost_set_owner;
>> vhost_reset_device_op vhost_reset_device;
>> + vhost_reset_vring_op vhost_reset_vring;
>> vhost_get_vq_index_op vhost_get_vq_index;
>> vhost_set_vring_enable_op vhost_set_vring_enable;
>> vhost_requires_shm_log_op vhost_requires_shm_log;
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 15/24] vhost-user: add op to enable or disable a single vring
2022-08-24 2:53 ` Jason Wang
@ 2022-08-24 3:09 ` Kangjie Xu
2022-08-24 9:02 ` Jason Wang
0 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-24 3:09 UTC (permalink / raw)
To: Jason Wang
Cc: Michael S. Tsirkin, qemu-devel, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/24 10:53, Jason Wang 写道:
>
> 在 2022/8/16 09:06, Kangjie Xu 写道:
>> The interface to set enable status for a single vring is lacked in
>> VhostOps, since the vhost_set_vring_enable_op will manipulate all
>> virtqueues in a device.
>>
>> Resetting a single vq will rely on this interface.
>>
>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>> ---
>> hw/virtio/vhost-user.c | 26 +++++++++++++++++++-------
>> include/hw/virtio/vhost-backend.h | 3 +++
>> 2 files changed, 22 insertions(+), 7 deletions(-)
>>
>> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
>> index 56033f7a92..8307976cda 100644
>> --- a/hw/virtio/vhost-user.c
>> +++ b/hw/virtio/vhost-user.c
>> @@ -1199,6 +1199,22 @@ static int vhost_user_set_vring_base(struct
>> vhost_dev *dev,
>> return vhost_set_vring(dev, VHOST_USER_SET_VRING_BASE, ring);
>> }
>> +static int vhost_user_set_single_vring_enable(struct vhost_dev *dev,
>> + int index,
>> + int enable)
>> +{
>> + if (index < dev->vq_index || index >= dev->vq_index + dev->nvqs) {
>> + return -EINVAL;
>> + }
>> +
>> + struct vhost_vring_state state = {
>> + .index = index,
>> + .num = enable,
>> + };
>> +
>> + return vhost_set_vring(dev, VHOST_USER_SET_VRING_ENABLE, &state);
>> +}
>> +
>> static int vhost_user_set_vring_enable(struct vhost_dev *dev, int
>> enable)
>> {
>> int i;
>> @@ -1208,13 +1224,8 @@ static int vhost_user_set_vring_enable(struct
>> vhost_dev *dev, int enable)
>> }
>> for (i = 0; i < dev->nvqs; ++i) {
>> - int ret;
>> - struct vhost_vring_state state = {
>> - .index = dev->vq_index + i,
>> - .num = enable,
>> - };
>> -
>> - ret = vhost_set_vring(dev, VHOST_USER_SET_VRING_ENABLE,
>> &state);
>
>
> Then I'd squash this into previous patch or re-roder to let this patch
> (vhost_user_set_single_vring_enable()) to be first.
>
> Thanks
>
>
Sorry, I don't get why we should re-order them, since these two patches
are independent.
Thanks
>> + int ret = vhost_user_set_single_vring_enable(dev,
>> dev->vq_index + i,
>> + enable);
>> if (ret < 0) {
>> /*
>> * Restoring the previous state is likely infeasible,
>> as well as
>> @@ -2668,6 +2679,7 @@ const VhostOps user_ops = {
>> .vhost_reset_vring = vhost_user_reset_vring,
>> .vhost_reset_device = vhost_user_reset_device,
>> .vhost_get_vq_index = vhost_user_get_vq_index,
>> + .vhost_set_single_vring_enable =
>> vhost_user_set_single_vring_enable,
>> .vhost_set_vring_enable = vhost_user_set_vring_enable,
>> .vhost_requires_shm_log = vhost_user_requires_shm_log,
>> .vhost_migration_done = vhost_user_migration_done,
>> diff --git a/include/hw/virtio/vhost-backend.h
>> b/include/hw/virtio/vhost-backend.h
>> index f23bf71a8d..38f6b752ff 100644
>> --- a/include/hw/virtio/vhost-backend.h
>> +++ b/include/hw/virtio/vhost-backend.h
>> @@ -83,6 +83,8 @@ typedef int (*vhost_reset_vring_op)(struct
>> vhost_dev *dev,
>> struct vhost_vring_state *ring);
>> typedef int (*vhost_reset_device_op)(struct vhost_dev *dev);
>> typedef int (*vhost_get_vq_index_op)(struct vhost_dev *dev, int idx);
>> +typedef int (*vhost_set_single_vring_enable_op)(struct vhost_dev *dev,
>> + int index, int enable);
>> typedef int (*vhost_set_vring_enable_op)(struct vhost_dev *dev,
>> int enable);
>> typedef bool (*vhost_requires_shm_log_op)(struct vhost_dev *dev);
>> @@ -158,6 +160,7 @@ typedef struct VhostOps {
>> vhost_reset_device_op vhost_reset_device;
>> vhost_reset_vring_op vhost_reset_vring;
>> vhost_get_vq_index_op vhost_get_vq_index;
>> + vhost_set_single_vring_enable_op vhost_set_single_vring_enable;
>> vhost_set_vring_enable_op vhost_set_vring_enable;
>> vhost_requires_shm_log_op vhost_requires_shm_log;
>> vhost_migration_done_op vhost_migration_done;
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 11/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
2022-08-24 2:40 ` Jason Wang
2022-08-24 2:46 ` Kangjie Xu
@ 2022-08-24 3:33 ` Kangjie Xu
2022-08-24 9:00 ` Jason Wang
1 sibling, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-24 3:33 UTC (permalink / raw)
To: Jason Wang
Cc: Michael S. Tsirkin, qemu-devel, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/24 10:40, Jason Wang 写道:
>
> 在 2022/8/16 09:06, Kangjie Xu 写道:
>> Introduce vhost_virtqueue_stop(), which can reset the virtqueue
>> in the device. Then it will unmap vrings and the desc of the
>> virtqueue.
>>
>> This patch only considers the case for vhost-kernel, when
>> NetClientDriver is NET_CLIENT_DRIVER_TAP.
>>
>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>> ---
>> hw/net/vhost_net.c | 21 +++++++++++++++++++++
>> include/net/vhost_net.h | 2 ++
>> 2 files changed, 23 insertions(+)
>>
>> diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
>> index ccac5b7a64..aa60dd901c 100644
>> --- a/hw/net/vhost_net.c
>> +++ b/hw/net/vhost_net.c
>> @@ -514,3 +514,24 @@ int vhost_net_set_mtu(struct vhost_net *net,
>> uint16_t mtu)
>> return vhost_ops->vhost_net_set_mtu(&net->dev, mtu);
>> }
>> +
>> +void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc,
>> + int vq_index)
>> +{
>> + VHostNetState *net = get_vhost_net(nc->peer);
>> + const VhostOps *vhost_ops = net->dev.vhost_ops;
>> + struct vhost_vring_file file = { .fd = -1 };
>> + int idx;
>> +
>> + assert(vhost_ops);
>> +
>> + idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index);
>> +
>> + if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) {
>> + file.index = idx;
>> + int r = vhost_net_set_backend(&net->dev, &file);
>> + assert(r >= 0);
>> + }
>
>
> Let's have a vhost_ops here instead of open code it.
>
> Thanks
>
I double-checked it, vhost_net_set_backend is already a wrapper of
vhost_ops->vhost_net_set_backend().
It seems that, to further simplify it, we can only add idx and fd to the
parameter list of vhost_net_set_backend().
Thanks
>
>> +
>> + vhost_dev_virtqueue_stop(&net->dev, vdev, idx);
>> +}
>> diff --git a/include/net/vhost_net.h b/include/net/vhost_net.h
>> index 387e913e4e..9b3aaf3814 100644
>> --- a/include/net/vhost_net.h
>> +++ b/include/net/vhost_net.h
>> @@ -48,4 +48,6 @@ uint64_t vhost_net_get_acked_features(VHostNetState
>> *net);
>> int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu);
>> +void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc,
>> + int vq_index);
>> #endif
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 16/24] vhost: vhost-user: update vhost_dev_virtqueue_stop()
2022-08-16 1:06 ` [PATCH v2 16/24] vhost: vhost-user: update vhost_dev_virtqueue_stop() Kangjie Xu
@ 2022-08-24 3:56 ` Jason Wang
2022-08-24 4:59 ` Kangjie Xu
0 siblings, 1 reply; 77+ messages in thread
From: Jason Wang @ 2022-08-24 3:56 UTC (permalink / raw)
To: Kangjie Xu, qemu-devel
Cc: mst, eduardo, marcel.apfelbaum, f4bug, wangyanan55, hengqi, xuanzhuo
在 2022/8/16 09:06, Kangjie Xu 写道:
> Update vhost_dev_virtqueue_stop() for vhost-user scenario.
>
> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> hw/virtio/vhost.c | 19 +++++++++++++++++++
> 1 file changed, 19 insertions(+)
>
> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index fc3f550c76..a0d6824353 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -1908,10 +1908,29 @@ int vhost_net_set_backend(struct vhost_dev *hdev,
> void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice *vdev,
> int idx)
> {
> + const VhostOps *vhost_ops = hdev->vhost_ops;
> + struct vhost_vring_state state;
> + int r;
> +
> + assert(vhost_ops);
> +
> + if (vhost_ops->vhost_reset_vring) {
> + state.index = hdev->vq_index + idx;
> + r = vhost_ops->vhost_reset_vring(hdev, &state);
> + if (r < 0) {
> + goto err_queue_reset;
> + }
> + }
> +
So this worries me:
1) having two similar functions
vhost_virtqueue_stop()
and
vhost_dev_virtqueue_stop()
It can easily confuse the people who want stop the device.
I think we need rename vhost_dev_virtqueue_stop() to
vhost_virqtueue_reset() since it has different semantic compared to stop:
1) stop means the virtqueue state is reserved, e.g the index could be
synced via get_vring_base()
2) reset means the virqtueue state is lost
Thanks
> vhost_virtqueue_unmap(hdev,
> vdev,
> hdev->vqs + idx,
> idx);
> +
> + return;
> +
> +err_queue_reset:
> + error_report("Error when stopping the qeuue.");
> }
>
> int vhost_dev_virtqueue_restart(struct vhost_dev *hdev, VirtIODevice *vdev,
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 17/24] vhost: vhost-user: update vhost_dev_virtqueue_restart()
2022-08-16 1:06 ` [PATCH v2 17/24] vhost: vhost-user: update vhost_dev_virtqueue_restart() Kangjie Xu
@ 2022-08-24 4:03 ` Jason Wang
2022-08-24 5:04 ` Kangjie Xu
0 siblings, 1 reply; 77+ messages in thread
From: Jason Wang @ 2022-08-24 4:03 UTC (permalink / raw)
To: Kangjie Xu, qemu-devel
Cc: mst, eduardo, marcel.apfelbaum, f4bug, wangyanan55, hengqi, xuanzhuo
在 2022/8/16 09:06, Kangjie Xu 写道:
> Update vhost_dev_virtqueue_restart() for vhost-user scenario.
>
> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> hw/virtio/vhost.c | 26 ++++++++++++++++++++++----
> 1 file changed, 22 insertions(+), 4 deletions(-)
>
> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index a0d6824353..bd90cfe13a 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -1937,11 +1937,29 @@ int vhost_dev_virtqueue_restart(struct vhost_dev *hdev, VirtIODevice *vdev,
> int idx)
> {
> const VhostOps *vhost_ops = hdev->vhost_ops;
> + int r;
>
> assert(vhost_ops);
>
> - return vhost_virtqueue_start(hdev,
> - vdev,
> - hdev->vqs + idx,
> - hdev->vq_index + idx);
> + r = vhost_virtqueue_start(hdev,
> + vdev,
> + hdev->vqs + idx,
> + hdev->vq_index + idx);
> + if (r < 0) {
> + goto err_start;
> + }
> +
> + if (vhost_ops->vhost_set_single_vring_enable) {
> + r = vhost_ops->vhost_set_single_vring_enable(hdev,
> + hdev->vq_index + idx,
> + 1);
Rethink about the name, I think the current names is kind of confusing:
.vhost_set_single_vring_enable =
vhost_user_set_single_vring_enable,
.vhost_set_vring_enable = vhost_user_set_vring_enable,
I wonder if it's better:
1) rename vhost_set_vring_enable to vhost_set_dev_enable()
then we can
2) use vhost_set_vring_enable() for per vq enabling?
Thanks
> + if (r < 0) {
> + goto err_start;
> + }
> + }
> +
> + return 0;
> +
> +err_start:
> + return r;
> }
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 18/24] vhost-net: vhost-user: update vhost_net_virtqueue_stop()
2022-08-16 1:06 ` [PATCH v2 18/24] vhost-net: vhost-user: update vhost_net_virtqueue_stop() Kangjie Xu
@ 2022-08-24 4:05 ` Jason Wang
2022-08-24 4:57 ` Kangjie Xu
0 siblings, 1 reply; 77+ messages in thread
From: Jason Wang @ 2022-08-24 4:05 UTC (permalink / raw)
To: Kangjie Xu, qemu-devel
Cc: mst, eduardo, marcel.apfelbaum, f4bug, wangyanan55, hengqi, xuanzhuo
在 2022/8/16 09:06, Kangjie Xu 写道:
> Update vhost_net_virtqueue_stop() for vhost-user scenario.
Let's explain why it is needed now or why it doesn't cause any issue or
it's a bug fix or not.
Thanks
>
> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> hw/net/vhost_net.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
> index 2ab67e875e..c0d408f3b4 100644
> --- a/hw/net/vhost_net.c
> +++ b/hw/net/vhost_net.c
> @@ -533,6 +533,10 @@ void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc,
> assert(r >= 0);
> }
>
> + if (net->nc->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
> + idx = idx - net->dev.vq_index;
> + }
> +
> vhost_dev_virtqueue_stop(&net->dev, vdev, idx);
> }
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 19/24] vhost-net: vhost-user: update vhost_net_virtqueue_restart()
2022-08-16 1:06 ` [PATCH v2 19/24] vhost-net: vhost-user: update vhost_net_virtqueue_restart() Kangjie Xu
@ 2022-08-24 4:06 ` Jason Wang
0 siblings, 0 replies; 77+ messages in thread
From: Jason Wang @ 2022-08-24 4:06 UTC (permalink / raw)
To: Kangjie Xu, qemu-devel
Cc: mst, eduardo, marcel.apfelbaum, f4bug, wangyanan55, hengqi, xuanzhuo
在 2022/8/16 09:06, Kangjie Xu 写道:
> Update vhost_net_virtqueue_restart() for vhost-user scenario.
Similar to previous patch, let's explain why it it needed.
Thanks
>
> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> hw/net/vhost_net.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
> index c0d408f3b4..778081e54a 100644
> --- a/hw/net/vhost_net.c
> +++ b/hw/net/vhost_net.c
> @@ -556,6 +556,10 @@ int vhost_net_virtqueue_restart(VirtIODevice *vdev, NetClientState *nc,
>
> idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index);
>
> + if (net->nc->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
> + idx = idx - net->dev.vq_index;
> + }
> +
> r = vhost_dev_virtqueue_restart(&net->dev, vdev, idx);
> if (r < 0) {
> goto err_start;
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (25 preceding siblings ...)
2022-08-23 1:49 ` Kangjie Xu
@ 2022-08-24 4:10 ` Jason Wang
2022-08-24 4:11 ` Jason Wang
27 siblings, 0 replies; 77+ messages in thread
From: Jason Wang @ 2022-08-24 4:10 UTC (permalink / raw)
To: Kangjie Xu, qemu-devel
Cc: mst, eduardo, marcel.apfelbaum, f4bug, wangyanan55, hengqi, xuanzhuo
在 2022/8/16 09:06, Kangjie Xu 写道:
> The virtio queue reset function has already been defined in the virtio spec 1.2.
> The relevant virtio spec information is here:
>
> https://github.com/oasis-tcs/virtio-spec/issues/124
> https://github.com/oasis-tcs/virtio-spec/issues/139
>
> This patch set is to support this function in QEMU. It consists of several parts:
> 1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
> 2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
> 3. Patches 13-19 support vq stop and vq restart for vhost-user.
> 4. Patches 20-22 support vq reset and re-enable for virtio-net.
> 5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.
>
> The process of virtqueue reset can be concluded as:
> 1. The virtqueue is disabled when VIRTIO_PCI_COMMON_Q_RESET is written.
> 2. Then the virtqueue can be optionally restarted(re-enabled).
>
> Since this patch set involves multiple modules and seems a bit messy, we briefly describe the
> calling process for different modes below.
> virtio-net:
> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> -> virtio_queue_reset() [virtio]
> -> virtio_net_queue_reset() [virtio-net]
> -> __virtio_queue_reset()
> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> -> set enabled, reset status of vq.
>
> vhost-kernel:
> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> -> virtio_queue_reset() [virtio]
> -> virtio_net_queue_reset() [virtio-net]
> -> vhost_net_virtqueue_stop() [vhost-net]
> -> vhost_net_set_backend() [vhost]
> -> vhost_dev_virtqueue_stop()
> -> vhost_virtqueue_unmap()
> -> __virtio_queue_reset()
> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> -> virtio_queue_enable() [virtio]
> -> virtio_net_queue_enable() [virtio-net]
> -> vhost_net_virtqueue_restart() [vhost-net]
> -> vhost_dev_virtqueue_restart() [vhost]
> -> vhost_virtqueue_start()
> -> vhost_net_set_backend()
> -> set enabled, reset status of vq.
>
> vhost-user:
> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> -> virtio_queue_reset() [virtio]
> -> virtio_net_queue_reset() [virtio-net]
> -> vhost_net_virtqueue_stop() [vhost-net]
> -> vhost_dev_virtqueue_stop() [vhost]
> -> vhost_user_reset_vring() [vhost-user]
> -> send VHOST_USER_RESET_VRING to the device
> -> vhost_virtqueue_unmap()
> -> __virtio_queue_reset()
> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> -> virtio_queue_enable() [virtio]
> -> virtio_net_queue_enable() [virtio-net]
> -> vhost_net_virtqueue_restart() [vhost-net]
> -> vhost_dev_virtqueue_restart() [vhost]
> -> vhost_virtqueue_start()
> -> vhost_user_set_single_vring_enable [vhost-user]
> -> send VHOST_USER_SET_VRING_ENABLE to the device
> -> set enabled, reset status of vq.
>
>
> Test environment:
> Host: 5.19.0-rc3 (With vq reset support)
> Qemu: QEMU emulator version 7.0.50
> Guest: 5.19.0-rc3 (With vq reset support)
> DPDK: 22.07-rc1 (With vq reset support)
> Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;
>
> The drvier can resize the virtio queue, then virtio queue reset function should
> be triggered.
>
> The default is split mode, modify Qemu virtio-net to add PACKED feature to
> test packed mode.
>
> Guest Kernel Patch:
> https://lore.kernel.org/bpf/20220801063902.129329-1-xuanzhuo@linux.alibaba.com/
>
> DPDK Patch:
> https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7
>
> Host Kernel Patch:
> https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3
>
> Looking forward to your review and comments. Thanks.
I suggest to split vhost-user part into another series since:
1) vhost-net kernel is ready for this future
2) vhost-user probably requires some extensions of the current protocol
Then it can speed up the merging and reduce the rebasing of a huge series.
Thanks
>
> Kangjie Xu (19):
> virtio: introduce virtio_queue_enable()
> virtio: core: vq reset feature negotation support
> virtio-pci: support queue enable
> vhost: extract the logic of unmapping the vrings and desc
> vhost: introduce vhost_dev_virtqueue_stop()
> vhost: introduce vhost_dev_virtqueue_restart()
> vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
> vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
> docs: vhost-user: add VHOST_USER_RESET_VRING message
> vhost-user: introduce vhost_reset_vring() interface
> vhost-user: add op to enable or disable a single vring
> vhost: vhost-user: update vhost_dev_virtqueue_stop()
> vhost: vhost-user: update vhost_dev_virtqueue_restart()
> vhost-net: vhost-user: update vhost_net_virtqueue_stop()
> vhost-net: vhost-user: update vhost_net_virtqueue_restart()
> virtio-net: introduce flush_or_purge_queued_packets()
> virtio-net: support queue_enable
> vhost: vhost-kernel: enable vq reset feature
> vhost: vhost-user: enable vq reset feature
>
> Xuan Zhuo (5):
> virtio: sync relevant definitions with linux
> virtio: introduce __virtio_queue_reset()
> virtio: introduce virtio_queue_reset()
> virtio-pci: support queue reset
> virtio-net: support queue reset
>
> docs/interop/vhost-user.rst | 10 +++
> hw/core/machine.c | 1 +
> hw/net/vhost_net.c | 79 +++++++++++++++++++
> hw/net/virtio-net.c | 58 ++++++++++++--
> hw/virtio/vhost-user.c | 67 ++++++++++++++--
> hw/virtio/vhost.c | 79 +++++++++++++++++--
> hw/virtio/virtio-pci.c | 20 +++++
> hw/virtio/virtio.c | 62 +++++++++++----
> include/hw/virtio/vhost-backend.h | 6 ++
> include/hw/virtio/vhost.h | 5 ++
> include/hw/virtio/virtio-pci.h | 1 +
> include/hw/virtio/virtio.h | 8 +-
> include/net/vhost_net.h | 4 +
> .../standard-headers/linux/virtio_config.h | 5 ++
> include/standard-headers/linux/virtio_pci.h | 2 +
> 15 files changed, 371 insertions(+), 36 deletions(-)
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
` (26 preceding siblings ...)
2022-08-24 4:10 ` Jason Wang
@ 2022-08-24 4:11 ` Jason Wang
27 siblings, 0 replies; 77+ messages in thread
From: Jason Wang @ 2022-08-24 4:11 UTC (permalink / raw)
To: Kangjie Xu, qemu-devel
Cc: mst, eduardo, marcel.apfelbaum, f4bug, wangyanan55, hengqi, xuanzhuo
在 2022/8/16 09:06, Kangjie Xu 写道:
> Host Kernel Patch:
> https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3
Btw, this patch could be posted right now. Then it would be more easier
for people to try you series.
Thanks
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 18/24] vhost-net: vhost-user: update vhost_net_virtqueue_stop()
2022-08-24 4:05 ` Jason Wang
@ 2022-08-24 4:57 ` Kangjie Xu
2022-08-24 9:04 ` Jason Wang
0 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-24 4:57 UTC (permalink / raw)
To: Jason Wang
Cc: Michael S. Tsirkin, qemu-devel, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/24 12:05, Jason Wang 写道:
>
> 在 2022/8/16 09:06, Kangjie Xu 写道:
>> Update vhost_net_virtqueue_stop() for vhost-user scenario.
>
>
> Let's explain why it is needed now or why it doesn't cause any issue
> or it's a bug fix or not.
>
> Thanks
>
This patch is to suppport vq reset for vhost-user.
We need this simply because the behavior of vhost_ops->get_vq_index() is
different in vhost-user and vhost-kernel.
vhost_user_get_vq_index(dev, idx) simply returns "idx".
vhost_kernel_get_vq_index(dev, idx) returns "idx - dev->vq_index".
Thanks
>
>>
>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>> ---
>> hw/net/vhost_net.c | 4 ++++
>> 1 file changed, 4 insertions(+)
>>
>> diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
>> index 2ab67e875e..c0d408f3b4 100644
>> --- a/hw/net/vhost_net.c
>> +++ b/hw/net/vhost_net.c
>> @@ -533,6 +533,10 @@ void vhost_net_virtqueue_stop(VirtIODevice
>> *vdev, NetClientState *nc,
>> assert(r >= 0);
>> }
>> + if (net->nc->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
>> + idx = idx - net->dev.vq_index;
>> + }
>> +
>> vhost_dev_virtqueue_stop(&net->dev, vdev, idx);
>> }
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 16/24] vhost: vhost-user: update vhost_dev_virtqueue_stop()
2022-08-24 3:56 ` Jason Wang
@ 2022-08-24 4:59 ` Kangjie Xu
0 siblings, 0 replies; 77+ messages in thread
From: Kangjie Xu @ 2022-08-24 4:59 UTC (permalink / raw)
To: Jason Wang
Cc: Michael S. Tsirkin, qemu-devel, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
[-- Attachment #1: Type: text/plain, Size: 2180 bytes --]
在 2022/8/24 11:56, Jason Wang 写道:
>
> 在 2022/8/16 09:06, Kangjie Xu 写道:
>> Update vhost_dev_virtqueue_stop() for vhost-user scenario.
>>
>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>> ---
>> hw/virtio/vhost.c | 19 +++++++++++++++++++
>> 1 file changed, 19 insertions(+)
>>
>> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
>> index fc3f550c76..a0d6824353 100644
>> --- a/hw/virtio/vhost.c
>> +++ b/hw/virtio/vhost.c
>> @@ -1908,10 +1908,29 @@ int vhost_net_set_backend(struct vhost_dev
>> *hdev,
>> void vhost_dev_virtqueue_stop(struct vhost_dev *hdev, VirtIODevice
>> *vdev,
>> int idx)
>> {
>> + const VhostOps *vhost_ops = hdev->vhost_ops;
>> + struct vhost_vring_state state;
>> + int r;
>> +
>> + assert(vhost_ops);
>> +
>> + if (vhost_ops->vhost_reset_vring) {
>> + state.index = hdev->vq_index + idx;
>> + r = vhost_ops->vhost_reset_vring(hdev, &state);
>> + if (r < 0) {
>> + goto err_queue_reset;
>> + }
>> + }
>> +
>
>
> So this worries me:
>
> 1) having two similar functions
>
> vhost_virtqueue_stop()
>
> and
>
> vhost_dev_virtqueue_stop()
>
> It can easily confuse the people who want stop the device.
>
> I think we need rename vhost_dev_virtqueue_stop() to
> vhost_virqtueue_reset() since it has different semantic compared to stop:
>
> 1) stop means the virtqueue state is reserved, e.g the index could be
> synced via get_vring_base()
> 2) reset means the virqtueue state is lost
>
> Thanks
>
>
Totally agree, will fix.
Thanks
>> vhost_virtqueue_unmap(hdev,
>> vdev,
>> hdev->vqs + idx,
>> idx);
>> +
>> + return;
>> +
>> +err_queue_reset:
>> + error_report("Error when stopping the qeuue.");
>> }
>> int vhost_dev_virtqueue_restart(struct vhost_dev *hdev,
>> VirtIODevice *vdev,
[-- Attachment #2: Type: text/html, Size: 5190 bytes --]
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 17/24] vhost: vhost-user: update vhost_dev_virtqueue_restart()
2022-08-24 4:03 ` Jason Wang
@ 2022-08-24 5:04 ` Kangjie Xu
0 siblings, 0 replies; 77+ messages in thread
From: Kangjie Xu @ 2022-08-24 5:04 UTC (permalink / raw)
To: Jason Wang
Cc: Michael S. Tsirkin, qemu-devel, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/24 12:03, Jason Wang 写道:
>
> 在 2022/8/16 09:06, Kangjie Xu 写道:
>> Update vhost_dev_virtqueue_restart() for vhost-user scenario.
>>
>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>> ---
>> hw/virtio/vhost.c | 26 ++++++++++++++++++++++----
>> 1 file changed, 22 insertions(+), 4 deletions(-)
>>
>> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
>> index a0d6824353..bd90cfe13a 100644
>> --- a/hw/virtio/vhost.c
>> +++ b/hw/virtio/vhost.c
>> @@ -1937,11 +1937,29 @@ int vhost_dev_virtqueue_restart(struct
>> vhost_dev *hdev, VirtIODevice *vdev,
>> int idx)
>> {
>> const VhostOps *vhost_ops = hdev->vhost_ops;
>> + int r;
>> assert(vhost_ops);
>> - return vhost_virtqueue_start(hdev,
>> - vdev,
>> - hdev->vqs + idx,
>> - hdev->vq_index + idx);
>> + r = vhost_virtqueue_start(hdev,
>> + vdev,
>> + hdev->vqs + idx,
>> + hdev->vq_index + idx);
>> + if (r < 0) {
>> + goto err_start;
>> + }
>> +
>> + if (vhost_ops->vhost_set_single_vring_enable) {
>> + r = vhost_ops->vhost_set_single_vring_enable(hdev,
>> + hdev->vq_index + idx,
>> + 1);
>
>
> Rethink about the name, I think the current names is kind of confusing:
>
> .vhost_set_single_vring_enable =
> vhost_user_set_single_vring_enable,
> .vhost_set_vring_enable = vhost_user_set_vring_enable,
>
> I wonder if it's better:
>
> 1) rename vhost_set_vring_enable to vhost_set_dev_enable()
>
> then we can
>
> 2) use vhost_set_vring_enable() for per vq enabling?
>
> Thanks
>
>
I agree. It looks better, I will refactor this part.
Thanks.
>> + if (r < 0) {
>> + goto err_start;
>> + }
>> + }
>> +
>> + return 0;
>> +
>> +err_start:
>> + return r;
>> }
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 06/24] virtio-pci: support queue reset
2022-08-23 7:52 ` Kangjie Xu
@ 2022-08-24 8:56 ` Jason Wang
0 siblings, 0 replies; 77+ messages in thread
From: Jason Wang @ 2022-08-24 8:56 UTC (permalink / raw)
To: Kangjie Xu
Cc: Michael S. Tsirkin, qemu-devel, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/23 15:52, Kangjie Xu 写道:
>
> 在 2022/8/23 15:40, Jason Wang 写道:
>>
>> 在 2022/8/16 09:06, Kangjie Xu 写道:
>>> From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>>
>>> PCI devices support vq reset.
>>>
>>> Based on this function, the driver can adjust the size of the ring, and
>>> quickly recycle the buffer in the ring.
>>>
>>> The migration of the virtio devices will not happen during a reset
>>> operation. This is becuase the global iothread lock is held. Migration
>>> thread also needs the lock. As a result, we do not need to migrate the
>>> reset state of VirtIOPCIQueue.
>>>
>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>>> ---
>>> hw/virtio/virtio-pci.c | 19 +++++++++++++++++++
>>> include/hw/virtio/virtio-pci.h | 1 +
>>> 2 files changed, 20 insertions(+)
>>>
>>> diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
>>> index 45327f0b31..ec8e92052f 100644
>>> --- a/hw/virtio/virtio-pci.c
>>> +++ b/hw/virtio/virtio-pci.c
>>> @@ -1246,6 +1246,9 @@ static uint64_t virtio_pci_common_read(void
>>> *opaque, hwaddr addr,
>>> case VIRTIO_PCI_COMMON_Q_USEDHI:
>>> val = proxy->vqs[vdev->queue_sel].used[1];
>>> break;
>>> + case VIRTIO_PCI_COMMON_Q_RESET:
>>> + val = proxy->vqs[vdev->queue_sel].reset;
>>> + break;
>>> default:
>>> val = 0;
>>> }
>>> @@ -1333,6 +1336,7 @@ static void virtio_pci_common_write(void
>>> *opaque, hwaddr addr,
>>> ((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << 32 |
>>> proxy->vqs[vdev->queue_sel].used[0]);
>>> proxy->vqs[vdev->queue_sel].enabled = 1;
>>> + proxy->vqs[vdev->queue_sel].reset = 0;
>>> } else {
>>> virtio_error(vdev, "wrong value for queue_enable
>>> %"PRIx64, val);
>>> }
>>> @@ -1355,6 +1359,20 @@ static void virtio_pci_common_write(void
>>> *opaque, hwaddr addr,
>>> case VIRTIO_PCI_COMMON_Q_USEDHI:
>>> proxy->vqs[vdev->queue_sel].used[1] = val;
>>> break;
>>> + case VIRTIO_PCI_COMMON_Q_RESET:
>>> + if (val == 1) {
>>> + /*
>>> + * With the global iothread lock taken, the migration
>>> will not
>>> + * happen until the virtqueue reset is done.
>>> + */
>>
>>
>> This comment applies to all other common cfg operation as well, So it
>> looks not necessary?
>>
> Get it.
>
>>
>>> + proxy->vqs[vdev->queue_sel].reset = 1;
>>> +
>>> + virtio_queue_reset(vdev, vdev->queue_sel);
>>> +
>>> + proxy->vqs[vdev->queue_sel].reset = 0;
>>> + proxy->vqs[vdev->queue_sel].enabled = 0;
>>> + }
>>> + break;
>>> default:
>>> break;
>>> }
>>> @@ -1950,6 +1968,7 @@ static void virtio_pci_reset(DeviceState *qdev)
>>> for (i = 0; i < VIRTIO_QUEUE_MAX; i++) {
>>> proxy->vqs[i].enabled = 0;
>>> + proxy->vqs[i].reset = 0;
>>> proxy->vqs[i].num = 0;
>>> proxy->vqs[i].desc[0] = proxy->vqs[i].desc[1] = 0;
>>> proxy->vqs[i].avail[0] = proxy->vqs[i].avail[1] = 0;
>>> diff --git a/include/hw/virtio/virtio-pci.h
>>> b/include/hw/virtio/virtio-pci.h
>>> index 2446dcd9ae..e9290e2b94 100644
>>> --- a/include/hw/virtio/virtio-pci.h
>>> +++ b/include/hw/virtio/virtio-pci.h
>>> @@ -117,6 +117,7 @@ typedef struct VirtIOPCIRegion {
>>> typedef struct VirtIOPCIQueue {
>>> uint16_t num;
>>> bool enabled;
>>> + bool reset;
>>
>>
>> Do we need to migrate this?
>>
>> Thanks
>>
> I think we do not need to migrate this because we hold the global
> iothread lock when virtqueue reset is triggered. The migration of
> these device states also needs this lock.
>
> On the other hand, the 'reset' state of virtqueue is same(is 0) before
> and after the process of resetting a virtqueue.
>
> Thus, the migration will not happen when we are resetting a virtqueue
> and we do not to migrate it.
Ok, let's add a comment above reset to explain this.
Thanks
>
> Thanks
>
>>
>>> uint32_t desc[2];
>>> uint32_t avail[2];
>>> uint32_t used[2];
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 07/24] virtio-pci: support queue enable
2022-08-23 8:20 ` Kangjie Xu
@ 2022-08-24 8:59 ` Jason Wang
2022-08-24 11:27 ` Kangjie Xu
0 siblings, 1 reply; 77+ messages in thread
From: Jason Wang @ 2022-08-24 8:59 UTC (permalink / raw)
To: Kangjie Xu
Cc: Michael S. Tsirkin, qemu-devel, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/23 16:20, Kangjie Xu 写道:
>
> 在 2022/8/23 15:44, Jason Wang 写道:
>>
>> 在 2022/8/16 09:06, Kangjie Xu 写道:
>>> PCI devices support vq enable.
>>
>>
>> Nit: it might be "support device specific vq enable"
>>
>>
> Get it.
>>>
>>> Based on this function, the driver can re-enable the virtqueue after
>>> the
>>> virtqueue is reset.
>>>
>>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>> ---
>>> hw/virtio/virtio-pci.c | 1 +
>>> 1 file changed, 1 insertion(+)
>>>
>>> diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
>>> index ec8e92052f..3d560e45ad 100644
>>> --- a/hw/virtio/virtio-pci.c
>>> +++ b/hw/virtio/virtio-pci.c
>>> @@ -1335,6 +1335,7 @@ static void virtio_pci_common_write(void
>>> *opaque, hwaddr addr,
>>> proxy->vqs[vdev->queue_sel].avail[0],
>>> ((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << 32 |
>>> proxy->vqs[vdev->queue_sel].used[0]);
>>> + virtio_queue_enable(vdev, vdev->queue_sel);
>>> proxy->vqs[vdev->queue_sel].enabled = 1;
>>> proxy->vqs[vdev->queue_sel].reset = 0;
>>
>>
>> Any reason we do it before the assignment of 1? It probably means the
>> device specific method can't depend on virtio_queue_enabled()?
>>
>> Thanks
>>
> Sorry, I don't get why device specific method can't depend on
> virtio_queue_enabled().
I meant if the device specific method call virtio_queue_enabled() it
will return false in this case, is this intended?
>
> Before virtio_queue_enable() is done, virtqueue should always be not
> ready and disabled.
>
> Otherwise, If we put it after the assignment of enabled to 1, the
> virtqueue may be accessed illegally and may cause panic, because the
> virtqueue is still being intialized and being configured.
How? Shouldn't we make transport ready before making device
virtqueue(device) ready?
Thanks
>
> Thanks
>
>>
>>> } else {
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 11/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
2022-08-24 3:33 ` Kangjie Xu
@ 2022-08-24 9:00 ` Jason Wang
0 siblings, 0 replies; 77+ messages in thread
From: Jason Wang @ 2022-08-24 9:00 UTC (permalink / raw)
To: Kangjie Xu
Cc: Michael S. Tsirkin, qemu-devel, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/24 11:33, Kangjie Xu 写道:
>
> 在 2022/8/24 10:40, Jason Wang 写道:
>>
>> 在 2022/8/16 09:06, Kangjie Xu 写道:
>>> Introduce vhost_virtqueue_stop(), which can reset the virtqueue
>>> in the device. Then it will unmap vrings and the desc of the
>>> virtqueue.
>>>
>>> This patch only considers the case for vhost-kernel, when
>>> NetClientDriver is NET_CLIENT_DRIVER_TAP.
>>>
>>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>> ---
>>> hw/net/vhost_net.c | 21 +++++++++++++++++++++
>>> include/net/vhost_net.h | 2 ++
>>> 2 files changed, 23 insertions(+)
>>>
>>> diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
>>> index ccac5b7a64..aa60dd901c 100644
>>> --- a/hw/net/vhost_net.c
>>> +++ b/hw/net/vhost_net.c
>>> @@ -514,3 +514,24 @@ int vhost_net_set_mtu(struct vhost_net *net,
>>> uint16_t mtu)
>>> return vhost_ops->vhost_net_set_mtu(&net->dev, mtu);
>>> }
>>> +
>>> +void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState *nc,
>>> + int vq_index)
>>> +{
>>> + VHostNetState *net = get_vhost_net(nc->peer);
>>> + const VhostOps *vhost_ops = net->dev.vhost_ops;
>>> + struct vhost_vring_file file = { .fd = -1 };
>>> + int idx;
>>> +
>>> + assert(vhost_ops);
>>> +
>>> + idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index);
>>> +
>>> + if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) {
>>> + file.index = idx;
>>> + int r = vhost_net_set_backend(&net->dev, &file);
>>> + assert(r >= 0);
>>> + }
>>
>>
>> Let's have a vhost_ops here instead of open code it.
>>
>> Thanks
>>
> I double-checked it, vhost_net_set_backend is already a wrapper of
> vhost_ops->vhost_net_set_backend().
>
> It seems that, to further simplify it, we can only add idx and fd to
> the parameter list of vhost_net_set_backend().
Ok, so we can leave it as is. (Probably need a vhost_net_ops in the future).
Thanks
>
> Thanks
>
>>
>>> +
>>> + vhost_dev_virtqueue_stop(&net->dev, vdev, idx);
>>> +}
>>> diff --git a/include/net/vhost_net.h b/include/net/vhost_net.h
>>> index 387e913e4e..9b3aaf3814 100644
>>> --- a/include/net/vhost_net.h
>>> +++ b/include/net/vhost_net.h
>>> @@ -48,4 +48,6 @@ uint64_t
>>> vhost_net_get_acked_features(VHostNetState *net);
>>> int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu);
>>> +void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState
>>> *nc,
>>> + int vq_index);
>>> #endif
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 12/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
2022-08-24 2:53 ` Kangjie Xu
@ 2022-08-24 9:01 ` Jason Wang
2022-08-24 9:03 ` Kangjie Xu
0 siblings, 1 reply; 77+ messages in thread
From: Jason Wang @ 2022-08-24 9:01 UTC (permalink / raw)
To: Kangjie Xu
Cc: Michael S. Tsirkin, qemu-devel, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/24 10:53, Kangjie Xu 写道:
>
> 在 2022/8/24 10:44, Jason Wang 写道:
>>
>> 在 2022/8/16 09:06, Kangjie Xu 写道:
>>> Introduce vhost_net_virtqueue_restart(), which can restart the
>>> virtqueue when the vhost net started running before. If it fails
>>> to restart the virtqueue, the device will be stopped.
>>>
>>> This patch only considers the case for vhost-kernel, when
>>> NetClientDriver is NET_CLIENT_DRIVER_TAP.
>>>
>>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>
>>
>> I would explain why current
>> vhost_net_start_one()/vhost_net_stop_one() can't work. Is it because
>> it works at queue pair level? If yes can we restructure the code and
>> try to reuse ?
>>
>> Thanks
>>
> Because vhost_net_start_one()/vhost_net_stop_one() works at device level.
>
> The queue pair level start/stop are vhost_virtqueue_start() and
> vhost_virtqueue_stop().
>
> What we can reuse is the vhost_virtqueue_start().
> vhost_virtqueue_stop() cannot be reused because it will destroy device.
Let's add this in the changelog or a comment in the code.
Thanks
>
> I think we do not need to restructure because we've already had an
> abstraction vhost_virtqueue_start().
>
> Thanks.
>
>>
>>> ---
>>> hw/net/vhost_net.c | 48
>>> +++++++++++++++++++++++++++++++++++++++++
>>> include/net/vhost_net.h | 2 ++
>>> 2 files changed, 50 insertions(+)
>>>
>>> diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
>>> index aa60dd901c..2ab67e875e 100644
>>> --- a/hw/net/vhost_net.c
>>> +++ b/hw/net/vhost_net.c
>>> @@ -535,3 +535,51 @@ void vhost_net_virtqueue_stop(VirtIODevice
>>> *vdev, NetClientState *nc,
>>> vhost_dev_virtqueue_stop(&net->dev, vdev, idx);
>>> }
>>> +
>>> +int vhost_net_virtqueue_restart(VirtIODevice *vdev, NetClientState
>>> *nc,
>>> + int vq_index)
>>> +{
>>> + VHostNetState *net = get_vhost_net(nc->peer);
>>> + const VhostOps *vhost_ops = net->dev.vhost_ops;
>>> + struct vhost_vring_file file = { };
>>> + int idx, r;
>>> +
>>> + if (!net->dev.started) {
>>> + return 0;
>>> + }
>>> +
>>> + assert(vhost_ops);
>>> +
>>> + idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index);
>>> +
>>> + r = vhost_dev_virtqueue_restart(&net->dev, vdev, idx);
>>> + if (r < 0) {
>>> + goto err_start;
>>> + }
>>> +
>>> + if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) {
>>> + file.index = idx;
>>> + file.fd = net->backend;
>>> + r = vhost_net_set_backend(&net->dev, &file);
>>> + if (r < 0) {
>>> + r = -errno;
>>> + goto err_start;
>>> + }
>>> + }
>>> +
>>> + return 0;
>>> +
>>> +err_start:
>>> + error_report("Error when restarting the queue.");
>>> +
>>> + if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) {
>>> + file.fd = -1;
>>> + file.index = idx;
>>> + int r = vhost_net_set_backend(&net->dev, &file);
>>> + assert(r >= 0);
>>> + }
>>> +
>>> + vhost_dev_stop(&net->dev, vdev);
>>> +
>>> + return r;
>>> +}
>>> diff --git a/include/net/vhost_net.h b/include/net/vhost_net.h
>>> index 9b3aaf3814..e11a297380 100644
>>> --- a/include/net/vhost_net.h
>>> +++ b/include/net/vhost_net.h
>>> @@ -50,4 +50,6 @@ int vhost_net_set_mtu(struct vhost_net *net,
>>> uint16_t mtu);
>>> void vhost_net_virtqueue_stop(VirtIODevice *vdev, NetClientState
>>> *nc,
>>> int vq_index);
>>> +int vhost_net_virtqueue_restart(VirtIODevice *vdev, NetClientState
>>> *nc,
>>> + int vq_index);
>>> #endif
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 15/24] vhost-user: add op to enable or disable a single vring
2022-08-24 3:09 ` Kangjie Xu
@ 2022-08-24 9:02 ` Jason Wang
2022-08-24 9:09 ` Kangjie Xu
0 siblings, 1 reply; 77+ messages in thread
From: Jason Wang @ 2022-08-24 9:02 UTC (permalink / raw)
To: Kangjie Xu
Cc: Michael S. Tsirkin, qemu-devel, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/24 11:09, Kangjie Xu 写道:
>
> 在 2022/8/24 10:53, Jason Wang 写道:
>>
>> 在 2022/8/16 09:06, Kangjie Xu 写道:
>>> The interface to set enable status for a single vring is lacked in
>>> VhostOps, since the vhost_set_vring_enable_op will manipulate all
>>> virtqueues in a device.
>>>
>>> Resetting a single vq will rely on this interface.
>>>
>>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>> ---
>>> hw/virtio/vhost-user.c | 26 +++++++++++++++++++-------
>>> include/hw/virtio/vhost-backend.h | 3 +++
>>> 2 files changed, 22 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
>>> index 56033f7a92..8307976cda 100644
>>> --- a/hw/virtio/vhost-user.c
>>> +++ b/hw/virtio/vhost-user.c
>>> @@ -1199,6 +1199,22 @@ static int vhost_user_set_vring_base(struct
>>> vhost_dev *dev,
>>> return vhost_set_vring(dev, VHOST_USER_SET_VRING_BASE, ring);
>>> }
>>> +static int vhost_user_set_single_vring_enable(struct vhost_dev *dev,
>>> + int index,
>>> + int enable)
>>> +{
>>> + if (index < dev->vq_index || index >= dev->vq_index + dev->nvqs) {
>>> + return -EINVAL;
>>> + }
>>> +
>>> + struct vhost_vring_state state = {
>>> + .index = index,
>>> + .num = enable,
>>> + };
>>> +
>>> + return vhost_set_vring(dev, VHOST_USER_SET_VRING_ENABLE, &state);
>>> +}
>>> +
>>> static int vhost_user_set_vring_enable(struct vhost_dev *dev, int
>>> enable)
>>> {
>>> int i;
>>> @@ -1208,13 +1224,8 @@ static int vhost_user_set_vring_enable(struct
>>> vhost_dev *dev, int enable)
>>> }
>>> for (i = 0; i < dev->nvqs; ++i) {
>>> - int ret;
>>> - struct vhost_vring_state state = {
>>> - .index = dev->vq_index + i,
>>> - .num = enable,
>>> - };
>>> -
>>> - ret = vhost_set_vring(dev, VHOST_USER_SET_VRING_ENABLE,
>>> &state);
>>
>>
>> Then I'd squash this into previous patch or re-roder to let this
>> patch (vhost_user_set_single_vring_enable()) to be first.
>>
>> Thanks
>>
>>
> Sorry, I don't get why we should re-order them, since these two
> patches are independent.
I meant it's not good to introduce some codes in patch 14 but delete
them in patch 15 (the above part for example).
Thanks
>
> Thanks
>
>>> + int ret = vhost_user_set_single_vring_enable(dev,
>>> dev->vq_index + i,
>>> + enable);
>>> if (ret < 0) {
>>> /*
>>> * Restoring the previous state is likely infeasible,
>>> as well as
>>> @@ -2668,6 +2679,7 @@ const VhostOps user_ops = {
>>> .vhost_reset_vring = vhost_user_reset_vring,
>>> .vhost_reset_device = vhost_user_reset_device,
>>> .vhost_get_vq_index = vhost_user_get_vq_index,
>>> + .vhost_set_single_vring_enable =
>>> vhost_user_set_single_vring_enable,
>>> .vhost_set_vring_enable = vhost_user_set_vring_enable,
>>> .vhost_requires_shm_log = vhost_user_requires_shm_log,
>>> .vhost_migration_done = vhost_user_migration_done,
>>> diff --git a/include/hw/virtio/vhost-backend.h
>>> b/include/hw/virtio/vhost-backend.h
>>> index f23bf71a8d..38f6b752ff 100644
>>> --- a/include/hw/virtio/vhost-backend.h
>>> +++ b/include/hw/virtio/vhost-backend.h
>>> @@ -83,6 +83,8 @@ typedef int (*vhost_reset_vring_op)(struct
>>> vhost_dev *dev,
>>> struct vhost_vring_state *ring);
>>> typedef int (*vhost_reset_device_op)(struct vhost_dev *dev);
>>> typedef int (*vhost_get_vq_index_op)(struct vhost_dev *dev, int idx);
>>> +typedef int (*vhost_set_single_vring_enable_op)(struct vhost_dev *dev,
>>> + int index, int
>>> enable);
>>> typedef int (*vhost_set_vring_enable_op)(struct vhost_dev *dev,
>>> int enable);
>>> typedef bool (*vhost_requires_shm_log_op)(struct vhost_dev *dev);
>>> @@ -158,6 +160,7 @@ typedef struct VhostOps {
>>> vhost_reset_device_op vhost_reset_device;
>>> vhost_reset_vring_op vhost_reset_vring;
>>> vhost_get_vq_index_op vhost_get_vq_index;
>>> + vhost_set_single_vring_enable_op vhost_set_single_vring_enable;
>>> vhost_set_vring_enable_op vhost_set_vring_enable;
>>> vhost_requires_shm_log_op vhost_requires_shm_log;
>>> vhost_migration_done_op vhost_migration_done;
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 12/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
2022-08-24 9:01 ` Jason Wang
@ 2022-08-24 9:03 ` Kangjie Xu
0 siblings, 0 replies; 77+ messages in thread
From: Kangjie Xu @ 2022-08-24 9:03 UTC (permalink / raw)
To: Jason Wang
Cc: Michael S. Tsirkin, qemu-devel, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/24 17:01, Jason Wang 写道:
>
> 在 2022/8/24 10:53, Kangjie Xu 写道:
>>
>> 在 2022/8/24 10:44, Jason Wang 写道:
>>>
>>> 在 2022/8/16 09:06, Kangjie Xu 写道:
>>>> Introduce vhost_net_virtqueue_restart(), which can restart the
>>>> virtqueue when the vhost net started running before. If it fails
>>>> to restart the virtqueue, the device will be stopped.
>>>>
>>>> This patch only considers the case for vhost-kernel, when
>>>> NetClientDriver is NET_CLIENT_DRIVER_TAP.
>>>>
>>>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>>
>>>
>>> I would explain why current
>>> vhost_net_start_one()/vhost_net_stop_one() can't work. Is it because
>>> it works at queue pair level? If yes can we restructure the code and
>>> try to reuse ?
>>>
>>> Thanks
>>>
>> Because vhost_net_start_one()/vhost_net_stop_one() works at device
>> level.
>>
>> The queue pair level start/stop are vhost_virtqueue_start() and
>> vhost_virtqueue_stop().
>>
>> What we can reuse is the vhost_virtqueue_start().
>> vhost_virtqueue_stop() cannot be reused because it will destroy device.
>
>
> Let's add this in the changelog or a comment in the code.
>
> Thanks
>
>
Will fix.
Thanks.
>>
>> I think we do not need to restructure because we've already had an
>> abstraction vhost_virtqueue_start().
>>
>> Thanks.
>>
>>>
>>>> ---
>>>> hw/net/vhost_net.c | 48
>>>> +++++++++++++++++++++++++++++++++++++++++
>>>> include/net/vhost_net.h | 2 ++
>>>> 2 files changed, 50 insertions(+)
>>>>
>>>> diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
>>>> index aa60dd901c..2ab67e875e 100644
>>>> --- a/hw/net/vhost_net.c
>>>> +++ b/hw/net/vhost_net.c
>>>> @@ -535,3 +535,51 @@ void vhost_net_virtqueue_stop(VirtIODevice
>>>> *vdev, NetClientState *nc,
>>>> vhost_dev_virtqueue_stop(&net->dev, vdev, idx);
>>>> }
>>>> +
>>>> +int vhost_net_virtqueue_restart(VirtIODevice *vdev, NetClientState
>>>> *nc,
>>>> + int vq_index)
>>>> +{
>>>> + VHostNetState *net = get_vhost_net(nc->peer);
>>>> + const VhostOps *vhost_ops = net->dev.vhost_ops;
>>>> + struct vhost_vring_file file = { };
>>>> + int idx, r;
>>>> +
>>>> + if (!net->dev.started) {
>>>> + return 0;
>>>> + }
>>>> +
>>>> + assert(vhost_ops);
>>>> +
>>>> + idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index);
>>>> +
>>>> + r = vhost_dev_virtqueue_restart(&net->dev, vdev, idx);
>>>> + if (r < 0) {
>>>> + goto err_start;
>>>> + }
>>>> +
>>>> + if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) {
>>>> + file.index = idx;
>>>> + file.fd = net->backend;
>>>> + r = vhost_net_set_backend(&net->dev, &file);
>>>> + if (r < 0) {
>>>> + r = -errno;
>>>> + goto err_start;
>>>> + }
>>>> + }
>>>> +
>>>> + return 0;
>>>> +
>>>> +err_start:
>>>> + error_report("Error when restarting the queue.");
>>>> +
>>>> + if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) {
>>>> + file.fd = -1;
>>>> + file.index = idx;
>>>> + int r = vhost_net_set_backend(&net->dev, &file);
>>>> + assert(r >= 0);
>>>> + }
>>>> +
>>>> + vhost_dev_stop(&net->dev, vdev);
>>>> +
>>>> + return r;
>>>> +}
>>>> diff --git a/include/net/vhost_net.h b/include/net/vhost_net.h
>>>> index 9b3aaf3814..e11a297380 100644
>>>> --- a/include/net/vhost_net.h
>>>> +++ b/include/net/vhost_net.h
>>>> @@ -50,4 +50,6 @@ int vhost_net_set_mtu(struct vhost_net *net,
>>>> uint16_t mtu);
>>>> void vhost_net_virtqueue_stop(VirtIODevice *vdev,
>>>> NetClientState *nc,
>>>> int vq_index);
>>>> +int vhost_net_virtqueue_restart(VirtIODevice *vdev, NetClientState
>>>> *nc,
>>>> + int vq_index);
>>>> #endif
>>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 18/24] vhost-net: vhost-user: update vhost_net_virtqueue_stop()
2022-08-24 4:57 ` Kangjie Xu
@ 2022-08-24 9:04 ` Jason Wang
2022-08-24 9:18 ` Kangjie Xu
0 siblings, 1 reply; 77+ messages in thread
From: Jason Wang @ 2022-08-24 9:04 UTC (permalink / raw)
To: Kangjie Xu
Cc: Michael S. Tsirkin, qemu-devel, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/24 12:57, Kangjie Xu 写道:
>
> 在 2022/8/24 12:05, Jason Wang 写道:
>>
>> 在 2022/8/16 09:06, Kangjie Xu 写道:
>>> Update vhost_net_virtqueue_stop() for vhost-user scenario.
>>
>>
>> Let's explain why it is needed now or why it doesn't cause any issue
>> or it's a bug fix or not.
>>
>> Thanks
>>
> This patch is to suppport vq reset for vhost-user.
>
> We need this simply because the behavior of vhost_ops->get_vq_index()
> is different in vhost-user and vhost-kernel.
>
> vhost_user_get_vq_index(dev, idx) simply returns "idx".
>
> vhost_kernel_get_vq_index(dev, idx) returns "idx - dev->vq_index".
>
> Thanks
Let's add them in the change-log in the next version.
But the question still, is this a bug fix (requires a Fixes tag)? If not
why do we need this now?
Thanks
>
>>
>>>
>>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>> ---
>>> hw/net/vhost_net.c | 4 ++++
>>> 1 file changed, 4 insertions(+)
>>>
>>> diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
>>> index 2ab67e875e..c0d408f3b4 100644
>>> --- a/hw/net/vhost_net.c
>>> +++ b/hw/net/vhost_net.c
>>> @@ -533,6 +533,10 @@ void vhost_net_virtqueue_stop(VirtIODevice
>>> *vdev, NetClientState *nc,
>>> assert(r >= 0);
>>> }
>>> + if (net->nc->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
>>> + idx = idx - net->dev.vq_index;
>>> + }
>>> +
>>> vhost_dev_virtqueue_stop(&net->dev, vdev, idx);
>>> }
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 15/24] vhost-user: add op to enable or disable a single vring
2022-08-24 9:02 ` Jason Wang
@ 2022-08-24 9:09 ` Kangjie Xu
0 siblings, 0 replies; 77+ messages in thread
From: Kangjie Xu @ 2022-08-24 9:09 UTC (permalink / raw)
To: Jason Wang
Cc: Michael S. Tsirkin, qemu-devel, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/24 17:02, Jason Wang 写道:
>
> 在 2022/8/24 11:09, Kangjie Xu 写道:
>>
>> 在 2022/8/24 10:53, Jason Wang 写道:
>>>
>>> 在 2022/8/16 09:06, Kangjie Xu 写道:
>>>> The interface to set enable status for a single vring is lacked in
>>>> VhostOps, since the vhost_set_vring_enable_op will manipulate all
>>>> virtqueues in a device.
>>>>
>>>> Resetting a single vq will rely on this interface.
>>>>
>>>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>>> ---
>>>> hw/virtio/vhost-user.c | 26 +++++++++++++++++++-------
>>>> include/hw/virtio/vhost-backend.h | 3 +++
>>>> 2 files changed, 22 insertions(+), 7 deletions(-)
>>>>
>>>> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
>>>> index 56033f7a92..8307976cda 100644
>>>> --- a/hw/virtio/vhost-user.c
>>>> +++ b/hw/virtio/vhost-user.c
>>>> @@ -1199,6 +1199,22 @@ static int vhost_user_set_vring_base(struct
>>>> vhost_dev *dev,
>>>> return vhost_set_vring(dev, VHOST_USER_SET_VRING_BASE, ring);
>>>> }
>>>> +static int vhost_user_set_single_vring_enable(struct vhost_dev
>>>> *dev,
>>>> + int index,
>>>> + int enable)
>>>> +{
>>>> + if (index < dev->vq_index || index >= dev->vq_index +
>>>> dev->nvqs) {
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + struct vhost_vring_state state = {
>>>> + .index = index,
>>>> + .num = enable,
>>>> + };
>>>> +
>>>> + return vhost_set_vring(dev, VHOST_USER_SET_VRING_ENABLE, &state);
>>>> +}
>>>> +
>>>> static int vhost_user_set_vring_enable(struct vhost_dev *dev, int
>>>> enable)
>>>> {
>>>> int i;
>>>> @@ -1208,13 +1224,8 @@ static int
>>>> vhost_user_set_vring_enable(struct vhost_dev *dev, int enable)
>>>> }
>>>> for (i = 0; i < dev->nvqs; ++i) {
>>>> - int ret;
>>>> - struct vhost_vring_state state = {
>>>> - .index = dev->vq_index + i,
>>>> - .num = enable,
>>>> - };
>>>> -
>>>> - ret = vhost_set_vring(dev, VHOST_USER_SET_VRING_ENABLE,
>>>> &state);
>>>
>>>
>>> Then I'd squash this into previous patch or re-roder to let this
>>> patch (vhost_user_set_single_vring_enable()) to be first.
>>>
>>> Thanks
>>>
>>>
>> Sorry, I don't get why we should re-order them, since these two
>> patches are independent.
>
>
> I meant it's not good to introduce some codes in patch 14 but delete
> them in patch 15 (the above part for example).
>
> Thanks
I get your point, but in fact, it seems that the deleded codes here in
patch 15 do not appear in patch 14....
Patch 14 is about vhost_user_reset_vring(), patch 15 is about
vhost_user_set_vring_enable().
Thanks.
>>
>> Thanks
>>
>>>> + int ret = vhost_user_set_single_vring_enable(dev,
>>>> dev->vq_index + i,
>>>> + enable);
>>>> if (ret < 0) {
>>>> /*
>>>> * Restoring the previous state is likely infeasible,
>>>> as well as
>>>> @@ -2668,6 +2679,7 @@ const VhostOps user_ops = {
>>>> .vhost_reset_vring = vhost_user_reset_vring,
>>>> .vhost_reset_device = vhost_user_reset_device,
>>>> .vhost_get_vq_index = vhost_user_get_vq_index,
>>>> + .vhost_set_single_vring_enable =
>>>> vhost_user_set_single_vring_enable,
>>>> .vhost_set_vring_enable = vhost_user_set_vring_enable,
>>>> .vhost_requires_shm_log = vhost_user_requires_shm_log,
>>>> .vhost_migration_done = vhost_user_migration_done,
>>>> diff --git a/include/hw/virtio/vhost-backend.h
>>>> b/include/hw/virtio/vhost-backend.h
>>>> index f23bf71a8d..38f6b752ff 100644
>>>> --- a/include/hw/virtio/vhost-backend.h
>>>> +++ b/include/hw/virtio/vhost-backend.h
>>>> @@ -83,6 +83,8 @@ typedef int (*vhost_reset_vring_op)(struct
>>>> vhost_dev *dev,
>>>> struct vhost_vring_state *ring);
>>>> typedef int (*vhost_reset_device_op)(struct vhost_dev *dev);
>>>> typedef int (*vhost_get_vq_index_op)(struct vhost_dev *dev, int
>>>> idx);
>>>> +typedef int (*vhost_set_single_vring_enable_op)(struct vhost_dev
>>>> *dev,
>>>> + int index, int
>>>> enable);
>>>> typedef int (*vhost_set_vring_enable_op)(struct vhost_dev *dev,
>>>> int enable);
>>>> typedef bool (*vhost_requires_shm_log_op)(struct vhost_dev *dev);
>>>> @@ -158,6 +160,7 @@ typedef struct VhostOps {
>>>> vhost_reset_device_op vhost_reset_device;
>>>> vhost_reset_vring_op vhost_reset_vring;
>>>> vhost_get_vq_index_op vhost_get_vq_index;
>>>> + vhost_set_single_vring_enable_op vhost_set_single_vring_enable;
>>>> vhost_set_vring_enable_op vhost_set_vring_enable;
>>>> vhost_requires_shm_log_op vhost_requires_shm_log;
>>>> vhost_migration_done_op vhost_migration_done;
>>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 18/24] vhost-net: vhost-user: update vhost_net_virtqueue_stop()
2022-08-24 9:04 ` Jason Wang
@ 2022-08-24 9:18 ` Kangjie Xu
0 siblings, 0 replies; 77+ messages in thread
From: Kangjie Xu @ 2022-08-24 9:18 UTC (permalink / raw)
To: Jason Wang
Cc: Michael S. Tsirkin, qemu-devel, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/24 17:04, Jason Wang 写道:
>
> 在 2022/8/24 12:57, Kangjie Xu 写道:
>>
>> 在 2022/8/24 12:05, Jason Wang 写道:
>>>
>>> 在 2022/8/16 09:06, Kangjie Xu 写道:
>>>> Update vhost_net_virtqueue_stop() for vhost-user scenario.
>>>
>>>
>>> Let's explain why it is needed now or why it doesn't cause any issue
>>> or it's a bug fix or not.
>>>
>>> Thanks
>>>
>> This patch is to suppport vq reset for vhost-user.
>>
>> We need this simply because the behavior of vhost_ops->get_vq_index()
>> is different in vhost-user and vhost-kernel.
>>
>> vhost_user_get_vq_index(dev, idx) simply returns "idx".
>>
>> vhost_kernel_get_vq_index(dev, idx) returns "idx - dev->vq_index".
>>
>> Thanks
>
>
> Let's add them in the change-log in the next version.
Sorry, i don't get what to be changed here, could you explain it?
>
> But the question still, is this a bug fix (requires a Fixes tag)? If
> not why do we need this now?
>
> Thanks
>
Actually, it is not a bugfix, it is simply intended to support vhost-user.
Because vhost_ops->get_vq_index returns different values for
vhost-kernel and vhost-user.
To align vhost-kernel and vhost-user and reuse the following code,
vhost_dev_virtqueue_stop(&net->dev, vdev, idx);
we process the 'idx' here for vhost-user specifically.
Thanks.
>
>>
>>>
>>>>
>>>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>>> ---
>>>> hw/net/vhost_net.c | 4 ++++
>>>> 1 file changed, 4 insertions(+)
>>>>
>>>> diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
>>>> index 2ab67e875e..c0d408f3b4 100644
>>>> --- a/hw/net/vhost_net.c
>>>> +++ b/hw/net/vhost_net.c
>>>> @@ -533,6 +533,10 @@ void vhost_net_virtqueue_stop(VirtIODevice
>>>> *vdev, NetClientState *nc,
>>>> assert(r >= 0);
>>>> }
>>>> + if (net->nc->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
>>>> + idx = idx - net->dev.vq_index;
>>>> + }
>>>> +
>>>> vhost_dev_virtqueue_stop(&net->dev, vdev, idx);
>>>> }
>>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 07/24] virtio-pci: support queue enable
2022-08-24 8:59 ` Jason Wang
@ 2022-08-24 11:27 ` Kangjie Xu
2022-08-25 2:52 ` Jason Wang
0 siblings, 1 reply; 77+ messages in thread
From: Kangjie Xu @ 2022-08-24 11:27 UTC (permalink / raw)
To: Jason Wang
Cc: Michael S. Tsirkin, qemu-devel, marcel.apfelbaum, f4bug, eduardo,
wangyanan55, hengqi, Xuan Zhuo
[-- Attachment #1: Type: text/plain, Size: 3331 bytes --]
在 2022/8/24 16:59, Jason Wang 写道:
>
> 在 2022/8/23 16:20, Kangjie Xu 写道:
>>
>> 在 2022/8/23 15:44, Jason Wang 写道:
>>>
>>> 在 2022/8/16 09:06, Kangjie Xu 写道:
>>>> PCI devices support vq enable.
>>>
>>>
>>> Nit: it might be "support device specific vq enable"
>>>
>>>
>> Get it.
>>>>
>>>> Based on this function, the driver can re-enable the virtqueue
>>>> after the
>>>> virtqueue is reset.
>>>>
>>>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>>> ---
>>>> hw/virtio/virtio-pci.c | 1 +
>>>> 1 file changed, 1 insertion(+)
>>>>
>>>> diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
>>>> index ec8e92052f..3d560e45ad 100644
>>>> --- a/hw/virtio/virtio-pci.c
>>>> +++ b/hw/virtio/virtio-pci.c
>>>> @@ -1335,6 +1335,7 @@ static void virtio_pci_common_write(void
>>>> *opaque, hwaddr addr,
>>>> proxy->vqs[vdev->queue_sel].avail[0],
>>>> ((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << 32 |
>>>> proxy->vqs[vdev->queue_sel].used[0]);
>>>> + virtio_queue_enable(vdev, vdev->queue_sel);
>>>> proxy->vqs[vdev->queue_sel].enabled = 1;
>>>> proxy->vqs[vdev->queue_sel].reset = 0;
>>>
>>>
>>> Any reason we do it before the assignment of 1? It probably means
>>> the device specific method can't depend on virtio_queue_enabled()?
>>>
>>> Thanks
>>>
>> Sorry, I don't get why device specific method can't depend on
>> virtio_queue_enabled().
>
>
> I meant if the device specific method call virtio_queue_enabled() it
> will return false in this case, is this intended?
>
Yes, I intend it to behave in this way.
>
>>
>> Before virtio_queue_enable() is done, virtqueue should always be not
>> ready and disabled.
>>
>> Otherwise, If we put it after the assignment of enabled to 1, the
>> virtqueue may be accessed illegally and may cause panic, because the
>> virtqueue is still being intialized and being configured.
>
>
> How? Shouldn't we make transport ready before making device
> virtqueue(device) ready?
>
> Thanks
>
I am not experienced in this field, could you tell me why we should make
the transport ready first?
I make the transport ready later than making device ready for two aspects:
1. In QEMU, the virtio_queue_enabled() is used only when we start the
device/queue pair (vhost_dev_start_one), or reading
VIRTIO_PCI_COMMON_Q_ENABLE. These two operations and resetting the queue
will *be synchronized* using iothread lock, so we do not need to worry
about the case currently.
2. Suppose we use virtio_queue_enabled() or access the enabled status
asynchronously, and we make the transport ready first.
After enabled set to true, and before virtio_queue_enable() is finished,
somewhere that call virtio_queue_enabled() or access the enabled
status of VirtIOPCIQueue. Then the caller will consider the virtqueue as
enabled(enabled = true in VirtIOPCIQueue). The caller might access the
virtqueue(access avail ring / desc table). But I think *the access here
is illegal* because the virtqueue might still be *unintialized* status.
Thus, from my perspective, to prevent illegal access, we need to make
transport ready after virtio_queue_enable().
Thanks
>
>>
>> Thanks
>>
>>>
>>>> } else {
>>
[-- Attachment #2: Type: text/html, Size: 5839 bytes --]
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 07/24] virtio-pci: support queue enable
2022-08-24 11:27 ` Kangjie Xu
@ 2022-08-25 2:52 ` Jason Wang
2022-08-25 3:38 ` Kangjie Xu
0 siblings, 1 reply; 77+ messages in thread
From: Jason Wang @ 2022-08-25 2:52 UTC (permalink / raw)
To: Kangjie Xu
Cc: Michael S. Tsirkin, qemu-devel, Marcel Apfelbaum,
Philippe Mathieu-Daudé,
Eduardo Habkost, wangyanan55, Heng Qi, Xuan Zhuo
On Wed, Aug 24, 2022 at 7:27 PM Kangjie Xu <kangjie.xu@linux.alibaba.com> wrote:
>
>
> 在 2022/8/24 16:59, Jason Wang 写道:
>
>
> 在 2022/8/23 16:20, Kangjie Xu 写道:
>
>
> 在 2022/8/23 15:44, Jason Wang 写道:
>
>
> 在 2022/8/16 09:06, Kangjie Xu 写道:
>
> PCI devices support vq enable.
>
>
>
> Nit: it might be "support device specific vq enable"
>
>
> Get it.
>
>
> Based on this function, the driver can re-enable the virtqueue after the
> virtqueue is reset.
>
> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> hw/virtio/virtio-pci.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
> index ec8e92052f..3d560e45ad 100644
> --- a/hw/virtio/virtio-pci.c
> +++ b/hw/virtio/virtio-pci.c
> @@ -1335,6 +1335,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,
> proxy->vqs[vdev->queue_sel].avail[0],
> ((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << 32 |
> proxy->vqs[vdev->queue_sel].used[0]);
> + virtio_queue_enable(vdev, vdev->queue_sel);
> proxy->vqs[vdev->queue_sel].enabled = 1;
> proxy->vqs[vdev->queue_sel].reset = 0;
>
>
>
> Any reason we do it before the assignment of 1? It probably means the device specific method can't depend on virtio_queue_enabled()?
>
> Thanks
>
> Sorry, I don't get why device specific method can't depend on virtio_queue_enabled().
>
>
>
> I meant if the device specific method call virtio_queue_enabled() it will return false in this case, is this intended?
>
> Yes, I intend it to behave in this way.
>
>
>
> Before virtio_queue_enable() is done, virtqueue should always be not ready and disabled.
>
> Otherwise, If we put it after the assignment of enabled to 1, the virtqueue may be accessed illegally and may cause panic, because the virtqueue is still being intialized and being configured.
>
>
>
> How? Shouldn't we make transport ready before making device virtqueue(device) ready?
>
> Thanks
>
> I am not experienced in this field, could you tell me why we should make the transport ready first?
That's a must for the device to work.
E.g for PCI device, I can't image the device is ready to work before
PCI is ready.
>
> I make the transport ready later than making device ready for two aspects:
>
> 1. In QEMU, the virtio_queue_enabled() is used only when we start the device/queue pair (vhost_dev_start_one), or reading VIRTIO_PCI_COMMON_Q_ENABLE. These two operations and resetting the queue will be synchronized using iothread lock, so we do not need to worry about the case currently.
Note that virtio_queue_enabled() is an exported helper, you can't
assume how it will be used in the future.
>
> 2. Suppose we use virtio_queue_enabled() or access the enabled status asynchronously, and we make the transport ready first.
>
> After enabled set to true, and before virtio_queue_enable() is finished, somewhere that call virtio_queue_enabled() or access the enabled status of VirtIOPCIQueue. Then the caller will consider the virtqueue as enabled(enabled = true in VirtIOPCIQueue). The caller might access the virtqueue(access avail ring / desc table). But I think the access here is illegal because the virtqueue might still be unintialized status.
>
How can this happen, won't we call device specific method after we set
queue_enabled as true? It's the charge of the device specific method
to synchronize the necessary external I/O in this case.
Thanks
> Thus, from my perspective, to prevent illegal access, we need to make transport ready after virtio_queue_enable().
>
> Thanks
>
>
>
> Thanks
>
>
> } else {
>
>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v2 07/24] virtio-pci: support queue enable
2022-08-25 2:52 ` Jason Wang
@ 2022-08-25 3:38 ` Kangjie Xu
0 siblings, 0 replies; 77+ messages in thread
From: Kangjie Xu @ 2022-08-25 3:38 UTC (permalink / raw)
To: Jason Wang
Cc: Michael S. Tsirkin, qemu-devel, eduardo, marcel.apfelbaum, f4bug,
wangyanan55, hengqi, Xuan Zhuo
在 2022/8/25 10:52, Jason Wang 写道:
> On Wed, Aug 24, 2022 at 7:27 PM Kangjie Xu <kangjie.xu@linux.alibaba.com> wrote:
>>
>> 在 2022/8/24 16:59, Jason Wang 写道:
>>
>>
>> 在 2022/8/23 16:20, Kangjie Xu 写道:
>>
>>
>> 在 2022/8/23 15:44, Jason Wang 写道:
>>
>>
>> 在 2022/8/16 09:06, Kangjie Xu 写道:
>>
>> PCI devices support vq enable.
>>
>>
>>
>> Nit: it might be "support device specific vq enable"
>>
>>
>> Get it.
>>
>>
>> Based on this function, the driver can re-enable the virtqueue after the
>> virtqueue is reset.
>>
>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>> ---
>> hw/virtio/virtio-pci.c | 1 +
>> 1 file changed, 1 insertion(+)
>>
>> diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
>> index ec8e92052f..3d560e45ad 100644
>> --- a/hw/virtio/virtio-pci.c
>> +++ b/hw/virtio/virtio-pci.c
>> @@ -1335,6 +1335,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,
>> proxy->vqs[vdev->queue_sel].avail[0],
>> ((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << 32 |
>> proxy->vqs[vdev->queue_sel].used[0]);
>> + virtio_queue_enable(vdev, vdev->queue_sel);
>> proxy->vqs[vdev->queue_sel].enabled = 1;
>> proxy->vqs[vdev->queue_sel].reset = 0;
>>
>>
>>
>> Any reason we do it before the assignment of 1? It probably means the device specific method can't depend on virtio_queue_enabled()?
>>
>> Thanks
>>
>> Sorry, I don't get why device specific method can't depend on virtio_queue_enabled().
>>
>>
>>
>> I meant if the device specific method call virtio_queue_enabled() it will return false in this case, is this intended?
>>
>> Yes, I intend it to behave in this way.
>>
>>
>>
>> Before virtio_queue_enable() is done, virtqueue should always be not ready and disabled.
>>
>> Otherwise, If we put it after the assignment of enabled to 1, the virtqueue may be accessed illegally and may cause panic, because the virtqueue is still being intialized and being configured.
>>
>>
>>
>> How? Shouldn't we make transport ready before making device virtqueue(device) ready?
>>
>> Thanks
>>
>> I am not experienced in this field, could you tell me why we should make the transport ready first?
> That's a must for the device to work.
>
> E.g for PCI device, I can't image the device is ready to work before
> PCI is ready.
>
>> I make the transport ready later than making device ready for two aspects:
>>
>> 1. In QEMU, the virtio_queue_enabled() is used only when we start the device/queue pair (vhost_dev_start_one), or reading VIRTIO_PCI_COMMON_Q_ENABLE. These two operations and resetting the queue will be synchronized using iothread lock, so we do not need to worry about the case currently.
> Note that virtio_queue_enabled() is an exported helper, you can't
> assume how it will be used in the future.
>
>> 2. Suppose we use virtio_queue_enabled() or access the enabled status asynchronously, and we make the transport ready first.
>>
>> After enabled set to true, and before virtio_queue_enable() is finished, somewhere that call virtio_queue_enabled() or access the enabled status of VirtIOPCIQueue. Then the caller will consider the virtqueue as enabled(enabled = true in VirtIOPCIQueue). The caller might access the virtqueue(access avail ring / desc table). But I think the access here is illegal because the virtqueue might still be unintialized status.
>>
> How can this happen, won't we call device specific method after we set
> queue_enabled as true? It's the charge of the device specific method
> to synchronize the necessary external I/O in this case.
>
> Thanks
I get your point.
You are right. I realized that I ignored the external I/O.
Will fix it.
Thanks
>
>> Thus, from my perspective, to prevent illegal access, we need to make transport ready after virtio_queue_enable().
>>
>> Thanks
>>
>>
>>
>> Thanks
>>
>>
>> } else {
>>
>>
^ permalink raw reply [flat|nested] 77+ messages in thread
end of thread, other threads:[~2022-08-25 3:39 UTC | newest]
Thread overview: 77+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-08-16 1:06 [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 01/24] virtio: sync relevant definitions with linux Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 02/24] virtio: introduce __virtio_queue_reset() Kangjie Xu
2022-08-23 7:31 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 03/24] virtio: introduce virtio_queue_reset() Kangjie Xu
2022-08-23 7:32 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 04/24] virtio: introduce virtio_queue_enable() Kangjie Xu
2022-08-23 7:37 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 05/24] virtio: core: vq reset feature negotation support Kangjie Xu
2022-08-23 7:34 ` Jason Wang
2022-08-23 7:42 ` Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 06/24] virtio-pci: support queue reset Kangjie Xu
2022-08-23 7:40 ` Jason Wang
2022-08-23 7:52 ` Kangjie Xu
2022-08-24 8:56 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 07/24] virtio-pci: support queue enable Kangjie Xu
2022-08-23 7:44 ` Jason Wang
2022-08-23 8:20 ` Kangjie Xu
2022-08-24 8:59 ` Jason Wang
2022-08-24 11:27 ` Kangjie Xu
2022-08-25 2:52 ` Jason Wang
2022-08-25 3:38 ` Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 08/24] vhost: extract the logic of unmapping the vrings and desc Kangjie Xu
2022-08-23 7:45 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 09/24] vhost: introduce vhost_dev_virtqueue_stop() Kangjie Xu
2022-08-23 7:52 ` Jason Wang
2022-08-23 8:03 ` Kangjie Xu
2022-08-24 2:40 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 10/24] vhost: introduce vhost_dev_virtqueue_restart() Kangjie Xu
2022-08-24 2:37 ` Jason Wang
2022-08-24 2:44 ` Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 11/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop() Kangjie Xu
2022-08-24 2:40 ` Jason Wang
2022-08-24 2:46 ` Kangjie Xu
2022-08-24 3:33 ` Kangjie Xu
2022-08-24 9:00 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 12/24] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart() Kangjie Xu
2022-08-24 2:44 ` Jason Wang
2022-08-24 2:53 ` Kangjie Xu
2022-08-24 9:01 ` Jason Wang
2022-08-24 9:03 ` Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 13/24] docs: vhost-user: add VHOST_USER_RESET_VRING message Kangjie Xu
2022-08-24 2:46 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 14/24] vhost-user: introduce vhost_reset_vring() interface Kangjie Xu
2022-08-24 2:50 ` Jason Wang
2022-08-24 3:03 ` Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 15/24] vhost-user: add op to enable or disable a single vring Kangjie Xu
2022-08-24 2:53 ` Jason Wang
2022-08-24 3:09 ` Kangjie Xu
2022-08-24 9:02 ` Jason Wang
2022-08-24 9:09 ` Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 16/24] vhost: vhost-user: update vhost_dev_virtqueue_stop() Kangjie Xu
2022-08-24 3:56 ` Jason Wang
2022-08-24 4:59 ` Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 17/24] vhost: vhost-user: update vhost_dev_virtqueue_restart() Kangjie Xu
2022-08-24 4:03 ` Jason Wang
2022-08-24 5:04 ` Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 18/24] vhost-net: vhost-user: update vhost_net_virtqueue_stop() Kangjie Xu
2022-08-24 4:05 ` Jason Wang
2022-08-24 4:57 ` Kangjie Xu
2022-08-24 9:04 ` Jason Wang
2022-08-24 9:18 ` Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 19/24] vhost-net: vhost-user: update vhost_net_virtqueue_restart() Kangjie Xu
2022-08-24 4:06 ` Jason Wang
2022-08-16 1:06 ` [PATCH v2 20/24] virtio-net: introduce flush_or_purge_queued_packets() Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 21/24] virtio-net: support queue reset Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 22/24] virtio-net: support queue_enable Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 23/24] vhost: vhost-kernel: enable vq reset feature Kangjie Xu
2022-08-16 1:06 ` [PATCH v2 24/24] vhost: vhost-user: " Kangjie Xu
2022-08-16 6:14 ` [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern Michael S. Tsirkin
2022-08-16 6:15 ` Xuan Zhuo
2022-08-16 6:22 ` Michael S. Tsirkin
2022-08-16 6:40 ` Xuan Zhuo
2022-08-17 6:46 ` Kangjie Xu
2022-08-23 1:49 ` Kangjie Xu
2022-08-24 4:10 ` Jason Wang
2022-08-24 4:11 ` Jason Wang
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.