[RFC PATCH 0/6] virtio_test: add support for vhost-vdpa

[RFC PATCH 0/6] virtio_test: add support for vhost-vdpa

[Stefano Garzarella]

The first 3 patches fix various problems I have encountered with
virtio_test (they may go without this series, but I included to allow you
to test the series).

Patch 4 is in preparation of patch 5, moving the feature negotiation when
we initialize the device.

Patch 5 add the support of vhost-vdpa in virtio_test

Patch 6 add vdpa_test.ko based on vdpa_sim_test.c, so we can reuse most of
the code coming from the vdpa_sim framework.

I tested in this way:

  $ modprobe vdpa_sim
  $ modprobe vhost-vdpa

  # load the vdpasim_test device
  $ insmod vdpa_test.ko

  # create a new vdpasim_test device
  $ vdpa dev add mgmtdev vdpasim_test name dev0

  # run the test using the /dev/vhost-vdpa-0 device
  $ ./virtio_test --vdpa /dev/vhost-vdpa-0
  spurious wakeups: 0x0 started=0x100000 completed=0x100000

Comments and suggestions are welcome :-)

Thanks,
Stefano

Stefano Garzarella (6):
  tools/virtio: fix build
  vhost_test: add $(srctree) on the included path
  virtio_test: call __virtio_unbreak_device
  tools/virtio: move feature negotiation in vdev_info_init()
  virtio_test: support vhost-vdpa device
  vdpasim: add vdpa_sim_test module

 tools/virtio/linux/kernel.h           |   2 +-
 tools/virtio/linux/virtio.h           |   2 +
 tools/virtio/linux/vringh.h           |   1 +
 drivers/vdpa/vdpa_sim/vdpa_sim_test.c | 203 ++++++++++++++++++++++++++
 tools/virtio/vdpa_test/vdpa_test.c    |   1 +
 tools/virtio/virtio_test.c            | 131 ++++++++++++++---
 tools/virtio/Makefile                 |  32 +++-
 tools/virtio/vdpa_test/Makefile       |   3 +
 tools/virtio/vhost_test/Makefile      |   2 +-
 9 files changed, 350 insertions(+), 27 deletions(-)
 create mode 100644 drivers/vdpa/vdpa_sim/vdpa_sim_test.c
 create mode 100644 tools/virtio/vdpa_test/vdpa_test.c
 create mode 100644 tools/virtio/vdpa_test/Makefile

-- 
2.36.1

[RFC PATCH 0/6] virtio_test: add support for vhost-vdpa

[Stefano Garzarella]

The first 3 patches fix various problems I have encountered with
virtio_test (they may go without this series, but I included to allow you
to test the series).

Patch 4 is in preparation of patch 5, moving the feature negotiation when
we initialize the device.

Patch 5 add the support of vhost-vdpa in virtio_test

Patch 6 add vdpa_test.ko based on vdpa_sim_test.c, so we can reuse most of
the code coming from the vdpa_sim framework.

I tested in this way:

  $ modprobe vdpa_sim
  $ modprobe vhost-vdpa

  # load the vdpasim_test device
  $ insmod vdpa_test.ko

  # create a new vdpasim_test device
  $ vdpa dev add mgmtdev vdpasim_test name dev0

  # run the test using the /dev/vhost-vdpa-0 device
  $ ./virtio_test --vdpa /dev/vhost-vdpa-0
  spurious wakeups: 0x0 started=0x100000 completed=0x100000

Comments and suggestions are welcome :-)

Thanks,
Stefano

Stefano Garzarella (6):
  tools/virtio: fix build
  vhost_test: add $(srctree) on the included path
  virtio_test: call __virtio_unbreak_device
  tools/virtio: move feature negotiation in vdev_info_init()
  virtio_test: support vhost-vdpa device
  vdpasim: add vdpa_sim_test module

 tools/virtio/linux/kernel.h           |   2 +-
 tools/virtio/linux/virtio.h           |   2 +
 tools/virtio/linux/vringh.h           |   1 +
 drivers/vdpa/vdpa_sim/vdpa_sim_test.c | 203 ++++++++++++++++++++++++++
 tools/virtio/vdpa_test/vdpa_test.c    |   1 +
 tools/virtio/virtio_test.c            | 131 ++++++++++++++---
 tools/virtio/Makefile                 |  32 +++-
 tools/virtio/vdpa_test/Makefile       |   3 +
 tools/virtio/vhost_test/Makefile      |   2 +-
 9 files changed, 350 insertions(+), 27 deletions(-)
 create mode 100644 drivers/vdpa/vdpa_sim/vdpa_sim_test.c
 create mode 100644 tools/virtio/vdpa_test/vdpa_test.c
 create mode 100644 tools/virtio/vdpa_test/Makefile

-- 
2.36.1

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[RFC PATCH 1/6] tools/virtio: fix build

[Stefano Garzarella]

Fix the build caused by the following changes:
- phys_addr_t is now defined in tools/include/linux/types.h
- dev_warn_once() is used in drivers/virtio/virtio_ring.c
- linux/uio.h included by vringh.h use INT_MAX defined in limits.h

Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
---
 tools/virtio/linux/kernel.h | 2 +-
 tools/virtio/linux/vringh.h | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/virtio/linux/kernel.h b/tools/virtio/linux/kernel.h
index 0b493542e61a..21593bf97755 100644
--- a/tools/virtio/linux/kernel.h
+++ b/tools/virtio/linux/kernel.h
@@ -29,7 +29,6 @@
 #define READ                    0
 #define WRITE                   1
 
-typedef unsigned long long phys_addr_t;
 typedef unsigned long long dma_addr_t;
 typedef size_t __kernel_size_t;
 typedef unsigned int __wsum;
@@ -136,6 +135,7 @@ static inline void *krealloc_array(void *p, size_t new_n, size_t new_size, gfp_t
 #endif
 #define dev_err(dev, format, ...) fprintf (stderr, format, ## __VA_ARGS__)
 #define dev_warn(dev, format, ...) fprintf (stderr, format, ## __VA_ARGS__)
+#define dev_warn_once(dev, format, ...) fprintf (stderr, format, ## __VA_ARGS__)
 
 #define min(x, y) ({				\
 	typeof(x) _min1 = (x);			\
diff --git a/tools/virtio/linux/vringh.h b/tools/virtio/linux/vringh.h
index 9348957be56e..e11c6aece734 100644
--- a/tools/virtio/linux/vringh.h
+++ b/tools/virtio/linux/vringh.h
@@ -1 +1,2 @@
+#include <limits.h>
 #include "../../../include/linux/vringh.h"
-- 
2.36.1

[RFC PATCH 1/6] tools/virtio: fix build

[Stefano Garzarella]

Fix the build caused by the following changes:
- phys_addr_t is now defined in tools/include/linux/types.h
- dev_warn_once() is used in drivers/virtio/virtio_ring.c
- linux/uio.h included by vringh.h use INT_MAX defined in limits.h

Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
---
 tools/virtio/linux/kernel.h | 2 +-
 tools/virtio/linux/vringh.h | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/virtio/linux/kernel.h b/tools/virtio/linux/kernel.h
index 0b493542e61a..21593bf97755 100644
--- a/tools/virtio/linux/kernel.h
+++ b/tools/virtio/linux/kernel.h
@@ -29,7 +29,6 @@
 #define READ                    0
 #define WRITE                   1
 
-typedef unsigned long long phys_addr_t;
 typedef unsigned long long dma_addr_t;
 typedef size_t __kernel_size_t;
 typedef unsigned int __wsum;
@@ -136,6 +135,7 @@ static inline void *krealloc_array(void *p, size_t new_n, size_t new_size, gfp_t
 #endif
 #define dev_err(dev, format, ...) fprintf (stderr, format, ## __VA_ARGS__)
 #define dev_warn(dev, format, ...) fprintf (stderr, format, ## __VA_ARGS__)
+#define dev_warn_once(dev, format, ...) fprintf (stderr, format, ## __VA_ARGS__)
 
 #define min(x, y) ({				\
 	typeof(x) _min1 = (x);			\
diff --git a/tools/virtio/linux/vringh.h b/tools/virtio/linux/vringh.h
index 9348957be56e..e11c6aece734 100644
--- a/tools/virtio/linux/vringh.h
+++ b/tools/virtio/linux/vringh.h
@@ -1 +1,2 @@
+#include <limits.h>
 #include "../../../include/linux/vringh.h"
-- 
2.36.1

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[RFC PATCH 2/6] vhost_test: add $(srctree) on the included path

[Stefano Garzarella]

Adding $(srctree) on the included path we can build vhost_test
also when the kernel is not built in the source tree (make O=...).

Use of EXTRA_CFLAGS is deprecated, so let's use ccflags-y.

Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
---
 tools/virtio/vhost_test/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/virtio/vhost_test/Makefile b/tools/virtio/vhost_test/Makefile
index 94d3aff987dc..df5ad39e2520 100644
--- a/tools/virtio/vhost_test/Makefile
+++ b/tools/virtio/vhost_test/Makefile
@@ -1,3 +1,3 @@
 # SPDX-License-Identifier: GPL-2.0-only
 obj-m += vhost_test.o
-EXTRA_CFLAGS += -Idrivers/vhost
+ccflags-y += -I$(srctree)/drivers/vhost
-- 
2.36.1

[RFC PATCH 2/6] vhost_test: add $(srctree) on the included path

[Stefano Garzarella]

Adding $(srctree) on the included path we can build vhost_test
also when the kernel is not built in the source tree (make O=...).

Use of EXTRA_CFLAGS is deprecated, so let's use ccflags-y.

Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
---
 tools/virtio/vhost_test/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/virtio/vhost_test/Makefile b/tools/virtio/vhost_test/Makefile
index 94d3aff987dc..df5ad39e2520 100644
--- a/tools/virtio/vhost_test/Makefile
+++ b/tools/virtio/vhost_test/Makefile
@@ -1,3 +1,3 @@
 # SPDX-License-Identifier: GPL-2.0-only
 obj-m += vhost_test.o
-EXTRA_CFLAGS += -Idrivers/vhost
+ccflags-y += -I$(srctree)/drivers/vhost
-- 
2.36.1

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[RFC PATCH 3/6] virtio_test: call __virtio_unbreak_device

[Stefano Garzarella]

Commit 8b4ec69d7e09 ("virtio: harden vring IRQ") initialize vq->broken
to true, so we need to call __virtio_unbreak_device() before starting
to use it.

Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
---
 tools/virtio/linux/virtio.h | 2 ++
 tools/virtio/virtio_test.c  | 1 +
 2 files changed, 3 insertions(+)

diff --git a/tools/virtio/linux/virtio.h b/tools/virtio/linux/virtio.h
index 363b98228301..feb720d4304f 100644
--- a/tools/virtio/linux/virtio.h
+++ b/tools/virtio/linux/virtio.h
@@ -66,4 +66,6 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
 				      const char *name);
 void vring_del_virtqueue(struct virtqueue *vq);
 
+void __virtio_unbreak_device(struct virtio_device *dev);
+
 #endif
diff --git a/tools/virtio/virtio_test.c b/tools/virtio/virtio_test.c
index 23f142af544a..765e64895dab 100644
--- a/tools/virtio/virtio_test.c
+++ b/tools/virtio/virtio_test.c
@@ -177,6 +177,7 @@ static void run_test(struct vdev_info *dev, struct vq_info *vq,
 	long long spurious = 0;
 	const bool random_batch = batch == RANDOM_BATCH;
 
+	__virtio_unbreak_device(&dev->vdev);
 	r = ioctl(dev->control, VHOST_TEST_RUN, &test);
 	assert(r >= 0);
 	if (!reset_n) {
-- 
2.36.1

[RFC PATCH 3/6] virtio_test: call __virtio_unbreak_device

[Stefano Garzarella]

Commit 8b4ec69d7e09 ("virtio: harden vring IRQ") initialize vq->broken
to true, so we need to call __virtio_unbreak_device() before starting
to use it.

Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
---
 tools/virtio/linux/virtio.h | 2 ++
 tools/virtio/virtio_test.c  | 1 +
 2 files changed, 3 insertions(+)

diff --git a/tools/virtio/linux/virtio.h b/tools/virtio/linux/virtio.h
index 363b98228301..feb720d4304f 100644
--- a/tools/virtio/linux/virtio.h
+++ b/tools/virtio/linux/virtio.h
@@ -66,4 +66,6 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
 				      const char *name);
 void vring_del_virtqueue(struct virtqueue *vq);
 
+void __virtio_unbreak_device(struct virtio_device *dev);
+
 #endif
diff --git a/tools/virtio/virtio_test.c b/tools/virtio/virtio_test.c
index 23f142af544a..765e64895dab 100644
--- a/tools/virtio/virtio_test.c
+++ b/tools/virtio/virtio_test.c
@@ -177,6 +177,7 @@ static void run_test(struct vdev_info *dev, struct vq_info *vq,
 	long long spurious = 0;
 	const bool random_batch = batch == RANDOM_BATCH;
 
+	__virtio_unbreak_device(&dev->vdev);
 	r = ioctl(dev->control, VHOST_TEST_RUN, &test);
 	assert(r >= 0);
 	if (!reset_n) {
-- 
2.36.1

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[RFC PATCH 4/6] tools/virtio: move feature negotiation in vdev_info_init()

[Stefano Garzarella]

The feature negotiation are per device, so it' s better to move them
earlier in vdev_info_init() since vhost_vq_setup() contains the code
to initialize virtqueue.

Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
---
 tools/virtio/virtio_test.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/tools/virtio/virtio_test.c b/tools/virtio/virtio_test.c
index 765e64895dab..2d8a3e881637 100644
--- a/tools/virtio/virtio_test.c
+++ b/tools/virtio/virtio_test.c
@@ -69,7 +69,6 @@ void vhost_vq_setup(struct vdev_info *dev, struct vq_info *info)
 {
 	struct vhost_vring_state state = { .index = info->idx };
 	struct vhost_vring_file file = { .index = info->idx };
-	unsigned long long features = dev->vdev.features;
 	struct vhost_vring_addr addr = {
 		.index = info->idx,
 		.desc_user_addr = (uint64_t)(unsigned long)info->vring.desc,
@@ -77,8 +76,6 @@ void vhost_vq_setup(struct vdev_info *dev, struct vq_info *info)
 		.used_user_addr = (uint64_t)(unsigned long)info->vring.used,
 	};
 	int r;
-	r = ioctl(dev->control, VHOST_SET_FEATURES, &features);
-	assert(r >= 0);
 	state.num = info->vring.num;
 	r = ioctl(dev->control, VHOST_SET_VRING_NUM, &state);
 	assert(r >= 0);
@@ -149,6 +146,8 @@ static void vdev_info_init(struct vdev_info* dev, unsigned long long features)
 	dev->mem->regions[0].memory_size = dev->buf_size;
 	r = ioctl(dev->control, VHOST_SET_MEM_TABLE, dev->mem);
 	assert(r >= 0);
+	r = ioctl(dev->control, VHOST_SET_FEATURES, &features);
+	assert(r >= 0);
 }
 
 /* TODO: this is pretty bad: we get a cache line bounce
-- 
2.36.1

[RFC PATCH 4/6] tools/virtio: move feature negotiation in vdev_info_init()

[Stefano Garzarella]

The feature negotiation are per device, so it' s better to move them
earlier in vdev_info_init() since vhost_vq_setup() contains the code
to initialize virtqueue.

Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
---
 tools/virtio/virtio_test.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/tools/virtio/virtio_test.c b/tools/virtio/virtio_test.c
index 765e64895dab..2d8a3e881637 100644
--- a/tools/virtio/virtio_test.c
+++ b/tools/virtio/virtio_test.c
@@ -69,7 +69,6 @@ void vhost_vq_setup(struct vdev_info *dev, struct vq_info *info)
 {
 	struct vhost_vring_state state = { .index = info->idx };
 	struct vhost_vring_file file = { .index = info->idx };
-	unsigned long long features = dev->vdev.features;
 	struct vhost_vring_addr addr = {
 		.index = info->idx,
 		.desc_user_addr = (uint64_t)(unsigned long)info->vring.desc,
@@ -77,8 +76,6 @@ void vhost_vq_setup(struct vdev_info *dev, struct vq_info *info)
 		.used_user_addr = (uint64_t)(unsigned long)info->vring.used,
 	};
 	int r;
-	r = ioctl(dev->control, VHOST_SET_FEATURES, &features);
-	assert(r >= 0);
 	state.num = info->vring.num;
 	r = ioctl(dev->control, VHOST_SET_VRING_NUM, &state);
 	assert(r >= 0);
@@ -149,6 +146,8 @@ static void vdev_info_init(struct vdev_info* dev, unsigned long long features)
 	dev->mem->regions[0].memory_size = dev->buf_size;
 	r = ioctl(dev->control, VHOST_SET_MEM_TABLE, dev->mem);
 	assert(r >= 0);
+	r = ioctl(dev->control, VHOST_SET_FEATURES, &features);
+	assert(r >= 0);
 }
 
 /* TODO: this is pretty bad: we get a cache line bounce
-- 
2.36.1

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[RFC PATCH 5/6] virtio_test: support vhost-vdpa device

[Stefano Garzarella]

The new --vdpa parameter can be used to run virtio_test with
the new vdpa_sim_test.ko that implements the device.

The main differences with vhost_test are:
- control of status register
- dma map messages
- VHOST_SET_MEM_TABLE not supported by vhost-vdpa
- VHOST_TEST_RUN not supported by vhost-vdpa

The --reset option is not supported for now when using vhost-vdpa.

Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
---
 tools/virtio/virtio_test.c | 127 +++++++++++++++++++++++++++++++------
 1 file changed, 109 insertions(+), 18 deletions(-)

diff --git a/tools/virtio/virtio_test.c b/tools/virtio/virtio_test.c
index 2d8a3e881637..91f983266d86 100644
--- a/tools/virtio/virtio_test.c
+++ b/tools/virtio/virtio_test.c
@@ -44,6 +44,8 @@ struct vdev_info {
 	void *buf;
 	size_t buf_size;
 	struct vhost_memory *mem;
+	bool vdpa;
+	uint64_t backend_features;
 };
 
 static const struct vhost_vring_file no_backend = { .fd = -1 },
@@ -64,6 +66,36 @@ void vq_callback(struct virtqueue *vq)
 {
 }
 
+static void vdpa_add_status(struct vdev_info *dev, uint8_t status)
+{
+	uint8_t current_status;
+	int r;
+
+	r = ioctl(dev->control, VHOST_VDPA_GET_STATUS, &current_status);
+	assert(r >= 0);
+	current_status |= status;
+	r = ioctl(dev->control, VHOST_VDPA_SET_STATUS, &current_status);
+	assert(r >= 0);
+	r = ioctl(dev->control, VHOST_VDPA_GET_STATUS, &current_status);
+	assert(r >= 0);
+	assert((current_status & status) == status);
+}
+
+static void vdpa_dma_map(struct vdev_info *dev, uint64_t iova, uint64_t size,
+			 uint64_t uaddr, uint8_t perm)
+{
+	struct vhost_msg_v2 msg = {};
+	int r;
+
+	msg.type = VHOST_IOTLB_MSG_V2;
+	msg.iotlb.iova = iova;
+	msg.iotlb.size = size;
+	msg.iotlb.uaddr = uaddr;
+	msg.iotlb.perm = perm;
+	msg.iotlb.type = VHOST_IOTLB_UPDATE;
+	r = write(dev->control, &msg, sizeof(msg));
+	assert(r == sizeof(msg));
+}
 
 void vhost_vq_setup(struct vdev_info *dev, struct vq_info *info)
 {
@@ -76,6 +108,12 @@ void vhost_vq_setup(struct vdev_info *dev, struct vq_info *info)
 		.used_user_addr = (uint64_t)(unsigned long)info->vring.used,
 	};
 	int r;
+	if (dev->vdpa) {
+		vdpa_dma_map(dev, (uint64_t)(unsigned long)info->ring,
+			     vring_size(info->vring.num, 4096),
+			     (uint64_t)(unsigned long)info->ring,
+			     VHOST_ACCESS_RW);
+	}
 	state.num = info->vring.num;
 	r = ioctl(dev->control, VHOST_SET_VRING_NUM, &state);
 	assert(r >= 0);
@@ -90,6 +128,11 @@ void vhost_vq_setup(struct vdev_info *dev, struct vq_info *info)
 	file.fd = info->call;
 	r = ioctl(dev->control, VHOST_SET_VRING_CALL, &file);
 	assert(r >= 0);
+	if (dev->vdpa) {
+		state.num = 1;
+		r = ioctl(dev->control, VHOST_VDPA_SET_VRING_ENABLE, &state);
+		assert(r >= 0);
+	}
 }
 
 static void vq_reset(struct vq_info *info, int num, struct virtio_device *vdev)
@@ -121,33 +164,61 @@ static void vq_info_add(struct vdev_info *dev, int num)
 	dev->nvqs++;
 }
 
-static void vdev_info_init(struct vdev_info* dev, unsigned long long features)
+static void vdev_info_init(struct vdev_info *dev, unsigned long long features,
+			   char *vdpa_dev)
 {
+	char *vhost_dev = "/dev/vhost-test";
 	int r;
 	memset(dev, 0, sizeof *dev);
 	dev->vdev.features = features;
+	if (vdpa_dev) {
+		dev->vdpa = true;
+		vhost_dev = vdpa_dev;
+	}
 	INIT_LIST_HEAD(&dev->vdev.vqs);
 	spin_lock_init(&dev->vdev.vqs_list_lock);
 	dev->buf_size = 1024;
 	dev->buf = malloc(dev->buf_size);
 	assert(dev->buf);
-        dev->control = open("/dev/vhost-test", O_RDWR);
+	dev->control = open(vhost_dev, O_RDWR);
 	assert(dev->control >= 0);
 	r = ioctl(dev->control, VHOST_SET_OWNER, NULL);
 	assert(r >= 0);
-	dev->mem = malloc(offsetof(struct vhost_memory, regions) +
-			  sizeof dev->mem->regions[0]);
-	assert(dev->mem);
-	memset(dev->mem, 0, offsetof(struct vhost_memory, regions) +
-                          sizeof dev->mem->regions[0]);
-	dev->mem->nregions = 1;
-	dev->mem->regions[0].guest_phys_addr = (long)dev->buf;
-	dev->mem->regions[0].userspace_addr = (long)dev->buf;
-	dev->mem->regions[0].memory_size = dev->buf_size;
-	r = ioctl(dev->control, VHOST_SET_MEM_TABLE, dev->mem);
-	assert(r >= 0);
+	if (!dev->vdpa) {
+		dev->mem = malloc(offsetof(struct vhost_memory, regions) +
+					   sizeof(dev->mem->regions[0]));
+		assert(dev->mem);
+		memset(dev->mem, 0, offsetof(struct vhost_memory, regions) +
+					   sizeof(dev->mem->regions[0]));
+		dev->mem->nregions = 1;
+		dev->mem->regions[0].guest_phys_addr = (long)dev->buf;
+		dev->mem->regions[0].userspace_addr = (long)dev->buf;
+		dev->mem->regions[0].memory_size = dev->buf_size;
+		r = ioctl(dev->control, VHOST_SET_MEM_TABLE, dev->mem);
+		assert(r >= 0);
+	} else {
+		uint8_t status = 0;
+
+		r = ioctl(dev->control, VHOST_GET_BACKEND_FEATURES,
+			  &dev->backend_features);
+		assert(r >= 0);
+		dev->backend_features &= 0x1ULL << VHOST_BACKEND_F_IOTLB_MSG_V2;
+		assert(dev->backend_features);
+		r = ioctl(dev->control, VHOST_SET_BACKEND_FEATURES,
+			  &dev->backend_features);
+		assert(r >= 0);
+		r = ioctl(dev->control, VHOST_VDPA_SET_STATUS, &status);
+		assert(r >= 0);
+		vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE |
+				     VIRTIO_CONFIG_S_DRIVER);
+		vdpa_dma_map(dev, (uint64_t)(unsigned long)dev->buf,
+			     dev->buf_size, (uint64_t)(unsigned long)dev->buf,
+			     VHOST_ACCESS_RW);
+	}
 	r = ioctl(dev->control, VHOST_SET_FEATURES, &features);
 	assert(r >= 0);
+	if (dev->vdpa)
+		vdpa_add_status(dev, VIRTIO_CONFIG_S_FEATURES_OK);
 }
 
 /* TODO: this is pretty bad: we get a cache line bounce
@@ -177,8 +248,13 @@ static void run_test(struct vdev_info *dev, struct vq_info *vq,
 	const bool random_batch = batch == RANDOM_BATCH;
 
 	__virtio_unbreak_device(&dev->vdev);
-	r = ioctl(dev->control, VHOST_TEST_RUN, &test);
-	assert(r >= 0);
+
+	if (!dev->vdpa) {
+		r = ioctl(dev->control, VHOST_TEST_RUN, &test);
+		assert(r >= 0);
+	} else {
+		vdpa_add_status(dev, VIRTIO_CONFIG_S_DRIVER_OK);
+	}
 	if (!reset_n) {
 		next_reset = INT_MAX;
 	}
@@ -268,8 +344,10 @@ static void run_test(struct vdev_info *dev, struct vq_info *vq,
 		}
 	}
 	test = 0;
-	r = ioctl(dev->control, VHOST_TEST_RUN, &test);
-	assert(r >= 0);
+	if (!dev->vdpa) {
+		r = ioctl(dev->control, VHOST_TEST_RUN, &test);
+		assert(r >= 0);
+	}
 	fprintf(stderr,
 		"spurious wakeups: 0x%llx started=0x%lx completed=0x%lx\n",
 		spurious, started, completed);
@@ -323,6 +401,11 @@ const struct option longopts[] = {
 		.val = 'r',
 		.has_arg = optional_argument,
 	},
+	{
+		.name = "vdpa",
+		.val = 'V',
+		.has_arg = required_argument,
+	},
 	{
 	}
 };
@@ -336,6 +419,7 @@ static void help(void)
 		" [--delayed-interrupt]"
 		" [--batch=random/N]"
 		" [--reset=N]"
+		" [--vdpa=/dev/vhost-vdpa-N]"
 		"\n");
 }
 
@@ -347,6 +431,7 @@ int main(int argc, char **argv)
 	long batch = 1, reset = 0;
 	int o;
 	bool delayed = false;
+	char *vdpa_dev = NULL;
 
 	for (;;) {
 		o = getopt_long(argc, argv, optstring, longopts, NULL);
@@ -389,6 +474,10 @@ int main(int argc, char **argv)
 				assert(reset < (long)INT_MAX + 1);
 			}
 			break;
+		case 'V':
+			vdpa_dev = optarg;
+			features |= 1ULL << VIRTIO_F_ACCESS_PLATFORM;
+			break;
 		default:
 			assert(0);
 			break;
@@ -396,7 +485,9 @@ int main(int argc, char **argv)
 	}
 
 done:
-	vdev_info_init(&dev, features);
+	//TODO: support reset for vdpa
+	assert(vdpa_dev && !reset);
+	vdev_info_init(&dev, features, vdpa_dev);
 	vq_info_add(&dev, 256);
 	run_test(&dev, &dev.vqs[0], delayed, batch, reset, 0x100000);
 	return 0;
-- 
2.36.1

[RFC PATCH 5/6] virtio_test: support vhost-vdpa device

[Stefano Garzarella]

The new --vdpa parameter can be used to run virtio_test with
the new vdpa_sim_test.ko that implements the device.

The main differences with vhost_test are:
- control of status register
- dma map messages
- VHOST_SET_MEM_TABLE not supported by vhost-vdpa
- VHOST_TEST_RUN not supported by vhost-vdpa

The --reset option is not supported for now when using vhost-vdpa.

Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
---
 tools/virtio/virtio_test.c | 127 +++++++++++++++++++++++++++++++------
 1 file changed, 109 insertions(+), 18 deletions(-)

diff --git a/tools/virtio/virtio_test.c b/tools/virtio/virtio_test.c
index 2d8a3e881637..91f983266d86 100644
--- a/tools/virtio/virtio_test.c
+++ b/tools/virtio/virtio_test.c
@@ -44,6 +44,8 @@ struct vdev_info {
 	void *buf;
 	size_t buf_size;
 	struct vhost_memory *mem;
+	bool vdpa;
+	uint64_t backend_features;
 };
 
 static const struct vhost_vring_file no_backend = { .fd = -1 },
@@ -64,6 +66,36 @@ void vq_callback(struct virtqueue *vq)
 {
 }
 
+static void vdpa_add_status(struct vdev_info *dev, uint8_t status)
+{
+	uint8_t current_status;
+	int r;
+
+	r = ioctl(dev->control, VHOST_VDPA_GET_STATUS, &current_status);
+	assert(r >= 0);
+	current_status |= status;
+	r = ioctl(dev->control, VHOST_VDPA_SET_STATUS, &current_status);
+	assert(r >= 0);
+	r = ioctl(dev->control, VHOST_VDPA_GET_STATUS, &current_status);
+	assert(r >= 0);
+	assert((current_status & status) == status);
+}
+
+static void vdpa_dma_map(struct vdev_info *dev, uint64_t iova, uint64_t size,
+			 uint64_t uaddr, uint8_t perm)
+{
+	struct vhost_msg_v2 msg = {};
+	int r;
+
+	msg.type = VHOST_IOTLB_MSG_V2;
+	msg.iotlb.iova = iova;
+	msg.iotlb.size = size;
+	msg.iotlb.uaddr = uaddr;
+	msg.iotlb.perm = perm;
+	msg.iotlb.type = VHOST_IOTLB_UPDATE;
+	r = write(dev->control, &msg, sizeof(msg));
+	assert(r == sizeof(msg));
+}
 
 void vhost_vq_setup(struct vdev_info *dev, struct vq_info *info)
 {
@@ -76,6 +108,12 @@ void vhost_vq_setup(struct vdev_info *dev, struct vq_info *info)
 		.used_user_addr = (uint64_t)(unsigned long)info->vring.used,
 	};
 	int r;
+	if (dev->vdpa) {
+		vdpa_dma_map(dev, (uint64_t)(unsigned long)info->ring,
+			     vring_size(info->vring.num, 4096),
+			     (uint64_t)(unsigned long)info->ring,
+			     VHOST_ACCESS_RW);
+	}
 	state.num = info->vring.num;
 	r = ioctl(dev->control, VHOST_SET_VRING_NUM, &state);
 	assert(r >= 0);
@@ -90,6 +128,11 @@ void vhost_vq_setup(struct vdev_info *dev, struct vq_info *info)
 	file.fd = info->call;
 	r = ioctl(dev->control, VHOST_SET_VRING_CALL, &file);
 	assert(r >= 0);
+	if (dev->vdpa) {
+		state.num = 1;
+		r = ioctl(dev->control, VHOST_VDPA_SET_VRING_ENABLE, &state);
+		assert(r >= 0);
+	}
 }
 
 static void vq_reset(struct vq_info *info, int num, struct virtio_device *vdev)
@@ -121,33 +164,61 @@ static void vq_info_add(struct vdev_info *dev, int num)
 	dev->nvqs++;
 }
 
-static void vdev_info_init(struct vdev_info* dev, unsigned long long features)
+static void vdev_info_init(struct vdev_info *dev, unsigned long long features,
+			   char *vdpa_dev)
 {
+	char *vhost_dev = "/dev/vhost-test";
 	int r;
 	memset(dev, 0, sizeof *dev);
 	dev->vdev.features = features;
+	if (vdpa_dev) {
+		dev->vdpa = true;
+		vhost_dev = vdpa_dev;
+	}
 	INIT_LIST_HEAD(&dev->vdev.vqs);
 	spin_lock_init(&dev->vdev.vqs_list_lock);
 	dev->buf_size = 1024;
 	dev->buf = malloc(dev->buf_size);
 	assert(dev->buf);
-        dev->control = open("/dev/vhost-test", O_RDWR);
+	dev->control = open(vhost_dev, O_RDWR);
 	assert(dev->control >= 0);
 	r = ioctl(dev->control, VHOST_SET_OWNER, NULL);
 	assert(r >= 0);
-	dev->mem = malloc(offsetof(struct vhost_memory, regions) +
-			  sizeof dev->mem->regions[0]);
-	assert(dev->mem);
-	memset(dev->mem, 0, offsetof(struct vhost_memory, regions) +
-                          sizeof dev->mem->regions[0]);
-	dev->mem->nregions = 1;
-	dev->mem->regions[0].guest_phys_addr = (long)dev->buf;
-	dev->mem->regions[0].userspace_addr = (long)dev->buf;
-	dev->mem->regions[0].memory_size = dev->buf_size;
-	r = ioctl(dev->control, VHOST_SET_MEM_TABLE, dev->mem);
-	assert(r >= 0);
+	if (!dev->vdpa) {
+		dev->mem = malloc(offsetof(struct vhost_memory, regions) +
+					   sizeof(dev->mem->regions[0]));
+		assert(dev->mem);
+		memset(dev->mem, 0, offsetof(struct vhost_memory, regions) +
+					   sizeof(dev->mem->regions[0]));
+		dev->mem->nregions = 1;
+		dev->mem->regions[0].guest_phys_addr = (long)dev->buf;
+		dev->mem->regions[0].userspace_addr = (long)dev->buf;
+		dev->mem->regions[0].memory_size = dev->buf_size;
+		r = ioctl(dev->control, VHOST_SET_MEM_TABLE, dev->mem);
+		assert(r >= 0);
+	} else {
+		uint8_t status = 0;
+
+		r = ioctl(dev->control, VHOST_GET_BACKEND_FEATURES,
+			  &dev->backend_features);
+		assert(r >= 0);
+		dev->backend_features &= 0x1ULL << VHOST_BACKEND_F_IOTLB_MSG_V2;
+		assert(dev->backend_features);
+		r = ioctl(dev->control, VHOST_SET_BACKEND_FEATURES,
+			  &dev->backend_features);
+		assert(r >= 0);
+		r = ioctl(dev->control, VHOST_VDPA_SET_STATUS, &status);
+		assert(r >= 0);
+		vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE |
+				     VIRTIO_CONFIG_S_DRIVER);
+		vdpa_dma_map(dev, (uint64_t)(unsigned long)dev->buf,
+			     dev->buf_size, (uint64_t)(unsigned long)dev->buf,
+			     VHOST_ACCESS_RW);
+	}
 	r = ioctl(dev->control, VHOST_SET_FEATURES, &features);
 	assert(r >= 0);
+	if (dev->vdpa)
+		vdpa_add_status(dev, VIRTIO_CONFIG_S_FEATURES_OK);
 }
 
 /* TODO: this is pretty bad: we get a cache line bounce
@@ -177,8 +248,13 @@ static void run_test(struct vdev_info *dev, struct vq_info *vq,
 	const bool random_batch = batch == RANDOM_BATCH;
 
 	__virtio_unbreak_device(&dev->vdev);
-	r = ioctl(dev->control, VHOST_TEST_RUN, &test);
-	assert(r >= 0);
+
+	if (!dev->vdpa) {
+		r = ioctl(dev->control, VHOST_TEST_RUN, &test);
+		assert(r >= 0);
+	} else {
+		vdpa_add_status(dev, VIRTIO_CONFIG_S_DRIVER_OK);
+	}
 	if (!reset_n) {
 		next_reset = INT_MAX;
 	}
@@ -268,8 +344,10 @@ static void run_test(struct vdev_info *dev, struct vq_info *vq,
 		}
 	}
 	test = 0;
-	r = ioctl(dev->control, VHOST_TEST_RUN, &test);
-	assert(r >= 0);
+	if (!dev->vdpa) {
+		r = ioctl(dev->control, VHOST_TEST_RUN, &test);
+		assert(r >= 0);
+	}
 	fprintf(stderr,
 		"spurious wakeups: 0x%llx started=0x%lx completed=0x%lx\n",
 		spurious, started, completed);
@@ -323,6 +401,11 @@ const struct option longopts[] = {
 		.val = 'r',
 		.has_arg = optional_argument,
 	},
+	{
+		.name = "vdpa",
+		.val = 'V',
+		.has_arg = required_argument,
+	},
 	{
 	}
 };
@@ -336,6 +419,7 @@ static void help(void)
 		" [--delayed-interrupt]"
 		" [--batch=random/N]"
 		" [--reset=N]"
+		" [--vdpa=/dev/vhost-vdpa-N]"
 		"\n");
 }
 
@@ -347,6 +431,7 @@ int main(int argc, char **argv)
 	long batch = 1, reset = 0;
 	int o;
 	bool delayed = false;
+	char *vdpa_dev = NULL;
 
 	for (;;) {
 		o = getopt_long(argc, argv, optstring, longopts, NULL);
@@ -389,6 +474,10 @@ int main(int argc, char **argv)
 				assert(reset < (long)INT_MAX + 1);
 			}
 			break;
+		case 'V':
+			vdpa_dev = optarg;
+			features |= 1ULL << VIRTIO_F_ACCESS_PLATFORM;
+			break;
 		default:
 			assert(0);
 			break;
@@ -396,7 +485,9 @@ int main(int argc, char **argv)
 	}
 
 done:
-	vdev_info_init(&dev, features);
+	//TODO: support reset for vdpa
+	assert(vdpa_dev && !reset);
+	vdev_info_init(&dev, features, vdpa_dev);
 	vq_info_add(&dev, 256);
 	run_test(&dev, &dev.vqs[0], delayed, batch, reset, 0x100000);
 	return 0;
-- 
2.36.1

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[RFC PATCH 6/6] vdpasim: add vdpa_sim_test module

[Stefano Garzarella]

This adds a test module for vhost-vdpa infrastructure to be used
with virtio_test.

Intentionally not tied to kbuild to prevent people from installing
and loading it accidentally.

Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
---
 drivers/vdpa/vdpa_sim/vdpa_sim_test.c | 203 ++++++++++++++++++++++++++
 tools/virtio/vdpa_test/vdpa_test.c    |   1 +
 tools/virtio/Makefile                 |  32 +++-
 tools/virtio/vdpa_test/Makefile       |   3 +
 4 files changed, 234 insertions(+), 5 deletions(-)
 create mode 100644 drivers/vdpa/vdpa_sim/vdpa_sim_test.c
 create mode 100644 tools/virtio/vdpa_test/vdpa_test.c
 create mode 100644 tools/virtio/vdpa_test/Makefile

diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim_test.c b/drivers/vdpa/vdpa_sim/vdpa_sim_test.c
new file mode 100644
index 000000000000..17628b1a1cdd
--- /dev/null
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim_test.c
@@ -0,0 +1,203 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * VDPA simulator for virtio-test.
+ *
+ * Copyright (c) 2022, Red Hat Inc. All rights reserved.
+ *
+ */
+
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <linux/vringh.h>
+#include <linux/vdpa.h>
+
+#include "vdpa_sim.h"
+
+#define DRV_VERSION  "0.1"
+#define DRV_AUTHOR   "Stefano Garzarella <sgarzare@redhat.com>"
+#define DRV_DESC     "vDPA Device Simulator for virtio-test device"
+#define DRV_LICENSE  "GPL v2"
+
+#define VDPASIM_TEST_FEATURES	(VDPASIM_FEATURES)
+#define VDPASIM_TEST_VIRTIO_ID	0
+
+/* 1 virtqueues, 1 address spaces, 1 virtqueue groups */
+#define VDPASIM_TEST_VQ_NUM	1
+#define VDPASIM_TEST_AS_NUM	1
+#define VDPASIM_TEST_GROUP_NUM	1
+
+static bool vdpasim_test_handle_req(struct vdpasim *vdpasim,
+				    struct vdpasim_virtqueue *vq)
+{
+	int ret;
+
+	ret = vringh_getdesc_iotlb(&vq->vring, &vq->out_iov, &vq->in_iov,
+				   &vq->head, GFP_ATOMIC);
+	if (ret != 1)
+		return false;
+
+	if (vq->out_iov.used == 0 || vq->in_iov.used > 0) {
+		dev_dbg(&vdpasim->vdpa.dev, "Unexpected descriptor format - out_iov: %u in_iov %u\n",
+			vq->out_iov.used, vq->in_iov.used);
+		return false;
+	}
+
+	if (vringh_kiov_length(&vq->out_iov) == 0) {
+		dev_dbg(&vdpasim->vdpa.dev, "Unexpected 0 len for out buffer\n");
+		return false;
+	}
+
+	vringh_complete_iotlb(&vq->vring, vq->head, 0);
+
+	return true;
+}
+
+static void vdpasim_test_work(struct work_struct *work)
+{
+	struct vdpasim *vdpasim = container_of(work, struct vdpasim, work);
+	bool reschedule = false;
+	int i;
+
+	spin_lock(&vdpasim->lock);
+
+	if (!(vdpasim->status & VIRTIO_CONFIG_S_DRIVER_OK))
+		goto out;
+
+	for (i = 0; i < VDPASIM_TEST_VQ_NUM; i++) {
+		struct vdpasim_virtqueue *vq = &vdpasim->vqs[i];
+		int reqs = 0;
+
+		if (!vq->ready)
+			continue;
+
+		while (vdpasim_test_handle_req(vdpasim, vq)) {
+			/* Make sure used is visible before rasing the interrupt. */
+			smp_wmb();
+
+			local_bh_disable();
+			if (vringh_need_notify_iotlb(&vq->vring) > 0)
+				vringh_notify(&vq->vring);
+			local_bh_enable();
+
+			if (++reqs > 4) {
+				reschedule = true;
+				break;
+			}
+		}
+	}
+out:
+	spin_unlock(&vdpasim->lock);
+
+	if (reschedule)
+		schedule_work(&vdpasim->work);
+}
+
+static void vdpasim_test_get_config(struct vdpasim *vdpasim, void *config)
+{
+	u64 *test_config = config;
+
+	*test_config = cpu_to_vdpasim64(vdpasim, 42);
+}
+
+static void vdpasim_test_mgmtdev_release(struct device *dev)
+{
+}
+
+static struct device vdpasim_test_mgmtdev = {
+	.init_name = "vdpasim_test",
+	.release = vdpasim_test_mgmtdev_release,
+};
+
+static int vdpasim_test_dev_add(struct vdpa_mgmt_dev *mdev, const char *name,
+				const struct vdpa_dev_set_config *config)
+{
+	struct vdpasim_dev_attr dev_attr = {};
+	struct vdpasim *simdev;
+	int ret;
+
+	dev_attr.mgmt_dev = mdev;
+	dev_attr.name = name;
+	dev_attr.id = VDPASIM_TEST_VIRTIO_ID;
+	dev_attr.supported_features = VDPASIM_TEST_FEATURES;
+	dev_attr.nvqs = VDPASIM_TEST_VQ_NUM;
+	dev_attr.ngroups = VDPASIM_TEST_GROUP_NUM;
+	dev_attr.nas = VDPASIM_TEST_AS_NUM;
+	dev_attr.config_size = sizeof(uint64_t);
+	dev_attr.get_config = vdpasim_test_get_config;
+	dev_attr.work_fn = vdpasim_test_work;
+	dev_attr.buffer_size = 0;
+
+	simdev = vdpasim_create(&dev_attr);
+	if (IS_ERR(simdev))
+		return PTR_ERR(simdev);
+
+	ret = _vdpa_register_device(&simdev->vdpa, VDPASIM_TEST_VQ_NUM);
+	if (ret)
+		goto put_dev;
+
+	return 0;
+
+put_dev:
+	put_device(&simdev->vdpa.dev);
+	return ret;
+}
+
+static void vdpasim_test_dev_del(struct vdpa_mgmt_dev *mdev,
+				 struct vdpa_device *dev)
+{
+	struct vdpasim *simdev = container_of(dev, struct vdpasim, vdpa);
+
+	_vdpa_unregister_device(&simdev->vdpa);
+}
+
+static const struct vdpa_mgmtdev_ops vdpasim_test_mgmtdev_ops = {
+	.dev_add = vdpasim_test_dev_add,
+	.dev_del = vdpasim_test_dev_del
+};
+
+static struct virtio_device_id id_table[] = {
+	{ VDPASIM_TEST_VIRTIO_ID, VIRTIO_DEV_ANY_ID },
+	{ 0 },
+};
+
+static struct vdpa_mgmt_dev mgmt_dev = {
+	.device = &vdpasim_test_mgmtdev,
+	.id_table = id_table,
+	.ops = &vdpasim_test_mgmtdev_ops,
+};
+
+static int __init vdpasim_test_init(void)
+{
+	int ret;
+
+	ret = device_register(&vdpasim_test_mgmtdev);
+	if (ret)
+		return ret;
+
+	ret = vdpa_mgmtdev_register(&mgmt_dev);
+	if (ret)
+		goto parent_err;
+
+	return 0;
+
+parent_err:
+	device_unregister(&vdpasim_test_mgmtdev);
+	return ret;
+}
+
+static void __exit vdpasim_test_exit(void)
+{
+	vdpa_mgmtdev_unregister(&mgmt_dev);
+	device_unregister(&vdpasim_test_mgmtdev);
+}
+
+module_init(vdpasim_test_init)
+module_exit(vdpasim_test_exit)
+
+MODULE_VERSION(DRV_VERSION);
+MODULE_LICENSE(DRV_LICENSE);
+MODULE_AUTHOR(DRV_AUTHOR);
+MODULE_DESCRIPTION(DRV_DESC);
diff --git a/tools/virtio/vdpa_test/vdpa_test.c b/tools/virtio/vdpa_test/vdpa_test.c
new file mode 100644
index 000000000000..779d2be2c9e5
--- /dev/null
+++ b/tools/virtio/vdpa_test/vdpa_test.c
@@ -0,0 +1 @@
+#include "vdpa_sim/vdpa_sim_test.c"
diff --git a/tools/virtio/Makefile b/tools/virtio/Makefile
index 1b25cc7c64bb..74a3e8d649a3 100644
--- a/tools/virtio/Makefile
+++ b/tools/virtio/Makefile
@@ -10,6 +10,7 @@ LDFLAGS += -pthread
 vpath %.c ../../drivers/virtio ../../drivers/vhost
 mod:
 	${MAKE} -C `pwd`/../.. M=`pwd`/vhost_test V=${V}
+	${MAKE} -C `pwd`/../.. M=`pwd`/vdpa_test V=${V}
 
 #oot: build vhost as an out of tree module for a distro kernel
 #no effort is taken to make it actually build or work, but tends to mostly work
@@ -18,26 +19,47 @@ mod:
 #resulting modules in production!
 OOT_KSRC=/lib/modules/$$(uname -r)/build
 OOT_VHOST=`pwd`/../../drivers/vhost
+OOT_VDPA=`pwd`/../../drivers/vdpa
 #Everyone depends on vhost
 #Tweak the below to enable more modules
 OOT_CONFIGS=\
 	CONFIG_VHOST=m \
+	CONFIG_VHOST_VDPA=m \
 	CONFIG_VHOST_NET=n \
 	CONFIG_VHOST_SCSI=n \
 	CONFIG_VHOST_VSOCK=n \
 	CONFIG_VHOST_RING=n
-OOT_BUILD=KCFLAGS="-I "${OOT_VHOST} ${MAKE} -C ${OOT_KSRC} V=${V}
+OOT_VDPA_CONFIGS=\
+	CONFIG_VDPA=m \
+	CONFIG_VDPA_SIM=m \
+	CONFIG_VDPA_SIM_NET=n \
+	CONFIG_VDPA_SIM_BLOCK=n \
+	CONFIG_VDPA_USER=n \
+	CONFIG_MLX5_VDPA=n \
+	CONFIG_MLX5_VDPA_NET=n \
+	CONFIG_IFCVF=n \
+	CONFIG_VP_VDPA=n \
+	CONFIG_ALIBABA_ENI_VDPA=n
+OOT_BUILD=KCFLAGS="-I "${OOT_VHOST}" -I "${OOT_VDPA}  ${MAKE} -C ${OOT_KSRC} V=${V}
 oot-build:
 	echo "UNSUPPORTED! Don't use the resulting modules in production!"
 	${OOT_BUILD} M=`pwd`/vhost_test
 	${OOT_BUILD} M=${OOT_VHOST} ${OOT_CONFIGS}
 
-oot-clean: oot-build
-oot: oot-build
+oot-vdpa-build: oot-build
+	echo "UNSUPPORTED! Don't use the resulting modules in production!"
+	${OOT_BUILD} M=`pwd`/vdpa_test
+	${OOT_BUILD} M=${OOT_VDPA} ${OOT_VDPA_CONFIGS}
+
+oot-clean: oot
+oot: oot-build oot-vdpa-build
 oot-clean: OOT_BUILD+=clean
 
-.PHONY: all test mod clean vhost oot oot-clean oot-build
-clean:
+.PHONY: all test mod clean vhost oot oot-clean oot-build oot-vdpa-build clean-vdpa
+clean-vdpa:
+	${RM} vdpa_test/*.o vdpa_test/.*.cmd vdpa_test/Module.symvers \
+              vdpa_test/modules.order
+clean: clean-vdpa
 	${RM} *.o vringh_test virtio_test vhost_test/*.o vhost_test/.*.cmd \
               vhost_test/Module.symvers vhost_test/modules.order *.d
 -include *.d
diff --git a/tools/virtio/vdpa_test/Makefile b/tools/virtio/vdpa_test/Makefile
new file mode 100644
index 000000000000..fce9f344d88f
--- /dev/null
+++ b/tools/virtio/vdpa_test/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-m += vdpa_test.o
+ccflags-y += -I$(srctree)/drivers/vdpa/
-- 
2.36.1

[RFC PATCH 6/6] vdpasim: add vdpa_sim_test module

[Stefano Garzarella]

This adds a test module for vhost-vdpa infrastructure to be used
with virtio_test.

Intentionally not tied to kbuild to prevent people from installing
and loading it accidentally.

Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
---
 drivers/vdpa/vdpa_sim/vdpa_sim_test.c | 203 ++++++++++++++++++++++++++
 tools/virtio/vdpa_test/vdpa_test.c    |   1 +
 tools/virtio/Makefile                 |  32 +++-
 tools/virtio/vdpa_test/Makefile       |   3 +
 4 files changed, 234 insertions(+), 5 deletions(-)
 create mode 100644 drivers/vdpa/vdpa_sim/vdpa_sim_test.c
 create mode 100644 tools/virtio/vdpa_test/vdpa_test.c
 create mode 100644 tools/virtio/vdpa_test/Makefile

diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim_test.c b/drivers/vdpa/vdpa_sim/vdpa_sim_test.c
new file mode 100644
index 000000000000..17628b1a1cdd
--- /dev/null
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim_test.c
@@ -0,0 +1,203 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * VDPA simulator for virtio-test.
+ *
+ * Copyright (c) 2022, Red Hat Inc. All rights reserved.
+ *
+ */
+
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <linux/vringh.h>
+#include <linux/vdpa.h>
+
+#include "vdpa_sim.h"
+
+#define DRV_VERSION  "0.1"
+#define DRV_AUTHOR   "Stefano Garzarella <sgarzare@redhat.com>"
+#define DRV_DESC     "vDPA Device Simulator for virtio-test device"
+#define DRV_LICENSE  "GPL v2"
+
+#define VDPASIM_TEST_FEATURES	(VDPASIM_FEATURES)
+#define VDPASIM_TEST_VIRTIO_ID	0
+
+/* 1 virtqueues, 1 address spaces, 1 virtqueue groups */
+#define VDPASIM_TEST_VQ_NUM	1
+#define VDPASIM_TEST_AS_NUM	1
+#define VDPASIM_TEST_GROUP_NUM	1
+
+static bool vdpasim_test_handle_req(struct vdpasim *vdpasim,
+				    struct vdpasim_virtqueue *vq)
+{
+	int ret;
+
+	ret = vringh_getdesc_iotlb(&vq->vring, &vq->out_iov, &vq->in_iov,
+				   &vq->head, GFP_ATOMIC);
+	if (ret != 1)
+		return false;
+
+	if (vq->out_iov.used == 0 || vq->in_iov.used > 0) {
+		dev_dbg(&vdpasim->vdpa.dev, "Unexpected descriptor format - out_iov: %u in_iov %u\n",
+			vq->out_iov.used, vq->in_iov.used);
+		return false;
+	}
+
+	if (vringh_kiov_length(&vq->out_iov) == 0) {
+		dev_dbg(&vdpasim->vdpa.dev, "Unexpected 0 len for out buffer\n");
+		return false;
+	}
+
+	vringh_complete_iotlb(&vq->vring, vq->head, 0);
+
+	return true;
+}
+
+static void vdpasim_test_work(struct work_struct *work)
+{
+	struct vdpasim *vdpasim = container_of(work, struct vdpasim, work);
+	bool reschedule = false;
+	int i;
+
+	spin_lock(&vdpasim->lock);
+
+	if (!(vdpasim->status & VIRTIO_CONFIG_S_DRIVER_OK))
+		goto out;
+
+	for (i = 0; i < VDPASIM_TEST_VQ_NUM; i++) {
+		struct vdpasim_virtqueue *vq = &vdpasim->vqs[i];
+		int reqs = 0;
+
+		if (!vq->ready)
+			continue;
+
+		while (vdpasim_test_handle_req(vdpasim, vq)) {
+			/* Make sure used is visible before rasing the interrupt. */
+			smp_wmb();
+
+			local_bh_disable();
+			if (vringh_need_notify_iotlb(&vq->vring) > 0)
+				vringh_notify(&vq->vring);
+			local_bh_enable();
+
+			if (++reqs > 4) {
+				reschedule = true;
+				break;
+			}
+		}
+	}
+out:
+	spin_unlock(&vdpasim->lock);
+
+	if (reschedule)
+		schedule_work(&vdpasim->work);
+}
+
+static void vdpasim_test_get_config(struct vdpasim *vdpasim, void *config)
+{
+	u64 *test_config = config;
+
+	*test_config = cpu_to_vdpasim64(vdpasim, 42);
+}
+
+static void vdpasim_test_mgmtdev_release(struct device *dev)
+{
+}
+
+static struct device vdpasim_test_mgmtdev = {
+	.init_name = "vdpasim_test",
+	.release = vdpasim_test_mgmtdev_release,
+};
+
+static int vdpasim_test_dev_add(struct vdpa_mgmt_dev *mdev, const char *name,
+				const struct vdpa_dev_set_config *config)
+{
+	struct vdpasim_dev_attr dev_attr = {};
+	struct vdpasim *simdev;
+	int ret;
+
+	dev_attr.mgmt_dev = mdev;
+	dev_attr.name = name;
+	dev_attr.id = VDPASIM_TEST_VIRTIO_ID;
+	dev_attr.supported_features = VDPASIM_TEST_FEATURES;
+	dev_attr.nvqs = VDPASIM_TEST_VQ_NUM;
+	dev_attr.ngroups = VDPASIM_TEST_GROUP_NUM;
+	dev_attr.nas = VDPASIM_TEST_AS_NUM;
+	dev_attr.config_size = sizeof(uint64_t);
+	dev_attr.get_config = vdpasim_test_get_config;
+	dev_attr.work_fn = vdpasim_test_work;
+	dev_attr.buffer_size = 0;
+
+	simdev = vdpasim_create(&dev_attr);
+	if (IS_ERR(simdev))
+		return PTR_ERR(simdev);
+
+	ret = _vdpa_register_device(&simdev->vdpa, VDPASIM_TEST_VQ_NUM);
+	if (ret)
+		goto put_dev;
+
+	return 0;
+
+put_dev:
+	put_device(&simdev->vdpa.dev);
+	return ret;
+}
+
+static void vdpasim_test_dev_del(struct vdpa_mgmt_dev *mdev,
+				 struct vdpa_device *dev)
+{
+	struct vdpasim *simdev = container_of(dev, struct vdpasim, vdpa);
+
+	_vdpa_unregister_device(&simdev->vdpa);
+}
+
+static const struct vdpa_mgmtdev_ops vdpasim_test_mgmtdev_ops = {
+	.dev_add = vdpasim_test_dev_add,
+	.dev_del = vdpasim_test_dev_del
+};
+
+static struct virtio_device_id id_table[] = {
+	{ VDPASIM_TEST_VIRTIO_ID, VIRTIO_DEV_ANY_ID },
+	{ 0 },
+};
+
+static struct vdpa_mgmt_dev mgmt_dev = {
+	.device = &vdpasim_test_mgmtdev,
+	.id_table = id_table,
+	.ops = &vdpasim_test_mgmtdev_ops,
+};
+
+static int __init vdpasim_test_init(void)
+{
+	int ret;
+
+	ret = device_register(&vdpasim_test_mgmtdev);
+	if (ret)
+		return ret;
+
+	ret = vdpa_mgmtdev_register(&mgmt_dev);
+	if (ret)
+		goto parent_err;
+
+	return 0;
+
+parent_err:
+	device_unregister(&vdpasim_test_mgmtdev);
+	return ret;
+}
+
+static void __exit vdpasim_test_exit(void)
+{
+	vdpa_mgmtdev_unregister(&mgmt_dev);
+	device_unregister(&vdpasim_test_mgmtdev);
+}
+
+module_init(vdpasim_test_init)
+module_exit(vdpasim_test_exit)
+
+MODULE_VERSION(DRV_VERSION);
+MODULE_LICENSE(DRV_LICENSE);
+MODULE_AUTHOR(DRV_AUTHOR);
+MODULE_DESCRIPTION(DRV_DESC);
diff --git a/tools/virtio/vdpa_test/vdpa_test.c b/tools/virtio/vdpa_test/vdpa_test.c
new file mode 100644
index 000000000000..779d2be2c9e5
--- /dev/null
+++ b/tools/virtio/vdpa_test/vdpa_test.c
@@ -0,0 +1 @@
+#include "vdpa_sim/vdpa_sim_test.c"
diff --git a/tools/virtio/Makefile b/tools/virtio/Makefile
index 1b25cc7c64bb..74a3e8d649a3 100644
--- a/tools/virtio/Makefile
+++ b/tools/virtio/Makefile
@@ -10,6 +10,7 @@ LDFLAGS += -pthread
 vpath %.c ../../drivers/virtio ../../drivers/vhost
 mod:
 	${MAKE} -C `pwd`/../.. M=`pwd`/vhost_test V=${V}
+	${MAKE} -C `pwd`/../.. M=`pwd`/vdpa_test V=${V}
 
 #oot: build vhost as an out of tree module for a distro kernel
 #no effort is taken to make it actually build or work, but tends to mostly work
@@ -18,26 +19,47 @@ mod:
 #resulting modules in production!
 OOT_KSRC=/lib/modules/$$(uname -r)/build
 OOT_VHOST=`pwd`/../../drivers/vhost
+OOT_VDPA=`pwd`/../../drivers/vdpa
 #Everyone depends on vhost
 #Tweak the below to enable more modules
 OOT_CONFIGS=\
 	CONFIG_VHOST=m \
+	CONFIG_VHOST_VDPA=m \
 	CONFIG_VHOST_NET=n \
 	CONFIG_VHOST_SCSI=n \
 	CONFIG_VHOST_VSOCK=n \
 	CONFIG_VHOST_RING=n
-OOT_BUILD=KCFLAGS="-I "${OOT_VHOST} ${MAKE} -C ${OOT_KSRC} V=${V}
+OOT_VDPA_CONFIGS=\
+	CONFIG_VDPA=m \
+	CONFIG_VDPA_SIM=m \
+	CONFIG_VDPA_SIM_NET=n \
+	CONFIG_VDPA_SIM_BLOCK=n \
+	CONFIG_VDPA_USER=n \
+	CONFIG_MLX5_VDPA=n \
+	CONFIG_MLX5_VDPA_NET=n \
+	CONFIG_IFCVF=n \
+	CONFIG_VP_VDPA=n \
+	CONFIG_ALIBABA_ENI_VDPA=n
+OOT_BUILD=KCFLAGS="-I "${OOT_VHOST}" -I "${OOT_VDPA}  ${MAKE} -C ${OOT_KSRC} V=${V}
 oot-build:
 	echo "UNSUPPORTED! Don't use the resulting modules in production!"
 	${OOT_BUILD} M=`pwd`/vhost_test
 	${OOT_BUILD} M=${OOT_VHOST} ${OOT_CONFIGS}
 
-oot-clean: oot-build
-oot: oot-build
+oot-vdpa-build: oot-build
+	echo "UNSUPPORTED! Don't use the resulting modules in production!"
+	${OOT_BUILD} M=`pwd`/vdpa_test
+	${OOT_BUILD} M=${OOT_VDPA} ${OOT_VDPA_CONFIGS}
+
+oot-clean: oot
+oot: oot-build oot-vdpa-build
 oot-clean: OOT_BUILD+=clean
 
-.PHONY: all test mod clean vhost oot oot-clean oot-build
-clean:
+.PHONY: all test mod clean vhost oot oot-clean oot-build oot-vdpa-build clean-vdpa
+clean-vdpa:
+	${RM} vdpa_test/*.o vdpa_test/.*.cmd vdpa_test/Module.symvers \
+              vdpa_test/modules.order
+clean: clean-vdpa
 	${RM} *.o vringh_test virtio_test vhost_test/*.o vhost_test/.*.cmd \
               vhost_test/Module.symvers vhost_test/modules.order *.d
 -include *.d
diff --git a/tools/virtio/vdpa_test/Makefile b/tools/virtio/vdpa_test/Makefile
new file mode 100644
index 000000000000..fce9f344d88f
--- /dev/null
+++ b/tools/virtio/vdpa_test/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-m += vdpa_test.o
+ccflags-y += -I$(srctree)/drivers/vdpa/
-- 
2.36.1

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC PATCH 3/6] virtio_test: call __virtio_unbreak_device

[Michael S. Tsirkin]

On Mon, Jul 04, 2022 at 07:16:58PM +0200, Stefano Garzarella wrote:
> Commit 8b4ec69d7e09 ("virtio: harden vring IRQ") initialize vq->broken
> to true, so we need to call __virtio_unbreak_device() before starting
> to use it.
> 
> Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>

I think this shouldn't be necessary with latest master.

> ---
>  tools/virtio/linux/virtio.h | 2 ++
>  tools/virtio/virtio_test.c  | 1 +
>  2 files changed, 3 insertions(+)
> 
> diff --git a/tools/virtio/linux/virtio.h b/tools/virtio/linux/virtio.h
> index 363b98228301..feb720d4304f 100644
> --- a/tools/virtio/linux/virtio.h
> +++ b/tools/virtio/linux/virtio.h
> @@ -66,4 +66,6 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
>  				      const char *name);
>  void vring_del_virtqueue(struct virtqueue *vq);
>  
> +void __virtio_unbreak_device(struct virtio_device *dev);
> +
>  #endif
> diff --git a/tools/virtio/virtio_test.c b/tools/virtio/virtio_test.c
> index 23f142af544a..765e64895dab 100644
> --- a/tools/virtio/virtio_test.c
> +++ b/tools/virtio/virtio_test.c
> @@ -177,6 +177,7 @@ static void run_test(struct vdev_info *dev, struct vq_info *vq,
>  	long long spurious = 0;
>  	const bool random_batch = batch == RANDOM_BATCH;
>  
> +	__virtio_unbreak_device(&dev->vdev);
>  	r = ioctl(dev->control, VHOST_TEST_RUN, &test);
>  	assert(r >= 0);
>  	if (!reset_n) {
> -- 
> 2.36.1

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC PATCH 3/6] virtio_test: call __virtio_unbreak_device

[Michael S. Tsirkin]

On Mon, Jul 04, 2022 at 07:16:58PM +0200, Stefano Garzarella wrote:
> Commit 8b4ec69d7e09 ("virtio: harden vring IRQ") initialize vq->broken
> to true, so we need to call __virtio_unbreak_device() before starting
> to use it.
> 
> Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>

I think this shouldn't be necessary with latest master.

> ---
>  tools/virtio/linux/virtio.h | 2 ++
>  tools/virtio/virtio_test.c  | 1 +
>  2 files changed, 3 insertions(+)
> 
> diff --git a/tools/virtio/linux/virtio.h b/tools/virtio/linux/virtio.h
> index 363b98228301..feb720d4304f 100644
> --- a/tools/virtio/linux/virtio.h
> +++ b/tools/virtio/linux/virtio.h
> @@ -66,4 +66,6 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
>  				      const char *name);
>  void vring_del_virtqueue(struct virtqueue *vq);
>  
> +void __virtio_unbreak_device(struct virtio_device *dev);
> +
>  #endif
> diff --git a/tools/virtio/virtio_test.c b/tools/virtio/virtio_test.c
> index 23f142af544a..765e64895dab 100644
> --- a/tools/virtio/virtio_test.c
> +++ b/tools/virtio/virtio_test.c
> @@ -177,6 +177,7 @@ static void run_test(struct vdev_info *dev, struct vq_info *vq,
>  	long long spurious = 0;
>  	const bool random_batch = batch == RANDOM_BATCH;
>  
> +	__virtio_unbreak_device(&dev->vdev);
>  	r = ioctl(dev->control, VHOST_TEST_RUN, &test);
>  	assert(r >= 0);
>  	if (!reset_n) {
> -- 
> 2.36.1

Re: [RFC PATCH 1/6] tools/virtio: fix build

[Michael S. Tsirkin]

On Mon, Jul 04, 2022 at 07:16:56PM +0200, Stefano Garzarella wrote:
> Fix the build caused by the following changes:
> - phys_addr_t is now defined in tools/include/linux/types.h
> - dev_warn_once() is used in drivers/virtio/virtio_ring.c
> - linux/uio.h included by vringh.h use INT_MAX defined in limits.h
> 
> Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>

Post this separately?

> ---
>  tools/virtio/linux/kernel.h | 2 +-
>  tools/virtio/linux/vringh.h | 1 +
>  2 files changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/virtio/linux/kernel.h b/tools/virtio/linux/kernel.h
> index 0b493542e61a..21593bf97755 100644
> --- a/tools/virtio/linux/kernel.h
> +++ b/tools/virtio/linux/kernel.h
> @@ -29,7 +29,6 @@
>  #define READ                    0
>  #define WRITE                   1
>  
> -typedef unsigned long long phys_addr_t;
>  typedef unsigned long long dma_addr_t;
>  typedef size_t __kernel_size_t;
>  typedef unsigned int __wsum;
> @@ -136,6 +135,7 @@ static inline void *krealloc_array(void *p, size_t new_n, size_t new_size, gfp_t
>  #endif
>  #define dev_err(dev, format, ...) fprintf (stderr, format, ## __VA_ARGS__)
>  #define dev_warn(dev, format, ...) fprintf (stderr, format, ## __VA_ARGS__)
> +#define dev_warn_once(dev, format, ...) fprintf (stderr, format, ## __VA_ARGS__)
>  
>  #define min(x, y) ({				\
>  	typeof(x) _min1 = (x);			\
> diff --git a/tools/virtio/linux/vringh.h b/tools/virtio/linux/vringh.h
> index 9348957be56e..e11c6aece734 100644
> --- a/tools/virtio/linux/vringh.h
> +++ b/tools/virtio/linux/vringh.h
> @@ -1 +1,2 @@
> +#include <limits.h>
>  #include "../../../include/linux/vringh.h"
> -- 
> 2.36.1

Re: [RFC PATCH 1/6] tools/virtio: fix build

[Michael S. Tsirkin]

On Mon, Jul 04, 2022 at 07:16:56PM +0200, Stefano Garzarella wrote:
> Fix the build caused by the following changes:
> - phys_addr_t is now defined in tools/include/linux/types.h
> - dev_warn_once() is used in drivers/virtio/virtio_ring.c
> - linux/uio.h included by vringh.h use INT_MAX defined in limits.h
> 
> Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>

Post this separately?

> ---
>  tools/virtio/linux/kernel.h | 2 +-
>  tools/virtio/linux/vringh.h | 1 +
>  2 files changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/virtio/linux/kernel.h b/tools/virtio/linux/kernel.h
> index 0b493542e61a..21593bf97755 100644
> --- a/tools/virtio/linux/kernel.h
> +++ b/tools/virtio/linux/kernel.h
> @@ -29,7 +29,6 @@
>  #define READ                    0
>  #define WRITE                   1
>  
> -typedef unsigned long long phys_addr_t;
>  typedef unsigned long long dma_addr_t;
>  typedef size_t __kernel_size_t;
>  typedef unsigned int __wsum;
> @@ -136,6 +135,7 @@ static inline void *krealloc_array(void *p, size_t new_n, size_t new_size, gfp_t
>  #endif
>  #define dev_err(dev, format, ...) fprintf (stderr, format, ## __VA_ARGS__)
>  #define dev_warn(dev, format, ...) fprintf (stderr, format, ## __VA_ARGS__)
> +#define dev_warn_once(dev, format, ...) fprintf (stderr, format, ## __VA_ARGS__)
>  
>  #define min(x, y) ({				\
>  	typeof(x) _min1 = (x);			\
> diff --git a/tools/virtio/linux/vringh.h b/tools/virtio/linux/vringh.h
> index 9348957be56e..e11c6aece734 100644
> --- a/tools/virtio/linux/vringh.h
> +++ b/tools/virtio/linux/vringh.h
> @@ -1 +1,2 @@
> +#include <limits.h>
>  #include "../../../include/linux/vringh.h"
> -- 
> 2.36.1

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC PATCH 3/6] virtio_test: call __virtio_unbreak_device

[Stefano Garzarella]

On Mon, Jul 04, 2022 at 03:06:39PM -0400, Michael S. Tsirkin wrote:
>On Mon, Jul 04, 2022 at 07:16:58PM +0200, Stefano Garzarella wrote:
>> Commit 8b4ec69d7e09 ("virtio: harden vring IRQ") initialize vq->broken
>> to true, so we need to call __virtio_unbreak_device() before starting
>> to use it.
>>
>> Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
>
>I think this shouldn't be necessary with latest master.

Right, c346dae4f3fb ("virtio: disable notification hardening by default")
is merged now.

I'll remove this patch.

Thanks,
Stefano

>
>> ---
>>  tools/virtio/linux/virtio.h | 2 ++
>>  tools/virtio/virtio_test.c  | 1 +
>>  2 files changed, 3 insertions(+)
>>
>> diff --git a/tools/virtio/linux/virtio.h b/tools/virtio/linux/virtio.h
>> index 363b98228301..feb720d4304f 100644
>> --- a/tools/virtio/linux/virtio.h
>> +++ b/tools/virtio/linux/virtio.h
>> @@ -66,4 +66,6 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
>>  				      const char *name);
>>  void vring_del_virtqueue(struct virtqueue *vq);
>>
>> +void __virtio_unbreak_device(struct virtio_device *dev);
>> +
>>  #endif
>> diff --git a/tools/virtio/virtio_test.c b/tools/virtio/virtio_test.c
>> index 23f142af544a..765e64895dab 100644
>> --- a/tools/virtio/virtio_test.c
>> +++ b/tools/virtio/virtio_test.c
>> @@ -177,6 +177,7 @@ static void run_test(struct vdev_info *dev, struct vq_info *vq,
>>  	long long spurious = 0;
>>  	const bool random_batch = batch == RANDOM_BATCH;
>>
>> +	__virtio_unbreak_device(&dev->vdev);
>>  	r = ioctl(dev->control, VHOST_TEST_RUN, &test);
>>  	assert(r >= 0);
>>  	if (!reset_n) {
>> --
>> 2.36.1
>

Re: [RFC PATCH 3/6] virtio_test: call __virtio_unbreak_device

[Stefano Garzarella]

On Mon, Jul 04, 2022 at 03:06:39PM -0400, Michael S. Tsirkin wrote:
>On Mon, Jul 04, 2022 at 07:16:58PM +0200, Stefano Garzarella wrote:
>> Commit 8b4ec69d7e09 ("virtio: harden vring IRQ") initialize vq->broken
>> to true, so we need to call __virtio_unbreak_device() before starting
>> to use it.
>>
>> Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
>
>I think this shouldn't be necessary with latest master.

Right, c346dae4f3fb ("virtio: disable notification hardening by default")
is merged now.

I'll remove this patch.

Thanks,
Stefano

>
>> ---
>>  tools/virtio/linux/virtio.h | 2 ++
>>  tools/virtio/virtio_test.c  | 1 +
>>  2 files changed, 3 insertions(+)
>>
>> diff --git a/tools/virtio/linux/virtio.h b/tools/virtio/linux/virtio.h
>> index 363b98228301..feb720d4304f 100644
>> --- a/tools/virtio/linux/virtio.h
>> +++ b/tools/virtio/linux/virtio.h
>> @@ -66,4 +66,6 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
>>  				      const char *name);
>>  void vring_del_virtqueue(struct virtqueue *vq);
>>
>> +void __virtio_unbreak_device(struct virtio_device *dev);
>> +
>>  #endif
>> diff --git a/tools/virtio/virtio_test.c b/tools/virtio/virtio_test.c
>> index 23f142af544a..765e64895dab 100644
>> --- a/tools/virtio/virtio_test.c
>> +++ b/tools/virtio/virtio_test.c
>> @@ -177,6 +177,7 @@ static void run_test(struct vdev_info *dev, struct vq_info *vq,
>>  	long long spurious = 0;
>>  	const bool random_batch = batch == RANDOM_BATCH;
>>
>> +	__virtio_unbreak_device(&dev->vdev);
>>  	r = ioctl(dev->control, VHOST_TEST_RUN, &test);
>>  	assert(r >= 0);
>>  	if (!reset_n) {
>> --
>> 2.36.1
>

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC PATCH 1/6] tools/virtio: fix build

[Stefano Garzarella]

On Mon, Jul 04, 2022 at 03:07:15PM -0400, Michael S. Tsirkin wrote:
>On Mon, Jul 04, 2022 at 07:16:56PM +0200, Stefano Garzarella wrote:
>> Fix the build caused by the following changes:
>> - phys_addr_t is now defined in tools/include/linux/types.h
>> - dev_warn_once() is used in drivers/virtio/virtio_ring.c
>> - linux/uio.h included by vringh.h use INT_MAX defined in limits.h
>>
>> Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
>
>Post this separately?

Yep, sure!

Stefano

Re: [RFC PATCH 1/6] tools/virtio: fix build

[Stefano Garzarella]

On Mon, Jul 04, 2022 at 03:07:15PM -0400, Michael S. Tsirkin wrote:
>On Mon, Jul 04, 2022 at 07:16:56PM +0200, Stefano Garzarella wrote:
>> Fix the build caused by the following changes:
>> - phys_addr_t is now defined in tools/include/linux/types.h
>> - dev_warn_once() is used in drivers/virtio/virtio_ring.c
>> - linux/uio.h included by vringh.h use INT_MAX defined in limits.h
>>
>> Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
>
>Post this separately?

Yep, sure!

Stefano

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC PATCH 0/6] virtio_test: add support for vhost-vdpa

[Jason Wang]

在 2022/7/5 01:16, Stefano Garzarella 写道:
> The first 3 patches fix various problems I have encountered with
> virtio_test (they may go without this series, but I included to allow you
> to test the series).
>
> Patch 4 is in preparation of patch 5, moving the feature negotiation when
> we initialize the device.
>
> Patch 5 add the support of vhost-vdpa in virtio_test
>
> Patch 6 add vdpa_test.ko based on vdpa_sim_test.c, so we can reuse most of
> the code coming from the vdpa_sim framework.
>
> I tested in this way:
>
>    $ modprobe vdpa_sim
>    $ modprobe vhost-vdpa
>
>    # load the vdpasim_test device
>    $ insmod vdpa_test.ko
>
>    # create a new vdpasim_test device
>    $ vdpa dev add mgmtdev vdpasim_test name dev0


I wonder what's the benefit of using a dedicated test device other than 
networking? (already a loopback device anyhow).

Thanks


>
>    # run the test using the /dev/vhost-vdpa-0 device
>    $ ./virtio_test --vdpa /dev/vhost-vdpa-0
>    spurious wakeups: 0x0 started=0x100000 completed=0x100000
>
> Comments and suggestions are welcome :-)
>
> Thanks,
> Stefano
>
> Stefano Garzarella (6):
>    tools/virtio: fix build
>    vhost_test: add $(srctree) on the included path
>    virtio_test: call __virtio_unbreak_device
>    tools/virtio: move feature negotiation in vdev_info_init()
>    virtio_test: support vhost-vdpa device
>    vdpasim: add vdpa_sim_test module
>
>   tools/virtio/linux/kernel.h           |   2 +-
>   tools/virtio/linux/virtio.h           |   2 +
>   tools/virtio/linux/vringh.h           |   1 +
>   drivers/vdpa/vdpa_sim/vdpa_sim_test.c | 203 ++++++++++++++++++++++++++
>   tools/virtio/vdpa_test/vdpa_test.c    |   1 +
>   tools/virtio/virtio_test.c            | 131 ++++++++++++++---
>   tools/virtio/Makefile                 |  32 +++-
>   tools/virtio/vdpa_test/Makefile       |   3 +
>   tools/virtio/vhost_test/Makefile      |   2 +-
>   9 files changed, 350 insertions(+), 27 deletions(-)
>   create mode 100644 drivers/vdpa/vdpa_sim/vdpa_sim_test.c
>   create mode 100644 tools/virtio/vdpa_test/vdpa_test.c
>   create mode 100644 tools/virtio/vdpa_test/Makefile
>

Re: [RFC PATCH 0/6] virtio_test: add support for vhost-vdpa

[Jason Wang]

在 2022/7/5 01:16, Stefano Garzarella 写道:
> The first 3 patches fix various problems I have encountered with
> virtio_test (they may go without this series, but I included to allow you
> to test the series).
>
> Patch 4 is in preparation of patch 5, moving the feature negotiation when
> we initialize the device.
>
> Patch 5 add the support of vhost-vdpa in virtio_test
>
> Patch 6 add vdpa_test.ko based on vdpa_sim_test.c, so we can reuse most of
> the code coming from the vdpa_sim framework.
>
> I tested in this way:
>
>    $ modprobe vdpa_sim
>    $ modprobe vhost-vdpa
>
>    # load the vdpasim_test device
>    $ insmod vdpa_test.ko
>
>    # create a new vdpasim_test device
>    $ vdpa dev add mgmtdev vdpasim_test name dev0


I wonder what's the benefit of using a dedicated test device other than 
networking? (already a loopback device anyhow).

Thanks


>
>    # run the test using the /dev/vhost-vdpa-0 device
>    $ ./virtio_test --vdpa /dev/vhost-vdpa-0
>    spurious wakeups: 0x0 started=0x100000 completed=0x100000
>
> Comments and suggestions are welcome :-)
>
> Thanks,
> Stefano
>
> Stefano Garzarella (6):
>    tools/virtio: fix build
>    vhost_test: add $(srctree) on the included path
>    virtio_test: call __virtio_unbreak_device
>    tools/virtio: move feature negotiation in vdev_info_init()
>    virtio_test: support vhost-vdpa device
>    vdpasim: add vdpa_sim_test module
>
>   tools/virtio/linux/kernel.h           |   2 +-
>   tools/virtio/linux/virtio.h           |   2 +
>   tools/virtio/linux/vringh.h           |   1 +
>   drivers/vdpa/vdpa_sim/vdpa_sim_test.c | 203 ++++++++++++++++++++++++++
>   tools/virtio/vdpa_test/vdpa_test.c    |   1 +
>   tools/virtio/virtio_test.c            | 131 ++++++++++++++---
>   tools/virtio/Makefile                 |  32 +++-
>   tools/virtio/vdpa_test/Makefile       |   3 +
>   tools/virtio/vhost_test/Makefile      |   2 +-
>   9 files changed, 350 insertions(+), 27 deletions(-)
>   create mode 100644 drivers/vdpa/vdpa_sim/vdpa_sim_test.c
>   create mode 100644 tools/virtio/vdpa_test/vdpa_test.c
>   create mode 100644 tools/virtio/vdpa_test/Makefile
>

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC PATCH 0/6] virtio_test: add support for vhost-vdpa

[Stefano Garzarella]

On Tue, Jul 12, 2022 at 04:34:11PM +0800, Jason Wang wrote:
>
>在 2022/7/5 01:16, Stefano Garzarella 写道:
>>The first 3 patches fix various problems I have encountered with
>>virtio_test (they may go without this series, but I included to allow you
>>to test the series).
>>
>>Patch 4 is in preparation of patch 5, moving the feature negotiation when
>>we initialize the device.
>>
>>Patch 5 add the support of vhost-vdpa in virtio_test
>>
>>Patch 6 add vdpa_test.ko based on vdpa_sim_test.c, so we can reuse most of
>>the code coming from the vdpa_sim framework.
>>
>>I tested in this way:
>>
>>   $ modprobe vdpa_sim
>>   $ modprobe vhost-vdpa
>>
>>   # load the vdpasim_test device
>>   $ insmod vdpa_test.ko
>>
>>   # create a new vdpasim_test device
>>   $ vdpa dev add mgmtdev vdpasim_test name dev0
>
>
>I wonder what's the benefit of using a dedicated test device other 
>than networking? (already a loopback device anyhow).
>

The main advantage I was thinking is that we can extend it at will to 
try to cover as much as possible all the paths that maybe with net we 
can't (indirect, reset, etc.).

Plus it should be easier to test vhost-vdpa this way without using QEMU.

Do you think this is redundant?

Thanks,
Stefano

Re: [RFC PATCH 0/6] virtio_test: add support for vhost-vdpa

[Stefano Garzarella]

On Tue, Jul 12, 2022 at 04:34:11PM +0800, Jason Wang wrote:
>
>在 2022/7/5 01:16, Stefano Garzarella 写道:
>>The first 3 patches fix various problems I have encountered with
>>virtio_test (they may go without this series, but I included to allow you
>>to test the series).
>>
>>Patch 4 is in preparation of patch 5, moving the feature negotiation when
>>we initialize the device.
>>
>>Patch 5 add the support of vhost-vdpa in virtio_test
>>
>>Patch 6 add vdpa_test.ko based on vdpa_sim_test.c, so we can reuse most of
>>the code coming from the vdpa_sim framework.
>>
>>I tested in this way:
>>
>>   $ modprobe vdpa_sim
>>   $ modprobe vhost-vdpa
>>
>>   # load the vdpasim_test device
>>   $ insmod vdpa_test.ko
>>
>>   # create a new vdpasim_test device
>>   $ vdpa dev add mgmtdev vdpasim_test name dev0
>
>
>I wonder what's the benefit of using a dedicated test device other 
>than networking? (already a loopback device anyhow).
>

The main advantage I was thinking is that we can extend it at will to 
try to cover as much as possible all the paths that maybe with net we 
can't (indirect, reset, etc.).

Plus it should be easier to test vhost-vdpa this way without using QEMU.

Do you think this is redundant?

Thanks,
Stefano

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC PATCH 0/6] virtio_test: add support for vhost-vdpa

[Jason Wang]

On Tue, Jul 12, 2022 at 4:46 PM Stefano Garzarella <sgarzare@redhat.com> wrote:
>
> On Tue, Jul 12, 2022 at 04:34:11PM +0800, Jason Wang wrote:
> >
> >在 2022/7/5 01:16, Stefano Garzarella 写道:
> >>The first 3 patches fix various problems I have encountered with
> >>virtio_test (they may go without this series, but I included to allow you
> >>to test the series).
> >>
> >>Patch 4 is in preparation of patch 5, moving the feature negotiation when
> >>we initialize the device.
> >>
> >>Patch 5 add the support of vhost-vdpa in virtio_test
> >>
> >>Patch 6 add vdpa_test.ko based on vdpa_sim_test.c, so we can reuse most of
> >>the code coming from the vdpa_sim framework.
> >>
> >>I tested in this way:
> >>
> >>   $ modprobe vdpa_sim
> >>   $ modprobe vhost-vdpa
> >>
> >>   # load the vdpasim_test device
> >>   $ insmod vdpa_test.ko
> >>
> >>   # create a new vdpasim_test device
> >>   $ vdpa dev add mgmtdev vdpasim_test name dev0
> >
> >
> >I wonder what's the benefit of using a dedicated test device other
> >than networking? (already a loopback device anyhow).
> >
>
> The main advantage I was thinking is that we can extend it at will to
> try to cover as much as possible all the paths that maybe with net we
> can't (indirect, reset, etc.).
>
> Plus it should be easier to test vhost-vdpa this way without using QEMU.
>
> Do you think this is redundant?

It's probably fine and it might be better to describe the reason in
the changelog.

THanks

>
> Thanks,
> Stefano
>

Re: [RFC PATCH 0/6] virtio_test: add support for vhost-vdpa

[Jason Wang]

On Tue, Jul 12, 2022 at 4:46 PM Stefano Garzarella <sgarzare@redhat.com> wrote:
>
> On Tue, Jul 12, 2022 at 04:34:11PM +0800, Jason Wang wrote:
> >
> >在 2022/7/5 01:16, Stefano Garzarella 写道:
> >>The first 3 patches fix various problems I have encountered with
> >>virtio_test (they may go without this series, but I included to allow you
> >>to test the series).
> >>
> >>Patch 4 is in preparation of patch 5, moving the feature negotiation when
> >>we initialize the device.
> >>
> >>Patch 5 add the support of vhost-vdpa in virtio_test
> >>
> >>Patch 6 add vdpa_test.ko based on vdpa_sim_test.c, so we can reuse most of
> >>the code coming from the vdpa_sim framework.
> >>
> >>I tested in this way:
> >>
> >>   $ modprobe vdpa_sim
> >>   $ modprobe vhost-vdpa
> >>
> >>   # load the vdpasim_test device
> >>   $ insmod vdpa_test.ko
> >>
> >>   # create a new vdpasim_test device
> >>   $ vdpa dev add mgmtdev vdpasim_test name dev0
> >
> >
> >I wonder what's the benefit of using a dedicated test device other
> >than networking? (already a loopback device anyhow).
> >
>
> The main advantage I was thinking is that we can extend it at will to
> try to cover as much as possible all the paths that maybe with net we
> can't (indirect, reset, etc.).
>
> Plus it should be easier to test vhost-vdpa this way without using QEMU.
>
> Do you think this is redundant?

It's probably fine and it might be better to describe the reason in
the changelog.

THanks

>
> Thanks,
> Stefano
>

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC PATCH 0/6] virtio_test: add support for vhost-vdpa

[Stefano Garzarella]

On Tue, Jul 12, 2022 at 04:47:10PM +0800, Jason Wang wrote:
>On Tue, Jul 12, 2022 at 4:46 PM Stefano Garzarella <sgarzare@redhat.com> wrote:
>>
>> On Tue, Jul 12, 2022 at 04:34:11PM +0800, Jason Wang wrote:
>> >
>> >在 2022/7/5 01:16, Stefano Garzarella 写道:
>> >>The first 3 patches fix various problems I have encountered with
>> >>virtio_test (they may go without this series, but I included to allow you
>> >>to test the series).
>> >>
>> >>Patch 4 is in preparation of patch 5, moving the feature negotiation when
>> >>we initialize the device.
>> >>
>> >>Patch 5 add the support of vhost-vdpa in virtio_test
>> >>
>> >>Patch 6 add vdpa_test.ko based on vdpa_sim_test.c, so we can reuse most of
>> >>the code coming from the vdpa_sim framework.
>> >>
>> >>I tested in this way:
>> >>
>> >>   $ modprobe vdpa_sim
>> >>   $ modprobe vhost-vdpa
>> >>
>> >>   # load the vdpasim_test device
>> >>   $ insmod vdpa_test.ko
>> >>
>> >>   # create a new vdpasim_test device
>> >>   $ vdpa dev add mgmtdev vdpasim_test name dev0
>> >
>> >
>> >I wonder what's the benefit of using a dedicated test device other
>> >than networking? (already a loopback device anyhow).
>> >
>>
>> The main advantage I was thinking is that we can extend it at will to
>> try to cover as much as possible all the paths that maybe with net we
>> can't (indirect, reset, etc.).
>>
>> Plus it should be easier to test vhost-vdpa this way without using QEMU.
>>
>> Do you think this is redundant?
>
>It's probably fine and it might be better to describe the reason in
>the changelog.

Okay, I'll do in the next version!

Thanks,
Stefano

Re: [RFC PATCH 0/6] virtio_test: add support for vhost-vdpa

[Stefano Garzarella]

On Tue, Jul 12, 2022 at 04:47:10PM +0800, Jason Wang wrote:
>On Tue, Jul 12, 2022 at 4:46 PM Stefano Garzarella <sgarzare@redhat.com> wrote:
>>
>> On Tue, Jul 12, 2022 at 04:34:11PM +0800, Jason Wang wrote:
>> >
>> >在 2022/7/5 01:16, Stefano Garzarella 写道:
>> >>The first 3 patches fix various problems I have encountered with
>> >>virtio_test (they may go without this series, but I included to allow you
>> >>to test the series).
>> >>
>> >>Patch 4 is in preparation of patch 5, moving the feature negotiation when
>> >>we initialize the device.
>> >>
>> >>Patch 5 add the support of vhost-vdpa in virtio_test
>> >>
>> >>Patch 6 add vdpa_test.ko based on vdpa_sim_test.c, so we can reuse most of
>> >>the code coming from the vdpa_sim framework.
>> >>
>> >>I tested in this way:
>> >>
>> >>   $ modprobe vdpa_sim
>> >>   $ modprobe vhost-vdpa
>> >>
>> >>   # load the vdpasim_test device
>> >>   $ insmod vdpa_test.ko
>> >>
>> >>   # create a new vdpasim_test device
>> >>   $ vdpa dev add mgmtdev vdpasim_test name dev0
>> >
>> >
>> >I wonder what's the benefit of using a dedicated test device other
>> >than networking? (already a loopback device anyhow).
>> >
>>
>> The main advantage I was thinking is that we can extend it at will to
>> try to cover as much as possible all the paths that maybe with net we
>> can't (indirect, reset, etc.).
>>
>> Plus it should be easier to test vhost-vdpa this way without using QEMU.
>>
>> Do you think this is redundant?
>
>It's probably fine and it might be better to describe the reason in
>the changelog.

Okay, I'll do in the next version!

Thanks,
Stefano

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization