All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] tools/virtio: Test virtual address range detection
@ 2022-02-21 16:15 ` David Woodhouse
  0 siblings, 0 replies; 24+ messages in thread
From: David Woodhouse @ 2022-02-21 16:15 UTC (permalink / raw)
  To: Michael S. Tsirkin, Jason Wang; +Cc: virtualization, linux-kernel

[-- Attachment #1: Type: text/plain, Size: 5004 bytes --]

As things stand, an application which wants to use vhost with a trivial
1:1 mapping of its virtual address space is forced to jump through hoops
to detect what the address range might be. The VHOST_SET_MEM_TABLE ioctl
helpfully doesn't fail immediately; you only get a failure *later* when
you attempt to set the backend, if the table *could* map to an address
which is out of range, even if no out-of-range address is actually
being referenced.

Since userspace is growing workarounds for this lovely kernel API, let's
ensure that we have a regression test that does things basically the same
way as https://gitlab.com/openconnect/openconnect/-/commit/443edd9d8826
does.

This is untested as I can't actually get virtio_test to work at all; it
just seems to deadlock on a spinlock. But it's getting the right answer
for the virtio range on x86_64 at least.

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
---

Please, tell me I don't need to do this. But if I *do*, it needs a
regression test in-kernel.

 tools/virtio/virtio_test.c | 109 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 106 insertions(+), 3 deletions(-)

diff --git a/tools/virtio/virtio_test.c b/tools/virtio/virtio_test.c
index cb3f29c09aff..e40eeeb05b71 100644
--- a/tools/virtio/virtio_test.c
+++ b/tools/virtio/virtio_test.c
@@ -11,6 +11,7 @@
 #include <sys/ioctl.h>
 #include <sys/stat.h>
 #include <sys/types.h>
+#include <sys/mman.h>
 #include <fcntl.h>
 #include <stdbool.h>
 #include <linux/virtio_types.h>
@@ -124,6 +125,109 @@ static void vq_info_add(struct vdev_info *dev, int num)
 	dev->nvqs++;
 }
 
+/*
+ * This is awful. The kernel doesn't let us just ask for a 1:1 mapping of
+ * our virtual address space; we have to *know* the minimum and maximum
+ * addresses. We can't test it directly with VHOST_SET_MEM_TABLE because
+ * that actually succeeds, and the failure only occurs later when we try
+ * to use a buffer at an address that *is* valid, but our memory table
+ * *could* point to addresses that aren't. Ewww.
+ *
+ * So... attempt to work out what TASK_SIZE is for the kernel we happen
+ * to be running on right now...
+ */
+
+static int testaddr(unsigned long addr)
+{
+	void *res = mmap((void *)addr, getpagesize(), PROT_NONE,
+			 MAP_FIXED|MAP_ANONYMOUS, -1, 0);
+	if (res == MAP_FAILED) {
+		if (errno == EEXIST || errno == EINVAL)
+			return 1;
+
+		/* We get ENOMEM for a bad virtual address */
+		return 0;
+	}
+	/* It shouldn't actually succeed without either MAP_SHARED or
+	 * MAP_PRIVATE in the flags, but just in case... */
+	munmap((void *)addr, getpagesize());
+	return 1;
+}
+
+static int find_vmem_range(struct vhost_memory *vmem)
+{
+	const unsigned long page_size = getpagesize();
+	unsigned long top;
+	unsigned long bottom;
+
+	top = -page_size;
+
+	if (testaddr(top)) {
+		vmem->regions[0].memory_size = top;
+		goto out;
+	}
+
+	/* 'top' is the lowest address known *not* to work */
+	bottom = top;
+	while (1) {
+		bottom >>= 1;
+		bottom &= ~(page_size - 1);
+		assert(bottom);
+
+		if (testaddr(bottom))
+			break;
+		top = bottom;
+	}
+
+	/* It's often a page or two below the boundary */
+	top -= page_size;
+	if (testaddr(top)) {
+		vmem->regions[0].memory_size = top;
+		goto out;
+	}
+	top -= page_size;
+	if (testaddr(top)) {
+		vmem->regions[0].memory_size = top;
+		goto out;
+	}
+
+	/* Now, bottom is the highest address known to work,
+	   and we must search between it and 'top' which is
+	   the lowest address known not to. */
+	while (bottom + page_size != top) {
+		unsigned long test = bottom + (top - bottom) / 2;
+		test &= ~(page_size - 1);
+
+		if (testaddr(test)) {
+			bottom = test;
+			continue;
+		}
+		test -= page_size;
+		if (testaddr(test)) {
+			vmem->regions[0].memory_size = test;
+			goto out;
+		}
+
+		test -= page_size;
+		if (testaddr(test)) {
+			vmem->regions[0].memory_size = test;
+			goto out;
+		}
+		top = test;
+	}
+	vmem->regions[0].memory_size = bottom;
+
+ out:
+	vmem->regions[0].guest_phys_addr = page_size;
+	vmem->regions[0].userspace_addr = page_size;
+	printf("Detected virtual address range 0x%lx-0x%lx\n",
+	       page_size,
+	       (unsigned long)(page_size + vmem->regions[0].memory_size));
+
+	return 0;
+}
+
+
 static void vdev_info_init(struct vdev_info* dev, unsigned long long features)
 {
 	int r;
@@ -143,9 +247,8 @@ static void vdev_info_init(struct vdev_info* dev, unsigned long long features)
 	memset(dev->mem, 0, offsetof(struct vhost_memory, regions) +
                           sizeof dev->mem->regions[0]);
 	dev->mem->nregions = 1;
-	dev->mem->regions[0].guest_phys_addr = (long)dev->buf;
-	dev->mem->regions[0].userspace_addr = (long)dev->buf;
-	dev->mem->regions[0].memory_size = dev->buf_size;
+	r = find_vmem_range(dev->mem);
+	assert(r >= 0);
 	r = ioctl(dev->control, VHOST_SET_MEM_TABLE, dev->mem);
 	assert(r >= 0);
 }



[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [PATCH] tools/virtio: Test virtual address range detection
@ 2022-02-21 16:15 ` David Woodhouse
  0 siblings, 0 replies; 24+ messages in thread
From: David Woodhouse @ 2022-02-21 16:15 UTC (permalink / raw)
  To: Michael S. Tsirkin, Jason Wang; +Cc: linux-kernel, virtualization


[-- Attachment #1.1: Type: text/plain, Size: 5004 bytes --]

As things stand, an application which wants to use vhost with a trivial
1:1 mapping of its virtual address space is forced to jump through hoops
to detect what the address range might be. The VHOST_SET_MEM_TABLE ioctl
helpfully doesn't fail immediately; you only get a failure *later* when
you attempt to set the backend, if the table *could* map to an address
which is out of range, even if no out-of-range address is actually
being referenced.

Since userspace is growing workarounds for this lovely kernel API, let's
ensure that we have a regression test that does things basically the same
way as https://gitlab.com/openconnect/openconnect/-/commit/443edd9d8826
does.

This is untested as I can't actually get virtio_test to work at all; it
just seems to deadlock on a spinlock. But it's getting the right answer
for the virtio range on x86_64 at least.

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
---

Please, tell me I don't need to do this. But if I *do*, it needs a
regression test in-kernel.

 tools/virtio/virtio_test.c | 109 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 106 insertions(+), 3 deletions(-)

diff --git a/tools/virtio/virtio_test.c b/tools/virtio/virtio_test.c
index cb3f29c09aff..e40eeeb05b71 100644
--- a/tools/virtio/virtio_test.c
+++ b/tools/virtio/virtio_test.c
@@ -11,6 +11,7 @@
 #include <sys/ioctl.h>
 #include <sys/stat.h>
 #include <sys/types.h>
+#include <sys/mman.h>
 #include <fcntl.h>
 #include <stdbool.h>
 #include <linux/virtio_types.h>
@@ -124,6 +125,109 @@ static void vq_info_add(struct vdev_info *dev, int num)
 	dev->nvqs++;
 }
 
+/*
+ * This is awful. The kernel doesn't let us just ask for a 1:1 mapping of
+ * our virtual address space; we have to *know* the minimum and maximum
+ * addresses. We can't test it directly with VHOST_SET_MEM_TABLE because
+ * that actually succeeds, and the failure only occurs later when we try
+ * to use a buffer at an address that *is* valid, but our memory table
+ * *could* point to addresses that aren't. Ewww.
+ *
+ * So... attempt to work out what TASK_SIZE is for the kernel we happen
+ * to be running on right now...
+ */
+
+static int testaddr(unsigned long addr)
+{
+	void *res = mmap((void *)addr, getpagesize(), PROT_NONE,
+			 MAP_FIXED|MAP_ANONYMOUS, -1, 0);
+	if (res == MAP_FAILED) {
+		if (errno == EEXIST || errno == EINVAL)
+			return 1;
+
+		/* We get ENOMEM for a bad virtual address */
+		return 0;
+	}
+	/* It shouldn't actually succeed without either MAP_SHARED or
+	 * MAP_PRIVATE in the flags, but just in case... */
+	munmap((void *)addr, getpagesize());
+	return 1;
+}
+
+static int find_vmem_range(struct vhost_memory *vmem)
+{
+	const unsigned long page_size = getpagesize();
+	unsigned long top;
+	unsigned long bottom;
+
+	top = -page_size;
+
+	if (testaddr(top)) {
+		vmem->regions[0].memory_size = top;
+		goto out;
+	}
+
+	/* 'top' is the lowest address known *not* to work */
+	bottom = top;
+	while (1) {
+		bottom >>= 1;
+		bottom &= ~(page_size - 1);
+		assert(bottom);
+
+		if (testaddr(bottom))
+			break;
+		top = bottom;
+	}
+
+	/* It's often a page or two below the boundary */
+	top -= page_size;
+	if (testaddr(top)) {
+		vmem->regions[0].memory_size = top;
+		goto out;
+	}
+	top -= page_size;
+	if (testaddr(top)) {
+		vmem->regions[0].memory_size = top;
+		goto out;
+	}
+
+	/* Now, bottom is the highest address known to work,
+	   and we must search between it and 'top' which is
+	   the lowest address known not to. */
+	while (bottom + page_size != top) {
+		unsigned long test = bottom + (top - bottom) / 2;
+		test &= ~(page_size - 1);
+
+		if (testaddr(test)) {
+			bottom = test;
+			continue;
+		}
+		test -= page_size;
+		if (testaddr(test)) {
+			vmem->regions[0].memory_size = test;
+			goto out;
+		}
+
+		test -= page_size;
+		if (testaddr(test)) {
+			vmem->regions[0].memory_size = test;
+			goto out;
+		}
+		top = test;
+	}
+	vmem->regions[0].memory_size = bottom;
+
+ out:
+	vmem->regions[0].guest_phys_addr = page_size;
+	vmem->regions[0].userspace_addr = page_size;
+	printf("Detected virtual address range 0x%lx-0x%lx\n",
+	       page_size,
+	       (unsigned long)(page_size + vmem->regions[0].memory_size));
+
+	return 0;
+}
+
+
 static void vdev_info_init(struct vdev_info* dev, unsigned long long features)
 {
 	int r;
@@ -143,9 +247,8 @@ static void vdev_info_init(struct vdev_info* dev, unsigned long long features)
 	memset(dev->mem, 0, offsetof(struct vhost_memory, regions) +
                           sizeof dev->mem->regions[0]);
 	dev->mem->nregions = 1;
-	dev->mem->regions[0].guest_phys_addr = (long)dev->buf;
-	dev->mem->regions[0].userspace_addr = (long)dev->buf;
-	dev->mem->regions[0].memory_size = dev->buf_size;
+	r = find_vmem_range(dev->mem);
+	assert(r >= 0);
 	r = ioctl(dev->control, VHOST_SET_MEM_TABLE, dev->mem);
 	assert(r >= 0);
 }



[-- Attachment #1.2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]

[-- Attachment #2: Type: text/plain, Size: 183 bytes --]

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
  2022-02-21 16:15 ` David Woodhouse
@ 2022-02-21 17:02   ` Stefano Garzarella
  -1 siblings, 0 replies; 24+ messages in thread
From: Stefano Garzarella @ 2022-02-21 17:02 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Michael S. Tsirkin, Jason Wang, virtualization, linux-kernel

On Mon, Feb 21, 2022 at 04:15:22PM +0000, David Woodhouse wrote:
>As things stand, an application which wants to use vhost with a trivial
>1:1 mapping of its virtual address space is forced to jump through hoops
>to detect what the address range might be. The VHOST_SET_MEM_TABLE ioctl
>helpfully doesn't fail immediately; you only get a failure *later* when
>you attempt to set the backend, if the table *could* map to an address
>which is out of range, even if no out-of-range address is actually
>being referenced.
>
>Since userspace is growing workarounds for this lovely kernel API, let's
>ensure that we have a regression test that does things basically the same
>way as https://gitlab.com/openconnect/openconnect/-/commit/443edd9d8826
>does.
>
>This is untested as I can't actually get virtio_test to work at all; it
>just seems to deadlock on a spinlock. But it's getting the right answer
>for the virtio range on x86_64 at least.

I had a similar issue with virtio_test and this simple patch [1] should 
fix the deadlock.

[1] 
https://lore.kernel.org/lkml/20220118150631.167015-1-sgarzare@redhat.com/

Stefano


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
@ 2022-02-21 17:02   ` Stefano Garzarella
  0 siblings, 0 replies; 24+ messages in thread
From: Stefano Garzarella @ 2022-02-21 17:02 UTC (permalink / raw)
  To: David Woodhouse; +Cc: virtualization, linux-kernel, Michael S. Tsirkin

On Mon, Feb 21, 2022 at 04:15:22PM +0000, David Woodhouse wrote:
>As things stand, an application which wants to use vhost with a trivial
>1:1 mapping of its virtual address space is forced to jump through hoops
>to detect what the address range might be. The VHOST_SET_MEM_TABLE ioctl
>helpfully doesn't fail immediately; you only get a failure *later* when
>you attempt to set the backend, if the table *could* map to an address
>which is out of range, even if no out-of-range address is actually
>being referenced.
>
>Since userspace is growing workarounds for this lovely kernel API, let's
>ensure that we have a regression test that does things basically the same
>way as https://gitlab.com/openconnect/openconnect/-/commit/443edd9d8826
>does.
>
>This is untested as I can't actually get virtio_test to work at all; it
>just seems to deadlock on a spinlock. But it's getting the right answer
>for the virtio range on x86_64 at least.

I had a similar issue with virtio_test and this simple patch [1] should 
fix the deadlock.

[1] 
https://lore.kernel.org/lkml/20220118150631.167015-1-sgarzare@redhat.com/

Stefano

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
  2022-02-21 17:02   ` Stefano Garzarella
@ 2022-02-21 17:18     ` David Woodhouse
  -1 siblings, 0 replies; 24+ messages in thread
From: David Woodhouse @ 2022-02-21 17:18 UTC (permalink / raw)
  To: Stefano Garzarella
  Cc: Michael S. Tsirkin, Jason Wang, virtualization, linux-kernel

[-- Attachment #1: Type: text/plain, Size: 2210 bytes --]

On Mon, 2022-02-21 at 18:02 +0100, Stefano Garzarella wrote:
> On Mon, Feb 21, 2022 at 04:15:22PM +0000, David Woodhouse wrote:
> > As things stand, an application which wants to use vhost with a trivial
> > 1:1 mapping of its virtual address space is forced to jump through hoops
> > to detect what the address range might be. The VHOST_SET_MEM_TABLE ioctl
> > helpfully doesn't fail immediately; you only get a failure *later* when
> > you attempt to set the backend, if the table *could* map to an address
> > which is out of range, even if no out-of-range address is actually
> > being referenced.
> > 
> > Since userspace is growing workarounds for this lovely kernel API, let's
> > ensure that we have a regression test that does things basically the same
> > way as 
> > https://gitlab.com/openconnect/openconnect/-/commit/443edd9d8826
> > 
> > does.
> > 
> > This is untested as I can't actually get virtio_test to work at all; it
> > just seems to deadlock on a spinlock. But it's getting the right answer
> > for the virtio range on x86_64 at least.
> 
> I had a similar issue with virtio_test and this simple patch [1] should 
> fix the deadlock.
> 
> [1] 
> https://lore.kernel.org/lkml/20220118150631.167015-1-sgarzare@redhat.com/=

Thanks.

[dwoodhou@i7 virtio]$ sudo ~/virtio_test
Detected virtual address range 0x1000-0x7ffffffff000
spurious wakeups: 0x0 started=0x100000 completed=0x100000

Although in some circumstances I also see a different build failure:

cc -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h   -c -o vringh_test.o vringh_test.c
In file included from ./linux/uio.h:3,
                 from ./linux/../../../include/linux/vringh.h:15,
                 from ./linux/vringh.h:1,
                 from vringh_test.c:9:
./linux/../../../include/linux/uio.h:10:10: fatal error: linux/mm_types.h: No such file or directory
   10 | #include <linux/mm_types.h>
      |          ^~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [<builtin>: vringh_test.o] Error 1

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
@ 2022-02-21 17:18     ` David Woodhouse
  0 siblings, 0 replies; 24+ messages in thread
From: David Woodhouse @ 2022-02-21 17:18 UTC (permalink / raw)
  To: Stefano Garzarella; +Cc: virtualization, linux-kernel, Michael S. Tsirkin


[-- Attachment #1.1: Type: text/plain, Size: 2210 bytes --]

On Mon, 2022-02-21 at 18:02 +0100, Stefano Garzarella wrote:
> On Mon, Feb 21, 2022 at 04:15:22PM +0000, David Woodhouse wrote:
> > As things stand, an application which wants to use vhost with a trivial
> > 1:1 mapping of its virtual address space is forced to jump through hoops
> > to detect what the address range might be. The VHOST_SET_MEM_TABLE ioctl
> > helpfully doesn't fail immediately; you only get a failure *later* when
> > you attempt to set the backend, if the table *could* map to an address
> > which is out of range, even if no out-of-range address is actually
> > being referenced.
> > 
> > Since userspace is growing workarounds for this lovely kernel API, let's
> > ensure that we have a regression test that does things basically the same
> > way as 
> > https://gitlab.com/openconnect/openconnect/-/commit/443edd9d8826
> > 
> > does.
> > 
> > This is untested as I can't actually get virtio_test to work at all; it
> > just seems to deadlock on a spinlock. But it's getting the right answer
> > for the virtio range on x86_64 at least.
> 
> I had a similar issue with virtio_test and this simple patch [1] should 
> fix the deadlock.
> 
> [1] 
> https://lore.kernel.org/lkml/20220118150631.167015-1-sgarzare@redhat.com/=

Thanks.

[dwoodhou@i7 virtio]$ sudo ~/virtio_test
Detected virtual address range 0x1000-0x7ffffffff000
spurious wakeups: 0x0 started=0x100000 completed=0x100000

Although in some circumstances I also see a different build failure:

cc -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h   -c -o vringh_test.o vringh_test.c
In file included from ./linux/uio.h:3,
                 from ./linux/../../../include/linux/vringh.h:15,
                 from ./linux/vringh.h:1,
                 from vringh_test.c:9:
./linux/../../../include/linux/uio.h:10:10: fatal error: linux/mm_types.h: No such file or directory
   10 | #include <linux/mm_types.h>
      |          ^~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [<builtin>: vringh_test.o] Error 1

[-- Attachment #1.2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]

[-- Attachment #2: Type: text/plain, Size: 183 bytes --]

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
  2022-02-21 16:15 ` David Woodhouse
@ 2022-02-22  3:25   ` Jason Wang
  -1 siblings, 0 replies; 24+ messages in thread
From: Jason Wang @ 2022-02-22  3:25 UTC (permalink / raw)
  To: David Woodhouse; +Cc: Michael S. Tsirkin, virtualization, linux-kernel

On Tue, Feb 22, 2022 at 12:17 AM David Woodhouse <dwmw2@infradead.org> wrote:
>
> As things stand, an application which wants to use vhost with a trivial
> 1:1 mapping of its virtual address space is forced to jump through hoops
> to detect what the address range might be. The VHOST_SET_MEM_TABLE ioctl
> helpfully doesn't fail immediately; you only get a failure *later* when
> you attempt to set the backend, if the table *could* map to an address
> which is out of range, even if no out-of-range address is actually
> being referenced.
>
> Since userspace is growing workarounds for this lovely kernel API, let's
> ensure that we have a regression test that does things basically the same
> way as https://gitlab.com/openconnect/openconnect/-/commit/443edd9d8826
> does.

I wonder if it's useful to have a small library that wraps vhost
kernel uAPI somewhere.

(In the future, we may want to let the kernel accept 1:1 mapping by
figuring out the illegal range by itself?)

Thanks

>
> This is untested as I can't actually get virtio_test to work at all; it
> just seems to deadlock on a spinlock. But it's getting the right answer
> for the virtio range on x86_64 at least.
>
> Signed-off-by: David Woodhouse <dwmw2@infradead.org>
> ---
>
> Please, tell me I don't need to do this. But if I *do*, it needs a
> regression test in-kernel.
>
>  tools/virtio/virtio_test.c | 109 ++++++++++++++++++++++++++++++++++++-
>  1 file changed, 106 insertions(+), 3 deletions(-)
>
> diff --git a/tools/virtio/virtio_test.c b/tools/virtio/virtio_test.c
> index cb3f29c09aff..e40eeeb05b71 100644
> --- a/tools/virtio/virtio_test.c
> +++ b/tools/virtio/virtio_test.c
> @@ -11,6 +11,7 @@
>  #include <sys/ioctl.h>
>  #include <sys/stat.h>
>  #include <sys/types.h>
> +#include <sys/mman.h>
>  #include <fcntl.h>
>  #include <stdbool.h>
>  #include <linux/virtio_types.h>
> @@ -124,6 +125,109 @@ static void vq_info_add(struct vdev_info *dev, int num)
>         dev->nvqs++;
>  }
>
> +/*
> + * This is awful. The kernel doesn't let us just ask for a 1:1 mapping of
> + * our virtual address space; we have to *know* the minimum and maximum
> + * addresses. We can't test it directly with VHOST_SET_MEM_TABLE because
> + * that actually succeeds, and the failure only occurs later when we try
> + * to use a buffer at an address that *is* valid, but our memory table
> + * *could* point to addresses that aren't. Ewww.
> + *
> + * So... attempt to work out what TASK_SIZE is for the kernel we happen
> + * to be running on right now...
> + */
> +
> +static int testaddr(unsigned long addr)
> +{
> +       void *res = mmap((void *)addr, getpagesize(), PROT_NONE,
> +                        MAP_FIXED|MAP_ANONYMOUS, -1, 0);
> +       if (res == MAP_FAILED) {
> +               if (errno == EEXIST || errno == EINVAL)
> +                       return 1;
> +
> +               /* We get ENOMEM for a bad virtual address */
> +               return 0;
> +       }
> +       /* It shouldn't actually succeed without either MAP_SHARED or
> +        * MAP_PRIVATE in the flags, but just in case... */
> +       munmap((void *)addr, getpagesize());
> +       return 1;
> +}
> +
> +static int find_vmem_range(struct vhost_memory *vmem)
> +{
> +       const unsigned long page_size = getpagesize();
> +       unsigned long top;
> +       unsigned long bottom;
> +
> +       top = -page_size;
> +
> +       if (testaddr(top)) {
> +               vmem->regions[0].memory_size = top;
> +               goto out;
> +       }
> +
> +       /* 'top' is the lowest address known *not* to work */
> +       bottom = top;
> +       while (1) {
> +               bottom >>= 1;
> +               bottom &= ~(page_size - 1);
> +               assert(bottom);
> +
> +               if (testaddr(bottom))
> +                       break;
> +               top = bottom;
> +       }
> +
> +       /* It's often a page or two below the boundary */
> +       top -= page_size;
> +       if (testaddr(top)) {
> +               vmem->regions[0].memory_size = top;
> +               goto out;
> +       }
> +       top -= page_size;
> +       if (testaddr(top)) {
> +               vmem->regions[0].memory_size = top;
> +               goto out;
> +       }
> +
> +       /* Now, bottom is the highest address known to work,
> +          and we must search between it and 'top' which is
> +          the lowest address known not to. */
> +       while (bottom + page_size != top) {
> +               unsigned long test = bottom + (top - bottom) / 2;
> +               test &= ~(page_size - 1);
> +
> +               if (testaddr(test)) {
> +                       bottom = test;
> +                       continue;
> +               }
> +               test -= page_size;
> +               if (testaddr(test)) {
> +                       vmem->regions[0].memory_size = test;
> +                       goto out;
> +               }
> +
> +               test -= page_size;
> +               if (testaddr(test)) {
> +                       vmem->regions[0].memory_size = test;
> +                       goto out;
> +               }
> +               top = test;
> +       }
> +       vmem->regions[0].memory_size = bottom;
> +
> + out:
> +       vmem->regions[0].guest_phys_addr = page_size;
> +       vmem->regions[0].userspace_addr = page_size;
> +       printf("Detected virtual address range 0x%lx-0x%lx\n",
> +              page_size,
> +              (unsigned long)(page_size + vmem->regions[0].memory_size));
> +
> +       return 0;
> +}
> +
> +
>  static void vdev_info_init(struct vdev_info* dev, unsigned long long features)
>  {
>         int r;
> @@ -143,9 +247,8 @@ static void vdev_info_init(struct vdev_info* dev, unsigned long long features)
>         memset(dev->mem, 0, offsetof(struct vhost_memory, regions) +
>                            sizeof dev->mem->regions[0]);
>         dev->mem->nregions = 1;
> -       dev->mem->regions[0].guest_phys_addr = (long)dev->buf;
> -       dev->mem->regions[0].userspace_addr = (long)dev->buf;
> -       dev->mem->regions[0].memory_size = dev->buf_size;
> +       r = find_vmem_range(dev->mem);
> +       assert(r >= 0);
>         r = ioctl(dev->control, VHOST_SET_MEM_TABLE, dev->mem);
>         assert(r >= 0);
>  }
>
>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
@ 2022-02-22  3:25   ` Jason Wang
  0 siblings, 0 replies; 24+ messages in thread
From: Jason Wang @ 2022-02-22  3:25 UTC (permalink / raw)
  To: David Woodhouse; +Cc: virtualization, linux-kernel, Michael S. Tsirkin

On Tue, Feb 22, 2022 at 12:17 AM David Woodhouse <dwmw2@infradead.org> wrote:
>
> As things stand, an application which wants to use vhost with a trivial
> 1:1 mapping of its virtual address space is forced to jump through hoops
> to detect what the address range might be. The VHOST_SET_MEM_TABLE ioctl
> helpfully doesn't fail immediately; you only get a failure *later* when
> you attempt to set the backend, if the table *could* map to an address
> which is out of range, even if no out-of-range address is actually
> being referenced.
>
> Since userspace is growing workarounds for this lovely kernel API, let's
> ensure that we have a regression test that does things basically the same
> way as https://gitlab.com/openconnect/openconnect/-/commit/443edd9d8826
> does.

I wonder if it's useful to have a small library that wraps vhost
kernel uAPI somewhere.

(In the future, we may want to let the kernel accept 1:1 mapping by
figuring out the illegal range by itself?)

Thanks

>
> This is untested as I can't actually get virtio_test to work at all; it
> just seems to deadlock on a spinlock. But it's getting the right answer
> for the virtio range on x86_64 at least.
>
> Signed-off-by: David Woodhouse <dwmw2@infradead.org>
> ---
>
> Please, tell me I don't need to do this. But if I *do*, it needs a
> regression test in-kernel.
>
>  tools/virtio/virtio_test.c | 109 ++++++++++++++++++++++++++++++++++++-
>  1 file changed, 106 insertions(+), 3 deletions(-)
>
> diff --git a/tools/virtio/virtio_test.c b/tools/virtio/virtio_test.c
> index cb3f29c09aff..e40eeeb05b71 100644
> --- a/tools/virtio/virtio_test.c
> +++ b/tools/virtio/virtio_test.c
> @@ -11,6 +11,7 @@
>  #include <sys/ioctl.h>
>  #include <sys/stat.h>
>  #include <sys/types.h>
> +#include <sys/mman.h>
>  #include <fcntl.h>
>  #include <stdbool.h>
>  #include <linux/virtio_types.h>
> @@ -124,6 +125,109 @@ static void vq_info_add(struct vdev_info *dev, int num)
>         dev->nvqs++;
>  }
>
> +/*
> + * This is awful. The kernel doesn't let us just ask for a 1:1 mapping of
> + * our virtual address space; we have to *know* the minimum and maximum
> + * addresses. We can't test it directly with VHOST_SET_MEM_TABLE because
> + * that actually succeeds, and the failure only occurs later when we try
> + * to use a buffer at an address that *is* valid, but our memory table
> + * *could* point to addresses that aren't. Ewww.
> + *
> + * So... attempt to work out what TASK_SIZE is for the kernel we happen
> + * to be running on right now...
> + */
> +
> +static int testaddr(unsigned long addr)
> +{
> +       void *res = mmap((void *)addr, getpagesize(), PROT_NONE,
> +                        MAP_FIXED|MAP_ANONYMOUS, -1, 0);
> +       if (res == MAP_FAILED) {
> +               if (errno == EEXIST || errno == EINVAL)
> +                       return 1;
> +
> +               /* We get ENOMEM for a bad virtual address */
> +               return 0;
> +       }
> +       /* It shouldn't actually succeed without either MAP_SHARED or
> +        * MAP_PRIVATE in the flags, but just in case... */
> +       munmap((void *)addr, getpagesize());
> +       return 1;
> +}
> +
> +static int find_vmem_range(struct vhost_memory *vmem)
> +{
> +       const unsigned long page_size = getpagesize();
> +       unsigned long top;
> +       unsigned long bottom;
> +
> +       top = -page_size;
> +
> +       if (testaddr(top)) {
> +               vmem->regions[0].memory_size = top;
> +               goto out;
> +       }
> +
> +       /* 'top' is the lowest address known *not* to work */
> +       bottom = top;
> +       while (1) {
> +               bottom >>= 1;
> +               bottom &= ~(page_size - 1);
> +               assert(bottom);
> +
> +               if (testaddr(bottom))
> +                       break;
> +               top = bottom;
> +       }
> +
> +       /* It's often a page or two below the boundary */
> +       top -= page_size;
> +       if (testaddr(top)) {
> +               vmem->regions[0].memory_size = top;
> +               goto out;
> +       }
> +       top -= page_size;
> +       if (testaddr(top)) {
> +               vmem->regions[0].memory_size = top;
> +               goto out;
> +       }
> +
> +       /* Now, bottom is the highest address known to work,
> +          and we must search between it and 'top' which is
> +          the lowest address known not to. */
> +       while (bottom + page_size != top) {
> +               unsigned long test = bottom + (top - bottom) / 2;
> +               test &= ~(page_size - 1);
> +
> +               if (testaddr(test)) {
> +                       bottom = test;
> +                       continue;
> +               }
> +               test -= page_size;
> +               if (testaddr(test)) {
> +                       vmem->regions[0].memory_size = test;
> +                       goto out;
> +               }
> +
> +               test -= page_size;
> +               if (testaddr(test)) {
> +                       vmem->regions[0].memory_size = test;
> +                       goto out;
> +               }
> +               top = test;
> +       }
> +       vmem->regions[0].memory_size = bottom;
> +
> + out:
> +       vmem->regions[0].guest_phys_addr = page_size;
> +       vmem->regions[0].userspace_addr = page_size;
> +       printf("Detected virtual address range 0x%lx-0x%lx\n",
> +              page_size,
> +              (unsigned long)(page_size + vmem->regions[0].memory_size));
> +
> +       return 0;
> +}
> +
> +
>  static void vdev_info_init(struct vdev_info* dev, unsigned long long features)
>  {
>         int r;
> @@ -143,9 +247,8 @@ static void vdev_info_init(struct vdev_info* dev, unsigned long long features)
>         memset(dev->mem, 0, offsetof(struct vhost_memory, regions) +
>                            sizeof dev->mem->regions[0]);
>         dev->mem->nregions = 1;
> -       dev->mem->regions[0].guest_phys_addr = (long)dev->buf;
> -       dev->mem->regions[0].userspace_addr = (long)dev->buf;
> -       dev->mem->regions[0].memory_size = dev->buf_size;
> +       r = find_vmem_range(dev->mem);
> +       assert(r >= 0);
>         r = ioctl(dev->control, VHOST_SET_MEM_TABLE, dev->mem);
>         assert(r >= 0);
>  }
>
>

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
  2022-02-21 17:18     ` David Woodhouse
@ 2022-02-22  6:31       ` Michael S. Tsirkin
  -1 siblings, 0 replies; 24+ messages in thread
From: Michael S. Tsirkin @ 2022-02-22  6:31 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Stefano Garzarella, Jason Wang, virtualization, linux-kernel

On Mon, Feb 21, 2022 at 05:18:48PM +0000, David Woodhouse wrote:
> On Mon, 2022-02-21 at 18:02 +0100, Stefano Garzarella wrote:
> > On Mon, Feb 21, 2022 at 04:15:22PM +0000, David Woodhouse wrote:
> > > As things stand, an application which wants to use vhost with a trivial
> > > 1:1 mapping of its virtual address space is forced to jump through hoops
> > > to detect what the address range might be. The VHOST_SET_MEM_TABLE ioctl
> > > helpfully doesn't fail immediately; you only get a failure *later* when
> > > you attempt to set the backend, if the table *could* map to an address
> > > which is out of range, even if no out-of-range address is actually
> > > being referenced.
> > > 
> > > Since userspace is growing workarounds for this lovely kernel API, let's
> > > ensure that we have a regression test that does things basically the same
> > > way as 
> > > https://gitlab.com/openconnect/openconnect/-/commit/443edd9d8826
> > > 
> > > does.
> > > 
> > > This is untested as I can't actually get virtio_test to work at all; it
> > > just seems to deadlock on a spinlock. But it's getting the right answer
> > > for the virtio range on x86_64 at least.
> > 
> > I had a similar issue with virtio_test and this simple patch [1] should 
> > fix the deadlock.
> > 
> > [1] 
> > https://lore.kernel.org/lkml/20220118150631.167015-1-sgarzare@redhat.com/=
> 
> Thanks.
> 
> [dwoodhou@i7 virtio]$ sudo ~/virtio_test
> Detected virtual address range 0x1000-0x7ffffffff000
> spurious wakeups: 0x0 started=0x100000 completed=0x100000
> 
> Although in some circumstances I also see a different build failure:
> 
> cc -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h   -c -o vringh_test.o vringh_test.c
> In file included from ./linux/uio.h:3,
>                  from ./linux/../../../include/linux/vringh.h:15,
>                  from ./linux/vringh.h:1,
>                  from vringh_test.c:9:
> ./linux/../../../include/linux/uio.h:10:10: fatal error: linux/mm_types.h: No such file or directory
>    10 | #include <linux/mm_types.h>
>       |          ^~~~~~~~~~~~~~~~~~
> compilation terminated.
> make: *** [<builtin>: vringh_test.o] Error 1

Which tree has this build failure? In mine linux/uio.h does not
include linux/mm_types.h.

-- 
MST


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
@ 2022-02-22  6:31       ` Michael S. Tsirkin
  0 siblings, 0 replies; 24+ messages in thread
From: Michael S. Tsirkin @ 2022-02-22  6:31 UTC (permalink / raw)
  To: David Woodhouse; +Cc: virtualization, linux-kernel

On Mon, Feb 21, 2022 at 05:18:48PM +0000, David Woodhouse wrote:
> On Mon, 2022-02-21 at 18:02 +0100, Stefano Garzarella wrote:
> > On Mon, Feb 21, 2022 at 04:15:22PM +0000, David Woodhouse wrote:
> > > As things stand, an application which wants to use vhost with a trivial
> > > 1:1 mapping of its virtual address space is forced to jump through hoops
> > > to detect what the address range might be. The VHOST_SET_MEM_TABLE ioctl
> > > helpfully doesn't fail immediately; you only get a failure *later* when
> > > you attempt to set the backend, if the table *could* map to an address
> > > which is out of range, even if no out-of-range address is actually
> > > being referenced.
> > > 
> > > Since userspace is growing workarounds for this lovely kernel API, let's
> > > ensure that we have a regression test that does things basically the same
> > > way as 
> > > https://gitlab.com/openconnect/openconnect/-/commit/443edd9d8826
> > > 
> > > does.
> > > 
> > > This is untested as I can't actually get virtio_test to work at all; it
> > > just seems to deadlock on a spinlock. But it's getting the right answer
> > > for the virtio range on x86_64 at least.
> > 
> > I had a similar issue with virtio_test and this simple patch [1] should 
> > fix the deadlock.
> > 
> > [1] 
> > https://lore.kernel.org/lkml/20220118150631.167015-1-sgarzare@redhat.com/=
> 
> Thanks.
> 
> [dwoodhou@i7 virtio]$ sudo ~/virtio_test
> Detected virtual address range 0x1000-0x7ffffffff000
> spurious wakeups: 0x0 started=0x100000 completed=0x100000
> 
> Although in some circumstances I also see a different build failure:
> 
> cc -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h   -c -o vringh_test.o vringh_test.c
> In file included from ./linux/uio.h:3,
>                  from ./linux/../../../include/linux/vringh.h:15,
>                  from ./linux/vringh.h:1,
>                  from vringh_test.c:9:
> ./linux/../../../include/linux/uio.h:10:10: fatal error: linux/mm_types.h: No such file or directory
>    10 | #include <linux/mm_types.h>
>       |          ^~~~~~~~~~~~~~~~~~
> compilation terminated.
> make: *** [<builtin>: vringh_test.o] Error 1

Which tree has this build failure? In mine linux/uio.h does not
include linux/mm_types.h.

-- 
MST

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
  2022-02-22  6:31       ` Michael S. Tsirkin
@ 2022-02-22  7:58         ` David Woodhouse
  -1 siblings, 0 replies; 24+ messages in thread
From: David Woodhouse @ 2022-02-22  7:58 UTC (permalink / raw)
  To: Michael S. Tsirkin, Matthew Wilcox (Oracle)
  Cc: Stefano Garzarella, Jason Wang, virtualization, linux-kernel

[-- Attachment #1: Type: text/plain, Size: 1608 bytes --]

On Tue, 2022-02-22 at 01:31 -0500, Michael S. Tsirkin wrote:
> On Mon, Feb 21, 2022 at 05:18:48PM +0000, David Woodhouse wrote:
> > 
> > [dwoodhou@i7 virtio]$ sudo ~/virtio_test
> > Detected virtual address range 0x1000-0x7ffffffff000
> > spurious wakeups: 0x0 started=0x100000 completed=0x100000
> > 
> > Although in some circumstances I also see a different build failure:
> > 
> > cc -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h   -c -o vringh_test.o vringh_test.c
> > In file included from ./linux/uio.h:3,
> >                  from ./linux/../../../include/linux/vringh.h:15,
> >                  from ./linux/vringh.h:1,
> >                  from vringh_test.c:9:
> > ./linux/../../../include/linux/uio.h:10:10: fatal error: linux/mm_types.h: No such file or directory
> >    10 | #include <linux/mm_types.h>
> >       |          ^~~~~~~~~~~~~~~~~~
> > compilation terminated.
> > make: *** [<builtin>: vringh_test.o] Error 1
> 
> Which tree has this build failure? In mine linux/uio.h does not
> include linux/mm_types.h.

Strictly it's
https://git.infradead.org/users/dwmw2/linux.git/shortlog/refs/heads/xen-evtchn-kernel
but I'm sure my part isn't relevant; it's just v5.17-rc5.

 $ git blame include/linux/uio.h | grep mm_types.h
d9c19d32d86fa (Matthew Wilcox (Oracle) 2021-10-18 10:39:06 -0400  10) #include <linux/mm_types.h>
 $ git describe --tags d9c19d32d86fa
v5.16-rc4-37-gd9c19d32d86f

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
@ 2022-02-22  7:58         ` David Woodhouse
  0 siblings, 0 replies; 24+ messages in thread
From: David Woodhouse @ 2022-02-22  7:58 UTC (permalink / raw)
  To: Michael S. Tsirkin, Matthew Wilcox (Oracle); +Cc: virtualization, linux-kernel


[-- Attachment #1.1: Type: text/plain, Size: 1608 bytes --]

On Tue, 2022-02-22 at 01:31 -0500, Michael S. Tsirkin wrote:
> On Mon, Feb 21, 2022 at 05:18:48PM +0000, David Woodhouse wrote:
> > 
> > [dwoodhou@i7 virtio]$ sudo ~/virtio_test
> > Detected virtual address range 0x1000-0x7ffffffff000
> > spurious wakeups: 0x0 started=0x100000 completed=0x100000
> > 
> > Although in some circumstances I also see a different build failure:
> > 
> > cc -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h   -c -o vringh_test.o vringh_test.c
> > In file included from ./linux/uio.h:3,
> >                  from ./linux/../../../include/linux/vringh.h:15,
> >                  from ./linux/vringh.h:1,
> >                  from vringh_test.c:9:
> > ./linux/../../../include/linux/uio.h:10:10: fatal error: linux/mm_types.h: No such file or directory
> >    10 | #include <linux/mm_types.h>
> >       |          ^~~~~~~~~~~~~~~~~~
> > compilation terminated.
> > make: *** [<builtin>: vringh_test.o] Error 1
> 
> Which tree has this build failure? In mine linux/uio.h does not
> include linux/mm_types.h.

Strictly it's
https://git.infradead.org/users/dwmw2/linux.git/shortlog/refs/heads/xen-evtchn-kernel
but I'm sure my part isn't relevant; it's just v5.17-rc5.

 $ git blame include/linux/uio.h | grep mm_types.h
d9c19d32d86fa (Matthew Wilcox (Oracle) 2021-10-18 10:39:06 -0400  10) #include <linux/mm_types.h>
 $ git describe --tags d9c19d32d86fa
v5.16-rc4-37-gd9c19d32d86f

[-- Attachment #1.2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]

[-- Attachment #2: Type: text/plain, Size: 183 bytes --]

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
  2022-02-22  7:58         ` David Woodhouse
@ 2022-02-22 23:18           ` Matthew Wilcox
  -1 siblings, 0 replies; 24+ messages in thread
From: Matthew Wilcox @ 2022-02-22 23:18 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Michael S. Tsirkin, Stefano Garzarella, Jason Wang,
	virtualization, linux-kernel

On Tue, Feb 22, 2022 at 07:58:33AM +0000, David Woodhouse wrote:
> On Tue, 2022-02-22 at 01:31 -0500, Michael S. Tsirkin wrote:
> > On Mon, Feb 21, 2022 at 05:18:48PM +0000, David Woodhouse wrote:
> > > 
> > > [dwoodhou@i7 virtio]$ sudo ~/virtio_test
> > > Detected virtual address range 0x1000-0x7ffffffff000
> > > spurious wakeups: 0x0 started=0x100000 completed=0x100000
> > > 
> > > Although in some circumstances I also see a different build failure:
> > > 
> > > cc -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h   -c -o vringh_test.o vringh_test.c

Trying to test this myself ...

$ cd tools/virtio/
$ make
...
cc -lpthread  virtio_test.o virtio_ring.o   -o virtio_test
/usr/bin/ld: virtio_ring.o: in function `spin_lock':
/home/willy/kernel/folio/tools/virtio/./linux/spinlock.h:16: undefined reference to `pthread_spin_lock'

So this is not the only problem here?

> > > In file included from ./linux/uio.h:3,
> > >                  from ./linux/../../../include/linux/vringh.h:15,
> > >                  from ./linux/vringh.h:1,
> > >                  from vringh_test.c:9:
> > > ./linux/../../../include/linux/uio.h:10:10: fatal error: linux/mm_types.h: No such file or directory
> > >    10 | #include <linux/mm_types.h>
> > >       |          ^~~~~~~~~~~~~~~~~~
> > > compilation terminated.
> > > make: *** [<builtin>: vringh_test.o] Error 1
> > 
> > Which tree has this build failure? In mine linux/uio.h does not
> > include linux/mm_types.h.
> 
> Strictly it's
> https://git.infradead.org/users/dwmw2/linux.git/shortlog/refs/heads/xen-evtchn-kernel
> but I'm sure my part isn't relevant; it's just v5.17-rc5.
> 
>  $ git blame include/linux/uio.h | grep mm_types.h
> d9c19d32d86fa (Matthew Wilcox (Oracle) 2021-10-18 10:39:06 -0400  10) #include <linux/mm_types.h>
>  $ git describe --tags d9c19d32d86fa
> v5.16-rc4-37-gd9c19d32d86f

grr.  Originally, I had this doing a typebusting cast, but hch objected,
so I had to include mm_types.h.  This should fix it ...

$ git diff
diff --git a/tools/virtio/linux/mm_types.h b/tools/virtio/linux/mm_types.h
new file mode 100644
index 000000000000..3b0fc9bc5b8f
--- /dev/null
+++ b/tools/virtio/linux/mm_types.h
@@ -0,0 +1,3 @@
+struct folio {
+       struct page page;
+};

At least, it makes it compile for me.

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
@ 2022-02-22 23:18           ` Matthew Wilcox
  0 siblings, 0 replies; 24+ messages in thread
From: Matthew Wilcox @ 2022-02-22 23:18 UTC (permalink / raw)
  To: David Woodhouse; +Cc: virtualization, linux-kernel, Michael S. Tsirkin

On Tue, Feb 22, 2022 at 07:58:33AM +0000, David Woodhouse wrote:
> On Tue, 2022-02-22 at 01:31 -0500, Michael S. Tsirkin wrote:
> > On Mon, Feb 21, 2022 at 05:18:48PM +0000, David Woodhouse wrote:
> > > 
> > > [dwoodhou@i7 virtio]$ sudo ~/virtio_test
> > > Detected virtual address range 0x1000-0x7ffffffff000
> > > spurious wakeups: 0x0 started=0x100000 completed=0x100000
> > > 
> > > Although in some circumstances I also see a different build failure:
> > > 
> > > cc -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h   -c -o vringh_test.o vringh_test.c

Trying to test this myself ...

$ cd tools/virtio/
$ make
...
cc -lpthread  virtio_test.o virtio_ring.o   -o virtio_test
/usr/bin/ld: virtio_ring.o: in function `spin_lock':
/home/willy/kernel/folio/tools/virtio/./linux/spinlock.h:16: undefined reference to `pthread_spin_lock'

So this is not the only problem here?

> > > In file included from ./linux/uio.h:3,
> > >                  from ./linux/../../../include/linux/vringh.h:15,
> > >                  from ./linux/vringh.h:1,
> > >                  from vringh_test.c:9:
> > > ./linux/../../../include/linux/uio.h:10:10: fatal error: linux/mm_types.h: No such file or directory
> > >    10 | #include <linux/mm_types.h>
> > >       |          ^~~~~~~~~~~~~~~~~~
> > > compilation terminated.
> > > make: *** [<builtin>: vringh_test.o] Error 1
> > 
> > Which tree has this build failure? In mine linux/uio.h does not
> > include linux/mm_types.h.
> 
> Strictly it's
> https://git.infradead.org/users/dwmw2/linux.git/shortlog/refs/heads/xen-evtchn-kernel
> but I'm sure my part isn't relevant; it's just v5.17-rc5.
> 
>  $ git blame include/linux/uio.h | grep mm_types.h
> d9c19d32d86fa (Matthew Wilcox (Oracle) 2021-10-18 10:39:06 -0400  10) #include <linux/mm_types.h>
>  $ git describe --tags d9c19d32d86fa
> v5.16-rc4-37-gd9c19d32d86f

grr.  Originally, I had this doing a typebusting cast, but hch objected,
so I had to include mm_types.h.  This should fix it ...

$ git diff
diff --git a/tools/virtio/linux/mm_types.h b/tools/virtio/linux/mm_types.h
new file mode 100644
index 000000000000..3b0fc9bc5b8f
--- /dev/null
+++ b/tools/virtio/linux/mm_types.h
@@ -0,0 +1,3 @@
+struct folio {
+       struct page page;
+};

At least, it makes it compile for me.
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
  2022-02-22 23:18           ` Matthew Wilcox
@ 2022-02-25 15:48             ` Matthew Wilcox
  -1 siblings, 0 replies; 24+ messages in thread
From: Matthew Wilcox @ 2022-02-25 15:48 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Michael S. Tsirkin, Stefano Garzarella, Jason Wang,
	virtualization, linux-kernel

On Tue, Feb 22, 2022 at 11:18:18PM +0000, Matthew Wilcox wrote:
> On Tue, Feb 22, 2022 at 07:58:33AM +0000, David Woodhouse wrote:
> > On Tue, 2022-02-22 at 01:31 -0500, Michael S. Tsirkin wrote:
> > > On Mon, Feb 21, 2022 at 05:18:48PM +0000, David Woodhouse wrote:
> > > > 
> > > > [dwoodhou@i7 virtio]$ sudo ~/virtio_test
> > > > Detected virtual address range 0x1000-0x7ffffffff000
> > > > spurious wakeups: 0x0 started=0x100000 completed=0x100000
> > > > 
> > > > Although in some circumstances I also see a different build failure:
> > > > 
> > > > cc -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h   -c -o vringh_test.o vringh_test.c
> 
> Trying to test this myself ...
> 
> $ cd tools/virtio/
> $ make
> ...
> cc -lpthread  virtio_test.o virtio_ring.o   -o virtio_test
> /usr/bin/ld: virtio_ring.o: in function `spin_lock':
> /home/willy/kernel/folio/tools/virtio/./linux/spinlock.h:16: undefined reference to `pthread_spin_lock'
> 
> So this is not the only problem here?
> 
> > > > In file included from ./linux/uio.h:3,
> > > >                  from ./linux/../../../include/linux/vringh.h:15,
> > > >                  from ./linux/vringh.h:1,
> > > >                  from vringh_test.c:9:
> > > > ./linux/../../../include/linux/uio.h:10:10: fatal error: linux/mm_types.h: No such file or directory
> > > >    10 | #include <linux/mm_types.h>
> > > >       |          ^~~~~~~~~~~~~~~~~~
> > > > compilation terminated.
> > > > make: *** [<builtin>: vringh_test.o] Error 1
> > > 
> > > Which tree has this build failure? In mine linux/uio.h does not
> > > include linux/mm_types.h.
> > 
> > Strictly it's
> > https://git.infradead.org/users/dwmw2/linux.git/shortlog/refs/heads/xen-evtchn-kernel
> > but I'm sure my part isn't relevant; it's just v5.17-rc5.
> > 
> >  $ git blame include/linux/uio.h | grep mm_types.h
> > d9c19d32d86fa (Matthew Wilcox (Oracle) 2021-10-18 10:39:06 -0400  10) #include <linux/mm_types.h>
> >  $ git describe --tags d9c19d32d86fa
> > v5.16-rc4-37-gd9c19d32d86f
> 
> grr.  Originally, I had this doing a typebusting cast, but hch objected,
> so I had to include mm_types.h.  This should fix it ...

ping?  Just noticed this one crop up in a "list of problems".  Should
I submit it myself?

> $ git diff
> diff --git a/tools/virtio/linux/mm_types.h b/tools/virtio/linux/mm_types.h
> new file mode 100644
> index 000000000000..3b0fc9bc5b8f
> --- /dev/null
> +++ b/tools/virtio/linux/mm_types.h
> @@ -0,0 +1,3 @@
> +struct folio {
> +       struct page page;
> +};
> 
> At least, it makes it compile for me.

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
@ 2022-02-25 15:48             ` Matthew Wilcox
  0 siblings, 0 replies; 24+ messages in thread
From: Matthew Wilcox @ 2022-02-25 15:48 UTC (permalink / raw)
  To: David Woodhouse; +Cc: virtualization, linux-kernel, Michael S. Tsirkin

On Tue, Feb 22, 2022 at 11:18:18PM +0000, Matthew Wilcox wrote:
> On Tue, Feb 22, 2022 at 07:58:33AM +0000, David Woodhouse wrote:
> > On Tue, 2022-02-22 at 01:31 -0500, Michael S. Tsirkin wrote:
> > > On Mon, Feb 21, 2022 at 05:18:48PM +0000, David Woodhouse wrote:
> > > > 
> > > > [dwoodhou@i7 virtio]$ sudo ~/virtio_test
> > > > Detected virtual address range 0x1000-0x7ffffffff000
> > > > spurious wakeups: 0x0 started=0x100000 completed=0x100000
> > > > 
> > > > Although in some circumstances I also see a different build failure:
> > > > 
> > > > cc -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h   -c -o vringh_test.o vringh_test.c
> 
> Trying to test this myself ...
> 
> $ cd tools/virtio/
> $ make
> ...
> cc -lpthread  virtio_test.o virtio_ring.o   -o virtio_test
> /usr/bin/ld: virtio_ring.o: in function `spin_lock':
> /home/willy/kernel/folio/tools/virtio/./linux/spinlock.h:16: undefined reference to `pthread_spin_lock'
> 
> So this is not the only problem here?
> 
> > > > In file included from ./linux/uio.h:3,
> > > >                  from ./linux/../../../include/linux/vringh.h:15,
> > > >                  from ./linux/vringh.h:1,
> > > >                  from vringh_test.c:9:
> > > > ./linux/../../../include/linux/uio.h:10:10: fatal error: linux/mm_types.h: No such file or directory
> > > >    10 | #include <linux/mm_types.h>
> > > >       |          ^~~~~~~~~~~~~~~~~~
> > > > compilation terminated.
> > > > make: *** [<builtin>: vringh_test.o] Error 1
> > > 
> > > Which tree has this build failure? In mine linux/uio.h does not
> > > include linux/mm_types.h.
> > 
> > Strictly it's
> > https://git.infradead.org/users/dwmw2/linux.git/shortlog/refs/heads/xen-evtchn-kernel
> > but I'm sure my part isn't relevant; it's just v5.17-rc5.
> > 
> >  $ git blame include/linux/uio.h | grep mm_types.h
> > d9c19d32d86fa (Matthew Wilcox (Oracle) 2021-10-18 10:39:06 -0400  10) #include <linux/mm_types.h>
> >  $ git describe --tags d9c19d32d86fa
> > v5.16-rc4-37-gd9c19d32d86f
> 
> grr.  Originally, I had this doing a typebusting cast, but hch objected,
> so I had to include mm_types.h.  This should fix it ...

ping?  Just noticed this one crop up in a "list of problems".  Should
I submit it myself?

> $ git diff
> diff --git a/tools/virtio/linux/mm_types.h b/tools/virtio/linux/mm_types.h
> new file mode 100644
> index 000000000000..3b0fc9bc5b8f
> --- /dev/null
> +++ b/tools/virtio/linux/mm_types.h
> @@ -0,0 +1,3 @@
> +struct folio {
> +       struct page page;
> +};
> 
> At least, it makes it compile for me.
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
  2022-02-25 15:48             ` Matthew Wilcox
@ 2022-02-25 18:25               ` Michael S. Tsirkin
  -1 siblings, 0 replies; 24+ messages in thread
From: Michael S. Tsirkin @ 2022-02-25 18:25 UTC (permalink / raw)
  To: Matthew Wilcox
  Cc: David Woodhouse, Stefano Garzarella, Jason Wang, virtualization,
	linux-kernel

On Fri, Feb 25, 2022 at 03:48:58PM +0000, Matthew Wilcox wrote:
> On Tue, Feb 22, 2022 at 11:18:18PM +0000, Matthew Wilcox wrote:
> > On Tue, Feb 22, 2022 at 07:58:33AM +0000, David Woodhouse wrote:
> > > On Tue, 2022-02-22 at 01:31 -0500, Michael S. Tsirkin wrote:
> > > > On Mon, Feb 21, 2022 at 05:18:48PM +0000, David Woodhouse wrote:
> > > > > 
> > > > > [dwoodhou@i7 virtio]$ sudo ~/virtio_test
> > > > > Detected virtual address range 0x1000-0x7ffffffff000
> > > > > spurious wakeups: 0x0 started=0x100000 completed=0x100000
> > > > > 
> > > > > Although in some circumstances I also see a different build failure:
> > > > > 
> > > > > cc -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h   -c -o vringh_test.o vringh_test.c
> > 
> > Trying to test this myself ...
> > 
> > $ cd tools/virtio/
> > $ make
> > ...
> > cc -lpthread  virtio_test.o virtio_ring.o   -o virtio_test
> > /usr/bin/ld: virtio_ring.o: in function `spin_lock':
> > /home/willy/kernel/folio/tools/virtio/./linux/spinlock.h:16: undefined reference to `pthread_spin_lock'
> > 
> > So this is not the only problem here?
> > 
> > > > > In file included from ./linux/uio.h:3,
> > > > >                  from ./linux/../../../include/linux/vringh.h:15,
> > > > >                  from ./linux/vringh.h:1,
> > > > >                  from vringh_test.c:9:
> > > > > ./linux/../../../include/linux/uio.h:10:10: fatal error: linux/mm_types.h: No such file or directory
> > > > >    10 | #include <linux/mm_types.h>
> > > > >       |          ^~~~~~~~~~~~~~~~~~
> > > > > compilation terminated.
> > > > > make: *** [<builtin>: vringh_test.o] Error 1
> > > > 
> > > > Which tree has this build failure? In mine linux/uio.h does not
> > > > include linux/mm_types.h.
> > > 
> > > Strictly it's
> > > https://git.infradead.org/users/dwmw2/linux.git/shortlog/refs/heads/xen-evtchn-kernel
> > > but I'm sure my part isn't relevant; it's just v5.17-rc5.
> > > 
> > >  $ git blame include/linux/uio.h | grep mm_types.h
> > > d9c19d32d86fa (Matthew Wilcox (Oracle) 2021-10-18 10:39:06 -0400  10) #include <linux/mm_types.h>
> > >  $ git describe --tags d9c19d32d86fa
> > > v5.16-rc4-37-gd9c19d32d86f
> > 
> > grr.  Originally, I had this doing a typebusting cast, but hch objected,
> > so I had to include mm_types.h.  This should fix it ...
> 
> ping?  Just noticed this one crop up in a "list of problems".  Should
> I submit it myself?

Pls do.

> > $ git diff
> > diff --git a/tools/virtio/linux/mm_types.h b/tools/virtio/linux/mm_types.h
> > new file mode 100644
> > index 000000000000..3b0fc9bc5b8f
> > --- /dev/null
> > +++ b/tools/virtio/linux/mm_types.h
> > @@ -0,0 +1,3 @@
> > +struct folio {
> > +       struct page page;
> > +};
> > 
> > At least, it makes it compile for me.


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
@ 2022-02-25 18:25               ` Michael S. Tsirkin
  0 siblings, 0 replies; 24+ messages in thread
From: Michael S. Tsirkin @ 2022-02-25 18:25 UTC (permalink / raw)
  To: Matthew Wilcox; +Cc: virtualization, David Woodhouse, linux-kernel

On Fri, Feb 25, 2022 at 03:48:58PM +0000, Matthew Wilcox wrote:
> On Tue, Feb 22, 2022 at 11:18:18PM +0000, Matthew Wilcox wrote:
> > On Tue, Feb 22, 2022 at 07:58:33AM +0000, David Woodhouse wrote:
> > > On Tue, 2022-02-22 at 01:31 -0500, Michael S. Tsirkin wrote:
> > > > On Mon, Feb 21, 2022 at 05:18:48PM +0000, David Woodhouse wrote:
> > > > > 
> > > > > [dwoodhou@i7 virtio]$ sudo ~/virtio_test
> > > > > Detected virtual address range 0x1000-0x7ffffffff000
> > > > > spurious wakeups: 0x0 started=0x100000 completed=0x100000
> > > > > 
> > > > > Although in some circumstances I also see a different build failure:
> > > > > 
> > > > > cc -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h   -c -o vringh_test.o vringh_test.c
> > 
> > Trying to test this myself ...
> > 
> > $ cd tools/virtio/
> > $ make
> > ...
> > cc -lpthread  virtio_test.o virtio_ring.o   -o virtio_test
> > /usr/bin/ld: virtio_ring.o: in function `spin_lock':
> > /home/willy/kernel/folio/tools/virtio/./linux/spinlock.h:16: undefined reference to `pthread_spin_lock'
> > 
> > So this is not the only problem here?
> > 
> > > > > In file included from ./linux/uio.h:3,
> > > > >                  from ./linux/../../../include/linux/vringh.h:15,
> > > > >                  from ./linux/vringh.h:1,
> > > > >                  from vringh_test.c:9:
> > > > > ./linux/../../../include/linux/uio.h:10:10: fatal error: linux/mm_types.h: No such file or directory
> > > > >    10 | #include <linux/mm_types.h>
> > > > >       |          ^~~~~~~~~~~~~~~~~~
> > > > > compilation terminated.
> > > > > make: *** [<builtin>: vringh_test.o] Error 1
> > > > 
> > > > Which tree has this build failure? In mine linux/uio.h does not
> > > > include linux/mm_types.h.
> > > 
> > > Strictly it's
> > > https://git.infradead.org/users/dwmw2/linux.git/shortlog/refs/heads/xen-evtchn-kernel
> > > but I'm sure my part isn't relevant; it's just v5.17-rc5.
> > > 
> > >  $ git blame include/linux/uio.h | grep mm_types.h
> > > d9c19d32d86fa (Matthew Wilcox (Oracle) 2021-10-18 10:39:06 -0400  10) #include <linux/mm_types.h>
> > >  $ git describe --tags d9c19d32d86fa
> > > v5.16-rc4-37-gd9c19d32d86f
> > 
> > grr.  Originally, I had this doing a typebusting cast, but hch objected,
> > so I had to include mm_types.h.  This should fix it ...
> 
> ping?  Just noticed this one crop up in a "list of problems".  Should
> I submit it myself?

Pls do.

> > $ git diff
> > diff --git a/tools/virtio/linux/mm_types.h b/tools/virtio/linux/mm_types.h
> > new file mode 100644
> > index 000000000000..3b0fc9bc5b8f
> > --- /dev/null
> > +++ b/tools/virtio/linux/mm_types.h
> > @@ -0,0 +1,3 @@
> > +struct folio {
> > +       struct page page;
> > +};
> > 
> > At least, it makes it compile for me.

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
  2022-02-22 23:18           ` Matthew Wilcox
@ 2022-03-16 14:41             ` Matthew Wilcox
  -1 siblings, 0 replies; 24+ messages in thread
From: Matthew Wilcox @ 2022-03-16 14:41 UTC (permalink / raw)
  To: David Woodhouse; +Cc: virtualization, linux-kernel, Michael S. Tsirkin

On Tue, Feb 22, 2022 at 11:18:18PM +0000, Matthew Wilcox wrote:
> On Tue, Feb 22, 2022 at 07:58:33AM +0000, David Woodhouse wrote:
> > On Tue, 2022-02-22 at 01:31 -0500, Michael S. Tsirkin wrote:
> > > On Mon, Feb 21, 2022 at 05:18:48PM +0000, David Woodhouse wrote:
> > > > 
> > > > [dwoodhou@i7 virtio]$ sudo ~/virtio_test
> > > > Detected virtual address range 0x1000-0x7ffffffff000
> > > > spurious wakeups: 0x0 started=0x100000 completed=0x100000
> > > > 
> > > > Although in some circumstances I also see a different build failure:
> > > > 
> > > > cc -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h   -c -o vringh_test.o vringh_test.c
> 
> Trying to test this myself ...
> 
> $ cd tools/virtio/
> $ make
> ...
> cc -lpthread  virtio_test.o virtio_ring.o   -o virtio_test
> /usr/bin/ld: virtio_ring.o: in function `spin_lock':
> /home/willy/kernel/folio/tools/virtio/./linux/spinlock.h:16: undefined reference to `pthread_spin_lock'
> 
> So this is not the only problem here?

FYI, this fixes it for me:

diff --git a/tools/virtio/Makefile b/tools/virtio/Makefile
index 0d7bbe49359d..83b6a522d0d2 100644
--- a/tools/virtio/Makefile
+++ b/tools/virtio/Makefile
@@ -5,7 +5,7 @@ virtio_test: virtio_ring.o virtio_test.o
 vringh_test: vringh_test.o vringh.o virtio_ring.o

 CFLAGS += -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h
-LDFLAGS += -lpthread
+LDFLAGS += -pthread
 vpath %.c ../../drivers/virtio ../../drivers/vhost
 mod:
        ${MAKE} -C `pwd`/../.. M=`pwd`/vhost_test V=${V}

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
@ 2022-03-16 14:41             ` Matthew Wilcox
  0 siblings, 0 replies; 24+ messages in thread
From: Matthew Wilcox @ 2022-03-16 14:41 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Michael S. Tsirkin, Stefano Garzarella, Jason Wang,
	virtualization, linux-kernel

On Tue, Feb 22, 2022 at 11:18:18PM +0000, Matthew Wilcox wrote:
> On Tue, Feb 22, 2022 at 07:58:33AM +0000, David Woodhouse wrote:
> > On Tue, 2022-02-22 at 01:31 -0500, Michael S. Tsirkin wrote:
> > > On Mon, Feb 21, 2022 at 05:18:48PM +0000, David Woodhouse wrote:
> > > > 
> > > > [dwoodhou@i7 virtio]$ sudo ~/virtio_test
> > > > Detected virtual address range 0x1000-0x7ffffffff000
> > > > spurious wakeups: 0x0 started=0x100000 completed=0x100000
> > > > 
> > > > Although in some circumstances I also see a different build failure:
> > > > 
> > > > cc -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h   -c -o vringh_test.o vringh_test.c
> 
> Trying to test this myself ...
> 
> $ cd tools/virtio/
> $ make
> ...
> cc -lpthread  virtio_test.o virtio_ring.o   -o virtio_test
> /usr/bin/ld: virtio_ring.o: in function `spin_lock':
> /home/willy/kernel/folio/tools/virtio/./linux/spinlock.h:16: undefined reference to `pthread_spin_lock'
> 
> So this is not the only problem here?

FYI, this fixes it for me:

diff --git a/tools/virtio/Makefile b/tools/virtio/Makefile
index 0d7bbe49359d..83b6a522d0d2 100644
--- a/tools/virtio/Makefile
+++ b/tools/virtio/Makefile
@@ -5,7 +5,7 @@ virtio_test: virtio_ring.o virtio_test.o
 vringh_test: vringh_test.o vringh.o virtio_ring.o

 CFLAGS += -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h
-LDFLAGS += -lpthread
+LDFLAGS += -pthread
 vpath %.c ../../drivers/virtio ../../drivers/vhost
 mod:
        ${MAKE} -C `pwd`/../.. M=`pwd`/vhost_test V=${V}


^ permalink raw reply related	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
  2022-03-16 14:41             ` Matthew Wilcox
@ 2022-03-20 11:01               ` Michael S. Tsirkin
  -1 siblings, 0 replies; 24+ messages in thread
From: Michael S. Tsirkin @ 2022-03-20 11:01 UTC (permalink / raw)
  To: Matthew Wilcox
  Cc: David Woodhouse, Stefano Garzarella, Jason Wang, virtualization,
	linux-kernel

On Wed, Mar 16, 2022 at 02:41:35PM +0000, Matthew Wilcox wrote:
> On Tue, Feb 22, 2022 at 11:18:18PM +0000, Matthew Wilcox wrote:
> > On Tue, Feb 22, 2022 at 07:58:33AM +0000, David Woodhouse wrote:
> > > On Tue, 2022-02-22 at 01:31 -0500, Michael S. Tsirkin wrote:
> > > > On Mon, Feb 21, 2022 at 05:18:48PM +0000, David Woodhouse wrote:
> > > > > 
> > > > > [dwoodhou@i7 virtio]$ sudo ~/virtio_test
> > > > > Detected virtual address range 0x1000-0x7ffffffff000
> > > > > spurious wakeups: 0x0 started=0x100000 completed=0x100000
> > > > > 
> > > > > Although in some circumstances I also see a different build failure:
> > > > > 
> > > > > cc -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h   -c -o vringh_test.o vringh_test.c
> > 
> > Trying to test this myself ...
> > 
> > $ cd tools/virtio/
> > $ make
> > ...
> > cc -lpthread  virtio_test.o virtio_ring.o   -o virtio_test
> > /usr/bin/ld: virtio_ring.o: in function `spin_lock':
> > /home/willy/kernel/folio/tools/virtio/./linux/spinlock.h:16: undefined reference to `pthread_spin_lock'
> > 
> > So this is not the only problem here?
> 
> FYI, this fixes it for me:
> 
> diff --git a/tools/virtio/Makefile b/tools/virtio/Makefile
> index 0d7bbe49359d..83b6a522d0d2 100644
> --- a/tools/virtio/Makefile
> +++ b/tools/virtio/Makefile
> @@ -5,7 +5,7 @@ virtio_test: virtio_ring.o virtio_test.o
>  vringh_test: vringh_test.o vringh.o virtio_ring.o
> 
>  CFLAGS += -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h
> -LDFLAGS += -lpthread
> +LDFLAGS += -pthread
>  vpath %.c ../../drivers/virtio ../../drivers/vhost
>  mod:
>         ${MAKE} -C `pwd`/../.. M=`pwd`/vhost_test V=${V}

gcc documentation seems to say this is neeed in cflags too:

       -pthread
           Define additional macros required for using the POSIX threads library.  You should use this option consistently for both
           compilation and linking.  This option is supported on GNU/Linux targets, most other Unix derivatives, and also on x86
           Cygwin and MinGW targets.

right?

-- 
MST


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
@ 2022-03-20 11:01               ` Michael S. Tsirkin
  0 siblings, 0 replies; 24+ messages in thread
From: Michael S. Tsirkin @ 2022-03-20 11:01 UTC (permalink / raw)
  To: Matthew Wilcox; +Cc: virtualization, David Woodhouse, linux-kernel

On Wed, Mar 16, 2022 at 02:41:35PM +0000, Matthew Wilcox wrote:
> On Tue, Feb 22, 2022 at 11:18:18PM +0000, Matthew Wilcox wrote:
> > On Tue, Feb 22, 2022 at 07:58:33AM +0000, David Woodhouse wrote:
> > > On Tue, 2022-02-22 at 01:31 -0500, Michael S. Tsirkin wrote:
> > > > On Mon, Feb 21, 2022 at 05:18:48PM +0000, David Woodhouse wrote:
> > > > > 
> > > > > [dwoodhou@i7 virtio]$ sudo ~/virtio_test
> > > > > Detected virtual address range 0x1000-0x7ffffffff000
> > > > > spurious wakeups: 0x0 started=0x100000 completed=0x100000
> > > > > 
> > > > > Although in some circumstances I also see a different build failure:
> > > > > 
> > > > > cc -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h   -c -o vringh_test.o vringh_test.c
> > 
> > Trying to test this myself ...
> > 
> > $ cd tools/virtio/
> > $ make
> > ...
> > cc -lpthread  virtio_test.o virtio_ring.o   -o virtio_test
> > /usr/bin/ld: virtio_ring.o: in function `spin_lock':
> > /home/willy/kernel/folio/tools/virtio/./linux/spinlock.h:16: undefined reference to `pthread_spin_lock'
> > 
> > So this is not the only problem here?
> 
> FYI, this fixes it for me:
> 
> diff --git a/tools/virtio/Makefile b/tools/virtio/Makefile
> index 0d7bbe49359d..83b6a522d0d2 100644
> --- a/tools/virtio/Makefile
> +++ b/tools/virtio/Makefile
> @@ -5,7 +5,7 @@ virtio_test: virtio_ring.o virtio_test.o
>  vringh_test: vringh_test.o vringh.o virtio_ring.o
> 
>  CFLAGS += -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h
> -LDFLAGS += -lpthread
> +LDFLAGS += -pthread
>  vpath %.c ../../drivers/virtio ../../drivers/vhost
>  mod:
>         ${MAKE} -C `pwd`/../.. M=`pwd`/vhost_test V=${V}

gcc documentation seems to say this is neeed in cflags too:

       -pthread
           Define additional macros required for using the POSIX threads library.  You should use this option consistently for both
           compilation and linking.  This option is supported on GNU/Linux targets, most other Unix derivatives, and also on x86
           Cygwin and MinGW targets.

right?

-- 
MST

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
  2022-02-22 23:18           ` Matthew Wilcox
@ 2022-03-20 11:03             ` Michael S. Tsirkin
  -1 siblings, 0 replies; 24+ messages in thread
From: Michael S. Tsirkin @ 2022-03-20 11:03 UTC (permalink / raw)
  To: Matthew Wilcox
  Cc: David Woodhouse, Stefano Garzarella, Jason Wang, virtualization,
	linux-kernel

On Tue, Feb 22, 2022 at 11:18:18PM +0000, Matthew Wilcox wrote:
> On Tue, Feb 22, 2022 at 07:58:33AM +0000, David Woodhouse wrote:
> > On Tue, 2022-02-22 at 01:31 -0500, Michael S. Tsirkin wrote:
> > > On Mon, Feb 21, 2022 at 05:18:48PM +0000, David Woodhouse wrote:
> > > > 
> > > > [dwoodhou@i7 virtio]$ sudo ~/virtio_test
> > > > Detected virtual address range 0x1000-0x7ffffffff000
> > > > spurious wakeups: 0x0 started=0x100000 completed=0x100000
> > > > 
> > > > Although in some circumstances I also see a different build failure:
> > > > 
> > > > cc -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h   -c -o vringh_test.o vringh_test.c
> 
> Trying to test this myself ...
> 
> $ cd tools/virtio/
> $ make
> ...
> cc -lpthread  virtio_test.o virtio_ring.o   -o virtio_test
> /usr/bin/ld: virtio_ring.o: in function `spin_lock':
> /home/willy/kernel/folio/tools/virtio/./linux/spinlock.h:16: undefined reference to `pthread_spin_lock'
> 
> So this is not the only problem here?

Could you let me know which system and gcc version have this problem, for inclusion
in the commit log?

> > > > In file included from ./linux/uio.h:3,
> > > >                  from ./linux/../../../include/linux/vringh.h:15,
> > > >                  from ./linux/vringh.h:1,
> > > >                  from vringh_test.c:9:
> > > > ./linux/../../../include/linux/uio.h:10:10: fatal error: linux/mm_types.h: No such file or directory
> > > >    10 | #include <linux/mm_types.h>
> > > >       |          ^~~~~~~~~~~~~~~~~~
> > > > compilation terminated.
> > > > make: *** [<builtin>: vringh_test.o] Error 1
> > > 
> > > Which tree has this build failure? In mine linux/uio.h does not
> > > include linux/mm_types.h.
> > 
> > Strictly it's
> > https://git.infradead.org/users/dwmw2/linux.git/shortlog/refs/heads/xen-evtchn-kernel
> > but I'm sure my part isn't relevant; it's just v5.17-rc5.
> > 
> >  $ git blame include/linux/uio.h | grep mm_types.h
> > d9c19d32d86fa (Matthew Wilcox (Oracle) 2021-10-18 10:39:06 -0400  10) #include <linux/mm_types.h>
> >  $ git describe --tags d9c19d32d86fa
> > v5.16-rc4-37-gd9c19d32d86f
> 
> grr.  Originally, I had this doing a typebusting cast, but hch objected,
> so I had to include mm_types.h.  This should fix it ...
> 
> $ git diff
> diff --git a/tools/virtio/linux/mm_types.h b/tools/virtio/linux/mm_types.h
> new file mode 100644
> index 000000000000..3b0fc9bc5b8f
> --- /dev/null
> +++ b/tools/virtio/linux/mm_types.h
> @@ -0,0 +1,3 @@
> +struct folio {
> +       struct page page;
> +};
> 
> At least, it makes it compile for me.


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH] tools/virtio: Test virtual address range detection
@ 2022-03-20 11:03             ` Michael S. Tsirkin
  0 siblings, 0 replies; 24+ messages in thread
From: Michael S. Tsirkin @ 2022-03-20 11:03 UTC (permalink / raw)
  To: Matthew Wilcox; +Cc: virtualization, David Woodhouse, linux-kernel

On Tue, Feb 22, 2022 at 11:18:18PM +0000, Matthew Wilcox wrote:
> On Tue, Feb 22, 2022 at 07:58:33AM +0000, David Woodhouse wrote:
> > On Tue, 2022-02-22 at 01:31 -0500, Michael S. Tsirkin wrote:
> > > On Mon, Feb 21, 2022 at 05:18:48PM +0000, David Woodhouse wrote:
> > > > 
> > > > [dwoodhou@i7 virtio]$ sudo ~/virtio_test
> > > > Detected virtual address range 0x1000-0x7ffffffff000
> > > > spurious wakeups: 0x0 started=0x100000 completed=0x100000
> > > > 
> > > > Although in some circumstances I also see a different build failure:
> > > > 
> > > > cc -g -O2 -Werror -Wno-maybe-uninitialized -Wall -I. -I../include/ -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -include ../../include/linux/kconfig.h   -c -o vringh_test.o vringh_test.c
> 
> Trying to test this myself ...
> 
> $ cd tools/virtio/
> $ make
> ...
> cc -lpthread  virtio_test.o virtio_ring.o   -o virtio_test
> /usr/bin/ld: virtio_ring.o: in function `spin_lock':
> /home/willy/kernel/folio/tools/virtio/./linux/spinlock.h:16: undefined reference to `pthread_spin_lock'
> 
> So this is not the only problem here?

Could you let me know which system and gcc version have this problem, for inclusion
in the commit log?

> > > > In file included from ./linux/uio.h:3,
> > > >                  from ./linux/../../../include/linux/vringh.h:15,
> > > >                  from ./linux/vringh.h:1,
> > > >                  from vringh_test.c:9:
> > > > ./linux/../../../include/linux/uio.h:10:10: fatal error: linux/mm_types.h: No such file or directory
> > > >    10 | #include <linux/mm_types.h>
> > > >       |          ^~~~~~~~~~~~~~~~~~
> > > > compilation terminated.
> > > > make: *** [<builtin>: vringh_test.o] Error 1
> > > 
> > > Which tree has this build failure? In mine linux/uio.h does not
> > > include linux/mm_types.h.
> > 
> > Strictly it's
> > https://git.infradead.org/users/dwmw2/linux.git/shortlog/refs/heads/xen-evtchn-kernel
> > but I'm sure my part isn't relevant; it's just v5.17-rc5.
> > 
> >  $ git blame include/linux/uio.h | grep mm_types.h
> > d9c19d32d86fa (Matthew Wilcox (Oracle) 2021-10-18 10:39:06 -0400  10) #include <linux/mm_types.h>
> >  $ git describe --tags d9c19d32d86fa
> > v5.16-rc4-37-gd9c19d32d86f
> 
> grr.  Originally, I had this doing a typebusting cast, but hch objected,
> so I had to include mm_types.h.  This should fix it ...
> 
> $ git diff
> diff --git a/tools/virtio/linux/mm_types.h b/tools/virtio/linux/mm_types.h
> new file mode 100644
> index 000000000000..3b0fc9bc5b8f
> --- /dev/null
> +++ b/tools/virtio/linux/mm_types.h
> @@ -0,0 +1,3 @@
> +struct folio {
> +       struct page page;
> +};
> 
> At least, it makes it compile for me.

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

^ permalink raw reply	[flat|nested] 24+ messages in thread

end of thread, other threads:[~2022-03-20 11:04 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-02-21 16:15 [PATCH] tools/virtio: Test virtual address range detection David Woodhouse
2022-02-21 16:15 ` David Woodhouse
2022-02-21 17:02 ` Stefano Garzarella
2022-02-21 17:02   ` Stefano Garzarella
2022-02-21 17:18   ` David Woodhouse
2022-02-21 17:18     ` David Woodhouse
2022-02-22  6:31     ` Michael S. Tsirkin
2022-02-22  6:31       ` Michael S. Tsirkin
2022-02-22  7:58       ` David Woodhouse
2022-02-22  7:58         ` David Woodhouse
2022-02-22 23:18         ` Matthew Wilcox
2022-02-22 23:18           ` Matthew Wilcox
2022-02-25 15:48           ` Matthew Wilcox
2022-02-25 15:48             ` Matthew Wilcox
2022-02-25 18:25             ` Michael S. Tsirkin
2022-02-25 18:25               ` Michael S. Tsirkin
2022-03-16 14:41           ` Matthew Wilcox
2022-03-16 14:41             ` Matthew Wilcox
2022-03-20 11:01             ` Michael S. Tsirkin
2022-03-20 11:01               ` Michael S. Tsirkin
2022-03-20 11:03           ` Michael S. Tsirkin
2022-03-20 11:03             ` Michael S. Tsirkin
2022-02-22  3:25 ` Jason Wang
2022-02-22  3:25   ` Jason Wang

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.