[PATCH kvm-unit-tests 00/18] merge x86 vmalloc and arm/powerpc alloc

[PATCH kvm-unit-tests 00/18] merge x86 vmalloc and arm/powerpc alloc

[Paolo Bonzini]

This series unifies the memory allocators in kvm-unit-tests:

- x86 startup code sets up phys_alloc using the e801 memory map
  information from multiboot.  phys_alloc remains active until
  the tests call setup_vm.  phys_alloc is very simple and is
  not able to free any memory.

- a new MMU-friendly alloc_ops implementation is provided based on
  lib/x86/vm.c.  The new alloc_ops relies on a physical page allocator
  (in lib/alloc_page.c), which is initialized from the final phys_alloc
  state, and a virtual address allocator.  The alloc_ops get physical
  pages and maps them into a contiguous virtual address space range.

The entry points for the architecture-dependent APIs are setup_mmu,
install_page and virt_to_pte_phys.  lib/x86/vm.c implements them
together with other x86-specific functions.

Changes from yesterday's RFC include:

- code style fixes when code was moved

- spinlocks in the page allocator

- an extra patch "x86: introduce pgd_t and pteval_t" to avoid abusing
  unsigned long / u64, and in general paying more attention to types

- quite a few bugfixes

- the final 7 patches that port ARM tests to use the new mechanism

- tested on i386 and PPC too

Possible extensions include implementing the new API for PPC/s390 (probably
using a soft TLB approach, similar to Linux, for PPC hashed page tables),
and supporting sub-page allocation in malloc.

Paolo

Paolo Bonzini (18):
  lib: simplify alloc_ops
  lib: start moving vmalloc to generic code
  lib: move page allocator here from x86
  alloc_page: fix off-by-one
  x86: leave VM setup out of vtd_init
  x86: prepare for extending setup.c
  x86: use alloc_phys
  alloc: implement free
  x86: introduce pgd_t and pteval_t
  x86: turn virt_to_phys_cr3 into virt_to_pte_phys
  vmalloc: convert to alloc_ops
  arm64: use adrp instead of adr
  arm: make pgtable.h private to mmu.c
  arm: get rid of pud
  arm: use better-matching types for virtual and physical addresses
  arm: only align CPU stacks to one page
  arm: do not allocate virtio buffers from the stack
  arm: enable vmalloc

 arm/Makefile.common           |   4 +
 arm/cstart64.S                |  25 ++--
 arm/sieve.c                   |   1 +
 lib/alloc.c                   | 202 ++++++++------------------------
 lib/alloc.h                   |  90 +--------------
 lib/alloc_page.c              | 131 +++++++++++++++++++++
 lib/alloc_page.h              |  16 +++
 lib/alloc_phys.c              | 132 +++++++++++++++++++++
 lib/alloc_phys.h              |  45 ++++++++
 lib/arm/asm/mmu-api.h         |  15 ++-
 lib/arm/asm/mmu.h             |   1 -
 lib/arm/asm/page.h            |   4 -
 lib/arm/asm/pgtable.h         |  36 +++---
 lib/arm/asm/thread_info.h     |   2 +-
 lib/arm/bitops.c              |   1 +
 lib/arm/mmu.c                 | 104 +++++++++++------
 lib/arm/setup.c               |   3 +-
 lib/arm64/asm/mmu.h           |   1 -
 lib/arm64/asm/page.h          |  12 +-
 lib/arm64/asm/pgtable-hwdef.h |  17 +--
 lib/arm64/asm/pgtable.h       |  15 +--
 lib/chr-testdev.c             |  17 ++-
 lib/libcflat.h                |   2 +
 lib/powerpc/setup.c           |   1 +
 lib/virtio-mmio.c             |   4 +-
 lib/vmalloc.c                 | 102 ++++++++++++++++
 lib/vmalloc.h                 |  17 +++
 lib/x86/asm/page.h            |   3 +
 lib/x86/intel-iommu.c         |   4 +-
 lib/x86/intel-iommu.h         |   1 -
 lib/x86/io.c                  |   2 +-
 lib/x86/isr.c                 |   5 +-
 lib/x86/setup.c               |  49 +++++---
 lib/x86/vm.c                  | 262 +++++++-----------------------------------
 lib/x86/vm.h                  |  41 +++----
 powerpc/Makefile.common       |   1 +
 s390x/Makefile                |   2 +-
 x86/Makefile.common           |   4 +
 x86/asyncpf.c                 |   7 +-
 x86/cstart.S                  |   4 +-
 x86/cstart64.S                |   4 +-
 x86/emulator.c                |   2 +
 x86/eventinj.c                |   4 +-
 x86/hypercall.c               |   1 +
 x86/hyperv_clock.c            |   1 +
 x86/hyperv_connections.c      |   1 +
 x86/hyperv_stimer.c           |   1 +
 x86/hyperv_synic.c            |   1 +
 x86/intel-iommu.c             |   5 +
 x86/pmu.c                     |   4 +-
 x86/rmap_chain.c              |   2 +
 x86/sieve.c                   |   6 +-
 x86/svm.c                     |   1 +
 x86/taskswitch.c              |   1 +
 x86/taskswitch2.c             |   2 +
 x86/vmx.c                     |   1 +
 x86/vmx_tests.c               |   2 +
 57 files changed, 777 insertions(+), 647 deletions(-)
 create mode 120000 arm/sieve.c
 create mode 100644 lib/alloc_page.c
 create mode 100644 lib/alloc_page.h
 create mode 100644 lib/alloc_phys.c
 create mode 100644 lib/alloc_phys.h
 create mode 100644 lib/vmalloc.c
 create mode 100644 lib/vmalloc.h

-- 
2.14.2

[PATCH kvm-unit-tests 01/18] lib: simplify alloc_ops

[Paolo Bonzini]

Layer malloc/calloc/free/memalign over a basic allocator.
The implementation of high-level routines remains the basic one
from before, but the callback interface is reduced to a single
function.  Since alloc.c now contains the physical address allocator
only, rename it.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 arm/Makefile.common           |  2 +-
 lib/alloc.h                   | 77 ++++++-------------------------------------
 lib/{alloc.c => alloc_phys.c} | 62 +++-------------------------------
 lib/alloc_phys.h              | 39 ++++++++++++++++++++++
 lib/arm/setup.c               |  1 +
 lib/powerpc/setup.c           |  1 +
 powerpc/Makefile.common       |  2 +-
 s390x/Makefile                |  2 +-
 8 files changed, 59 insertions(+), 127 deletions(-)
 rename lib/{alloc.c => alloc_phys.c} (68%)
 create mode 100644 lib/alloc_phys.h

diff --git a/arm/Makefile.common b/arm/Makefile.common
index 7e5f527..6979a24 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -34,7 +34,7 @@ asm-offsets = lib/$(ARCH)/asm-offsets.h
 include $(SRCDIR)/scripts/asm-offsets.mak
 
 cflatobjs += lib/util.o
-cflatobjs += lib/alloc.o
+cflatobjs += lib/alloc_phys.o
 cflatobjs += lib/devicetree.o
 cflatobjs += lib/pci.o
 cflatobjs += lib/pci-host-generic.o
diff --git a/lib/alloc.h b/lib/alloc.h
index 81f5369..24f85b4 100644
--- a/lib/alloc.h
+++ b/lib/alloc.h
@@ -23,33 +23,31 @@
 #include "libcflat.h"
 
 struct alloc_ops {
-	void *(*malloc)(size_t size);
-	void *(*calloc)(size_t nmemb, size_t size);
-	void (*free)(void *ptr);
 	void *(*memalign)(size_t alignment, size_t size);
 };
 
-/*
- * alloc_ops is initialized to early_alloc_ops
- */
 extern struct alloc_ops *alloc_ops;
 
+/*
+ * Our malloc implementation is currently so simple that it can just
+ * be inlined. :)
+ */
 static inline void *malloc(size_t size)
 {
-	assert(alloc_ops && alloc_ops->malloc);
-	return alloc_ops->malloc(size);
+	assert(alloc_ops && alloc_ops->memalign);
+	return alloc_ops->memalign(sizeof(long), size);
 }
 
 static inline void *calloc(size_t nmemb, size_t size)
 {
-	assert(alloc_ops && alloc_ops->calloc);
-	return alloc_ops->calloc(nmemb, size);
+	void *ptr = malloc(nmemb * size);
+	if (ptr)
+		memset(ptr, 0, nmemb * size);
+	return ptr;
 }
 
 static inline void free(void *ptr)
 {
-	assert(alloc_ops && alloc_ops->free);
-	alloc_ops->free(ptr);
 }
 
 static inline void *memalign(size_t alignment, size_t size)
@@ -58,59 +56,4 @@ static inline void *memalign(size_t alignment, size_t size)
 	return alloc_ops->memalign(alignment, size);
 }
 
-/*
- * phys_alloc is a very simple allocator which allows physical memory
- * to be partitioned into regions until all memory is allocated.
- *
- * Note: This is such a simple allocator that there is no way to free
- * a region. For more complicated memory management a single region
- * can be allocated, but then have its memory managed by a more
- * sophisticated allocator, e.g. a page allocator.
- */
-#define DEFAULT_MINIMUM_ALIGNMENT 32
-
-/*
- * phys_alloc_init creates the initial free memory region of size @size
- * at @base. The minimum alignment is set to DEFAULT_MINIMUM_ALIGNMENT.
- */
-extern void phys_alloc_init(phys_addr_t base, phys_addr_t size);
-
-/*
- * phys_alloc_set_minimum_alignment sets the minimum alignment to
- * @align.
- */
-extern void phys_alloc_set_minimum_alignment(phys_addr_t align);
-
-/*
- * phys_alloc_aligned returns the base address of a region of size @size,
- * where the address is aligned to @align, or INVALID_PHYS_ADDR if there
- * isn't enough free memory to satisfy the request.
- */
-extern phys_addr_t phys_alloc_aligned(phys_addr_t size, phys_addr_t align);
-
-/*
- * phys_zalloc_aligned is like phys_alloc_aligned, but zeros the memory
- * before returning the address.
- */
-extern phys_addr_t phys_zalloc_aligned(phys_addr_t size, phys_addr_t align);
-
-/*
- * phys_alloc returns the base address of a region of size @size, or
- * INVALID_PHYS_ADDR if there isn't enough free memory to satisfy the
- * request.
- */
-extern phys_addr_t phys_alloc(phys_addr_t size);
-
-/*
- * phys_zalloc is like phys_alloc, but zeros the memory before returning.
- */
-extern phys_addr_t phys_zalloc(phys_addr_t size);
-
-/*
- * phys_alloc_show outputs all currently allocated regions with the
- * following format
- *   <start_addr>-<end_addr> [<USED|FREE>]
- */
-extern void phys_alloc_show(void);
-
 #endif /* _ALLOC_H_ */
diff --git a/lib/alloc.c b/lib/alloc_phys.c
similarity index 68%
rename from lib/alloc.c
rename to lib/alloc_phys.c
index d553a7e..3972277 100644
--- a/lib/alloc.c
+++ b/lib/alloc_phys.c
@@ -2,6 +2,9 @@
  * Copyright (C) 2014, Red Hat Inc, Andrew Jones <drjones@redhat.com>
  *
  * This work is licensed under the terms of the GNU LGPL, version 2.
+ *
+ * This is a simple allocator that provides contiguous physical addresses
+ * with byte granularity.
  */
 #include "alloc.h"
 #include "asm/spinlock.h"
@@ -9,6 +12,8 @@
 
 #define PHYS_ALLOC_NR_REGIONS	256
 
+#define DEFAULT_MINIMUM_ALIGNMENT	32
+
 struct phys_alloc_region {
 	phys_addr_t base;
 	phys_addr_t size;
@@ -102,60 +107,6 @@ static phys_addr_t phys_alloc_aligned_safe(phys_addr_t size,
 	return addr;
 }
 
-static phys_addr_t phys_zalloc_aligned_safe(phys_addr_t size,
-					    phys_addr_t align, bool safe)
-{
-	phys_addr_t addr = phys_alloc_aligned_safe(size, align, safe);
-	if (addr == INVALID_PHYS_ADDR)
-		return addr;
-
-	memset(phys_to_virt(addr), 0, size);
-	return addr;
-}
-
-phys_addr_t phys_alloc_aligned(phys_addr_t size, phys_addr_t align)
-{
-	return phys_alloc_aligned_safe(size, align, false);
-}
-
-phys_addr_t phys_zalloc_aligned(phys_addr_t size, phys_addr_t align)
-{
-	return phys_zalloc_aligned_safe(size, align, false);
-}
-
-phys_addr_t phys_alloc(phys_addr_t size)
-{
-	return phys_alloc_aligned(size, align_min);
-}
-
-phys_addr_t phys_zalloc(phys_addr_t size)
-{
-	return phys_zalloc_aligned(size, align_min);
-}
-
-static void *early_malloc(size_t size)
-{
-	phys_addr_t addr = phys_alloc_aligned_safe(size, align_min, true);
-	if (addr == INVALID_PHYS_ADDR)
-		return NULL;
-
-	return phys_to_virt(addr);
-}
-
-static void *early_calloc(size_t nmemb, size_t size)
-{
-	phys_addr_t addr = phys_zalloc_aligned_safe(nmemb * size,
-						    align_min, true);
-	if (addr == INVALID_PHYS_ADDR)
-		return NULL;
-
-	return phys_to_virt(addr);
-}
-
-static void early_free(void *ptr __unused)
-{
-}
-
 static void *early_memalign(size_t alignment, size_t size)
 {
 	phys_addr_t addr;
@@ -170,9 +121,6 @@ static void *early_memalign(size_t alignment, size_t size)
 }
 
 static struct alloc_ops early_alloc_ops = {
-	.malloc = early_malloc,
-	.calloc = early_calloc,
-	.free = early_free,
 	.memalign = early_memalign,
 };
 
diff --git a/lib/alloc_phys.h b/lib/alloc_phys.h
new file mode 100644
index 0000000..f09d667
--- /dev/null
+++ b/lib/alloc_phys.h
@@ -0,0 +1,39 @@
+#ifndef _ALLOC_PHYS_H_
+#define _ALLOC_PHYS_H_
+/*
+ * phys_alloc is a very simple allocator which allows physical memory
+ * to be partitioned into regions until all memory is allocated.
+ *
+ * Note: This is such a simple allocator that there is no way to free
+ * a region. For more complicated memory management a single region
+ * can be allocated, but then have its memory managed by a more
+ * sophisticated allocator, e.g. a page allocator.
+ *
+ * Copyright (C) 2014, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#include "libcflat.h"
+
+#define DEFAULT_MINIMUM_ALIGNMENT 32
+
+/*
+ * phys_alloc_init creates the initial free memory region of size @size
+ * at @base. The minimum alignment is set to DEFAULT_MINIMUM_ALIGNMENT.
+ */
+extern void phys_alloc_init(phys_addr_t base, phys_addr_t size);
+
+/*
+ * phys_alloc_set_minimum_alignment sets the minimum alignment to
+ * @align.
+ */
+extern void phys_alloc_set_minimum_alignment(phys_addr_t align);
+
+/*
+ * phys_alloc_show outputs all currently allocated regions with the
+ * following format
+ *   <start_addr>-<end_addr> [<USED|FREE>]
+ */
+extern void phys_alloc_show(void);
+
+#endif /* _ALLOC_PHYS_H_ */
diff --git a/lib/arm/setup.c b/lib/arm/setup.c
index a0b1795..0e2bab4 100644
--- a/lib/arm/setup.c
+++ b/lib/arm/setup.c
@@ -14,6 +14,7 @@
 #include <libfdt/libfdt.h>
 #include <devicetree.h>
 #include <alloc.h>
+#include <alloc_phys.h>
 #include <argv.h>
 #include <asm/thread_info.h>
 #include <asm/setup.h>
diff --git a/lib/powerpc/setup.c b/lib/powerpc/setup.c
index 20a1e37..d8f4327 100644
--- a/lib/powerpc/setup.c
+++ b/lib/powerpc/setup.c
@@ -14,6 +14,7 @@
 #include <libfdt/libfdt.h>
 #include <devicetree.h>
 #include <alloc.h>
+#include <alloc_phys.h>
 #include <argv.h>
 #include <asm/setup.h>
 #include <asm/page.h>
diff --git a/powerpc/Makefile.common b/powerpc/Makefile.common
index c4df2e9..126590a 100644
--- a/powerpc/Makefile.common
+++ b/powerpc/Makefile.common
@@ -30,7 +30,7 @@ asm-offsets = lib/$(ARCH)/asm-offsets.h
 include $(SRCDIR)/scripts/asm-offsets.mak
 
 cflatobjs += lib/util.o
-cflatobjs += lib/alloc.o
+cflatobjs += lib/alloc_phys.o
 cflatobjs += lib/devicetree.o
 cflatobjs += lib/powerpc/io.o
 cflatobjs += lib/powerpc/hcall.o
diff --git a/s390x/Makefile b/s390x/Makefile
index 573cba2..f585faf 100644
--- a/s390x/Makefile
+++ b/s390x/Makefile
@@ -21,7 +21,7 @@ asm-offsets = lib/$(ARCH)/asm-offsets.h
 include $(SRCDIR)/scripts/asm-offsets.mak
 
 cflatobjs += lib/util.o
-cflatobjs += lib/alloc.o
+cflatobjs += lib/alloc_phys.o
 cflatobjs += lib/s390x/io.o
 cflatobjs += lib/s390x/stack.o
 cflatobjs += lib/s390x/sclp-ascii.o
-- 
2.14.2

[PATCH kvm-unit-tests 02/18] lib: start moving vmalloc to generic code

[Paolo Bonzini]

For now, vmalloc provides a primitive that allocates contiguous virtual
address.  Together with a page allocator that allocates single
physical memory pages, it will provide an implementation of
alloc_ops for when an MMU is enabled.

Before doing that, however, we need to move the page allocator and
give lib/alloc.c's malloc feature parity with lib/x86/vm.c's vmalloc.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 lib/vmalloc.c       | 31 +++++++++++++++++++++++++++++++
 lib/vmalloc.h       |  8 ++++++++
 lib/x86/vm.c        | 25 ++++++-------------------
 lib/x86/vm.h        |  2 --
 x86/Makefile.common |  1 +
 x86/emulator.c      |  1 +
 x86/eventinj.c      |  1 +
 x86/taskswitch.c    |  1 +
 x86/taskswitch2.c   |  1 +
 x86/vmx_tests.c     |  1 +
 10 files changed, 51 insertions(+), 21 deletions(-)
 create mode 100644 lib/vmalloc.c
 create mode 100644 lib/vmalloc.h

diff --git a/lib/vmalloc.c b/lib/vmalloc.c
new file mode 100644
index 0000000..06298f4
--- /dev/null
+++ b/lib/vmalloc.c
@@ -0,0 +1,31 @@
+/*
+ * Copyright (C) 2012, 2017, Red Hat Inc.
+ *
+ * This allocator provides contiguous physical addresses with page
+ * granularity.
+ */
+
+#include "libcflat.h"
+#include "asm/spinlock.h"
+#include "asm/page.h"
+
+static struct spinlock lock;
+static void *vfree_top = 0;
+
+void *alloc_vpages(ulong nr)
+{
+	spin_lock(&lock);
+	vfree_top -= PAGE_SIZE * nr;
+	spin_unlock(&lock);
+	return vfree_top;
+}
+
+void *alloc_vpage(void)
+{
+	return alloc_vpages(1);
+}
+
+void init_alloc_vpage(void *top)
+{
+	vfree_top = top;
+}
diff --git a/lib/vmalloc.h b/lib/vmalloc.h
new file mode 100644
index 0000000..4941fef
--- /dev/null
+++ b/lib/vmalloc.h
@@ -0,0 +1,8 @@
+#ifndef VMALLOC_H
+#define VMALLOC_H 1
+
+extern void *alloc_vpages(ulong nr);
+extern void *alloc_vpage(void);
+extern void init_alloc_vpage(void *top);
+
+#endif
diff --git a/lib/x86/vm.c b/lib/x86/vm.c
index 9b5f922..b975a73 100644
--- a/lib/x86/vm.c
+++ b/lib/x86/vm.c
@@ -1,9 +1,9 @@
 #include "fwcfg.h"
 #include "vm.h"
 #include "libcflat.h"
+#include "vmalloc.h"
 
 static void *free = 0;
-static void *vfree_top = 0;
 
 static void free_memory(void *mem, unsigned long size)
 {
@@ -271,7 +271,7 @@ static void setup_mmu(unsigned long len)
     /* 0 - 2G memory, 2G-3G valloc area, 3G-4G mmio */
     setup_mmu_range(cr3, 0, len);
     setup_mmu_range(cr3, 3ul << 30, (1ul << 30));
-    vfree_top = (void*)(3ul << 30);
+    init_alloc_vpage((void*)(3ul << 30));
 #endif
 
     write_cr3(virt_to_phys(cr3));
@@ -302,9 +302,8 @@ void *vmalloc(unsigned long size)
     size += sizeof(unsigned long);
 
     size = (size + PAGE_SIZE - 1) & ~(PAGE_SIZE - 1);
-    vfree_top -= size;
-    mem = p = vfree_top;
     pages = size / PAGE_SIZE;
+    mem = p = alloc_vpages(pages);
     while (pages--) {
 	install_page(phys_to_virt(read_cr3()), virt_to_phys(alloc_page()), p);
 	p += PAGE_SIZE;
@@ -336,11 +335,10 @@ void *vmap(unsigned long long phys, unsigned long size)
     unsigned pages;
 
     size = (size + PAGE_SIZE - 1) & ~(PAGE_SIZE - 1);
-    vfree_top -= size;
-    phys &= ~(unsigned long long)(PAGE_SIZE - 1);
-
-    mem = p = vfree_top;
     pages = size / PAGE_SIZE;
+    mem = p = alloc_vpages(pages);
+
+    phys &= ~(unsigned long long)(PAGE_SIZE - 1);
     while (pages--) {
 	install_page(phys_to_virt(read_cr3()), phys, p);
 	phys += PAGE_SIZE;
@@ -348,14 +346,3 @@ void *vmap(unsigned long long phys, unsigned long size)
     }
     return mem;
 }
-
-void *alloc_vpages(ulong nr)
-{
-	vfree_top -= PAGE_SIZE * nr;
-	return vfree_top;
-}
-
-void *alloc_vpage(void)
-{
-    return alloc_vpages(1);
-}
diff --git a/lib/x86/vm.h b/lib/x86/vm.h
index 7457b70..543da33 100644
--- a/lib/x86/vm.h
+++ b/lib/x86/vm.h
@@ -11,8 +11,6 @@ void setup_5level_page_table();
 void *vmalloc(unsigned long size);
 void vfree(void *mem);
 void *vmap(unsigned long long phys, unsigned long size);
-void *alloc_vpage(void);
-void *alloc_vpages(ulong nr);
 uint64_t virt_to_phys_cr3(void *mem);
 
 struct pte_search {
diff --git a/x86/Makefile.common b/x86/Makefile.common
index e96812b..f3f3742 100644
--- a/x86/Makefile.common
+++ b/x86/Makefile.common
@@ -4,6 +4,7 @@ all: directories test_cases
 
 cflatobjs += lib/pci.o
 cflatobjs += lib/pci-edu.o
+cflatobjs += lib/vmalloc.o
 cflatobjs += lib/x86/setup.o
 cflatobjs += lib/x86/io.o
 cflatobjs += lib/x86/smp.o
diff --git a/x86/emulator.c b/x86/emulator.c
index 8d262d8..4fb0f32 100644
--- a/x86/emulator.c
+++ b/x86/emulator.c
@@ -4,6 +4,7 @@
 #include "desc.h"
 #include "types.h"
 #include "processor.h"
+#include "vmalloc.h"
 
 #define memset __builtin_memset
 #define TESTDEV_IO_PORT 0xe0
diff --git a/x86/eventinj.c b/x86/eventinj.c
index 665eb61..61e1b96 100644
--- a/x86/eventinj.c
+++ b/x86/eventinj.c
@@ -5,6 +5,7 @@
 #include "isr.h"
 #include "apic.h"
 #include "apic-defs.h"
+#include "vmalloc.h"
 
 #ifdef __x86_64__
 #  define R "r"
diff --git a/x86/taskswitch.c b/x86/taskswitch.c
index 01483a1..889831e 100644
--- a/x86/taskswitch.c
+++ b/x86/taskswitch.c
@@ -7,6 +7,7 @@
 
 #include "libcflat.h"
 #include "x86/desc.h"
+#include "vmalloc.h"
 
 #define TSS_RETURN		(FIRST_SPARE_SEL)
 
diff --git a/x86/taskswitch2.c b/x86/taskswitch2.c
index bb7345b..e442fa6 100644
--- a/x86/taskswitch2.c
+++ b/x86/taskswitch2.c
@@ -4,6 +4,7 @@
 #include "apic.h"
 #include "processor.h"
 #include "vm.h"
+#include "vmalloc.h"
 
 #define MAIN_TSS_SEL (FIRST_SPARE_SEL + 0)
 #define VM86_TSS_SEL (FIRST_SPARE_SEL + 8)
diff --git a/x86/vmx_tests.c b/x86/vmx_tests.c
index 4a3e94b..d9bbe10 100644
--- a/x86/vmx_tests.c
+++ b/x86/vmx_tests.c
@@ -12,6 +12,7 @@
 #include "desc.h"
 #include "apic.h"
 #include "types.h"
+#include "vmalloc.h"
 
 #define NONCANONICAL            0xaaaaaaaaaaaaaaaaull
 
-- 
2.14.2

[PATCH kvm-unit-tests 03/18] lib: move page allocator here from x86

[Paolo Bonzini]

This is another step in porting the x86 (v)malloc implementation to
other architectures.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 lib/alloc_page.c         | 130 +++++++++++++++++++++++++++++++++++++++++++++++
 lib/alloc_page.h         |  16 ++++++
 lib/x86/intel-iommu.c    |   2 +
 lib/x86/intel-iommu.h    |   1 -
 lib/x86/vm.c             | 108 +--------------------------------------
 lib/x86/vm.h             |   4 --
 x86/Makefile.common      |   1 +
 x86/emulator.c           |   1 +
 x86/eventinj.c           |   3 +-
 x86/hypercall.c          |   1 +
 x86/hyperv_clock.c       |   1 +
 x86/hyperv_connections.c |   1 +
 x86/hyperv_stimer.c      |   1 +
 x86/hyperv_synic.c       |   1 +
 x86/intel-iommu.c        |   2 +
 x86/rmap_chain.c         |   1 +
 x86/svm.c                |   1 +
 x86/taskswitch2.c        |   1 +
 x86/vmx.c                |   1 +
 x86/vmx_tests.c          |   1 +
 20 files changed, 166 insertions(+), 112 deletions(-)
 create mode 100644 lib/alloc_page.c
 create mode 100644 lib/alloc_page.h

diff --git a/lib/alloc_page.c b/lib/alloc_page.c
new file mode 100644
index 0000000..15f6721
--- /dev/null
+++ b/lib/alloc_page.c
@@ -0,0 +1,130 @@
+/*
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ *
+ * This is a simple allocator that provides contiguous physical addresses
+ * with page granularity.
+ */
+#include "libcflat.h"
+#include <asm/page.h>
+#include <asm/io.h>
+#include <asm/spinlock.h>
+
+static struct spinlock lock;
+static void *freelist = 0;
+
+void free_pages(void *mem, unsigned long size)
+{
+	void *old_freelist;
+	void *end;
+
+	assert_msg((unsigned long) mem % PAGE_SIZE == 0,
+		   "mem not page aligned: %p", mem);
+
+	assert_msg(size % PAGE_SIZE == 0, "size not page aligned: %#lx", size);
+
+	assert_msg(size == 0 || mem + size > mem,
+		   "mem + size overflow: %p + %#lx", mem, size);
+
+	if (size == 0) {
+		freelist = NULL;
+		return;
+	}
+
+	spin_lock(&lock);
+	old_freelist = freelist;
+	freelist = mem;
+	end = mem + size;
+	while (mem + PAGE_SIZE != end) {
+		*(void **)mem = (mem + PAGE_SIZE);
+		mem += PAGE_SIZE;
+	}
+
+	*(void **)mem = old_freelist;
+	spin_unlock(&lock);
+}
+
+void *alloc_page()
+{
+	void *p;
+
+	if (!freelist)
+		return 0;
+
+	spin_lock(&lock);
+	p = freelist;
+	freelist = *(void **)freelist;
+	spin_unlock(&lock);
+
+	return p;
+}
+
+/*
+ * Allocates (1 << order) physically contiguous and naturally aligned pages.
+ * Returns NULL if there's no memory left.
+ */
+void *alloc_pages(unsigned long order)
+{
+	/* Generic list traversal. */
+	void *prev;
+	void *curr = NULL;
+	void *next = freelist;
+
+	/* Looking for a run of length (1 << order). */
+	unsigned long run = 0;
+	const unsigned long n = 1ul << order;
+	const unsigned long align_mask = (n << PAGE_SHIFT) - 1;
+	void *run_start = NULL;
+	void *run_prev = NULL;
+	unsigned long run_next_pa = 0;
+	unsigned long pa;
+
+	assert(order < sizeof(unsigned long) * 8);
+
+	spin_lock(&lock);
+	for (;;) {
+		prev = curr;
+		curr = next;
+
+		if (!curr) {
+			run_start = NULL;
+			break;
+		}
+
+		next = *((void **) curr);
+		pa = virt_to_phys(curr);
+
+		if (run == 0) {
+			if (!(pa & align_mask)) {
+				run_start = curr;
+				run_prev = prev;
+				run_next_pa = pa + PAGE_SIZE;
+				run = 1;
+			}
+		} else if (pa == run_next_pa) {
+			run_next_pa += PAGE_SIZE;
+			run += 1;
+		} else {
+			run = 0;
+		}
+
+		if (run == n) {
+			if (run_prev)
+				*((void **) run_prev) = next;
+			else
+				freelist = next;
+			break;
+		}
+	}
+	spin_unlock(&lock);
+	return run_start;
+}
+
+
+void free_page(void *page)
+{
+	spin_lock(&lock);
+	*(void **)page = freelist;
+	freelist = page;
+	spin_unlock(&lock);
+}
+
diff --git a/lib/alloc_page.h b/lib/alloc_page.h
new file mode 100644
index 0000000..1884c7a
--- /dev/null
+++ b/lib/alloc_page.h
@@ -0,0 +1,16 @@
+/*
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ *
+ * This is a simple allocator that provides contiguous physical addresses
+ * with byte granularity.
+ */
+
+#ifndef ALLOC_PAGE_H
+#define ALLOC_PAGE_H 1
+
+void *alloc_page();
+void *alloc_pages(unsigned long order);
+void free_page(void *page);
+void free_pages(void *mem, unsigned long size);
+
+#endif
diff --git a/lib/x86/intel-iommu.c b/lib/x86/intel-iommu.c
index 9fdbd3b..af92fab 100644
--- a/lib/x86/intel-iommu.c
+++ b/lib/x86/intel-iommu.c
@@ -12,8 +12,10 @@
 
 #include "intel-iommu.h"
 #include "libcflat.h"
+#include "vm.h"
 #include "pci.h"
 #include "atomic.h"
+#include "alloc_page.h"
 
 /*
  * VT-d in QEMU currently only support 39 bits address width, which is
diff --git a/lib/x86/intel-iommu.h b/lib/x86/intel-iommu.h
index 885be53..05b9744 100644
--- a/lib/x86/intel-iommu.h
+++ b/lib/x86/intel-iommu.h
@@ -16,7 +16,6 @@
 #define __INTEL_IOMMU_H__
 
 #include "libcflat.h"
-#include "vm.h"
 #include "isr.h"
 #include "smp.h"
 #include "desc.h"
diff --git a/lib/x86/vm.c b/lib/x86/vm.c
index b975a73..d2a3c59 100644
--- a/lib/x86/vm.c
+++ b/lib/x86/vm.c
@@ -2,111 +2,7 @@
 #include "vm.h"
 #include "libcflat.h"
 #include "vmalloc.h"
-
-static void *free = 0;
-
-static void free_memory(void *mem, unsigned long size)
-{
-	void *end;
-
-	assert_msg((unsigned long) mem % PAGE_SIZE == 0,
-		   "mem not page aligned: %p", mem);
-
-	assert_msg(size % PAGE_SIZE == 0, "size not page aligned: %#lx", size);
-
-	assert_msg(size == 0 || mem + size > mem,
-		   "mem + size overflow: %p + %#lx", mem, size);
-
-	if (size == 0) {
-		free = NULL;
-		return;
-	}
-
-	free = mem;
-	end = mem + size;
-	while (mem + PAGE_SIZE != end) {
-		*(void **)mem = (mem + PAGE_SIZE);
-		mem += PAGE_SIZE;
-	}
-
-	*(void **)mem = NULL;
-}
-
-void *alloc_page()
-{
-    void *p;
-
-    if (!free)
-	return 0;
-
-    p = free;
-    free = *(void **)free;
-
-    return p;
-}
-
-/*
- * Allocates (1 << order) physically contiguous and naturally aligned pages.
- * Returns NULL if there's no memory left.
- */
-void *alloc_pages(unsigned long order)
-{
-	/* Generic list traversal. */
-	void *prev;
-	void *curr = NULL;
-	void *next = free;
-
-	/* Looking for a run of length (1 << order). */
-	unsigned long run = 0;
-	const unsigned long n = 1ul << order;
-	const unsigned long align_mask = (n << PAGE_SHIFT) - 1;
-	void *run_start = NULL;
-	void *run_prev = NULL;
-	unsigned long run_next_pa = 0;
-	unsigned long pa;
-
-	assert(order < sizeof(unsigned long) * 8);
-
-	for (;;) {
-		prev = curr;
-		curr = next;
-		next = curr ? *((void **) curr) : NULL;
-
-		if (!curr)
-			return 0;
-
-		pa = virt_to_phys(curr);
-
-		if (run == 0) {
-			if (!(pa & align_mask)) {
-				run_start = curr;
-				run_prev = prev;
-				run_next_pa = pa + PAGE_SIZE;
-				run = 1;
-			}
-		} else if (pa == run_next_pa) {
-			run_next_pa += PAGE_SIZE;
-			run += 1;
-		} else {
-			run = 0;
-		}
-
-		if (run == n) {
-			if (run_prev)
-				*((void **) run_prev) = next;
-			else
-				free = next;
-			return run_start;
-		}
-	}
-}
-
-
-void free_page(void *page)
-{
-    *(void **)page = free;
-    free = page;
-}
+#include "alloc_page.h"
 
 extern char edata;
 static unsigned long end_of_memory;
@@ -290,7 +186,7 @@ void setup_vm()
 {
     assert(!end_of_memory);
     end_of_memory = fwcfg_get_u64(FW_CFG_RAM_SIZE);
-    free_memory(&edata, end_of_memory - (unsigned long)&edata);
+    free_pages(&edata, end_of_memory - (unsigned long)&edata);
     setup_mmu(end_of_memory);
 }
 
diff --git a/lib/x86/vm.h b/lib/x86/vm.h
index 543da33..4a7888b 100644
--- a/lib/x86/vm.h
+++ b/lib/x86/vm.h
@@ -40,10 +40,6 @@ unsigned long *install_pte(unsigned long *cr3,
                            unsigned long pte,
                            unsigned long *pt_page);
 
-void *alloc_page();
-void *alloc_pages(unsigned long order);
-void free_page(void *page);
-
 unsigned long *install_large_page(unsigned long *cr3,unsigned long phys,
                                   void *virt);
 unsigned long *install_page(unsigned long *cr3, unsigned long phys, void *virt);
diff --git a/x86/Makefile.common b/x86/Makefile.common
index f3f3742..fce0801 100644
--- a/x86/Makefile.common
+++ b/x86/Makefile.common
@@ -5,6 +5,7 @@ all: directories test_cases
 cflatobjs += lib/pci.o
 cflatobjs += lib/pci-edu.o
 cflatobjs += lib/vmalloc.o
+cflatobjs += lib/alloc_page.o
 cflatobjs += lib/x86/setup.o
 cflatobjs += lib/x86/io.o
 cflatobjs += lib/x86/smp.o
diff --git a/x86/emulator.c b/x86/emulator.c
index 4fb0f32..e6f27cc 100644
--- a/x86/emulator.c
+++ b/x86/emulator.c
@@ -5,6 +5,7 @@
 #include "types.h"
 #include "processor.h"
 #include "vmalloc.h"
+#include "alloc_page.h"
 
 #define memset __builtin_memset
 #define TESTDEV_IO_PORT 0xe0
diff --git a/x86/eventinj.c b/x86/eventinj.c
index 61e1b96..247604c 100644
--- a/x86/eventinj.c
+++ b/x86/eventinj.c
@@ -1,11 +1,12 @@
 #include "libcflat.h"
-#include "processor.h"
 #include "vm.h"
+#include "processor.h"
 #include "desc.h"
 #include "isr.h"
 #include "apic.h"
 #include "apic-defs.h"
 #include "vmalloc.h"
+#include "alloc_page.h"
 
 #ifdef __x86_64__
 #  define R "r"
diff --git a/x86/hypercall.c b/x86/hypercall.c
index 9380f78..37fec8e 100644
--- a/x86/hypercall.c
+++ b/x86/hypercall.c
@@ -1,6 +1,7 @@
 #include "libcflat.h"
 #include "vm.h"
 #include "desc.h"
+#include "alloc_page.h"
 
 #define KVM_HYPERCALL_INTEL ".byte 0x0f,0x01,0xc1"
 #define KVM_HYPERCALL_AMD ".byte 0x0f,0x01,0xd9"
diff --git a/x86/hyperv_clock.c b/x86/hyperv_clock.c
index 9497269..b72e357 100644
--- a/x86/hyperv_clock.c
+++ b/x86/hyperv_clock.c
@@ -4,6 +4,7 @@
 #include "processor.h"
 #include "hyperv.h"
 #include "vm.h"
+#include "alloc_page.h"
 
 #define MAX_CPU 4
 #define TICKS_PER_SEC (1000000000 / 100)
diff --git a/x86/hyperv_connections.c b/x86/hyperv_connections.c
index b061048..5d541c9 100644
--- a/x86/hyperv_connections.c
+++ b/x86/hyperv_connections.c
@@ -5,6 +5,7 @@
 #include "atomic.h"
 #include "hyperv.h"
 #include "bitops.h"
+#include "alloc_page.h"
 
 #define MAX_CPUS 64
 
diff --git a/x86/hyperv_stimer.c b/x86/hyperv_stimer.c
index bd7f966..b6332fd 100644
--- a/x86/hyperv_stimer.c
+++ b/x86/hyperv_stimer.c
@@ -9,6 +9,7 @@
 #include "atomic.h"
 #include "hyperv.h"
 #include "asm/barrier.h"
+#include "alloc_page.h"
 
 #define MAX_CPUS 4
 
diff --git a/x86/hyperv_synic.c b/x86/hyperv_synic.c
index 1ccf1a0..9a5f53f 100644
--- a/x86/hyperv_synic.c
+++ b/x86/hyperv_synic.c
@@ -8,6 +8,7 @@
 #include "smp.h"
 #include "atomic.h"
 #include "hyperv.h"
+#include "alloc_page.h"
 
 #define MAX_CPUS 4
 
diff --git a/x86/intel-iommu.c b/x86/intel-iommu.c
index 610cc65..82b8234 100644
--- a/x86/intel-iommu.c
+++ b/x86/intel-iommu.c
@@ -13,6 +13,8 @@
 #include "intel-iommu.h"
 #include "pci-edu.h"
 #include "x86/apic.h"
+#include "vm.h"
+#include "alloc_page.h"
 
 #define VTD_TEST_DMAR_4B ("DMAR 4B memcpy test")
 #define VTD_TEST_IR_MSI ("IR MSI")
diff --git a/x86/rmap_chain.c b/x86/rmap_chain.c
index 7bf6275..97591f9 100644
--- a/x86/rmap_chain.c
+++ b/x86/rmap_chain.c
@@ -4,6 +4,7 @@
 #include "fwcfg.h"
 #include "vm.h"
 #include "smp.h"
+#include "alloc_page.h"
 
 int main (void)
 {
diff --git a/x86/svm.c b/x86/svm.c
index 25ac0ce..162632c 100644
--- a/x86/svm.c
+++ b/x86/svm.c
@@ -6,6 +6,7 @@
 #include "vm.h"
 #include "smp.h"
 #include "types.h"
+#include "alloc_page.h"
 
 /* for the nested page table*/
 u64 *pml4e;
diff --git a/x86/taskswitch2.c b/x86/taskswitch2.c
index e442fa6..8714ade 100644
--- a/x86/taskswitch2.c
+++ b/x86/taskswitch2.c
@@ -5,6 +5,7 @@
 #include "processor.h"
 #include "vm.h"
 #include "vmalloc.h"
+#include "alloc_page.h"
 
 #define MAIN_TSS_SEL (FIRST_SPARE_SEL + 0)
 #define VM86_TSS_SEL (FIRST_SPARE_SEL + 8)
diff --git a/x86/vmx.c b/x86/vmx.c
index 786c70f..cc76b5b 100644
--- a/x86/vmx.c
+++ b/x86/vmx.c
@@ -30,6 +30,7 @@
 
 #include "libcflat.h"
 #include "processor.h"
+#include "alloc_page.h"
 #include "vm.h"
 #include "desc.h"
 #include "vmx.h"
diff --git a/x86/vmx_tests.c b/x86/vmx_tests.c
index d9bbe10..1c9c747 100644
--- a/x86/vmx_tests.c
+++ b/x86/vmx_tests.c
@@ -13,6 +13,7 @@
 #include "apic.h"
 #include "types.h"
 #include "vmalloc.h"
+#include "alloc_page.h"
 
 #define NONCANONICAL            0xaaaaaaaaaaaaaaaaull
 
-- 
2.14.2

[PATCH kvm-unit-tests 04/18] alloc_page: fix off-by-one

[Paolo Bonzini]

It is okay to free up to the very last page of virtual address space,
in which case mem+size is zero.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 lib/alloc_page.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/alloc_page.c b/lib/alloc_page.c
index 15f6721..52992ad 100644
--- a/lib/alloc_page.c
+++ b/lib/alloc_page.c
@@ -22,7 +22,8 @@ void free_pages(void *mem, unsigned long size)
 
 	assert_msg(size % PAGE_SIZE == 0, "size not page aligned: %#lx", size);
 
-	assert_msg(size == 0 || mem + size > mem,
+	assert_msg(size == 0 || (uintptr_t)mem == -size ||
+		   (uintptr_t)mem + size > (uintptr_t)mem,
 		   "mem + size overflow: %p + %#lx", mem, size);
 
 	if (size == 0) {
-- 
2.14.2

[PATCH kvm-unit-tests 05/18] x86: leave VM setup out of vtd_init

[Paolo Bonzini]

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 lib/x86/intel-iommu.c | 4 ----
 x86/intel-iommu.c     | 3 +++
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/lib/x86/intel-iommu.c b/lib/x86/intel-iommu.c
index af92fab..3f3f211 100644
--- a/lib/x86/intel-iommu.c
+++ b/lib/x86/intel-iommu.c
@@ -12,7 +12,6 @@
 
 #include "intel-iommu.h"
 #include "libcflat.h"
-#include "vm.h"
 #include "pci.h"
 #include "atomic.h"
 #include "alloc_page.h"
@@ -360,9 +359,6 @@ void vtd_setup_ioapic_irq(struct pci_dev *dev, int vector,
 
 void vtd_init(void)
 {
-	setup_vm();
-	smp_init();
-
 	vtd_reg_base = ioremap(Q35_HOST_BRIDGE_IOMMU_ADDR, PAGE_SIZE);
 
 	vtd_dump_init_info();
diff --git a/x86/intel-iommu.c b/x86/intel-iommu.c
index 82b8234..f24170d 100644
--- a/x86/intel-iommu.c
+++ b/x86/intel-iommu.c
@@ -132,6 +132,9 @@ static void vtd_test_ir(void)
 
 int main(int argc, char *argv[])
 {
+	setup_vm();
+	smp_init();
+
 	vtd_init();
 
 	report_prefix_push("vtd_init");
-- 
2.14.2

[PATCH kvm-unit-tests 06/18] x86: prepare for extending setup.c

[Paolo Bonzini]

Rename setup_get_initrd to setup_multiboot as we will get memory map
information from there, and since we are at it setup_environ to
setup_libcflat.

Use C structs for multiboot information instead of offsets.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 lib/x86/setup.c | 45 ++++++++++++++++++++++++++++++++-------------
 x86/cstart.S    |  4 ++--
 x86/cstart64.S  |  4 ++--
 3 files changed, 36 insertions(+), 17 deletions(-)

diff --git a/lib/x86/setup.c b/lib/x86/setup.c
index 804ba3a..22257a7 100644
--- a/lib/x86/setup.c
+++ b/lib/x86/setup.c
@@ -6,11 +6,32 @@
  * This work is licensed under the terms of the GNU LGPL, version 2.
  */
 #include "libcflat.h"
+#include "fwcfg.h"
+#include "alloc_phys.h"
 
-#define MBI_MODS_COUNT	20
-#define MBI_MODS_ADDR	24
-#define MB_MOD_START	 0
-#define MB_MOD_END	 4
+extern char edata;
+
+struct mbi_bootinfo {
+	u32 flags;
+	u32 mem_lower;
+	u32 mem_upper;
+	u32 boot_device;
+	u32 cmdline;
+	u32 mods_count;
+	u32 mods_addr;
+	u32 reserved[5];   /* 28-47 */
+	u32 mmap_addr;
+	u32 reserved0[3];  /* 52-63 */
+	u32 bootloader;
+	u32 reserved1[5];  /* 68-87 */
+	u32 size;
+};
+
+struct mbi_module {
+	u32 start, end;
+	u32 cmdline;
+	u32 unused;
+};
 
 #define ENV_SIZE 16384
 
@@ -21,22 +42,20 @@ u32 initrd_size;
 
 static char env[ENV_SIZE];
 
-void setup_get_initrd(u8 *bootinfo)
+void setup_multiboot(struct mbi_bootinfo *bootinfo)
 {
-	u32 *mods_addr, *mod_start, *mod_end;
+	struct mbi_module *mods;
 
-	if (*((u32 *)&bootinfo[MBI_MODS_COUNT]) != 1)
+	if (bootinfo->mods_count != 1)
 		return;
 
-	mods_addr = (u32 *)&bootinfo[MBI_MODS_ADDR];
-	mod_start = (u32 *)(ulong)(*mods_addr + MB_MOD_START);
-	mod_end = (u32 *)(ulong)(*mods_addr + MB_MOD_END);
+	mods = (struct mbi_module *)(uintptr_t) bootinfo->mods_addr;
 
-	initrd = (char *)(ulong)*mod_start;
-	initrd_size = *mod_end - *mod_start;
+	initrd = (char *)(uintptr_t) mods->start;
+	initrd_size = mods->end - mods->start;
 }
 
-void setup_environ(void)
+void setup_libcflat(void)
 {
 	if (initrd) {
 		/* environ is currently the only file in the initrd */
diff --git a/x86/cstart.S b/x86/cstart.S
index a3c26a4..143317e 100644
--- a/x86/cstart.S
+++ b/x86/cstart.S
@@ -95,8 +95,8 @@ MSR_GS_BASE = 0xc0000101
 .globl start
 start:
         push %ebx
-        call setup_get_initrd
-        call setup_environ
+        call setup_multiboot
+        call setup_libcflat
         mov mb_cmdline(%ebx), %eax
         mov %eax, __args
         call __setup_args
diff --git a/x86/cstart64.S b/x86/cstart64.S
index 8e4a1f3..21db10f 100644
--- a/x86/cstart64.S
+++ b/x86/cstart64.S
@@ -227,8 +227,8 @@ start64:
 	call enable_x2apic
 	mov mb_boot_info(%rip), %rbx
 	mov %rbx, %rdi
-	call setup_get_initrd
-	call setup_environ
+	call setup_multiboot
+	call setup_libcflat
 	mov mb_cmdline(%rbx), %eax
 	mov %rax, __args(%rip)
 	call __setup_args
-- 
2.14.2

[PATCH kvm-unit-tests 07/18] x86: use alloc_phys

[Paolo Bonzini]

As the next step in making vmalloc/vfree available to every other architecture,
register the available memory with phys_alloc for x86 too, and add a service
to return the unused phys_alloc region.  This makes setup_vm architecture-
independent.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 lib/alloc_page.c    |  2 +-
 lib/alloc_phys.c    |  6 ++++++
 lib/alloc_phys.h    |  6 ++++++
 lib/libcflat.h      |  2 ++
 lib/vmalloc.c       | 15 +++++++++++++++
 lib/vmalloc.h       |  3 +++
 lib/x86/setup.c     |  4 ++++
 lib/x86/vm.c        | 27 ++++++++-------------------
 lib/x86/vm.h        |  1 -
 x86/Makefile.common |  1 +
 10 files changed, 46 insertions(+), 21 deletions(-)

diff --git a/lib/alloc_page.c b/lib/alloc_page.c
index 52992ad..f0d46d7 100644
--- a/lib/alloc_page.c
+++ b/lib/alloc_page.c
@@ -5,6 +5,7 @@
  * with page granularity.
  */
 #include "libcflat.h"
+#include "alloc_phys.h"
 #include <asm/page.h>
 #include <asm/io.h>
 #include <asm/spinlock.h>
@@ -128,4 +129,3 @@ void free_page(void *page)
 	freelist = page;
 	spin_unlock(&lock);
 }
-
diff --git a/lib/alloc_phys.c b/lib/alloc_phys.c
index 3972277..6befb5c 100644
--- a/lib/alloc_phys.c
+++ b/lib/alloc_phys.c
@@ -107,6 +107,12 @@ static phys_addr_t phys_alloc_aligned_safe(phys_addr_t size,
 	return addr;
 }
 
+void phys_alloc_get_unused(phys_addr_t *p_base, phys_addr_t *p_top)
+{
+	*p_base = base;
+	*p_top = top;
+}
+
 static void *early_memalign(size_t alignment, size_t size)
 {
 	phys_addr_t addr;
diff --git a/lib/alloc_phys.h b/lib/alloc_phys.h
index f09d667..848d3db 100644
--- a/lib/alloc_phys.h
+++ b/lib/alloc_phys.h
@@ -36,4 +36,10 @@ extern void phys_alloc_set_minimum_alignment(phys_addr_t align);
  */
 extern void phys_alloc_show(void);
 
+/*
+ * phys_alloc_get_unused returns the addresses for the still-unused part
+ * of the initial free memory region passed to phys_alloc_init.
+ */
+extern void phys_alloc_get_unused(phys_addr_t *p_base, phys_addr_t *p_top);
+
 #endif /* _ALLOC_PHYS_H_ */
diff --git a/lib/libcflat.h b/lib/libcflat.h
index a5c4290..fb46541 100644
--- a/lib/libcflat.h
+++ b/lib/libcflat.h
@@ -156,4 +156,6 @@ static inline bool is_power_of_2(unsigned long n)
 void binstr(unsigned long x, char out[BINSTR_SZ]);
 void print_binstr(unsigned long x);
 
+extern void setup_vm(void);
+
 #endif
diff --git a/lib/vmalloc.c b/lib/vmalloc.c
index 06298f4..efeb100 100644
--- a/lib/vmalloc.c
+++ b/lib/vmalloc.c
@@ -8,9 +8,14 @@
 #include "libcflat.h"
 #include "asm/spinlock.h"
 #include "asm/page.h"
+#include "asm/io.h"
+#include "alloc_phys.h"
+#include "alloc_page.h"
+#include "vmalloc.h"
 
 static struct spinlock lock;
 static void *vfree_top = 0;
+static void *page_root;
 
 void *alloc_vpages(ulong nr)
 {
@@ -29,3 +34,13 @@ void init_alloc_vpage(void *top)
 {
 	vfree_top = top;
 }
+
+void setup_vm()
+{
+	phys_addr_t base, top;
+	phys_alloc_get_unused(&base, &top);
+	base = (base + PAGE_SIZE - 1) & -PAGE_SIZE;
+	top = top & -PAGE_SIZE;
+	free_pages(phys_to_virt(base), top - base);
+	page_root = setup_mmu(top);
+}
diff --git a/lib/vmalloc.h b/lib/vmalloc.h
index 4941fef..285a0fc 100644
--- a/lib/vmalloc.h
+++ b/lib/vmalloc.h
@@ -4,5 +4,8 @@
 extern void *alloc_vpages(ulong nr);
 extern void *alloc_vpage(void);
 extern void init_alloc_vpage(void *top);
+extern void setup_vm();
+
+extern void *setup_mmu(phys_addr_t top);
 
 #endif
diff --git a/lib/x86/setup.c b/lib/x86/setup.c
index 22257a7..f6a8ea8 100644
--- a/lib/x86/setup.c
+++ b/lib/x86/setup.c
@@ -53,6 +53,10 @@ void setup_multiboot(struct mbi_bootinfo *bootinfo)
 
 	initrd = (char *)(uintptr_t) mods->start;
 	initrd_size = mods->end - mods->start;
+
+	/* TODO: use e820 */
+	u64 end_of_memory = bootinfo->mem_upper * 1024ull;
+	phys_alloc_init((uintptr_t) &edata, end_of_memory - (uintptr_t)edata);
 }
 
 void setup_libcflat(void)
diff --git a/lib/x86/vm.c b/lib/x86/vm.c
index d2a3c59..a5d3aeb 100644
--- a/lib/x86/vm.c
+++ b/lib/x86/vm.c
@@ -1,12 +1,8 @@
-#include "fwcfg.h"
 #include "vm.h"
 #include "libcflat.h"
 #include "vmalloc.h"
 #include "alloc_page.h"
 
-extern char edata;
-static unsigned long end_of_memory;
-
 unsigned long *install_pte(unsigned long *cr3,
 			   int pte_level,
 			   void *virt,
@@ -149,23 +145,23 @@ static void setup_mmu_range(unsigned long *cr3, unsigned long start,
 	install_pages(cr3, phys, max - phys, (void *)(ulong)phys);
 }
 
-static void setup_mmu(unsigned long len)
+void *setup_mmu(phys_addr_t end_of_memory)
 {
     unsigned long *cr3 = alloc_page();
 
     memset(cr3, 0, PAGE_SIZE);
 
 #ifdef __x86_64__
-    if (len < (1ul << 32))
-        len = (1ul << 32);  /* map mmio 1:1 */
+    if (end_of_memory < (1ul << 32))
+        end_of_memory = (1ul << 32);  /* map mmio 1:1 */
 
-    setup_mmu_range(cr3, 0, len);
+    setup_mmu_range(cr3, 0, end_of_memory);
 #else
-    if (len > (1ul << 31))
-	    len = (1ul << 31);
+    if (end_of_memory > (1ul << 31))
+	    end_of_memory = (1ul << 31);
 
     /* 0 - 2G memory, 2G-3G valloc area, 3G-4G mmio */
-    setup_mmu_range(cr3, 0, len);
+    setup_mmu_range(cr3, 0, end_of_memory);
     setup_mmu_range(cr3, 3ul << 30, (1ul << 30));
     init_alloc_vpage((void*)(3ul << 30));
 #endif
@@ -180,14 +176,7 @@ static void setup_mmu(unsigned long len)
     printf("cr0 = %lx\n", read_cr0());
     printf("cr3 = %lx\n", read_cr3());
     printf("cr4 = %lx\n", read_cr4());
-}
-
-void setup_vm()
-{
-    assert(!end_of_memory);
-    end_of_memory = fwcfg_get_u64(FW_CFG_RAM_SIZE);
-    free_pages(&edata, end_of_memory - (unsigned long)&edata);
-    setup_mmu(end_of_memory);
+    return cr3;
 }
 
 void *vmalloc(unsigned long size)
diff --git a/lib/x86/vm.h b/lib/x86/vm.h
index 4a7888b..c3211eb 100644
--- a/lib/x86/vm.h
+++ b/lib/x86/vm.h
@@ -5,7 +5,6 @@
 #include "asm/page.h"
 #include "asm/io.h"
 
-void setup_vm();
 void setup_5level_page_table();
 
 void *vmalloc(unsigned long size);
diff --git a/x86/Makefile.common b/x86/Makefile.common
index fce0801..f0fbadf 100644
--- a/x86/Makefile.common
+++ b/x86/Makefile.common
@@ -6,6 +6,7 @@ cflatobjs += lib/pci.o
 cflatobjs += lib/pci-edu.o
 cflatobjs += lib/vmalloc.o
 cflatobjs += lib/alloc_page.o
+cflatobjs += lib/alloc_phys.o
 cflatobjs += lib/x86/setup.o
 cflatobjs += lib/x86/io.o
 cflatobjs += lib/x86/smp.o
-- 
2.14.2

[PATCH kvm-unit-tests 08/18] alloc: implement free

[Paolo Bonzini]

Track block size in lib/alloc.c and not in alloc_ops.  alloc_ops->free
gets the size from the malloc implementation.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 arm/Makefile.common     |  1 +
 lib/alloc.c             | 66 +++++++++++++++++++++++++++++++++++++++++++++++++
 lib/alloc.h             | 33 ++++++-------------------
 lib/alloc_phys.c        | 23 +++++++++--------
 powerpc/Makefile.common |  1 +
 5 files changed, 86 insertions(+), 38 deletions(-)
 create mode 100644 lib/alloc.c

diff --git a/arm/Makefile.common b/arm/Makefile.common
index 6979a24..e09c3e2 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -35,6 +35,7 @@ include $(SRCDIR)/scripts/asm-offsets.mak
 
 cflatobjs += lib/util.o
 cflatobjs += lib/alloc_phys.o
+cflatobjs += lib/alloc.o
 cflatobjs += lib/devicetree.o
 cflatobjs += lib/pci.o
 cflatobjs += lib/pci-host-generic.o
diff --git a/lib/alloc.c b/lib/alloc.c
new file mode 100644
index 0000000..61cb292
--- /dev/null
+++ b/lib/alloc.c
@@ -0,0 +1,66 @@
+#include "alloc.h"
+
+void *malloc(size_t size)
+{
+	return memalign(sizeof(long), size);
+}
+
+void *calloc(size_t nmemb, size_t size)
+{
+	void *ptr = malloc(nmemb * size);
+	if (ptr)
+		memset(ptr, 0, nmemb * size);
+	return ptr;
+}
+
+#define METADATA_EXTRA	(2 * sizeof(uintptr_t))
+#define OFS_SLACK	(-2 * sizeof(uintptr_t))
+#define OFS_SIZE	(-sizeof(uintptr_t))
+
+static inline void *block_begin(void *mem)
+{
+	uintptr_t slack = *(uintptr_t *)(mem + OFS_SLACK);
+	return mem - slack;
+}
+
+static inline uintptr_t block_size(void *mem)
+{
+	return *(uintptr_t *)(mem + OFS_SIZE);
+}
+
+void free(void *ptr)
+{
+	if (!alloc_ops->free)
+		return;
+
+	void *base = block_begin(ptr);
+	uintptr_t sz = block_size(ptr);
+
+	alloc_ops->free(base, sz);
+}
+
+void *memalign(size_t alignment, size_t size)
+{
+	void *p;
+	uintptr_t blkalign;
+	uintptr_t mem;
+
+	assert(alloc_ops && alloc_ops->memalign);
+	if (alignment <= sizeof(uintptr_t))
+		alignment = sizeof(uintptr_t);
+	else
+		size += alignment - 1;
+
+	blkalign = MAX(alignment, alloc_ops->align_min);
+	size = ALIGN(size + METADATA_EXTRA, alloc_ops->align_min);
+	p = alloc_ops->memalign(blkalign, size);
+
+	/* Leave room for metadata before aligning the result.  */
+	mem = (uintptr_t)p + METADATA_EXTRA;
+	mem = ALIGN(mem, alignment);
+
+	/* Write the metadata */
+	*(uintptr_t *)(mem + OFS_SLACK) = mem - (uintptr_t)p;
+	*(uintptr_t *)(mem + OFS_SIZE) = size;
+	return (void *)mem;
+}
diff --git a/lib/alloc.h b/lib/alloc.h
index 24f85b4..f560c4a 100644
--- a/lib/alloc.h
+++ b/lib/alloc.h
@@ -24,36 +24,17 @@
 
 struct alloc_ops {
 	void *(*memalign)(size_t alignment, size_t size);
+	void (*free)(void *ptr, size_t size);
+	size_t align_min;
 };
 
 extern struct alloc_ops *alloc_ops;
 
-/*
- * Our malloc implementation is currently so simple that it can just
- * be inlined. :)
- */
-static inline void *malloc(size_t size)
-{
-	assert(alloc_ops && alloc_ops->memalign);
-	return alloc_ops->memalign(sizeof(long), size);
-}
-
-static inline void *calloc(size_t nmemb, size_t size)
-{
-	void *ptr = malloc(nmemb * size);
-	if (ptr)
-		memset(ptr, 0, nmemb * size);
-	return ptr;
-}
+void *malloc(size_t size);
+void *calloc(size_t nmemb, size_t size);
+void free(void *ptr);
+void *memalign(size_t alignment, size_t size);
 
-static inline void free(void *ptr)
-{
-}
-
-static inline void *memalign(size_t alignment, size_t size)
-{
-	assert(alloc_ops && alloc_ops->memalign);
-	return alloc_ops->memalign(alignment, size);
-}
+extern struct alloc_ops *alloc_ops;
 
 #endif /* _ALLOC_H_ */
diff --git a/lib/alloc_phys.c b/lib/alloc_phys.c
index 6befb5c..d09395d 100644
--- a/lib/alloc_phys.c
+++ b/lib/alloc_phys.c
@@ -23,7 +23,15 @@ static struct phys_alloc_region regions[PHYS_ALLOC_NR_REGIONS];
 static int nr_regions;
 
 static struct spinlock lock;
-static phys_addr_t base, top, align_min;
+static phys_addr_t base, top;
+
+static void *early_memalign(size_t alignment, size_t size);
+static struct alloc_ops early_alloc_ops = {
+	.memalign = early_memalign,
+	.align_min = DEFAULT_MINIMUM_ALIGNMENT
+};
+
+struct alloc_ops *alloc_ops = &early_alloc_ops;
 
 void phys_alloc_show(void)
 {
@@ -31,7 +39,7 @@ void phys_alloc_show(void)
 
 	spin_lock(&lock);
 	printf("phys_alloc minimum alignment: %#" PRIx64 "\n",
-		(u64)align_min);
+		(u64)early_alloc_ops.align_min);
 	for (i = 0; i < nr_regions; ++i)
 		printf("%016" PRIx64 "-%016" PRIx64 " [%s]\n",
 			(u64)regions[i].base,
@@ -47,7 +55,6 @@ void phys_alloc_init(phys_addr_t base_addr, phys_addr_t size)
 	spin_lock(&lock);
 	base = base_addr;
 	top = base + size;
-	align_min = DEFAULT_MINIMUM_ALIGNMENT;
 	nr_regions = 0;
 	spin_unlock(&lock);
 }
@@ -56,7 +63,7 @@ void phys_alloc_set_minimum_alignment(phys_addr_t align)
 {
 	assert(align && !(align & (align - 1)));
 	spin_lock(&lock);
-	align_min = align;
+	early_alloc_ops.align_min = align;
 	spin_unlock(&lock);
 }
 
@@ -74,8 +81,6 @@ static phys_addr_t phys_alloc_aligned_safe(phys_addr_t size,
 	if (safe && sizeof(long) == 4)
 		top_safe = MIN(top_safe, 1ULL << 32);
 
-	align = MAX(align, align_min);
-
 	addr = ALIGN(base, align);
 	size += addr - base;
 
@@ -125,9 +130,3 @@ static void *early_memalign(size_t alignment, size_t size)
 
 	return phys_to_virt(addr);
 }
-
-static struct alloc_ops early_alloc_ops = {
-	.memalign = early_memalign,
-};
-
-struct alloc_ops *alloc_ops = &early_alloc_ops;
diff --git a/powerpc/Makefile.common b/powerpc/Makefile.common
index 126590a..21dea40 100644
--- a/powerpc/Makefile.common
+++ b/powerpc/Makefile.common
@@ -31,6 +31,7 @@ include $(SRCDIR)/scripts/asm-offsets.mak
 
 cflatobjs += lib/util.o
 cflatobjs += lib/alloc_phys.o
+cflatobjs += lib/alloc.o
 cflatobjs += lib/devicetree.o
 cflatobjs += lib/powerpc/io.o
 cflatobjs += lib/powerpc/hcall.o
-- 
2.14.2

[PATCH kvm-unit-tests 09/18] x86: introduce pgd_t and pteval_t

[Paolo Bonzini]

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 lib/x86/asm/page.h |  3 +++
 lib/x86/vm.c       | 60 ++++++++++++++++++++++++------------------------------
 lib/x86/vm.h       | 30 +++++++++++++--------------
 3 files changed, 44 insertions(+), 49 deletions(-)

diff --git a/lib/x86/asm/page.h b/lib/x86/asm/page.h
index 562594d..073580a 100644
--- a/lib/x86/asm/page.h
+++ b/lib/x86/asm/page.h
@@ -10,6 +10,9 @@
 #include <linux/const.h>
 #include <bitops.h>
 
+typedef unsigned long pteval_t;
+typedef unsigned long pgd_t;
+
 #define PAGE_SHIFT	12
 #define PAGE_SIZE	(_AC(1,UL) << PAGE_SHIFT)
 #define PAGE_MASK	(~(PAGE_SIZE-1))
diff --git a/lib/x86/vm.c b/lib/x86/vm.c
index a5d3aeb..0fcc94c 100644
--- a/lib/x86/vm.c
+++ b/lib/x86/vm.c
@@ -3,20 +3,20 @@
 #include "vmalloc.h"
 #include "alloc_page.h"
 
-unsigned long *install_pte(unsigned long *cr3,
-			   int pte_level,
-			   void *virt,
-			   unsigned long pte,
-			   unsigned long *pt_page)
+pteval_t *install_pte(pgd_t *cr3,
+		      int pte_level,
+		      void *virt,
+		      pteval_t pte,
+		      pteval_t *pt_page)
 {
     int level;
-    unsigned long *pt = cr3;
+    pteval_t *pt = cr3;
     unsigned offset;
 
     for (level = PAGE_LEVEL; level > pte_level; --level) {
-	offset = PGDIR_OFFSET((unsigned long)virt, level);
+	offset = PGDIR_OFFSET((uintptr_t)virt, level);
 	if (!(pt[offset] & PT_PRESENT_MASK)) {
-	    unsigned long *new_pt = pt_page;
+	    pteval_t *new_pt = pt_page;
             if (!new_pt)
                 new_pt = alloc_page();
             else
@@ -26,7 +26,7 @@ unsigned long *install_pte(unsigned long *cr3,
 	}
 	pt = phys_to_virt(pt[offset] & PT_ADDR_MASK);
     }
-    offset = PGDIR_OFFSET((unsigned long)virt, level);
+    offset = PGDIR_OFFSET((uintptr_t)virt, level);
     pt[offset] = pte;
     return &pt[offset];
 }
@@ -35,19 +35,19 @@ unsigned long *install_pte(unsigned long *cr3,
  * Finds last PTE in the mapping of @virt that's at or above @lowest_level. The
  * returned PTE isn't necessarily present, but its parent is.
  */
-struct pte_search find_pte_level(unsigned long *cr3, void *virt,
+struct pte_search find_pte_level(pgd_t *cr3, void *virt,
 				 int lowest_level)
 {
-	unsigned long *pt = cr3, pte;
+	pteval_t *pt = cr3, pte;
 	unsigned offset;
-	unsigned long shift;
+	unsigned shift;
 	struct pte_search r;
 
 	assert(lowest_level >= 1 && lowest_level <= PAGE_LEVEL);
 
 	for (r.level = PAGE_LEVEL;; --r.level) {
 		shift = (r.level - 1) * PGDIR_WIDTH + 12;
-		offset = ((unsigned long)virt >> shift) & PGDIR_MASK;
+		offset = ((uintptr_t)virt >> shift) & PGDIR_MASK;
 		r.pte = &pt[offset];
 		pte = *r.pte;
 
@@ -68,7 +68,7 @@ struct pte_search find_pte_level(unsigned long *cr3, void *virt,
  * Returns the leaf PTE in the mapping of @virt (i.e., 4K PTE or a present huge
  * PTE). Returns NULL if no leaf PTE exists.
  */
-unsigned long *get_pte(unsigned long *cr3, void *virt)
+pteval_t *get_pte(pgd_t *cr3, void *virt)
 {
 	struct pte_search search;
 
@@ -81,7 +81,7 @@ unsigned long *get_pte(unsigned long *cr3, void *virt)
  * Returns NULL if the PT at @pte_level isn't present (i.e., the mapping at
  * @pte_level - 1 isn't present).
  */
-unsigned long *get_pte_level(unsigned long *cr3, void *virt, int pte_level)
+pteval_t *get_pte_level(pgd_t *cr3, void *virt, int pte_level)
 {
 	struct pte_search search;
 
@@ -89,27 +89,22 @@ unsigned long *get_pte_level(unsigned long *cr3, void *virt, int pte_level)
 	return search.level == pte_level ? search.pte : NULL;
 }
 
-unsigned long *install_large_page(unsigned long *cr3,
-				  unsigned long phys,
-				  void *virt)
+pteval_t *install_large_page(pgd_t *cr3, phys_addr_t phys, void *virt)
 {
     return install_pte(cr3, 2, virt,
 		       phys | PT_PRESENT_MASK | PT_WRITABLE_MASK | PT_USER_MASK | PT_PAGE_SIZE_MASK, 0);
 }
 
-unsigned long *install_page(unsigned long *cr3,
-			    unsigned long phys,
-			    void *virt)
+pteval_t *install_page(pgd_t *cr3, phys_addr_t phys, void *virt)
 {
     return install_pte(cr3, 1, virt, phys | PT_PRESENT_MASK | PT_WRITABLE_MASK | PT_USER_MASK, 0);
 }
 
-void install_pages(unsigned long *cr3, unsigned long phys, unsigned long len,
-		   void *virt)
+void install_pages(pgd_t *cr3, phys_addr_t phys, size_t len, void *virt)
 {
-	unsigned long max = (u64)len + (u64)phys;
+	phys_addr_t max = (u64)len + (u64)phys;
 	assert(phys % PAGE_SIZE == 0);
-	assert((unsigned long) virt % PAGE_SIZE == 0);
+	assert((uintptr_t) virt % PAGE_SIZE == 0);
 	assert(len % PAGE_SIZE == 0);
 
 	while (phys + PAGE_SIZE <= max) {
@@ -119,21 +114,20 @@ void install_pages(unsigned long *cr3, unsigned long phys, unsigned long len,
 	}
 }
 
-bool any_present_pages(unsigned long *cr3, void *virt, unsigned long len)
+bool any_present_pages(pgd_t *cr3, void *virt, size_t len)
 {
-	unsigned long max = (unsigned long) virt + len;
-	unsigned long curr;
+	uintptr_t max = (uintptr_t) virt + len;
+	uintptr_t curr;
 
-	for (curr = (unsigned long) virt; curr < max; curr += PAGE_SIZE) {
-		unsigned long *ptep = get_pte(cr3, (void *) curr);
+	for (curr = (uintptr_t) virt; curr < max; curr += PAGE_SIZE) {
+		pteval_t *ptep = get_pte(cr3, (void *) curr);
 		if (ptep && (*ptep & PT_PRESENT_MASK))
 			return true;
 	}
 	return false;
 }
 
-static void setup_mmu_range(unsigned long *cr3, unsigned long start,
-			    unsigned long len)
+static void setup_mmu_range(pgd_t *cr3, phys_addr_t start, size_t len)
 {
 	u64 max = (u64)len + (u64)start;
 	u64 phys = start;
@@ -147,7 +141,7 @@ static void setup_mmu_range(unsigned long *cr3, unsigned long start,
 
 void *setup_mmu(phys_addr_t end_of_memory)
 {
-    unsigned long *cr3 = alloc_page();
+    pgd_t *cr3 = alloc_page();
 
     memset(cr3, 0, PAGE_SIZE);
 
diff --git a/lib/x86/vm.h b/lib/x86/vm.h
index c3211eb..ac0529a 100644
--- a/lib/x86/vm.h
+++ b/lib/x86/vm.h
@@ -14,7 +14,7 @@ uint64_t virt_to_phys_cr3(void *mem);
 
 struct pte_search {
 	int level;
-	unsigned long *pte;
+	pteval_t *pte;
 };
 
 static inline bool found_huge_pte(struct pte_search search)
@@ -29,22 +29,20 @@ static inline bool found_leaf_pte(struct pte_search search)
 	return search.level == 1 || found_huge_pte(search);
 }
 
-struct pte_search find_pte_level(unsigned long *cr3, void *virt,
+struct pte_search find_pte_level(pgd_t *cr3, void *virt,
 				 int lowest_level);
-unsigned long *get_pte(unsigned long *cr3, void *virt);
-unsigned long *get_pte_level(unsigned long *cr3, void *virt, int pte_level);
-unsigned long *install_pte(unsigned long *cr3,
-                           int pte_level,
-                           void *virt,
-                           unsigned long pte,
-                           unsigned long *pt_page);
-
-unsigned long *install_large_page(unsigned long *cr3,unsigned long phys,
-                                  void *virt);
-unsigned long *install_page(unsigned long *cr3, unsigned long phys, void *virt);
-void install_pages(unsigned long *cr3, unsigned long phys, unsigned long len,
-		   void *virt);
-bool any_present_pages(unsigned long *cr3, void *virt, unsigned long len);
+pteval_t *get_pte(pgd_t *cr3, void *virt);
+pteval_t *get_pte_level(pgd_t *cr3, void *virt, int pte_level);
+pteval_t *install_pte(pgd_t *cr3,
+		      int pte_level,
+		      void *virt,
+		      pteval_t pte,
+		      pteval_t *pt_page);
+
+pteval_t *install_large_page(pgd_t *cr3, phys_addr_t phys, void *virt);
+pteval_t *install_page(pgd_t *cr3, phys_addr_t phys, void *virt);
+void install_pages(pgd_t *cr3, phys_addr_t phys, size_t len, void *virt);
+bool any_present_pages(pgd_t *cr3, void *virt, size_t len);
 
 static inline void *current_page_table(void)
 {
-- 
2.14.2

[PATCH kvm-unit-tests 10/18] x86: turn virt_to_phys_cr3 into virt_to_pte_phys

[Paolo Bonzini]

For consistency with install_page, make it accept a root page table
pointer.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 lib/vmalloc.h | 1 +
 lib/x86/vm.c  | 4 ++--
 lib/x86/vm.h  | 1 -
 x86/asyncpf.c | 3 ++-
 4 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/lib/vmalloc.h b/lib/vmalloc.h
index 285a0fc..ec3fe36 100644
--- a/lib/vmalloc.h
+++ b/lib/vmalloc.h
@@ -7,5 +7,6 @@ extern void init_alloc_vpage(void *top);
 extern void setup_vm();
 
 extern void *setup_mmu(phys_addr_t top);
+extern phys_addr_t virt_to_pte_phys(pgd_t *pgtable, void *virt);
 
 #endif
diff --git a/lib/x86/vm.c b/lib/x86/vm.c
index 0fcc94c..223f14a 100644
--- a/lib/x86/vm.c
+++ b/lib/x86/vm.c
@@ -192,9 +192,9 @@ void *vmalloc(unsigned long size)
     return mem;
 }
 
-uint64_t virt_to_phys_cr3(void *mem)
+phys_addr_t virt_to_pte_phys(pgd_t *cr3, void *mem)
 {
-    return (*get_pte(phys_to_virt(read_cr3()), mem) & PT_ADDR_MASK) + ((ulong)mem & (PAGE_SIZE - 1));
+    return (*get_pte(cr3, mem) & PT_ADDR_MASK) + ((ulong)mem & (PAGE_SIZE - 1));
 }
 
 void vfree(void *mem)
diff --git a/lib/x86/vm.h b/lib/x86/vm.h
index ac0529a..1d8b805 100644
--- a/lib/x86/vm.h
+++ b/lib/x86/vm.h
@@ -10,7 +10,6 @@ void setup_5level_page_table();
 void *vmalloc(unsigned long size);
 void vfree(void *mem);
 void *vmap(unsigned long long phys, unsigned long size);
-uint64_t virt_to_phys_cr3(void *mem);
 
 struct pte_search {
 	int level;
diff --git a/x86/asyncpf.c b/x86/asyncpf.c
index f04d7cb..ee8baf9 100644
--- a/x86/asyncpf.c
+++ b/x86/asyncpf.c
@@ -23,6 +23,7 @@
 #include "x86/vm.h"
 
 #include "libcflat.h"
+#include "vmalloc.h"
 #include <stdint.h>
 
 #define KVM_PV_REASON_PAGE_NOT_PRESENT 1
@@ -55,7 +56,7 @@ static void pf_isr(struct ex_regs *r)
 			report("unexpected #PF at %#lx", false, read_cr2());
 			break;
 		case KVM_PV_REASON_PAGE_NOT_PRESENT:
-			phys = virt_to_phys_cr3(virt);
+			phys = virt_to_pte_phys(phys_to_virt(read_cr3()), virt);
 			install_pte(phys_to_virt(read_cr3()), 1, virt, phys, 0);
 			write_cr3(read_cr3());
 			report("Got not present #PF token %lx virt addr %p phys addr %#" PRIx64,
-- 
2.14.2

[PATCH kvm-unit-tests 11/18] vmalloc: convert to alloc_ops

[Paolo Bonzini]

The final step moves the vmalloc and vmap implementation to generic code,
rewriting vmalloc and vfree as an alloc_ops implementation that is
installed by setup_vm.

This way all architectures can benefit from it just by calling setup_vm
and providing the implementation of install_page.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 lib/alloc.c         |  1 +
 lib/alloc.h         |  2 --
 lib/vmalloc.c       | 56 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 lib/vmalloc.h       |  5 +++++
 lib/x86/io.c        |  2 +-
 lib/x86/isr.c       |  5 +++--
 lib/x86/vm.c        | 48 ---------------------------------------------
 lib/x86/vm.h        |  5 -----
 x86/Makefile.common |  1 +
 x86/asyncpf.c       |  4 +++-
 x86/pmu.c           |  4 ++--
 x86/rmap_chain.c    |  1 +
 x86/sieve.c         |  6 +++---
 13 files changed, 76 insertions(+), 64 deletions(-)

diff --git a/lib/alloc.c b/lib/alloc.c
index 61cb292..ecdbbc4 100644
--- a/lib/alloc.c
+++ b/lib/alloc.c
@@ -1,4 +1,5 @@
 #include "alloc.h"
+#include "asm/page.h"
 
 void *malloc(size_t size)
 {
diff --git a/lib/alloc.h b/lib/alloc.h
index f560c4a..c44d459 100644
--- a/lib/alloc.h
+++ b/lib/alloc.h
@@ -35,6 +35,4 @@ void *calloc(size_t nmemb, size_t size);
 void free(void *ptr);
 void *memalign(size_t alignment, size_t size);
 
-extern struct alloc_ops *alloc_ops;
-
 #endif /* _ALLOC_H_ */
diff --git a/lib/vmalloc.c b/lib/vmalloc.c
index efeb100..ca36bf8 100644
--- a/lib/vmalloc.c
+++ b/lib/vmalloc.c
@@ -9,6 +9,7 @@
 #include "asm/spinlock.h"
 #include "asm/page.h"
 #include "asm/io.h"
+#include "alloc.h"
 #include "alloc_phys.h"
 #include "alloc_page.h"
 #include "vmalloc.h"
@@ -35,12 +36,67 @@ void init_alloc_vpage(void *top)
 	vfree_top = top;
 }
 
+void *vmap(phys_addr_t phys, size_t size)
+{
+	void *mem, *p;
+	unsigned pages;
+
+	size = (size + PAGE_SIZE - 1) & ~(PAGE_SIZE - 1);
+	pages = size / PAGE_SIZE;
+	mem = p = alloc_vpages(pages);
+
+	phys &= ~(unsigned long long)(PAGE_SIZE - 1);
+	while (pages--) {
+		install_page(page_root, phys, p);
+		phys += PAGE_SIZE;
+		p += PAGE_SIZE;
+	}
+	return mem;
+}
+
+static void *vm_memalign(size_t alignment, size_t size)
+{
+	void *mem, *p;
+	unsigned pages;
+
+	assert(alignment <= PAGE_SIZE);
+	size = (size + PAGE_SIZE - 1) & ~(PAGE_SIZE - 1);
+	pages = size / PAGE_SIZE;
+	mem = p = alloc_vpages(pages);
+	while (pages--) {
+		phys_addr_t pa = virt_to_phys(alloc_page());
+		install_page(page_root, pa, p);
+		p += PAGE_SIZE;
+	}
+	return mem;
+}
+
+static void vm_free(void *mem, size_t size)
+{
+	while (size) {
+		free_page(phys_to_virt(virt_to_pte_phys(page_root, mem)));
+		mem += PAGE_SIZE;
+		size -= PAGE_SIZE;
+	}
+}
+
+static struct alloc_ops vmalloc_ops = {
+	.memalign = vm_memalign,
+	.free = vm_free,
+	.align_min = PAGE_SIZE,
+};
+
 void setup_vm()
 {
 	phys_addr_t base, top;
+
+	if (alloc_ops == &vmalloc_ops)
+		return;
+
 	phys_alloc_get_unused(&base, &top);
 	base = (base + PAGE_SIZE - 1) & -PAGE_SIZE;
 	top = top & -PAGE_SIZE;
 	free_pages(phys_to_virt(base), top - base);
 	page_root = setup_mmu(top);
+	alloc_ops = &vmalloc_ops;
 }
diff --git a/lib/vmalloc.h b/lib/vmalloc.h
index ec3fe36..b4cde65 100644
--- a/lib/vmalloc.h
+++ b/lib/vmalloc.h
@@ -1,6 +1,8 @@
 #ifndef VMALLOC_H
 #define VMALLOC_H 1
 
+#include <asm/page.h>
+
 extern void *alloc_vpages(ulong nr);
 extern void *alloc_vpage(void);
 extern void init_alloc_vpage(void *top);
@@ -8,5 +10,8 @@ extern void setup_vm();
 
 extern void *setup_mmu(phys_addr_t top);
 extern phys_addr_t virt_to_pte_phys(pgd_t *pgtable, void *virt);
+extern pteval_t *install_page(pgd_t *pgtable, phys_addr_t phys, void *virt);
+
+void *vmap(phys_addr_t phys, size_t size);
 
 #endif
diff --git a/lib/x86/io.c b/lib/x86/io.c
index cc5ac58..7e1c16d 100644
--- a/lib/x86/io.c
+++ b/lib/x86/io.c
@@ -1,8 +1,8 @@
 #include "libcflat.h"
-#include "vm.h"
 #include "smp.h"
 #include "asm/io.h"
 #include "asm/page.h"
+#include "vmalloc.h"
 #ifndef USE_SERIAL
 #define USE_SERIAL
 #endif
diff --git a/lib/x86/isr.c b/lib/x86/isr.c
index 9b1d505..53c0c2b 100644
--- a/lib/x86/isr.c
+++ b/lib/x86/isr.c
@@ -1,6 +1,7 @@
 #include "libcflat.h"
+#include "processor.h"
 #include "isr.h"
-#include "vm.h"
+#include "alloc.h"
 #include "desc.h"
 
 extern char isr_entry_point[];
@@ -65,7 +66,7 @@ asm (
 
 void handle_irq(unsigned vec, void (*func)(isr_regs_t *regs))
 {
-    u8 *thunk = vmalloc(50);
+    u8 *thunk = malloc(50);
 
     set_idt_entry(vec, thunk, 0);
 
diff --git a/lib/x86/vm.c b/lib/x86/vm.c
index 223f14a..73d9be4 100644
--- a/lib/x86/vm.c
+++ b/lib/x86/vm.c
@@ -173,55 +173,7 @@ void *setup_mmu(phys_addr_t end_of_memory)
     return cr3;
 }
 
-void *vmalloc(unsigned long size)
-{
-    void *mem, *p;
-    unsigned pages;
-
-    size += sizeof(unsigned long);
-
-    size = (size + PAGE_SIZE - 1) & ~(PAGE_SIZE - 1);
-    pages = size / PAGE_SIZE;
-    mem = p = alloc_vpages(pages);
-    while (pages--) {
-	install_page(phys_to_virt(read_cr3()), virt_to_phys(alloc_page()), p);
-	p += PAGE_SIZE;
-    }
-    *(unsigned long *)mem = size;
-    mem += sizeof(unsigned long);
-    return mem;
-}
-
 phys_addr_t virt_to_pte_phys(pgd_t *cr3, void *mem)
 {
     return (*get_pte(cr3, mem) & PT_ADDR_MASK) + ((ulong)mem & (PAGE_SIZE - 1));
 }
-
-void vfree(void *mem)
-{
-    unsigned long size = ((unsigned long *)mem)[-1];
-
-    while (size) {
-	free_page(phys_to_virt(*get_pte(phys_to_virt(read_cr3()), mem) & PT_ADDR_MASK));
-	mem += PAGE_SIZE;
-	size -= PAGE_SIZE;
-    }
-}
-
-void *vmap(unsigned long long phys, unsigned long size)
-{
-    void *mem, *p;
-    unsigned pages;
-
-    size = (size + PAGE_SIZE - 1) & ~(PAGE_SIZE - 1);
-    pages = size / PAGE_SIZE;
-    mem = p = alloc_vpages(pages);
-
-    phys &= ~(unsigned long long)(PAGE_SIZE - 1);
-    while (pages--) {
-	install_page(phys_to_virt(read_cr3()), phys, p);
-	phys += PAGE_SIZE;
-	p += PAGE_SIZE;
-    }
-    return mem;
-}
diff --git a/lib/x86/vm.h b/lib/x86/vm.h
index 1d8b805..ae1b769 100644
--- a/lib/x86/vm.h
+++ b/lib/x86/vm.h
@@ -7,10 +7,6 @@
 
 void setup_5level_page_table();
 
-void *vmalloc(unsigned long size);
-void vfree(void *mem);
-void *vmap(unsigned long long phys, unsigned long size);
-
 struct pte_search {
 	int level;
 	pteval_t *pte;
@@ -39,7 +35,6 @@ pteval_t *install_pte(pgd_t *cr3,
 		      pteval_t *pt_page);
 
 pteval_t *install_large_page(pgd_t *cr3, phys_addr_t phys, void *virt);
-pteval_t *install_page(pgd_t *cr3, phys_addr_t phys, void *virt);
 void install_pages(pgd_t *cr3, phys_addr_t phys, size_t len, void *virt);
 bool any_present_pages(pgd_t *cr3, void *virt, size_t len);
 
diff --git a/x86/Makefile.common b/x86/Makefile.common
index f0fbadf..5f7eac4 100644
--- a/x86/Makefile.common
+++ b/x86/Makefile.common
@@ -4,6 +4,7 @@ all: directories test_cases
 
 cflatobjs += lib/pci.o
 cflatobjs += lib/pci-edu.o
+cflatobjs += lib/alloc.o
 cflatobjs += lib/vmalloc.o
 cflatobjs += lib/alloc_page.o
 cflatobjs += lib/alloc_phys.o
diff --git a/x86/asyncpf.c b/x86/asyncpf.c
index ee8baf9..de3d592 100644
--- a/x86/asyncpf.c
+++ b/x86/asyncpf.c
@@ -22,6 +22,8 @@
 #include "x86/isr.h"
 #include "x86/vm.h"
 
+#include "asm/page.h"
+#include "alloc.h"
 #include "libcflat.h"
 #include "vmalloc.h"
 #include <stdint.h>
@@ -95,7 +97,7 @@ int main(int ac, char **av)
 	wrmsr(MSR_KVM_ASYNC_PF_EN, virt_to_phys((void*)&apf_reason) |
 			KVM_ASYNC_PF_SEND_ALWAYS | KVM_ASYNC_PF_ENABLED);
 	printf("alloc memory\n");
-	buf = vmalloc(MEM);
+	buf = malloc(MEM);
 	irq_enable();
 	while(loop--) {
 		printf("start loop\n");
diff --git a/x86/pmu.c b/x86/pmu.c
index b56b61e..12a5880 100644
--- a/x86/pmu.c
+++ b/x86/pmu.c
@@ -5,7 +5,7 @@
 #include "x86/apic.h"
 #include "x86/desc.h"
 #include "x86/isr.h"
-#include "x86/vm.h"
+#include "alloc.h"
 
 #include "libcflat.h"
 #include <stdint.h>
@@ -380,7 +380,7 @@ int main(int ac, char **av)
 	setup_vm();
 	setup_idt();
 	handle_irq(PC_VECTOR, cnt_overflow);
-	buf = vmalloc(N*64);
+	buf = malloc(N*64);
 
 	eax.full = id.a;
 	ebx.full = id.b;
diff --git a/x86/rmap_chain.c b/x86/rmap_chain.c
index 97591f9..3433dda 100644
--- a/x86/rmap_chain.c
+++ b/x86/rmap_chain.c
@@ -3,6 +3,7 @@
 #include "libcflat.h"
 #include "fwcfg.h"
 #include "vm.h"
+#include "vmalloc.h"
 #include "smp.h"
 #include "alloc_page.h"
 
diff --git a/x86/sieve.c b/x86/sieve.c
index 6cbcd6d..2ee2919 100644
--- a/x86/sieve.c
+++ b/x86/sieve.c
@@ -1,4 +1,4 @@
-#include "vm.h"
+#include "alloc.h"
 #include "libcflat.h"
 
 int sieve(char* data, int size)
@@ -42,9 +42,9 @@ int main()
     setup_vm();
     test_sieve("mapped", static_data, STATIC_SIZE);
     for (i = 0; i < 3; ++i) {
-	v = vmalloc(VSIZE);
+	v = malloc(VSIZE);
 	test_sieve("virtual", v, VSIZE);
-	vfree(v);
+	free(v);
     }
 
     return 0;
-- 
2.14.2

[PATCH kvm-unit-tests 12/18] arm64: use adrp instead of adr

[Paolo Bonzini]

We are pretty close to exhausting the 128KiB range allowed by adr.
Use adrp+add, or adrp+an offset, to allow for a wider range to avoid
that objcopy has to truncate relocations.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 arm/cstart64.S | 25 ++++++++++++++-----------
 1 file changed, 14 insertions(+), 11 deletions(-)

diff --git a/arm/cstart64.S b/arm/cstart64.S
index d8062f4..0a26587 100644
--- a/arm/cstart64.S
+++ b/arm/cstart64.S
@@ -24,8 +24,8 @@ start:
 	mov	x4, #1
 	msr	spsel, x4
 	isb
-	adr     x4, stackptr
-	mov	sp, x4
+	adrp    x4, stackptr
+	add     sp, x4, :lo12:stackptr
 	stp	x0, x1, [sp, #-16]!
 
 	/* Enable FP/ASIMD */
@@ -40,16 +40,19 @@ start:
 	bl	setup
 
 	/* run the test */
-	adr	x0, __argc
-	ldr	x0, [x0]
-	adr	x1, __argv
-	adr	x2, __environ
+	adrp	x0, __argc
+	ldr	x0, [x0, :lo12:__argc]
+	adrp	x1, __argv
+	add	x1, x1, :lo12:__argv
+	adrp	x2, __environ
+	add	x2, x2, :lo12:__environ
 	bl	main
 	bl	exit
 	b	halt
 
 exceptions_init:
-	adr	x0, vector_table
+	adrp	x0, vector_table
+	add	x0, x0, :lo12:vector_table
 	msr	vbar_el1, x0
 	isb
 	ret
@@ -66,13 +69,13 @@ secondary_entry:
 	bl	exceptions_init
 
 	/* enable the MMU */
-	adr	x0, mmu_idmap
-	ldr	x0, [x0]
+	adrp	x0, mmu_idmap
+	ldr	x0, [x0, :lo12:mmu_idmap]
 	bl	asm_mmu_enable
 
 	/* set the stack */
-	adr	x1, secondary_data
-	ldr	x0, [x1]
+	adrp	x0, secondary_data
+	ldr	x0, [x0, :lo12:secondary_data]
 	mov	sp, x0
 
 	/* finish init in C code */
-- 
2.14.2

[PATCH kvm-unit-tests 13/18] arm: make pgtable.h private to mmu.c

[Paolo Bonzini]

The pgd/pmd/pud definitions are not needed outside the architecture-
dependent MMU functions.  In fact, it probably would be simpler to
have completely different files for 32-bit and 64-bit ARM.  However,
as a first and possibly less controversial step, hide them inside
that file.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 lib/arm/asm/mmu-api.h | 4 ++++
 lib/arm/asm/mmu.h     | 1 -
 lib/arm/asm/pgtable.h | 4 ----
 lib/arm/bitops.c      | 1 +
 lib/arm/mmu.c         | 6 ++++++
 lib/arm64/asm/mmu.h   | 1 -
 6 files changed, 11 insertions(+), 6 deletions(-)

diff --git a/lib/arm/asm/mmu-api.h b/lib/arm/asm/mmu-api.h
index bf0e6a0..060e5cd 100644
--- a/lib/arm/asm/mmu-api.h
+++ b/lib/arm/asm/mmu-api.h
@@ -1,5 +1,9 @@
 #ifndef __ASMARM_MMU_API_H_
 #define __ASMARM_MMU_API_H_
+
+#include <asm/page.h>
+#include <stdbool.h>
+
 extern pgd_t *mmu_idmap;
 extern unsigned int mmu_disabled_cpu_count;
 extern bool __mmu_enabled(void);
diff --git a/lib/arm/asm/mmu.h b/lib/arm/asm/mmu.h
index 2bb0cde..a31e19c 100644
--- a/lib/arm/asm/mmu.h
+++ b/lib/arm/asm/mmu.h
@@ -5,7 +5,6 @@
  *
  * This work is licensed under the terms of the GNU LGPL, version 2.
  */
-#include <asm/pgtable.h>
 #include <asm/barrier.h>
 
 #define PTE_USER		L_PTE_USER
diff --git a/lib/arm/asm/pgtable.h b/lib/arm/asm/pgtable.h
index 5af4b9b..a38f5da 100644
--- a/lib/arm/asm/pgtable.h
+++ b/lib/arm/asm/pgtable.h
@@ -14,10 +14,6 @@
  *
  * This work is licensed under the terms of the GNU GPL, version 2.
  */
-#include <alloc.h>
-#include <asm/setup.h>
-#include <asm/page.h>
-#include <asm/pgtable-hwdef.h>
 
 #define pgd_none(pgd)		(!pgd_val(pgd))
 #define pud_none(pud)		(!pud_val(pud))
diff --git a/lib/arm/bitops.c b/lib/arm/bitops.c
index 9ab241b..b17d76a 100644
--- a/lib/arm/bitops.c
+++ b/lib/arm/bitops.c
@@ -6,6 +6,7 @@
  *
  * This work is licensed under the terms of the GNU GPL, version 2.
  */
+#include "libcflat.h"
 #include <bitops.h>
 #include <asm/barrier.h>
 #include <asm/mmu.h>
diff --git a/lib/arm/mmu.c b/lib/arm/mmu.c
index 1b7e4fb..227e25b 100644
--- a/lib/arm/mmu.c
+++ b/lib/arm/mmu.c
@@ -9,6 +9,12 @@
 #include <asm/thread_info.h>
 #include <asm/cpumask.h>
 #include <asm/mmu.h>
+#include <asm/setup.h>
+#include <asm/page.h>
+
+#include "alloc.h"
+#include <asm/pgtable-hwdef.h>
+#include <asm/pgtable.h>
 
 extern unsigned long etext;
 
diff --git a/lib/arm64/asm/mmu.h b/lib/arm64/asm/mmu.h
index 3bc31c9..9df99cc 100644
--- a/lib/arm64/asm/mmu.h
+++ b/lib/arm64/asm/mmu.h
@@ -5,7 +5,6 @@
  *
  * This work is licensed under the terms of the GNU LGPL, version 2.
  */
-#include <asm/pgtable.h>
 #include <asm/barrier.h>
 
 #define PMD_SECT_UNCACHED	PMD_ATTRINDX(MT_DEVICE_nGnRE)
-- 
2.14.2

[PATCH kvm-unit-tests 14/18] arm: get rid of pud

[Paolo Bonzini]

We do not need four levels of page tables in either 32-bit or 64-bit
ARM, so remove the useless indirection.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 lib/arm/asm/page.h            |  4 ----
 lib/arm/asm/pgtable.h         | 22 ++++++++--------------
 lib/arm/mmu.c                 |  3 +--
 lib/arm64/asm/page.h          | 12 +++---------
 lib/arm64/asm/pgtable-hwdef.h | 17 ++---------------
 lib/arm64/asm/pgtable.h       | 10 ++--------
 6 files changed, 16 insertions(+), 52 deletions(-)

diff --git a/lib/arm/asm/page.h b/lib/arm/asm/page.h
index 3802641..fc1b30e 100644
--- a/lib/arm/asm/page.h
+++ b/lib/arm/asm/page.h
@@ -34,10 +34,6 @@ typedef struct { pteval_t pgprot; } pgprot_t;
 #define __pgd(x)		((pgd_t) { (x) } )
 #define __pgprot(x)		((pgprot_t) { (x) } )
 
-typedef struct { pgd_t pgd; } pud_t;
-#define pud_val(x)		(pgd_val((x).pgd))
-#define __pud(x)		((pud_t) { __pgd(x) } )
-
 #ifndef __virt_to_phys
 #define __phys_to_virt(x)	((unsigned long) (x))
 #define __virt_to_phys(x)	(x)
diff --git a/lib/arm/asm/pgtable.h b/lib/arm/asm/pgtable.h
index a38f5da..27603db 100644
--- a/lib/arm/asm/pgtable.h
+++ b/lib/arm/asm/pgtable.h
@@ -4,7 +4,6 @@
  * Adapted from arch/arm/include/asm/pgtable.h
  *              arch/arm/include/asm/pgtable-3level.h
  *              arch/arm/include/asm/pgalloc.h
- *              include/asm-generic/pgtable-nopud.h
  *
  * Note: some Linux function APIs have been modified. Nothing crazy,
  *       but if a function took, for example, an mm_struct, then
@@ -16,7 +15,6 @@
  */
 
 #define pgd_none(pgd)		(!pgd_val(pgd))
-#define pud_none(pud)		(!pud_val(pud))
 #define pmd_none(pmd)		(!pmd_val(pmd))
 #define pte_none(pte)		(!pte_val(pte))
 
@@ -32,19 +30,15 @@ static inline pgd_t *pgd_alloc(void)
 	return pgd;
 }
 
-#define pud_offset(pgd, addr)	((pud_t *)pgd)
-#define pud_free(pud)
-#define pud_alloc(pgd, addr)	pud_offset(pgd, addr)
-
-static inline pmd_t *pud_page_vaddr(pud_t pud)
+static inline pmd_t *pgd_page_vaddr(pgd_t pgd)
 {
-	return __va(pud_val(pud) & PHYS_MASK & (s32)PAGE_MASK);
+	return __va(pgd_val(pgd) & PHYS_MASK & (s32)PAGE_MASK);
 }
 
 #define pmd_index(addr) \
 	(((addr) >> PMD_SHIFT) & (PTRS_PER_PMD - 1))
-#define pmd_offset(pud, addr) \
-	(pud_page_vaddr(*(pud)) + pmd_index(addr))
+#define pmd_offset(pgd, addr) \
+	(pgd_page_vaddr(*(pgd)) + pmd_index(addr))
 
 #define pmd_free(pmd) free(pmd)
 static inline pmd_t *pmd_alloc_one(void)
@@ -53,13 +47,13 @@ static inline pmd_t *pmd_alloc_one(void)
 	memset(pmd, 0, PTRS_PER_PMD * sizeof(pmd_t));
 	return pmd;
 }
-static inline pmd_t *pmd_alloc(pud_t *pud, unsigned long addr)
+static inline pmd_t *pmd_alloc(pgd_t *pgd, unsigned long addr)
 {
-	if (pud_none(*pud)) {
+	if (pgd_none(*pgd)) {
 		pmd_t *pmd = pmd_alloc_one();
-		pud_val(*pud) = __pa(pmd) | PMD_TYPE_TABLE;
+		pgd_val(*pgd) = __pa(pmd) | PMD_TYPE_TABLE;
 	}
-	return pmd_offset(pud, addr);
+	return pmd_offset(pgd, addr);
 }
 
 static inline pte_t *pmd_page_vaddr(pmd_t pmd)
diff --git a/lib/arm/mmu.c b/lib/arm/mmu.c
index 227e25b..4e7581d 100644
--- a/lib/arm/mmu.c
+++ b/lib/arm/mmu.c
@@ -81,8 +81,7 @@ void mmu_set_range_ptes(pgd_t *pgtable, unsigned long virt_offset,
 
 	for (; vaddr < virt_end; vaddr += PAGE_SIZE, paddr += PAGE_SIZE) {
 		pgd_t *pgd = pgd_offset(pgtable, vaddr);
-		pud_t *pud = pud_alloc(pgd, vaddr);
-		pmd_t *pmd = pmd_alloc(pud, vaddr);
+		pmd_t *pmd = pmd_alloc(pgd, vaddr);
 		pte_t *pte = pte_alloc(pmd, vaddr);
 
 		pte_val(*pte) = paddr;
diff --git a/lib/arm64/asm/page.h b/lib/arm64/asm/page.h
index 35ea0ed..01b4cf6 100644
--- a/lib/arm64/asm/page.h
+++ b/lib/arm64/asm/page.h
@@ -3,7 +3,6 @@
 /*
  * Adapted from
  *   arch/arm64/include/asm/pgtable-types.h
- *   include/asm-generic/pgtable-nopud.h
  *   include/asm-generic/pgtable-nopmd.h
  *
  * Copyright (C) 2017, Red Hat Inc, Andrew Jones <drjones@redhat.com>
@@ -28,7 +27,6 @@
 
 typedef u64 pteval_t;
 typedef u64 pmdval_t;
-typedef u64 pudval_t;
 typedef u64 pgdval_t;
 typedef struct { pteval_t pte; } pte_t;
 typedef struct { pgdval_t pgd; } pgd_t;
@@ -42,13 +40,9 @@ typedef struct { pteval_t pgprot; } pgprot_t;
 #define __pgd(x)		((pgd_t) { (x) } )
 #define __pgprot(x)		((pgprot_t) { (x) } )
 
-typedef struct { pgd_t pgd; } pud_t;
-#define pud_val(x)		(pgd_val((x).pgd))
-#define __pud(x)		((pud_t) { __pgd(x) } )
-
-typedef struct { pud_t pud; } pmd_t;
-#define pmd_val(x)		(pud_val((x).pud))
-#define __pmd(x)		((pmd_t) { __pud(x) } )
+typedef struct { pgd_t pgd; } pmd_t;
+#define pmd_val(x)		(pgd_val((x).pgd))
+#define __pmd(x)		((pmd_t) { __pgd(x) } )
 
 #ifndef __virt_to_phys
 #define __phys_to_virt(x)	((unsigned long) (x))
diff --git a/lib/arm64/asm/pgtable-hwdef.h b/lib/arm64/asm/pgtable-hwdef.h
index 8e9c295..045a3ce 100644
--- a/lib/arm64/asm/pgtable-hwdef.h
+++ b/lib/arm64/asm/pgtable-hwdef.h
@@ -22,13 +22,8 @@
 #define PGDIR_MASK		(~(PGDIR_SIZE-1))
 #define PTRS_PER_PGD		(1 << (VA_BITS - PGDIR_SHIFT))
 
-/* From include/asm-generic/pgtable-nopud.h */
-#define PUD_SHIFT		PGDIR_SHIFT
-#define PTRS_PER_PUD		1
-#define PUD_SIZE		(UL(1) << PUD_SHIFT)
-#define PUD_MASK		(~(PUD_SIZE-1))
 /* From include/asm-generic/pgtable-nopmd.h */
-#define PMD_SHIFT		PUD_SHIFT
+#define PMD_SHIFT		PGDIR_SHIFT
 #define PTRS_PER_PMD		1
 #define PMD_SIZE		(UL(1) << PMD_SHIFT)
 #define PMD_MASK		(~(PMD_SIZE-1))
@@ -43,15 +38,7 @@
 /*
  * Hardware page table definitions.
  *
- * Level 1 descriptor (PUD).
- */
-#define PUD_TYPE_TABLE		(_AT(pudval_t, 3) << 0)
-#define PUD_TABLE_BIT		(_AT(pgdval_t, 1) << 1)
-#define PUD_TYPE_MASK		(_AT(pgdval_t, 3) << 0)
-#define PUD_TYPE_SECT		(_AT(pgdval_t, 1) << 0)
-
-/*
- * Level 2 descriptor (PMD).
+ * Level 1 descriptor (PMD).
  */
 #define PMD_TYPE_MASK		(_AT(pmdval_t, 3) << 0)
 #define PMD_TYPE_FAULT		(_AT(pmdval_t, 0) << 0)
diff --git a/lib/arm64/asm/pgtable.h b/lib/arm64/asm/pgtable.h
index 217cdb6..14d7e3c 100644
--- a/lib/arm64/asm/pgtable.h
+++ b/lib/arm64/asm/pgtable.h
@@ -3,7 +3,6 @@
 /*
  * Adapted from arch/arm64/include/asm/pgtable.h
  *              include/asm-generic/pgtable-nopmd.h
- *              include/asm-generic/pgtable-nopud.h
  *              include/linux/mm.h
  *
  * Note: some Linux function APIs have been modified. Nothing crazy,
@@ -20,7 +19,6 @@
 #include <asm/pgtable-hwdef.h>
 
 #define pgd_none(pgd)		(!pgd_val(pgd))
-#define pud_none(pud)		(!pud_val(pud))
 #define pmd_none(pmd)		(!pmd_val(pmd))
 #define pte_none(pte)		(!pte_val(pte))
 
@@ -36,13 +34,9 @@ static inline pgd_t *pgd_alloc(void)
 	return pgd;
 }
 
-#define pud_offset(pgd, addr)	((pud_t *)pgd)
-#define pud_free(pud)
-#define pud_alloc(pgd, addr)	pud_offset(pgd, addr)
-
-#define pmd_offset(pud, addr)	((pmd_t *)pud)
+#define pmd_offset(pgd, addr)	((pmd_t *)pgd)
 #define pmd_free(pmd)
-#define pmd_alloc(pud, addr)	pmd_offset(pud, addr)
+#define pmd_alloc(pgd, addr)	pmd_offset(pgd, addr)
 
 static inline pte_t *pmd_page_vaddr(pmd_t pmd)
 {
-- 
2.14.2

[PATCH kvm-unit-tests 15/18] arm: use better-matching types for virtual and physical addresses

[Paolo Bonzini]

Make virtual addresses use uintptr_t and physical addresses
use phys_addr_t.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 lib/arm/asm/mmu-api.h | 10 +++++-----
 lib/arm/mmu.c         | 34 +++++++++++++++-------------------
 2 files changed, 20 insertions(+), 24 deletions(-)

diff --git a/lib/arm/asm/mmu-api.h b/lib/arm/asm/mmu-api.h
index 060e5cd..623f77f 100644
--- a/lib/arm/asm/mmu-api.h
+++ b/lib/arm/asm/mmu-api.h
@@ -16,11 +16,11 @@ extern void mmu_mark_disabled(int cpu);
 extern void mmu_enable(pgd_t *pgtable);
 extern void mmu_disable(void);
 extern void mmu_enable_idmap(void);
-extern void mmu_init_io_sect(pgd_t *pgtable, unsigned long virt_offset);
-extern void mmu_set_range_sect(pgd_t *pgtable, unsigned long virt_offset,
-			       unsigned long phys_start, unsigned long phys_end,
+
+extern void mmu_set_range_sect(pgd_t *pgtable, uintptr_t virt_offset,
+			       phys_addr_t phys_start, phys_addr_t phys_end,
 			       pgprot_t prot);
-extern void mmu_set_range_ptes(pgd_t *pgtable, unsigned long virt_offset,
-			       unsigned long phys_start, unsigned long phys_end,
+extern void mmu_set_range_ptes(pgd_t *pgtable, uintptr_t virt_offset,
+			       phys_addr_t phys_start, phys_addr_t phys_end,
 			       pgprot_t prot);
 #endif
diff --git a/lib/arm/mmu.c b/lib/arm/mmu.c
index 4e7581d..f575dac 100644
--- a/lib/arm/mmu.c
+++ b/lib/arm/mmu.c
@@ -71,13 +71,13 @@ void mmu_disable(void)
 	asm_mmu_disable();
 }
 
-void mmu_set_range_ptes(pgd_t *pgtable, unsigned long virt_offset,
-			unsigned long phys_start, unsigned long phys_end,
+void mmu_set_range_ptes(pgd_t *pgtable, uintptr_t virt_offset,
+			phys_addr_t phys_start, phys_addr_t phys_end,
 			pgprot_t prot)
 {
-	unsigned long vaddr = virt_offset & PAGE_MASK;
-	unsigned long paddr = phys_start & PAGE_MASK;
-	unsigned long virt_end = phys_end - paddr + vaddr;
+	phys_addr_t paddr = phys_start & PAGE_MASK;
+	uintptr_t vaddr = virt_offset & PAGE_MASK;
+	uintptr_t virt_end = phys_end - paddr + vaddr;
 
 	for (; vaddr < virt_end; vaddr += PAGE_SIZE, paddr += PAGE_SIZE) {
 		pgd_t *pgd = pgd_offset(pgtable, vaddr);
@@ -90,13 +90,13 @@ void mmu_set_range_ptes(pgd_t *pgtable, unsigned long virt_offset,
 	}
 }
 
-void mmu_set_range_sect(pgd_t *pgtable, unsigned long virt_offset,
-			unsigned long phys_start, unsigned long phys_end,
+void mmu_set_range_sect(pgd_t *pgtable, uintptr_t virt_offset,
+			phys_addr_t phys_start, phys_addr_t phys_end,
 			pgprot_t prot)
 {
-	unsigned long vaddr = virt_offset & PGDIR_MASK;
-	unsigned long paddr = phys_start & PGDIR_MASK;
-	unsigned long virt_end = phys_end - paddr + vaddr;
+	phys_addr_t paddr = phys_start & PGDIR_MASK;
+	uintptr_t vaddr = virt_offset & PGDIR_MASK;
+	uintptr_t virt_end = phys_end - paddr + vaddr;
 
 	for (; vaddr < virt_end; vaddr += PGDIR_SIZE, paddr += PGDIR_SIZE) {
 		pgd_t *pgd = pgd_offset(pgtable, vaddr);
@@ -107,22 +107,18 @@ void mmu_set_range_sect(pgd_t *pgtable, unsigned long virt_offset,
 }
 
 
-void mmu_init_io_sect(pgd_t *pgtable, unsigned long virt_offset)
-{
-	mmu_set_range_sect(pgtable, virt_offset,
-		PHYS_IO_OFFSET, PHYS_IO_END,
-		__pgprot(PMD_SECT_UNCACHED | PMD_SECT_USER));
-}
 
 void mmu_enable_idmap(void)
 {
-	unsigned long phys_end = sizeof(long) == 8 || !(PHYS_END >> 32)
+	uintptr_t phys_end = sizeof(long) == 8 || !(PHYS_END >> 32)
 						? PHYS_END : 0xfffff000;
-	unsigned long code_end = (unsigned long)&etext;
+	uintptr_t code_end = (uintptr_t)&etext;
 
 	mmu_idmap = pgd_alloc();
 
-	mmu_init_io_sect(mmu_idmap, PHYS_IO_OFFSET);
+	mmu_set_range_sect(mmu_idmap, PHYS_IO_OFFSET,
+		PHYS_IO_OFFSET, PHYS_IO_END,
+		__pgprot(PMD_SECT_UNCACHED | PMD_SECT_USER));
 
 	/* armv8 requires code shared between EL1 and EL0 to be read-only */
 	mmu_set_range_ptes(mmu_idmap, PHYS_OFFSET,
-- 
2.14.2

[PATCH kvm-unit-tests 16/18] arm: only align CPU stacks to one page

[Paolo Bonzini]

vmalloc is not able to provide allocations with alignment greater than
PAGE_SIZE, so avoid them.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 lib/arm/asm/thread_info.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/arm/asm/thread_info.h b/lib/arm/asm/thread_info.h
index 9f0c14d..c2ecebb 100644
--- a/lib/arm/asm/thread_info.h
+++ b/lib/arm/asm/thread_info.h
@@ -38,7 +38,7 @@
 
 static inline void *thread_stack_alloc(void)
 {
-	void *sp = memalign(THREAD_SIZE, THREAD_SIZE);
+	void *sp = memalign(PAGE_SIZE, THREAD_SIZE);
 	return sp + THREAD_START_SP;
 }
 
-- 
2.14.2

[PATCH kvm-unit-tests 17/18] arm: do not allocate virtio buffers from the stack

[Paolo Bonzini]

The stack will be in virtual memory space starting from the next patch,
so the address of automatic variables will not be a physical address
anymore.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 lib/chr-testdev.c | 17 ++++++++---------
 1 file changed, 8 insertions(+), 9 deletions(-)

diff --git a/lib/chr-testdev.c b/lib/chr-testdev.c
index c19424f..ca4a10b 100644
--- a/lib/chr-testdev.c
+++ b/lib/chr-testdev.c
@@ -30,20 +30,19 @@ static void __testdev_send(char *buf, unsigned int len)
 void chr_testdev_exit(int code)
 {
 	unsigned int len;
-	char buf[8];
-
-	snprintf(buf, sizeof(buf), "%dq", code);
-	len = strlen(buf);
+	static char buf[8];
 
 	spin_lock(&lock);
 
-	if (!vcon)
-		goto out;
+	snprintf(buf, sizeof(buf), "%dq", code);
+	len = strlen(buf);
 
-	__testdev_send(buf, len);
+	if (vcon) {
+		__testdev_send(buf, len);
+		printf("this shouldn't have happened!");
+	}
 
-out:
-	spin_unlock(&lock);
+	while (1);
 }
 
 void chr_testdev_init(void)
-- 
2.14.2

[PATCH kvm-unit-tests 18/18] arm: enable vmalloc

[Paolo Bonzini]

Unlike x86, ARM always enables virtual memory so it always switches early from
phys_alloc to vmalloc.  The changes mostly involve using alloc_page directly
whenever physical addresses are needed, and of course implementing the
architecture-dependent callbacks that vmalloc needs.

The 32-bit root page table doesn't need a full page, but it is simpler to just
allocate one.  With split files 32-bit and 64-bit, we could just place
mmu_idmap in .bss.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 arm/Makefile.common     |  3 +++
 arm/sieve.c             |  1 +
 lib/arm/asm/mmu-api.h   |  1 -
 lib/arm/asm/pgtable.h   | 10 +++++---
 lib/arm/mmu.c           | 67 ++++++++++++++++++++++++++++++++++++++-----------
 lib/arm/setup.c         |  2 +-
 lib/arm64/asm/pgtable.h |  5 ++--
 lib/virtio-mmio.c       |  4 ++-
 8 files changed, 70 insertions(+), 23 deletions(-)
 create mode 120000 arm/sieve.c

diff --git a/arm/Makefile.common b/arm/Makefile.common
index e09c3e2..0a039cf 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -15,6 +15,7 @@ tests-common += $(TEST_DIR)/pci-test.flat
 tests-common += $(TEST_DIR)/pmu.flat
 tests-common += $(TEST_DIR)/gic.flat
 tests-common += $(TEST_DIR)/psci.flat
+tests-common += $(TEST_DIR)/sieve.flat
 
 tests-all = $(tests-common) $(tests)
 all: directories $(tests-all)
@@ -35,6 +36,8 @@ include $(SRCDIR)/scripts/asm-offsets.mak
 
 cflatobjs += lib/util.o
 cflatobjs += lib/alloc_phys.o
+cflatobjs += lib/alloc_page.o
+cflatobjs += lib/vmalloc.o
 cflatobjs += lib/alloc.o
 cflatobjs += lib/devicetree.o
 cflatobjs += lib/pci.o
diff --git a/arm/sieve.c b/arm/sieve.c
new file mode 120000
index 0000000..8f14a5c
--- /dev/null
+++ b/arm/sieve.c
@@ -0,0 +1 @@
+../x86/sieve.c
\ No newline at end of file
diff --git a/lib/arm/asm/mmu-api.h b/lib/arm/asm/mmu-api.h
index 623f77f..df3ccf7 100644
--- a/lib/arm/asm/mmu-api.h
+++ b/lib/arm/asm/mmu-api.h
@@ -15,7 +15,6 @@ extern void mmu_mark_enabled(int cpu);
 extern void mmu_mark_disabled(int cpu);
 extern void mmu_enable(pgd_t *pgtable);
 extern void mmu_disable(void);
-extern void mmu_enable_idmap(void);
 
 extern void mmu_set_range_sect(pgd_t *pgtable, uintptr_t virt_offset,
 			       phys_addr_t phys_start, phys_addr_t phys_end,
diff --git a/lib/arm/asm/pgtable.h b/lib/arm/asm/pgtable.h
index 27603db..a95e630 100644
--- a/lib/arm/asm/pgtable.h
+++ b/lib/arm/asm/pgtable.h
@@ -40,10 +40,11 @@ static inline pmd_t *pgd_page_vaddr(pgd_t pgd)
 #define pmd_offset(pgd, addr) \
 	(pgd_page_vaddr(*(pgd)) + pmd_index(addr))
 
-#define pmd_free(pmd) free(pmd)
+#define pmd_free(pmd) free_page(pmd)
 static inline pmd_t *pmd_alloc_one(void)
 {
-	pmd_t *pmd = memalign(PAGE_SIZE, PTRS_PER_PMD * sizeof(pmd_t));
+	assert(PTRS_PER_PMD * sizeof(pmd_t) == PAGE_SIZE);
+	pmd_t *pmd = alloc_page();
 	memset(pmd, 0, PTRS_PER_PMD * sizeof(pmd_t));
 	return pmd;
 }
@@ -66,10 +67,11 @@ static inline pte_t *pmd_page_vaddr(pmd_t pmd)
 #define pte_offset(pmd, addr) \
 	(pmd_page_vaddr(*(pmd)) + pte_index(addr))
 
-#define pte_free(pte) free(pte)
+#define pte_free(pte) free_page(pte)
 static inline pte_t *pte_alloc_one(void)
 {
-	pte_t *pte = memalign(PAGE_SIZE, PTRS_PER_PTE * sizeof(pte_t));
+	assert(PTRS_PER_PTE * sizeof(pte_t) == PAGE_SIZE);
+	pte_t *pte = alloc_page();
 	memset(pte, 0, PTRS_PER_PTE * sizeof(pte_t));
 	return pte;
 }
diff --git a/lib/arm/mmu.c b/lib/arm/mmu.c
index f575dac..21bcf3a 100644
--- a/lib/arm/mmu.c
+++ b/lib/arm/mmu.c
@@ -13,6 +13,8 @@
 #include <asm/page.h>
 
 #include "alloc.h"
+#include "alloc_page.h"
+#include "vmalloc.h"
 #include <asm/pgtable-hwdef.h>
 #include <asm/pgtable.h>
 
@@ -71,6 +73,43 @@ void mmu_disable(void)
 	asm_mmu_disable();
 }
 
+static pteval_t *get_pte(pgd_t *pgtable, uintptr_t vaddr)
+{
+	pgd_t *pgd = pgd_offset(pgtable, vaddr);
+	pmd_t *pmd = pmd_alloc(pgd, vaddr);
+	pte_t *pte = pte_alloc(pmd, vaddr);
+
+	return &pte_val(*pte);
+}
+
+static pteval_t *install_pte(pgd_t *pgtable, uintptr_t vaddr, pteval_t pte)
+{
+	pteval_t *p_pte = get_pte(pgtable, vaddr);
+	*p_pte = pte;
+	return p_pte;
+}
+
+static pteval_t *install_page_prot(pgd_t *pgtable, phys_addr_t phys,
+				   uintptr_t vaddr, pgprot_t prot)
+{
+	pteval_t pte = phys;
+	pte |= PTE_TYPE_PAGE | PTE_AF | PTE_SHARED;
+	pte |= pgprot_val(prot);
+	return install_pte(pgtable, vaddr, pte);
+}
+
+pteval_t *install_page(pgd_t *pgtable, phys_addr_t phys, void *virt)
+{
+	return install_page_prot(pgtable, phys, (uintptr_t)virt,
+				 __pgprot(PTE_WBWA | PTE_USER));
+}
+
+phys_addr_t virt_to_pte_phys(pgd_t *pgtable, void *mem)
+{
+	return (*get_pte(pgtable, (uintptr_t)mem) & PHYS_MASK & -PAGE_SIZE)
+		+ ((ulong)mem & (PAGE_SIZE - 1));
+}
+
 void mmu_set_range_ptes(pgd_t *pgtable, uintptr_t virt_offset,
 			phys_addr_t phys_start, phys_addr_t phys_end,
 			pgprot_t prot)
@@ -79,15 +118,8 @@ void mmu_set_range_ptes(pgd_t *pgtable, uintptr_t virt_offset,
 	uintptr_t vaddr = virt_offset & PAGE_MASK;
 	uintptr_t virt_end = phys_end - paddr + vaddr;
 
-	for (; vaddr < virt_end; vaddr += PAGE_SIZE, paddr += PAGE_SIZE) {
-		pgd_t *pgd = pgd_offset(pgtable, vaddr);
-		pmd_t *pmd = pmd_alloc(pgd, vaddr);
-		pte_t *pte = pte_alloc(pmd, vaddr);
-
-		pte_val(*pte) = paddr;
-		pte_val(*pte) |= PTE_TYPE_PAGE | PTE_AF | PTE_SHARED;
-		pte_val(*pte) |= pgprot_val(prot);
-	}
+	for (; vaddr < virt_end; vaddr += PAGE_SIZE, paddr += PAGE_SIZE)
+		install_page_prot(pgtable, paddr, vaddr, prot);
 }
 
 void mmu_set_range_sect(pgd_t *pgtable, uintptr_t virt_offset,
@@ -107,14 +139,20 @@ void mmu_set_range_sect(pgd_t *pgtable, uintptr_t virt_offset,
 }
 
 
-
-void mmu_enable_idmap(void)
+void *setup_mmu(phys_addr_t phys_end)
 {
-	uintptr_t phys_end = sizeof(long) == 8 || !(PHYS_END >> 32)
-						? PHYS_END : 0xfffff000;
 	uintptr_t code_end = (uintptr_t)&etext;
 
-	mmu_idmap = pgd_alloc();
+	/* 0G-1G = I/O, 1G-3G = identity, 3G-4G = vmalloc */
+	if (phys_end > (3ul << 30))
+		phys_end = 3ul << 30;
+
+#ifdef __aarch64__
+	init_alloc_vpage((void*)(4ul << 30));
+#endif
+
+	mmu_idmap = alloc_page();
+	memset(mmu_idmap, 0, PAGE_SIZE);
 
 	mmu_set_range_sect(mmu_idmap, PHYS_IO_OFFSET,
 		PHYS_IO_OFFSET, PHYS_IO_END,
@@ -130,4 +168,5 @@ void mmu_enable_idmap(void)
 		__pgprot(PTE_WBWA | PTE_USER));
 
 	mmu_enable(mmu_idmap);
+	return mmu_idmap;
 }
diff --git a/lib/arm/setup.c b/lib/arm/setup.c
index 0e2bab4..326e78f 100644
--- a/lib/arm/setup.c
+++ b/lib/arm/setup.c
@@ -108,7 +108,7 @@ static void mem_init(phys_addr_t freemem_start)
 	phys_alloc_init(freemem_start, primary.end - freemem_start);
 	phys_alloc_set_minimum_alignment(SMP_CACHE_BYTES);
 
-	mmu_enable_idmap();
+	setup_vm();
 }
 
 void setup(const void *fdt)
diff --git a/lib/arm64/asm/pgtable.h b/lib/arm64/asm/pgtable.h
index 14d7e3c..941a850 100644
--- a/lib/arm64/asm/pgtable.h
+++ b/lib/arm64/asm/pgtable.h
@@ -48,10 +48,11 @@ static inline pte_t *pmd_page_vaddr(pmd_t pmd)
 #define pte_offset(pmd, addr) \
 	(pmd_page_vaddr(*(pmd)) + pte_index(addr))
 
-#define pte_free(pte) free(pte)
+#define pte_free(pte) free_page(pte)
 static inline pte_t *pte_alloc_one(void)
 {
-	pte_t *pte = memalign(PAGE_SIZE, PTRS_PER_PTE * sizeof(pte_t));
+	assert(PTRS_PER_PTE * sizeof(pte_t) == PAGE_SIZE);
+	pte_t *pte = alloc_page();
 	memset(pte, 0, PTRS_PER_PTE * sizeof(pte_t));
 	return pte;
 }
diff --git a/lib/virtio-mmio.c b/lib/virtio-mmio.c
index e4a92f1..e5e8f66 100644
--- a/lib/virtio-mmio.c
+++ b/lib/virtio-mmio.c
@@ -7,6 +7,7 @@
  */
 #include "libcflat.h"
 #include "devicetree.h"
+#include "alloc_page.h"
 #include "alloc.h"
 #include "asm/page.h"
 #include "asm/io.h"
@@ -53,7 +54,8 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev,
 	unsigned num = VIRTIO_MMIO_QUEUE_NUM_MIN;
 
 	vq = calloc(1, sizeof(*vq));
-	queue = memalign(PAGE_SIZE, VIRTIO_MMIO_QUEUE_SIZE_MIN);
+	assert(VIRTIO_MMIO_QUEUE_SIZE_MIN <= 2*PAGE_SIZE);
+	queue = alloc_pages(1);
 	assert(vq && queue);
 
 	writel(index, vm_dev->base + VIRTIO_MMIO_QUEUE_SEL);
-- 
2.14.2

Re: [PATCH kvm-unit-tests 16/18] arm: only align CPU stacks to one page

[Andrew Jones]

On Tue, Oct 24, 2017 at 05:21:43PM +0200, Paolo Bonzini wrote:
> vmalloc is not able to provide allocations with alignment greater than
> PAGE_SIZE, so avoid them.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  lib/arm/asm/thread_info.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/lib/arm/asm/thread_info.h b/lib/arm/asm/thread_info.h
> index 9f0c14d..c2ecebb 100644
> --- a/lib/arm/asm/thread_info.h
> +++ b/lib/arm/asm/thread_info.h
> @@ -38,7 +38,7 @@
>  
>  static inline void *thread_stack_alloc(void)
>  {
> -	void *sp = memalign(THREAD_SIZE, THREAD_SIZE);
> +	void *sp = memalign(PAGE_SIZE, THREAD_SIZE);
>  	return sp + THREAD_START_SP;
>  }
>

This reduces all the secondary cpu's stacks on arm from 16K to 4K.
Also, THREAD_SIZE is page aligned, so I'm wondering what's breaking
here?

Thanks,
drew

Re: [PATCH kvm-unit-tests 17/18] arm: do not allocate virtio buffers from the stack

[Andrew Jones]

On Tue, Oct 24, 2017 at 05:21:44PM +0200, Paolo Bonzini wrote:
> The stack will be in virtual memory space starting from the next patch,
> so the address of automatic variables will not be a physical address
> anymore.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  lib/chr-testdev.c | 17 ++++++++---------
>  1 file changed, 8 insertions(+), 9 deletions(-)
> 
> diff --git a/lib/chr-testdev.c b/lib/chr-testdev.c
> index c19424f..ca4a10b 100644
> --- a/lib/chr-testdev.c
> +++ b/lib/chr-testdev.c
> @@ -30,20 +30,19 @@ static void __testdev_send(char *buf, unsigned int len)
>  void chr_testdev_exit(int code)
>  {
>  	unsigned int len;
> -	char buf[8];
> -
> -	snprintf(buf, sizeof(buf), "%dq", code);
> -	len = strlen(buf);
> +	static char buf[8];
>  
>  	spin_lock(&lock);
>  
> -	if (!vcon)
> -		goto out;
> +	snprintf(buf, sizeof(buf), "%dq", code);
> +	len = strlen(buf);
>  
> -	__testdev_send(buf, len);
> +	if (vcon) {
> +		__testdev_send(buf, len);
> +		printf("this shouldn't have happened!");
> +	}
>  
> -out:
> -	spin_unlock(&lock);
> +	while (1);

I'd rather we continue to exit gracefully from chr_testdev_exit(),
allowing callers to unconditionally call it, and then do their own
"shouldn't have happened" printing, or fallback to another way to
exit. That way when a unit test is run without the chr-testdev
provided, exit() can still work, just differently. Granted we only
have halt() for that fallback right now, but I've exploited the fact
that chr_testdev_exit() will just return, without the chr-testdev
device present, several times in temporary hacks.

Thanks,
drew


>  }
>  
>  void chr_testdev_init(void)
> -- 
> 2.14.2
> 
> 

Re: [PATCH kvm-unit-tests 16/18] arm: only align CPU stacks to one page

[Paolo Bonzini]

On 25/10/2017 10:40, Andrew Jones wrote:
> On Tue, Oct 24, 2017 at 05:21:43PM +0200, Paolo Bonzini wrote:
>> vmalloc is not able to provide allocations with alignment greater than
>> PAGE_SIZE, so avoid them.
>>
>> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
>> ---
>>  lib/arm/asm/thread_info.h | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/lib/arm/asm/thread_info.h b/lib/arm/asm/thread_info.h
>> index 9f0c14d..c2ecebb 100644
>> --- a/lib/arm/asm/thread_info.h
>> +++ b/lib/arm/asm/thread_info.h
>> @@ -38,7 +38,7 @@
>>  
>>  static inline void *thread_stack_alloc(void)
>>  {
>> -	void *sp = memalign(THREAD_SIZE, THREAD_SIZE);
>> +	void *sp = memalign(PAGE_SIZE, THREAD_SIZE);
>>  	return sp + THREAD_START_SP;
>>  }
>>
> 
> This reduces all the secondary cpu's stacks on arm from 16K to 4K.
> Also, THREAD_SIZE is page aligned, so I'm wondering what's breaking
> here?

No, they stay 16K but only page-aligned; the prototype is

    void *memalign(size_t alignment, size_t size)

vmalloc is not able to satisfy memalign with alignment greater than a
page size, but the memory blocks can be as large as you want.

Paolo

Re: [PATCH kvm-unit-tests 16/18] arm: only align CPU stacks to one page

[Andrew Jones]

On Wed, Oct 25, 2017 at 11:23:01AM +0200, Paolo Bonzini wrote:
> On 25/10/2017 10:40, Andrew Jones wrote:
> > On Tue, Oct 24, 2017 at 05:21:43PM +0200, Paolo Bonzini wrote:
> >> vmalloc is not able to provide allocations with alignment greater than
> >> PAGE_SIZE, so avoid them.
> >>
> >> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> >> ---
> >>  lib/arm/asm/thread_info.h | 2 +-
> >>  1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/lib/arm/asm/thread_info.h b/lib/arm/asm/thread_info.h
> >> index 9f0c14d..c2ecebb 100644
> >> --- a/lib/arm/asm/thread_info.h
> >> +++ b/lib/arm/asm/thread_info.h
> >> @@ -38,7 +38,7 @@
> >>  
> >>  static inline void *thread_stack_alloc(void)
> >>  {
> >> -	void *sp = memalign(THREAD_SIZE, THREAD_SIZE);
> >> +	void *sp = memalign(PAGE_SIZE, THREAD_SIZE);
> >>  	return sp + THREAD_START_SP;
> >>  }
> >>
> > 
> > This reduces all the secondary cpu's stacks on arm from 16K to 4K.
> > Also, THREAD_SIZE is page aligned, so I'm wondering what's breaking
> > here?
> 
> No, they stay 16K but only page-aligned; the prototype is
> 
>     void *memalign(size_t alignment, size_t size)
> 
> vmalloc is not able to satisfy memalign with alignment greater than a
> page size, but the memory blocks can be as large as you want.

Argh, sorry for the noise. I was obviously still asleep and lazy when
I skimmed the series this morning...

Thanks,
drew

Re: [PATCH kvm-unit-tests 17/18] arm: do not allocate virtio buffers from the stack

[Paolo Bonzini]

On 25/10/2017 10:48, Andrew Jones wrote:
> I'd rather we continue to exit gracefully from chr_testdev_exit(),
> allowing callers to unconditionally call it, and then do their own
> "shouldn't have happened" printing, or fallback to another way to
> exit. That way when a unit test is run without the chr-testdev
> provided, exit() can still work, just differently. Granted we only
> have halt() for that fallback right now, but I've exploited the fact
> that chr_testdev_exit() will just return, without the chr-testdev
> device present, several times in temporary hacks.

Sure, that's certainly okay.  The patch is even simpler, and I should 
really have sent this one first:

diff --git a/lib/chr-testdev.c b/lib/chr-testdev.c
index c19424f..1450826 100644
--- a/lib/chr-testdev.c
+++ b/lib/chr-testdev.c
@@ -30,16 +30,15 @@ static void __testdev_send(char *buf, unsigned int len)
 void chr_testdev_exit(int code)
 {
 	unsigned int len;
-	char buf[8];
-
-	snprintf(buf, sizeof(buf), "%dq", code);
-	len = strlen(buf);
+	static char buf[8];
 
 	spin_lock(&lock);
-
 	if (!vcon)
 		goto out;
 
+	snprintf(buf, sizeof(buf), "%dq", code);
+	len = strlen(buf);
+
 	__testdev_send(buf, len);
 
 out:


Paolo

> 
> Thanks,
> drew
> 
> 
>>  }
>>  
>>  void chr_testdev_init(void)
>> -- 
>> 2.14.2
>>
>>