[PATCH 0/5] KVM: arm64: Selftest IPA fixes

[PATCH 0/5] KVM: arm64: Selftest IPA fixes

[Marc Zyngier]

Now that the common variety of fruity arm64 systems makes a pretty
remarkable KVM host, I have decided to run the collection of selftests
on it.

Nothing works. Oh no!

As it turns out, the tests have a notion of "default mode" (36bit PA,
4kB), which cannot work in general on arm64 because there no such
thing as an IPA size that would be valid everywhere (apart from the
minimal 32bit), nor a guaranteed to be supported page size.

This small series is a first attempt at making these things computed
at runtime by making the default something else on systems that do not
support it. It also makes the supported page sizes dynamically
discovered. The initial patch addresses an issue that has already been
addressed separately, and is only there so that I don't get shouted at
by some robot.

With that, the selftests do run on the M1, with the exception of the
memslot tests that are freaked out by the use of 16kB pages on the
host and 4kB in the guest. Maybe I'll implement some form of 16kB
support in the future.

Marc Zyngier (5):
  KVM: selftests: Fix vm_compute_max_gfn on !x86
  KVM: selftests: Initialise default mode in each test
  KVM: selftests: arm64: Introduce a variable default IPA size
  KVM: selftests: arm64: Check for supported page sizes
  KVM: selftests: arm64: Add support for VM_MODE_P36V48_{4K,64K}

 .../selftests/kvm/aarch64/arch_timer.c        |  3 +
 .../selftests/kvm/aarch64/debug-exceptions.c  |  3 +
 .../selftests/kvm/aarch64/get-reg-list.c      |  3 +
 .../selftests/kvm/aarch64/psci_cpu_on_test.c  |  3 +
 .../testing/selftests/kvm/aarch64/vgic_init.c |  3 +
 .../testing/selftests/kvm/include/kvm_util.h  | 15 ++--
 .../selftests/kvm/kvm_binary_stats_test.c     |  3 +
 .../selftests/kvm/kvm_create_max_vcpus.c      |  3 +
 .../selftests/kvm/lib/aarch64/processor.c     |  8 ++
 tools/testing/selftests/kvm/lib/guest_modes.c | 77 +++++++++++++++++--
 tools/testing/selftests/kvm/lib/kvm_util.c    |  6 ++
 .../testing/selftests/kvm/memslot_perf_test.c |  4 +
 tools/testing/selftests/kvm/rseq_test.c       |  3 +
 .../selftests/kvm/set_memory_region_test.c    |  4 +
 tools/testing/selftests/kvm/steal_time.c      |  3 +
 15 files changed, 131 insertions(+), 10 deletions(-)

-- 
2.30.2


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 1/5] KVM: selftests: Fix vm_compute_max_gfn on !x86

[Marc Zyngier]

Compiling the selftestts on arm64 leads to this:

gcc -Wall -Wstrict-prototypes -Wuninitialized -O2 -g -std=gnu99 -fno-stack-protector -fno-PIE -I../../../../tools/include -I../../../../tools/arch/arm64/include -I../../../../usr/include/ -Iinclude -Ilib -Iinclude/aarch64 -I..   -c lib/kvm_util.c -o /home/maz/arm-platforms/tools/testing/selftests/kvm/lib/kvm_util.o
In file included from lib/kvm_util.c:10:
include/kvm_util.h: In function ‘vm_compute_max_gfn’:
include/kvm_util.h:79:21: error: invalid use of undefined type ‘struct kvm_vm’
   79 |  return ((1ULL << vm->pa_bits) >> vm->page_shift) - 1;
      |                     ^~
include/kvm_util.h:79:37: error: invalid use of undefined type ‘struct kvm_vm’
   79 |  return ((1ULL << vm->pa_bits) >> vm->page_shift) - 1;
      |                                     ^~
[...]

This is all because struct kvm_vm is not defined yet (only declared).
Sidestep the whole issue by making vm_compute_max_gfn() a macro.

Fixes: c8cc43c1eae2 ("selftests: KVM: avoid failures due to reserved HyperTransport region")
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 tools/testing/selftests/kvm/include/kvm_util.h | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
index da2b702da71a..c74241ddf8b1 100644
--- a/tools/testing/selftests/kvm/include/kvm_util.h
+++ b/tools/testing/selftests/kvm/include/kvm_util.h
@@ -74,10 +74,11 @@ enum vm_guest_mode {
 #if defined(__x86_64__)
 unsigned long vm_compute_max_gfn(struct kvm_vm *vm);
 #else
-static inline unsigned long vm_compute_max_gfn(struct kvm_vm *vm)
-{
-	return ((1ULL << vm->pa_bits) >> vm->page_shift) - 1;
-}
+#define vm_compute_max_gfn(vm)						\
+	({								\
+		struct kvm_vm *__vm = vm;				\
+		((1ULL << __vm->pa_bits) >> __vm->page_shift) - 1;	\
+	})
 #endif
 
 #define MIN_PAGE_SIZE		(1U << MIN_PAGE_SHIFT)
-- 
2.30.2


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 2/5] KVM: selftests: Initialise default mode in each test

[Marc Zyngier]

As we are going to add support for a variable default mode on arm64,
let's make sure it is setup first by sprinkling a number of calls
to get_modes_append_default() when the test starts.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 tools/testing/selftests/kvm/aarch64/arch_timer.c       | 3 +++
 tools/testing/selftests/kvm/aarch64/debug-exceptions.c | 3 +++
 tools/testing/selftests/kvm/aarch64/get-reg-list.c     | 3 +++
 tools/testing/selftests/kvm/aarch64/psci_cpu_on_test.c | 3 +++
 tools/testing/selftests/kvm/aarch64/vgic_init.c        | 3 +++
 tools/testing/selftests/kvm/kvm_binary_stats_test.c    | 3 +++
 tools/testing/selftests/kvm/kvm_create_max_vcpus.c     | 3 +++
 tools/testing/selftests/kvm/memslot_perf_test.c        | 4 ++++
 tools/testing/selftests/kvm/rseq_test.c                | 3 +++
 tools/testing/selftests/kvm/set_memory_region_test.c   | 4 ++++
 tools/testing/selftests/kvm/steal_time.c               | 3 +++
 11 files changed, 35 insertions(+)

diff --git a/tools/testing/selftests/kvm/aarch64/arch_timer.c b/tools/testing/selftests/kvm/aarch64/arch_timer.c
index bf6a45b0b8dc..22f5b1d85135 100644
--- a/tools/testing/selftests/kvm/aarch64/arch_timer.c
+++ b/tools/testing/selftests/kvm/aarch64/arch_timer.c
@@ -35,6 +35,7 @@
 #include "arch_timer.h"
 #include "gic.h"
 #include "vgic.h"
+#include "guest_modes.h"
 
 #define NR_VCPUS_DEF			4
 #define NR_TEST_ITERS_DEF		5
@@ -460,6 +461,8 @@ int main(int argc, char *argv[])
 {
 	struct kvm_vm *vm;
 
+	guest_modes_append_default();
+
 	/* Tell stdout not to buffer its content */
 	setbuf(stdout, NULL);
 
diff --git a/tools/testing/selftests/kvm/aarch64/debug-exceptions.c b/tools/testing/selftests/kvm/aarch64/debug-exceptions.c
index ea189d83abf7..26fd0a87a5dc 100644
--- a/tools/testing/selftests/kvm/aarch64/debug-exceptions.c
+++ b/tools/testing/selftests/kvm/aarch64/debug-exceptions.c
@@ -2,6 +2,7 @@
 #include <test_util.h>
 #include <kvm_util.h>
 #include <processor.h>
+#include <guest_modes.h>
 
 #define VCPU_ID 0
 
@@ -200,6 +201,8 @@ int main(int argc, char *argv[])
 	struct ucall uc;
 	int stage;
 
+	guest_modes_append_default();
+
 	vm = vm_create_default(VCPU_ID, 0, guest_code);
 	ucall_init(vm, NULL);
 
diff --git a/tools/testing/selftests/kvm/aarch64/get-reg-list.c b/tools/testing/selftests/kvm/aarch64/get-reg-list.c
index cc898181faab..f97932439f90 100644
--- a/tools/testing/selftests/kvm/aarch64/get-reg-list.c
+++ b/tools/testing/selftests/kvm/aarch64/get-reg-list.c
@@ -33,6 +33,7 @@
 #include "kvm_util.h"
 #include "test_util.h"
 #include "processor.h"
+#include "guest_modes.h"
 
 static struct kvm_reg_list *reg_list;
 static __u64 *blessed_reg, blessed_n;
@@ -588,6 +589,8 @@ int main(int ac, char **av)
 	int i, ret = 0;
 	pid_t pid;
 
+	guest_modes_append_default();
+
 	for (i = 1; i < ac; ++i) {
 		if (strcmp(av[i], "--core-reg-fixup") == 0)
 			fixup_core_regs = true;
diff --git a/tools/testing/selftests/kvm/aarch64/psci_cpu_on_test.c b/tools/testing/selftests/kvm/aarch64/psci_cpu_on_test.c
index 4c5f6814030f..94f951b0edcb 100644
--- a/tools/testing/selftests/kvm/aarch64/psci_cpu_on_test.c
+++ b/tools/testing/selftests/kvm/aarch64/psci_cpu_on_test.c
@@ -16,6 +16,7 @@
 #include "kvm_util.h"
 #include "processor.h"
 #include "test_util.h"
+#include "guest_modes.h"
 
 #define VCPU_ID_SOURCE 0
 #define VCPU_ID_TARGET 1
@@ -76,6 +77,8 @@ int main(void)
 	struct kvm_vm *vm;
 	struct ucall uc;
 
+	guest_modes_append_default();
+
 	vm = vm_create(VM_MODE_DEFAULT, DEFAULT_GUEST_PHY_PAGES, O_RDWR);
 	kvm_vm_elf_load(vm, program_invocation_name);
 	ucall_init(vm, NULL);
diff --git a/tools/testing/selftests/kvm/aarch64/vgic_init.c b/tools/testing/selftests/kvm/aarch64/vgic_init.c
index 34379c98d2f4..f2c235453aa9 100644
--- a/tools/testing/selftests/kvm/aarch64/vgic_init.c
+++ b/tools/testing/selftests/kvm/aarch64/vgic_init.c
@@ -14,6 +14,7 @@
 #include "kvm_util.h"
 #include "processor.h"
 #include "vgic.h"
+#include "guest_modes.h"
 
 #define NR_VCPUS		4
 
@@ -699,6 +700,8 @@ int main(int ac, char **av)
 	int ret;
 	int pa_bits;
 
+	guest_modes_append_default();
+
 	pa_bits = vm_guest_mode_params[VM_MODE_DEFAULT].pa_bits;
 	max_phys_size = 1ULL << pa_bits;
 
diff --git a/tools/testing/selftests/kvm/kvm_binary_stats_test.c b/tools/testing/selftests/kvm/kvm_binary_stats_test.c
index 17f65d514915..0dc30a7ba0b3 100644
--- a/tools/testing/selftests/kvm/kvm_binary_stats_test.c
+++ b/tools/testing/selftests/kvm/kvm_binary_stats_test.c
@@ -17,6 +17,7 @@
 #include "test_util.h"
 
 #include "kvm_util.h"
+#include "guest_modes.h"
 #include "asm/kvm.h"
 #include "linux/kvm.h"
 
@@ -207,6 +208,8 @@ int main(int argc, char *argv[])
 	int max_vm = DEFAULT_NUM_VM;
 	int max_vcpu = DEFAULT_NUM_VCPU;
 
+	guest_modes_append_default();
+
 	/* Get the number of VMs and VCPUs that would be created for testing. */
 	if (argc > 1) {
 		max_vm = strtol(argv[1], NULL, 0);
diff --git a/tools/testing/selftests/kvm/kvm_create_max_vcpus.c b/tools/testing/selftests/kvm/kvm_create_max_vcpus.c
index aed9dc3ca1e9..7068a9f14d0c 100644
--- a/tools/testing/selftests/kvm/kvm_create_max_vcpus.c
+++ b/tools/testing/selftests/kvm/kvm_create_max_vcpus.c
@@ -17,6 +17,7 @@
 #include "test_util.h"
 
 #include "kvm_util.h"
+#include "guest_modes.h"
 #include "asm/kvm.h"
 #include "linux/kvm.h"
 
@@ -51,6 +52,8 @@ int main(int argc, char *argv[])
 	pr_info("KVM_CAP_MAX_VCPU_ID: %d\n", kvm_max_vcpu_id);
 	pr_info("KVM_CAP_MAX_VCPUS: %d\n", kvm_max_vcpus);
 
+	guest_modes_append_default();
+
 	/*
 	 * Check that we're allowed to open nr_fds_wanted file descriptors and
 	 * try raising the limits if needed.
diff --git a/tools/testing/selftests/kvm/memslot_perf_test.c b/tools/testing/selftests/kvm/memslot_perf_test.c
index 1727f75e0c2c..ef4ffce5d170 100644
--- a/tools/testing/selftests/kvm/memslot_perf_test.c
+++ b/tools/testing/selftests/kvm/memslot_perf_test.c
@@ -25,6 +25,8 @@
 #include <kvm_util.h>
 #include <processor.h>
 
+#include "guest_modes.h"
+
 #define VCPU_ID 0
 
 #define MEM_SIZE		((512U << 20) + 4096)
@@ -1015,6 +1017,8 @@ int main(int argc, char *argv[])
 	if (!parse_args(argc, argv, &targs))
 		return -1;
 
+	guest_modes_append_default();
+
 	rbestslottime.slottimens = 0;
 	for (tctr = targs.tfirst; tctr <= targs.tlast; tctr++) {
 		const struct test_data *data = &tests[tctr];
diff --git a/tools/testing/selftests/kvm/rseq_test.c b/tools/testing/selftests/kvm/rseq_test.c
index 4158da0da2bb..d2440fc72f0d 100644
--- a/tools/testing/selftests/kvm/rseq_test.c
+++ b/tools/testing/selftests/kvm/rseq_test.c
@@ -19,6 +19,7 @@
 #include "kvm_util.h"
 #include "processor.h"
 #include "test_util.h"
+#include "guest_modes.h"
 
 #define VCPU_ID 0
 
@@ -223,6 +224,8 @@ int main(int argc, char *argv[])
 
 	sys_rseq(0);
 
+	guest_modes_append_default();
+
 	/*
 	 * Create and run a dummy VM that immediately exits to userspace via
 	 * GUEST_SYNC, while concurrently migrating the process by setting its
diff --git a/tools/testing/selftests/kvm/set_memory_region_test.c b/tools/testing/selftests/kvm/set_memory_region_test.c
index 72a1c9b4882c..ff4cc7e796e7 100644
--- a/tools/testing/selftests/kvm/set_memory_region_test.c
+++ b/tools/testing/selftests/kvm/set_memory_region_test.c
@@ -17,6 +17,8 @@
 #include <kvm_util.h>
 #include <processor.h>
 
+#include "guest_modes.h"
+
 #define VCPU_ID 0
 
 /*
@@ -413,6 +415,8 @@ int main(int argc, char *argv[])
 	int i, loops;
 #endif
 
+	guest_modes_append_default();
+
 	/* Tell stdout not to buffer its content */
 	setbuf(stdout, NULL);
 
diff --git a/tools/testing/selftests/kvm/steal_time.c b/tools/testing/selftests/kvm/steal_time.c
index 62f2eb9ee3d5..5200044f572d 100644
--- a/tools/testing/selftests/kvm/steal_time.c
+++ b/tools/testing/selftests/kvm/steal_time.c
@@ -16,6 +16,7 @@
 #include "test_util.h"
 #include "kvm_util.h"
 #include "processor.h"
+#include "guest_modes.h"
 
 #define NR_VCPUS		4
 #define ST_GPA_BASE		(1 << 30)
@@ -273,6 +274,8 @@ int main(int ac, char **av)
 	pthread_attr_setaffinity_np(&attr, sizeof(cpu_set_t), &cpuset);
 	pthread_setaffinity_np(pthread_self(), sizeof(cpu_set_t), &cpuset);
 
+	guest_modes_append_default();
+
 	/* Create a one VCPU guest and an identity mapped memslot for the steal time structure */
 	vm = vm_create_default(0, 0, guest_code);
 	gpages = vm_calc_num_guest_pages(VM_MODE_DEFAULT, STEAL_TIME_SIZE * NR_VCPUS);
-- 
2.30.2


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 3/5] KVM: selftests: arm64: Introduce a variable default IPA size

[Marc Zyngier]

Contrary to popular belief, there is no such thing as a default
IPA size on arm64. Anything goes, and implementations are the
usual Wild West.

The selftest infrastructure default to 40bit IPA, which obviously
doesn't work for some systems out there.

Turn VM_MODE_DEFAULT from a constant into a variable, and let
guest_modes_append_default() populate it, depending on what
the HW can do. In order to preserve the current behaviour, we
still pick 40bits IPA as the default if it is available, and
the largest supported IPA space otherwise.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 .../testing/selftests/kvm/include/kvm_util.h  |  4 ++-
 tools/testing/selftests/kvm/lib/guest_modes.c | 28 +++++++++++++++++--
 2 files changed, 29 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
index c74241ddf8b1..d2ba830a1faf 100644
--- a/tools/testing/selftests/kvm/include/kvm_util.h
+++ b/tools/testing/selftests/kvm/include/kvm_util.h
@@ -53,7 +53,9 @@ enum vm_guest_mode {
 
 #if defined(__aarch64__)
 
-#define VM_MODE_DEFAULT			VM_MODE_P40V48_4K
+extern enum vm_guest_mode vm_mode_default;
+
+#define VM_MODE_DEFAULT			vm_mode_default
 #define MIN_PAGE_SHIFT			12U
 #define ptes_per_page(page_size)	((page_size) / 8)
 
diff --git a/tools/testing/selftests/kvm/lib/guest_modes.c b/tools/testing/selftests/kvm/lib/guest_modes.c
index c330f414ef96..fadc99bac69c 100644
--- a/tools/testing/selftests/kvm/lib/guest_modes.c
+++ b/tools/testing/selftests/kvm/lib/guest_modes.c
@@ -4,22 +4,46 @@
  */
 #include "guest_modes.h"
 
+#ifdef __aarch64__
+enum vm_guest_mode vm_mode_default;
+#endif
+
 struct guest_mode guest_modes[NUM_VM_MODES];
 
 void guest_modes_append_default(void)
 {
+#ifndef __aarch64__
 	guest_mode_append(VM_MODE_DEFAULT, true, true);
-
+#endif
 #ifdef __aarch64__
-	guest_mode_append(VM_MODE_P40V48_64K, true, true);
 	{
 		unsigned int limit = kvm_check_cap(KVM_CAP_ARM_VM_IPA_SIZE);
+		int i;
+
+		vm_mode_default = NUM_VM_MODES;
+
 		if (limit >= 52)
 			guest_mode_append(VM_MODE_P52V48_64K, true, true);
 		if (limit >= 48) {
 			guest_mode_append(VM_MODE_P48V48_4K, true, true);
 			guest_mode_append(VM_MODE_P48V48_64K, true, true);
 		}
+		if (limit >= 40) {
+			guest_mode_append(VM_MODE_P40V48_4K, true, true);
+			guest_mode_append(VM_MODE_P40V48_64K, true, true);
+			vm_mode_default = VM_MODE_P40V48_4K;
+		}
+
+		/* Pick the largest supported IPA size */
+		for (i = 0;
+		     vm_mode_default == NUM_VM_MODES && i < NUM_VM_MODES;
+		     i++) {
+			if (guest_modes[i].supported)
+				vm_mode_default = i;
+		}
+
+		TEST_ASSERT(vm_mode_default != NUM_VM_MODES,
+			    "No supported mode!");
 	}
 #endif
 #ifdef __s390x__
-- 
2.30.2


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 4/5] KVM: selftests: arm64: Check for supported page sizes

[Marc Zyngier]

Just as arm64 implemenations don't necessary support all IPA
ranges, they don't  all support the same page sizes either. Fun.

Create a dummy VM to snapshot the page sizes supported by the
host, and filter the supported modes.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 tools/testing/selftests/kvm/lib/guest_modes.c | 51 ++++++++++++++++---
 1 file changed, 45 insertions(+), 6 deletions(-)

diff --git a/tools/testing/selftests/kvm/lib/guest_modes.c b/tools/testing/selftests/kvm/lib/guest_modes.c
index fadc99bac69c..8db9ea2c4032 100644
--- a/tools/testing/selftests/kvm/lib/guest_modes.c
+++ b/tools/testing/selftests/kvm/lib/guest_modes.c
@@ -5,7 +5,42 @@
 #include "guest_modes.h"
 
 #ifdef __aarch64__
+#include "processor.h"
 enum vm_guest_mode vm_mode_default;
+static void get_supported_psz(uint32_t ipa,
+			      bool *ps4k, bool *ps16k, bool *ps64k)
+{
+	struct kvm_vcpu_init preferred_init;
+	int kvm_fd, vm_fd, vcpu_fd, err;
+	uint64_t val;
+	struct kvm_one_reg reg = {
+		.id	= KVM_ARM64_SYS_REG(SYS_ID_AA64MMFR0_EL1),
+		.addr	= (uint64_t)&val,
+	};
+
+	kvm_fd = open_kvm_dev_path_or_exit();
+	vm_fd = ioctl(kvm_fd, KVM_CREATE_VM, ipa);
+	TEST_ASSERT(vm_fd >= 0, "Can't create VM");
+
+	vcpu_fd = ioctl(vm_fd, KVM_CREATE_VCPU, 0);
+	TEST_ASSERT(vcpu_fd >= 0, "Can't create vcpu");
+
+	err = ioctl(vm_fd, KVM_ARM_PREFERRED_TARGET, &preferred_init);
+	TEST_ASSERT(err == 0, "Can't get target");
+	err = ioctl(vcpu_fd, KVM_ARM_VCPU_INIT, &preferred_init);
+	TEST_ASSERT(err == 0, "Can't get init vcpu");
+
+	err = ioctl(vcpu_fd, KVM_GET_ONE_REG, &reg);
+	TEST_ASSERT(err == 0, "Can't get MMFR0");
+
+	*ps4k = ((val >> 28) & 0xf) != 0xf;
+	*ps64k = ((val >> 24) & 0xf) == 0;
+	*ps16k = ((val >> 20) & 0xf) != 0;
+
+	close(vcpu_fd);
+	close(vm_fd);
+	close(kvm_fd);
+}
 #endif
 
 struct guest_mode guest_modes[NUM_VM_MODES];
@@ -18,20 +53,24 @@ void guest_modes_append_default(void)
 #ifdef __aarch64__
 	{
 		unsigned int limit = kvm_check_cap(KVM_CAP_ARM_VM_IPA_SIZE);
+		bool ps4k, ps16k, ps64k;
 		int i;
 
+		get_supported_psz(limit, &ps4k, &ps16k, &ps64k);
+
 		vm_mode_default = NUM_VM_MODES;
 
 		if (limit >= 52)
-			guest_mode_append(VM_MODE_P52V48_64K, true, true);
+			guest_mode_append(VM_MODE_P52V48_64K, ps64k, ps64k);
 		if (limit >= 48) {
-			guest_mode_append(VM_MODE_P48V48_4K, true, true);
-			guest_mode_append(VM_MODE_P48V48_64K, true, true);
+			guest_mode_append(VM_MODE_P48V48_4K, ps4k, ps4k);
+			guest_mode_append(VM_MODE_P48V48_64K, ps64k, ps64k);
 		}
 		if (limit >= 40) {
-			guest_mode_append(VM_MODE_P40V48_4K, true, true);
-			guest_mode_append(VM_MODE_P40V48_64K, true, true);
-			vm_mode_default = VM_MODE_P40V48_4K;
+			guest_mode_append(VM_MODE_P40V48_4K, ps4k, ps4k);
+			guest_mode_append(VM_MODE_P40V48_64K, ps64k, ps64k);
+			if (ps4k)
+				vm_mode_default = VM_MODE_P40V48_4K;
 		}
 
 		/* Pick the largest supported IPA size */
-- 
2.30.2


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 5/5] KVM: selftests: arm64: Add support for VM_MODE_P36V48_{4K, 64K}

[Marc Zyngier]

Some of the arm64 systems out there have an IPA space that is
positively tiny. Nonetheless, they make great KVM hosts.

Add support for 36bit IPA support with 4kB pages, which makes
some of the fruity machines happy. Whilst we're at it, add support
for 64kB pages as well, though these boxes have no support for it.

16kB is left as a exercise for the eager reviewer.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 tools/testing/selftests/kvm/include/kvm_util.h      | 2 ++
 tools/testing/selftests/kvm/lib/aarch64/processor.c | 8 ++++++++
 tools/testing/selftests/kvm/lib/guest_modes.c       | 4 ++++
 tools/testing/selftests/kvm/lib/kvm_util.c          | 6 ++++++
 4 files changed, 20 insertions(+)

diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
index d2ba830a1faf..a8f7605ddf48 100644
--- a/tools/testing/selftests/kvm/include/kvm_util.h
+++ b/tools/testing/selftests/kvm/include/kvm_util.h
@@ -48,6 +48,8 @@ enum vm_guest_mode {
 	VM_MODE_PXXV48_4K,	/* For 48bits VA but ANY bits PA */
 	VM_MODE_P47V64_4K,
 	VM_MODE_P44V64_4K,
+	VM_MODE_P36V48_4K,
+	VM_MODE_P36V48_64K,
 	NUM_VM_MODES,
 };
 
diff --git a/tools/testing/selftests/kvm/lib/aarch64/processor.c b/tools/testing/selftests/kvm/lib/aarch64/processor.c
index b4eeeafd2a70..a313fbed53a0 100644
--- a/tools/testing/selftests/kvm/lib/aarch64/processor.c
+++ b/tools/testing/selftests/kvm/lib/aarch64/processor.c
@@ -264,6 +264,14 @@ void aarch64_vcpu_setup(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_vcpu_init
 		tcr_el1 |= 1ul << 14; /* TG0 = 64KB */
 		tcr_el1 |= 2ul << 32; /* IPS = 40 bits */
 		break;
+	case VM_MODE_P36V48_4K:
+		tcr_el1 |= 0ul << 14; /* TG0 = 4KB */
+		tcr_el1 |= 1ul << 32; /* IPS = 36 bits */
+		break;
+	case VM_MODE_P36V48_64K:
+		tcr_el1 |= 1ul << 14; /* TG0 = 64KB */
+		tcr_el1 |= 1ul << 32; /* IPS = 36 bits */
+		break;
 	default:
 		TEST_FAIL("Unknown guest mode, mode: 0x%x", vm->mode);
 	}
diff --git a/tools/testing/selftests/kvm/lib/guest_modes.c b/tools/testing/selftests/kvm/lib/guest_modes.c
index 8db9ea2c4032..f1d3b0c012c7 100644
--- a/tools/testing/selftests/kvm/lib/guest_modes.c
+++ b/tools/testing/selftests/kvm/lib/guest_modes.c
@@ -72,6 +72,10 @@ void guest_modes_append_default(void)
 			if (ps4k)
 				vm_mode_default = VM_MODE_P40V48_4K;
 		}
+		if (limit >= 36) {
+			guest_mode_append(VM_MODE_P36V48_4K, ps4k, ps4k);
+			guest_mode_append(VM_MODE_P36V48_64K, ps64k, ps64k);
+		}
 
 		/* Pick the largest supported IPA size */
 		for (i = 0;
diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c
index daf6fdb217a7..3dc554430082 100644
--- a/tools/testing/selftests/kvm/lib/kvm_util.c
+++ b/tools/testing/selftests/kvm/lib/kvm_util.c
@@ -172,6 +172,8 @@ const char *vm_guest_mode_string(uint32_t i)
 		[VM_MODE_PXXV48_4K]	= "PA-bits:ANY, VA-bits:48,  4K pages",
 		[VM_MODE_P47V64_4K]	= "PA-bits:47,  VA-bits:64,  4K pages",
 		[VM_MODE_P44V64_4K]	= "PA-bits:44,  VA-bits:64,  4K pages",
+		[VM_MODE_P36V48_4K]	= "PA-bits:36,  VA-bits:48,  4K pages",
+		[VM_MODE_P36V48_64K]	= "PA-bits:36,  VA-bits:48, 64K pages",
 	};
 	_Static_assert(sizeof(strings)/sizeof(char *) == NUM_VM_MODES,
 		       "Missing new mode strings?");
@@ -191,6 +193,8 @@ const struct vm_guest_mode_params vm_guest_mode_params[] = {
 	[VM_MODE_PXXV48_4K]	= {  0,  0,  0x1000, 12 },
 	[VM_MODE_P47V64_4K]	= { 47, 64,  0x1000, 12 },
 	[VM_MODE_P44V64_4K]	= { 44, 64,  0x1000, 12 },
+	[VM_MODE_P36V48_4K]	= { 36, 48,  0x1000, 12 },
+	[VM_MODE_P36V48_64K]	= { 36, 48, 0x10000, 16 },
 };
 _Static_assert(sizeof(vm_guest_mode_params)/sizeof(struct vm_guest_mode_params) == NUM_VM_MODES,
 	       "Missing new mode params?");
@@ -252,9 +256,11 @@ struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm)
 		vm->pgtable_levels = 3;
 		break;
 	case VM_MODE_P40V48_4K:
+	case VM_MODE_P36V48_4K:
 		vm->pgtable_levels = 4;
 		break;
 	case VM_MODE_P40V48_64K:
+	case VM_MODE_P36V48_64K:
 		vm->pgtable_levels = 3;
 		break;
 	case VM_MODE_PXXV48_4K:
-- 
2.30.2


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 3/5] KVM: selftests: arm64: Introduce a variable default IPA size

[Andrew Jones]

On Thu, Dec 16, 2021 at 12:31:33PM +0000, Marc Zyngier wrote:
> Contrary to popular belief, there is no such thing as a default
> IPA size on arm64. Anything goes, and implementations are the
> usual Wild West.
> 
> The selftest infrastructure default to 40bit IPA, which obviously
> doesn't work for some systems out there.
> 
> Turn VM_MODE_DEFAULT from a constant into a variable, and let
> guest_modes_append_default() populate it, depending on what
> the HW can do. In order to preserve the current behaviour, we
> still pick 40bits IPA as the default if it is available, and
> the largest supported IPA space otherwise.
> 
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> ---
>  .../testing/selftests/kvm/include/kvm_util.h  |  4 ++-
>  tools/testing/selftests/kvm/lib/guest_modes.c | 28 +++++++++++++++++--
>  2 files changed, 29 insertions(+), 3 deletions(-)
> 
> diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
> index c74241ddf8b1..d2ba830a1faf 100644
> --- a/tools/testing/selftests/kvm/include/kvm_util.h
> +++ b/tools/testing/selftests/kvm/include/kvm_util.h
> @@ -53,7 +53,9 @@ enum vm_guest_mode {
>  
>  #if defined(__aarch64__)
>  
> -#define VM_MODE_DEFAULT			VM_MODE_P40V48_4K
> +extern enum vm_guest_mode vm_mode_default;
> +
> +#define VM_MODE_DEFAULT			vm_mode_default
>  #define MIN_PAGE_SHIFT			12U
>  #define ptes_per_page(page_size)	((page_size) / 8)
>  
> diff --git a/tools/testing/selftests/kvm/lib/guest_modes.c b/tools/testing/selftests/kvm/lib/guest_modes.c
> index c330f414ef96..fadc99bac69c 100644
> --- a/tools/testing/selftests/kvm/lib/guest_modes.c
> +++ b/tools/testing/selftests/kvm/lib/guest_modes.c
> @@ -4,22 +4,46 @@
>   */
>  #include "guest_modes.h"
>  
> +#ifdef __aarch64__
> +enum vm_guest_mode vm_mode_default;
> +#endif
> +
>  struct guest_mode guest_modes[NUM_VM_MODES];
>  
>  void guest_modes_append_default(void)
>  {
> +#ifndef __aarch64__
>  	guest_mode_append(VM_MODE_DEFAULT, true, true);
> -
> +#endif
>  #ifdef __aarch64__
> -	guest_mode_append(VM_MODE_P40V48_64K, true, true);
>  	{
>  		unsigned int limit = kvm_check_cap(KVM_CAP_ARM_VM_IPA_SIZE);
> +		int i;
> +
> +		vm_mode_default = NUM_VM_MODES;
> +
>  		if (limit >= 52)
>  			guest_mode_append(VM_MODE_P52V48_64K, true, true);
>  		if (limit >= 48) {
>  			guest_mode_append(VM_MODE_P48V48_4K, true, true);
>  			guest_mode_append(VM_MODE_P48V48_64K, true, true);
>  		}
> +		if (limit >= 40) {
> +			guest_mode_append(VM_MODE_P40V48_4K, true, true);
> +			guest_mode_append(VM_MODE_P40V48_64K, true, true);
> +			vm_mode_default = VM_MODE_P40V48_4K;
> +		}
> +
> +		/* Pick the largest supported IPA size */

The guest_modes array isn't sorted from smallest to largest PA addresses,
although it could be.

> +		for (i = 0;
> +		     vm_mode_default == NUM_VM_MODES && i < NUM_VM_MODES;
> +		     i++) {

Feel free to put this on one line.

> +			if (guest_modes[i].supported)

A bit safer would be to check both .supported and .enabled.

> +				vm_mode_default = i;
> +		}
> +
> +		TEST_ASSERT(vm_mode_default != NUM_VM_MODES,
> +			    "No supported mode!");
>  	}
>  #endif
>  #ifdef __s390x__
> -- 
> 2.30.2
>

Thanks,
drew


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 4/5] KVM: selftests: arm64: Check for supported page sizes

[Andrew Jones]

On Thu, Dec 16, 2021 at 12:31:34PM +0000, Marc Zyngier wrote:
> Just as arm64 implemenations don't necessary support all IPA
> ranges, they don't  all support the same page sizes either. Fun.
> 
> Create a dummy VM to snapshot the page sizes supported by the
> host, and filter the supported modes.
> 
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> ---
>  tools/testing/selftests/kvm/lib/guest_modes.c | 51 ++++++++++++++++---
>  1 file changed, 45 insertions(+), 6 deletions(-)
> 
> diff --git a/tools/testing/selftests/kvm/lib/guest_modes.c b/tools/testing/selftests/kvm/lib/guest_modes.c
> index fadc99bac69c..8db9ea2c4032 100644
> --- a/tools/testing/selftests/kvm/lib/guest_modes.c
> +++ b/tools/testing/selftests/kvm/lib/guest_modes.c
> @@ -5,7 +5,42 @@
>  #include "guest_modes.h"
>  
>  #ifdef __aarch64__
> +#include "processor.h"
>  enum vm_guest_mode vm_mode_default;
> +static void get_supported_psz(uint32_t ipa,
> +			      bool *ps4k, bool *ps16k, bool *ps64k)
> +{
> +	struct kvm_vcpu_init preferred_init;
> +	int kvm_fd, vm_fd, vcpu_fd, err;
> +	uint64_t val;
> +	struct kvm_one_reg reg = {
> +		.id	= KVM_ARM64_SYS_REG(SYS_ID_AA64MMFR0_EL1),
> +		.addr	= (uint64_t)&val,
> +	};
> +
> +	kvm_fd = open_kvm_dev_path_or_exit();
> +	vm_fd = ioctl(kvm_fd, KVM_CREATE_VM, ipa);
> +	TEST_ASSERT(vm_fd >= 0, "Can't create VM");
> +
> +	vcpu_fd = ioctl(vm_fd, KVM_CREATE_VCPU, 0);
> +	TEST_ASSERT(vcpu_fd >= 0, "Can't create vcpu");
> +
> +	err = ioctl(vm_fd, KVM_ARM_PREFERRED_TARGET, &preferred_init);
> +	TEST_ASSERT(err == 0, "Can't get target");
> +	err = ioctl(vcpu_fd, KVM_ARM_VCPU_INIT, &preferred_init);
> +	TEST_ASSERT(err == 0, "Can't get init vcpu");
> +
> +	err = ioctl(vcpu_fd, KVM_GET_ONE_REG, &reg);
> +	TEST_ASSERT(err == 0, "Can't get MMFR0");
> +
> +	*ps4k = ((val >> 28) & 0xf) != 0xf;
> +	*ps64k = ((val >> 24) & 0xf) == 0;
> +	*ps16k = ((val >> 20) & 0xf) != 0;
> +
> +	close(vcpu_fd);
> +	close(vm_fd);
> +	close(kvm_fd);
> +}

I think I'd prefer stashing this function in lib/aarch64/processor.c and
naming it aarch64_get_supported_page_sizes.

>  #endif
>  
>  struct guest_mode guest_modes[NUM_VM_MODES];
> @@ -18,20 +53,24 @@ void guest_modes_append_default(void)
>  #ifdef __aarch64__
>  	{
>  		unsigned int limit = kvm_check_cap(KVM_CAP_ARM_VM_IPA_SIZE);
> +		bool ps4k, ps16k, ps64k;
>  		int i;
>  
> +		get_supported_psz(limit, &ps4k, &ps16k, &ps64k);
> +
>  		vm_mode_default = NUM_VM_MODES;
>  
>  		if (limit >= 52)
> -			guest_mode_append(VM_MODE_P52V48_64K, true, true);
> +			guest_mode_append(VM_MODE_P52V48_64K, ps64k, ps64k);
>  		if (limit >= 48) {
> -			guest_mode_append(VM_MODE_P48V48_4K, true, true);
> -			guest_mode_append(VM_MODE_P48V48_64K, true, true);
> +			guest_mode_append(VM_MODE_P48V48_4K, ps4k, ps4k);
> +			guest_mode_append(VM_MODE_P48V48_64K, ps64k, ps64k);
>  		}
>  		if (limit >= 40) {
> -			guest_mode_append(VM_MODE_P40V48_4K, true, true);
> -			guest_mode_append(VM_MODE_P40V48_64K, true, true);
> -			vm_mode_default = VM_MODE_P40V48_4K;
> +			guest_mode_append(VM_MODE_P40V48_4K, ps4k, ps4k);
> +			guest_mode_append(VM_MODE_P40V48_64K, ps64k, ps64k);
> +			if (ps4k)
> +				vm_mode_default = VM_MODE_P40V48_4K;
>  		}
>  
>  		/* Pick the largest supported IPA size */
> -- 
> 2.30.2
>

Thanks,
drew 


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 5/5] KVM: selftests: arm64: Add support for VM_MODE_P36V48_{4K,64K}

[Andrew Jones]

On Thu, Dec 16, 2021 at 12:31:35PM +0000, Marc Zyngier wrote:
> Some of the arm64 systems out there have an IPA space that is
> positively tiny. Nonetheless, they make great KVM hosts.
> 
> Add support for 36bit IPA support with 4kB pages, which makes
> some of the fruity machines happy. Whilst we're at it, add support
> for 64kB pages as well, though these boxes have no support for it.
> 
> 16kB is left as a exercise for the eager reviewer.
> 
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> ---
>  tools/testing/selftests/kvm/include/kvm_util.h      | 2 ++
>  tools/testing/selftests/kvm/lib/aarch64/processor.c | 8 ++++++++
>  tools/testing/selftests/kvm/lib/guest_modes.c       | 4 ++++
>  tools/testing/selftests/kvm/lib/kvm_util.c          | 6 ++++++
>  4 files changed, 20 insertions(+)
>

Reviewed-by: Andrew Jones <drjones@redhat.com>


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 2/5] KVM: selftests: Initialise default mode in each test

[Andrew Jones]

On Thu, Dec 16, 2021 at 12:31:32PM +0000, Marc Zyngier wrote:
> As we are going to add support for a variable default mode on arm64,
> let's make sure it is setup first by sprinkling a number of calls
> to get_modes_append_default() when the test starts.
> 
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> ---
>  tools/testing/selftests/kvm/aarch64/arch_timer.c       | 3 +++
>  tools/testing/selftests/kvm/aarch64/debug-exceptions.c | 3 +++
>  tools/testing/selftests/kvm/aarch64/get-reg-list.c     | 3 +++
>  tools/testing/selftests/kvm/aarch64/psci_cpu_on_test.c | 3 +++
>  tools/testing/selftests/kvm/aarch64/vgic_init.c        | 3 +++
>  tools/testing/selftests/kvm/kvm_binary_stats_test.c    | 3 +++
>  tools/testing/selftests/kvm/kvm_create_max_vcpus.c     | 3 +++
>  tools/testing/selftests/kvm/memslot_perf_test.c        | 4 ++++
>  tools/testing/selftests/kvm/rseq_test.c                | 3 +++
>  tools/testing/selftests/kvm/set_memory_region_test.c   | 4 ++++
>  tools/testing/selftests/kvm/steal_time.c               | 3 +++

I wish there was a better way to set the defaults for each test
without requiring a function call to be put at the beginning of
each test. Maybe we should create a constructor function? I.e.

diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index f307c9f61981..603e09be12ae 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -148,7 +148,7 @@ LDFLAGS += -pthread $(no-pie-option) $(pgste-option)
 # $(TEST_GEN_PROGS) starts with $(OUTPUT)/
 include ../lib.mk
 
-STATIC_LIBS := $(OUTPUT)/libkvm.a
+STATIC_LIBS := lib/init.o $(OUTPUT)/libkvm.a
 LIBKVM_C := $(filter %.c,$(LIBKVM))
 LIBKVM_S := $(filter %.S,$(LIBKVM))
 LIBKVM_C_OBJ := $(patsubst %.c, $(OUTPUT)/%.o, $(LIBKVM_C))
diff --git a/tools/testing/selftests/kvm/lib/init.c b/tools/testing/selftests/kvm/lib/init.c
new file mode 100644
index 000000000000..6f92a85aa263
--- /dev/null
+++ b/tools/testing/selftests/kvm/lib/init.c
@@ -0,0 +1,6 @@
+#include "guest_modes.h"
+
+void __attribute__((constructor)) main_init(void)
+{
+#ifdef __aarch64__
+       guest_modes_set_default();
+#endif
+}


Thanks,
drew


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel